TSA agents forced crying 4-year-old to undergo pat-down after hug - <AUTHOR> http://www.washingtonpost.com/national/grandma-tsa-agents-forced-crying-4-year-old-to-undergo-tsa-pat-down-at-kan-airport-after-hug/2012/04/25/gIQAojLohT_story.html

<comment><USER2> Honestly I find the victim's quotes and attitudes toward this incident very disturbing. For example:

> claiming TSA treated her daughter "no better than if she had been a > terrorist."

> "There was no common sense and there was no compassion," Croft said. "That > was our biggest fault with the whole thing — not that they are following > security procedures, because I understand that they have to do that."

So basically, it's fine to treat _terrorists_ this way, just not me and my family, because we aren't terrorists (but that Muslim family over there might be). Similarly, I don't mind following security procedures, because those are all for the good, I just don't think they should apply to people like us.

How can they fail to see that this is _how the system works_? This is what it is designed to do. There was no mistake made here. This is what you voted for.

No. The problem is not in how we apply these invasive, wasteful, and humiliating security procedures. The security procedures _are_ the problem. The problem is not in who in particular is labeled a suspected terrorist. It's how we treat _people_.

I think the company I keep makes me think that everyone must share my perspective that the TSA is a huge waste of money and time -- makes me forget that a lot of Americans actually believe these things are making them safer and working as intended, and that incidents like this are just small accidents when the security apparatus gets turned on the wrong people. It's a sobering realization.

<comment><USER3> This is a situation that has many different points...

A) Yes the grandmother's attitude was biased with those quotes and further highlights why America is in this situation in the first place. Americans and terrorists are both made up of people who believe stupid crap and treat others based on those beliefs. This is not a situation we can fix.

B) Yes the TSA Agents actually acted like they should have. If I was trying to smuggle something in I would certainly have a kid along to go through security first and then trained such that were my smuggling attempt to fail to attempt some sort of backup plan by running up to me distressed to grab the item from me. So their reaction was entirely smart to re-screen her. This is also not a situation that could be fixed.

C) This whole incident further proves that their process is just not going to work like they want it to. On paper its great, but in reality its just not going to fly. When their detection capabilities are reliant upon either shooting X rays or molestation you're going to end up pissing off enough people to get your company voted off the corporate island. This is a situation that we can fix, get rid of a lot of the security theatre and point B and C goes away.

</comment></comment><comment><USER4> The worst thing we've learned is that all we have to say is "we followed standard procedures" and we get a free pass, regardless of how egregious standard procedures or implementations may be.

<comment><USER5> I think the TSA clerks here can legitimately be charged with child molestation - follow my logic:

When I have been patted down by TSA they are always careful to ask for my consent before they sexual assault me. Since I am past the age of consent I can give it to them. Here the TSA is relying on the parents to provide the consent to sexually assault their child. I don't think a parent can legally give that consent. If I were a DA I would charge both the TSA agent and the parent with criminal child abuse.

<comment><USER6> Under your logic teens can't be arrested.(you get patted down(aka molested) in the process)

</comment></comment></comment><comment><USER7> "She said TSA agents had shouted at the girl, telling her to calm down and saying the suspect wasn't cooperating."

What suspect? Suspected of what? This kind of pervasive cop-speak strips language of relevant meaning.

<comment><USER8> Oh, cop-speak preserve deep meaning. It clearly communicates the mindset of the speaker, which is an important part of language. When considered in context (not spoken by a cop), it telegraphs the mental illness of the speaker.

</comment></comment><comment><USER9> My suspicion is that, in the (presumed) absence of actual terrorists, the screeners are tested using 'secret shopper' style undercover people. They might try a wide range of scenarios, from 'oops, I forgot I left that gun in my trouser pocket'[1] through to 'can little Molly sneak an extra-large toothpaste?'

Anyone failing the test gets demoted/retrained/shouted at, until all compassion and common sense is burned out.

[1] From what I recall of the recent ex-TSA director, they fail even the most egregious of suitcase weapon plants, including firearms.

Edit: It doesn't appear to get much of a mention in Kip Hawley's Editorial in the WSJ[2], but [http://abcnews.go.com/Blotter/loaded-gun-slips-past-tsa- scre...](http://abcnews.go.com/Blotter/loaded-gun-slips-past-tsa- screeners/story?id=12412458) mentions figures of easily 50+%, from 2010

[2] [http://online.wsj.com/article/SB1000142405270230381540457733...](http://online.wsj.com/article/SB10001424052702303815404577335783535660546.html)

<comment><USER10> I have a hard time believing that they'd be doing testing in live airports with real (or real-looking) firearms. It's just not nearly controlled enough to be safe. Imagine a wackjob grabbing the improvised knife they always carry in their bag and trying to be a hero.

<comment><USER9> I edited my original post to include a link to some 2010 stats, and just found [http://www.nbcdfw.com/news/local/TSA-Agent-Slips-Through- DFW...](http://www.nbcdfw.com/news/local/TSA-Agent-Slips-Through-DFW-Body- Scanner-With-a-Gun-116497568.html) from sept '11 as well.

I'm not sure how they could run realistic tests in any other way than with (simulated) contraband in a public screening queue, ideally with different people and situations each time.

<comment><USER8> They insert simulated images of contraband in the scanner monitors.

</comment></comment></comment></comment><comment><USER11> I flew for the first time last weekend since the TSA implemented the pat- downs.

I went through the backscatter while the TSA scanned my bags, one being a stylus with a substantial aluminum shaft. I was paranoid it would be deemed a weapon. During the screening, I witnessed a pregnant woman go through an examination.

I do not feel safer. In fact, my fear has shifted.

</comment><comment><USER6> So We shouldn't have pat-downs for kids? this article reminds me of the saying "Security is only as good as your lowest hanging fruit".

<comment><USER12> Your desire to touch my low-hanging fruit is the problem here.

<comment><USER6> If certain people are exempt from pat-downs then pat-downs are useless.

<comment><USER13> This presupposes that there is no mechanism for assessing the relative threat of a passenger. Basically every passenger is an equal threat and so everyone needs to be processed identically.

Of course that is a non-sensical assumption but 'profiling' is considered a worse evil and so the TSA terrifies little children in order to avoid the evils of profiling.

<comment><USER14> > Of course that is a non-sensical assumption

Maybe it is, maybe it isn't, but there is no "of course" about it! While I think the moral indignation about profiling is justified, the real fear is that a system that relies heavily on profiling is very weak to attack by learning the profile and planning accordingly.

</comment></comment></comment></comment></comment><comment><USER15> It looks like the trick is just to buy your way through: [http://www.latimes.com/news/local/la-me-tsa- screeners-201204...](http://www.latimes.com/news/local/la-me-tsa- screeners-20120426,0,4508192.story)

</comment><comment><USER16> (My opinion is that the whole system is stupid, pointless and performs some theatre at the windows while the doors are wide open. Someone determined will be able to get through, or just attack somewhere else. All you want to do is catch the idiots, which pre-911 screening did just fine. The response to terrorism is to not be terrorized and to live well, not to amplify the little things they actually do.)

We should be very careful about "common sense" and screeners making their own minds up about things. It is far better that they follow a consistent set of rules. This for example is why the liquids rule is a good one from a screening point of view. There is no discretion left to the screeners and they don't have to entertain arguments about which liquids are harmless and which aren't, types of liquids, size of containers etc. Think carefully - do you really want each screener enforcing their own views? How trivial do you think they would be to outsmart by someone determined?

The American political system especially as espoused in the media where trivial things are inflated to make stories, there is relentless hounding of politicians who are rational, it is cheaper to have controversial talking heads instead of research and reporting, someone always has to be to blame, and issues are presented as black or white soundbites means that no politician can reasonably try to rein in the system or have detailed discussions on risks, spending, tradeoffs etc.

</comment><comment><USER17> This is what happens when you give minimum wage idiots that would normally be working at McDonalds fake badges and power. I know for a fact there are a few good, knowledgeable, professional TSA people at every airport, but the majority of them around the country are not. The majority of them are completely incompetent, completely unqualified, and glad they get to tell someone what do for 8 hours.

</comment><comment><USER18> First candidate to promise to dismantle TSA will sweep the presidential election.

</comment><comment><USER19> Just out of curiosity, can someone explain to me why TSA articles tend to make their way to the front page of HN. I know there are some pretty random topics that HN'ers are interested in, but usually there is at least some small link to tech/startups/hacking. Not sure what the connection is here.

<comment><USER20> Security.

</comment></comment><comment><USER21> Where do you get 50,000 TSA agents to do this sort of thing? That is the question that has to be asked by all Americans; [http://www.caseyresearch.com/articles/ascendence- sociopaths-...](http://www.caseyresearch.com/articles/ascendence-sociopaths- us-governance)

</comment><comment><USER22> They should be charged with harassment and physical assault of a child.

</comment><comment><USER23> On the one hand, the TSA agents could have been more sensitive. On the other hand, the parents could probably have prevented the whole thing by not letting the girl run off to hug grandma.

My dad was career miltary and so was my ex. I spent much of my life around environments where security was taken seriously. You just don't let kids goof off in such environments. Somewhat similarly, I also grew up with shotguns hung on the wall above my parents' bed. They were not loaded but there were two shells laying in the groove on the barrel so they could be loaded quickly if necessary. The rest of the ammo was locked up in dad's closet and kids did not go in dad's closet. Guns were not something children played with and there were no accidents because the adults in the house just did not allow it to happen.

</comment><comment><USER24> _Isabella had just learned about "stranger danger" at school_

What? Does she use a time traveling bus to visit 1990? I thought schools had wised up to the fact that virtually all harm to children (excluding traffic fatalities) is brought people the child knows.

I hate the TSA, but I see the dark humor of fear mongering on both sides of this story: Americans' fear of terrorists and children's instilled fear of strangers. As a traveler, I'm tired of the former, and as an adult male who happens to like children and strikes up conversations with strangers, I'm tired of the latter.

<comment><USER25> Attempted child abductions happen all the time (source: constant police reports in my local media). It's insane to suggest we don't teach kids street smarts just because you like talking to strangers.

<comment><USER24> _It's insane to suggest we don't teach kids street smarts_

I never said anything of the sort.</comment></comment></comment>


The Trello Tech Stack - <AUTHOR> http://blog.fogcreek.com/the-trello-tech-stack/ <comment><USER3> _Use things that are going to work great in two years._

This is one of those remarks that is obvious to someone who knows what it means but mysterious to someone who doesn't. So what does it really mean?

\- Don't use a technology that, no matter how good, is just too niche for widespread adoption?

\- Don't use a technology that we'll have trouble finding programmers to support?

\- Look for stuff that will take better advantage of impending improvements in hardware and communications?

\- Avoid frameworks for which support may wane?

\- Avoid technology that we may have to support ourselves?

\- Avoid anything proprietary whose owner may disappear?

\- Make sure that whatever you choose, it will work well on mobile technologies, even as native apps?

\- Choose technologies abstract enough to minimize our costs but not too abstract to be inefficient?

\- Any combination of the above?

\- What considerations have I missed?

<comment><USER4> My (minimal) criteria is it has to be packaged in Debian.

A couple years ago I considered both MongoDB and CouchDB immature for that reason. The recent confusion of CouchDB/Couchbase etc. shows that was a reasonable view.

<comment><USER5> I'm not sure when they started, but 10gen now packages and distributes their own MongoDB Debian and Redhat packages. It seems like the right move, so that they're not beholden to the update schedule of the distros.

<comment><USER6> Yes!!! The development cycle of software like MongoDB, RabbitMQ and so on, is much faster than that of Debian or any other Linux distro. The Debian package is fine for dabbling or low volume use, but for any serious app, you MUST go direct to the developers and use their latest stable release. That is what they support best on their mailing lists, and that is where most bugs are already fixed.

A lot of software development teams are releasing their own RPM and .DEB Linux binary packages for just that reason, to encourage people to use up to date packages instead of the stale OS distro packages.

In a way, it's rather like security updates. Who would refuse to install security updates because it's not part of the Ubuntu 10.4 LTS release? Almost nobody even thinks of doing that. So why would you use old obsolete releases of mission critical software?

<comment><USER7> > why would you use old obsolete releases of mission critical software?

"If it ain't broke, don't fix it"

Because it's mission critical, and you can't afford for it to break. Once you hit a certain complexity, upgrades almost always break something:

APIs change. Undefined behavior changes. New bugs are introduced. A feature critical to your app starts performing worse. The above changes break something else you depend on (libraries, proxies, etc.)

Upgrading to a significantly changed version of a mission-critical app/library/language is a lot of work, and is sometimes impossible: many projects couldn't be reasonably ported to Python 3 if they wanted to; a lot of important libraries don't work on Python 3.

This is exactly why bug and security fixes are often backported into old versions. Python 2.5 is still receiving security patches. Apache 1.3 was maintained for years after 2.0 was considered stable.

<comment><USER5> Yes, especially with a database you can't just pull in new updates and expect them to work. It involves reading the release notes and doing a lot of testing. (much of it is automated)

</comment></comment></comment></comment></comment></comment><comment><USER8> Not to pick on him, because I think the overall received wisdom on new tech has shifted and morphed a great deal over the last few years, no doubt including Joel's...but I can't resist pointing out that, my how times have changed...

<http://www.joelonsoftware.com/items/2006/09/01.html>

In other words, it's nice to see Joel greenlight something like this, I'd say it's kind of a sign of the times in terms of the industry's overall comfort level with what would be termed 'hip' technology, or 'new' or whatever moniker you want to attach to it.

<comment><USER9> To Joel's credit, that was over 5 years ago - which is, like, 35 dog-years ago (which are even shorter than tech-years).

<comment><USER8> Agreed, completely, just pointing out, again, time's is different...

</comment></comment></comment><comment><USER10> This part caught my eye:

"The Socket.io server currently has some problems with scaling up to more than 10K simultaneous client connections when using multiple processes and the Redis store, and the client has some issues that can cause it to open multiple connections to the same server, or not know that its connection has been severed."

I wonder if they ran into redis's hard-coded 10k connection upper-limit. As it turns out, their configuration for "unlimited" connections actually has a cap of 10k. I believe in master this is going away, but if you need more than 10k connections on redis <= 2.4, you need to manually patch the daemon, in case anyone else runs into this.

</comment><comment><USER11> How vulnerable are all javascript approaches in terms of injection type attacks? Do apps like node and mongo effectively prevent them, or is it still possible to shoot yourself in the foot? I've read some off hand comments along the lines of "as long as you're not a total moron you have nothing to worry about", but that sounds a lot like what was said about sql injections and xss before exploiting them went mainstream and it turned out everyones apps were filled with them. Has anyone audited a real world app built with a stack like this and come away with any experience to share?

<comment><USER12> You CAN do injections to MongoDB code. An injection is basically 'allow user input to interfere with code' so for mongoDB assuming the query is a string you can do '{name: ' + user_input + '}'. and user_input, without sanitizing it (which is simpler, just converting it to a string) could be: {'$where': ...}

[http://www.mongodb.org/display/DOCS/Do+I+Have+to+Worry+About...](http://www.mongodb.org/display/DOCS/Do+I+Have+to+Worry+About+SQL+Injection)

<comment><USER13> Unlike SQL, which you have to build as a string, the natural approach in JavaScript (even for junior devs) is to use an object literal to build the query. And then you get escaping for free.

</comment></comment></comment><comment><USER12> Just had to share this:

Our stack is Redis, MongoDB, Nginx, SCSS, HAML, Coffee, Rails and NodeJS. I'm extremely happy with these choices.

Recently me and a friend did a small weekend project: www.bubblefap.com (nsfw) The design and code is a homage to ugliness. We only used PHP-ActiveRecord and that's it. and I had so much fun!

I just hacked away! I was cowboy coding, hacking away, and I didn't need to think about frameworks and architecture and integration with fog and hacking Rack to support flash uploads. Oh, good times :-)

</comment><comment><USER14> Node.js, Redis, _and_ MongoDB? They've gone full web-scale!

In all seriousness, though, it looks like they are using Redis for exactly the right reasons, and the larger architecture is pretty much the definition of a sane forward-looking design.

</comment><comment><USER15> I think this is the best write-up I've seen of a full Javascript/CoffeeScript stack.

I've always been hesitant to get too far away from my LAPP(ython) stack, but I'll almost certainly be hacking something together with these components to see how I like writing everything in CoffeeScript.

</comment><comment><USER16> Nice job, guys. I love this architecture. This looks a lot like my dream stack for future web work.

Would love to hear some more about the "bleeding" experiences you've had.

<comment><USER17> Socket.io: <https://github.com/LearnBoost/socket.io/issues/686>

MongoDB (on FreeBSD): <https://jira.mongodb.org/browse/SERVER-3927>

Redis: <https://github.com/antirez/redis/issues/91>

. . . are a few.

<comment><USER18> dodger: I'd be curious to know where y'all bled over CoffeeScript or Backbone, if you don't mind sharing.

<comment><USER17> Actually, those have been great (and thank you for making them)! We had an outage during beta because somebody forgot to wrap an array comprehension assignment (CoffeeScript) and we didn't test right. Backbone has been really good, and while we had some complaints, I think many have been resolved in more recent versions - we're on something ancient.

<comment><USER19> > "we're on something ancient"

Isn't that one of the major drawbacks of using a bleeding edge Tech Stack? Albeit mitigated by choosing to use code that is easy to comprehend/maintain (like backbone!)

What do you think you will do? E.g. With backbone: Merge up, status quo -- maintain your branch, change libraries, or something else?

<comment><USER17> Agreed, it made some necessary, non-backward-compatible changes as a result of being such a young project. I think it is likely that we will upgrade to a newer version soon.

</comment></comment></comment></comment></comment></comment><comment><USER20> "We custom-built a client-side Model cache to handle updates and simplify client-side Model reuse."

Is this related at all to backbone.iosync.js?* Or, if not, is it something that Fog Creek will be open to sharing in the future?

* <https://github.com/logicalparadox/backbone.iobind>

</comment><comment><USER21> Question: They tout MongoDB's "generally fast" read speed. Any idea what this is in relation to? In "fast", are they suggesting that it is faster than a typical RDBMS? If so, does anyone know of any supporting benchmarks or breakdowns of how they accomplish this and to what extent?

Its news to me.

</comment><comment><USER22> Does anyone know of any open-source projects out there using a similar stack? I'm particularly interested in learning more about client-side view rendering with Backbone and Mustache... server side can be any of Node, Rails or Django.

<comment><USER23> I've been using CoffeScript and backbone.js a lot lately, it really has changed my approach to web apps (keep the server light, just transfer json back and forth). The approach that has worked the best for me is to keep pages such as the landing and authentication as traditional rendered pages and save the client side rendering for the user-authenticated 'meaty' parts of the app.

If you want to look a couple really-small examples I have a couple apps on github I used for learning backbone.js/coffeescript. They both use the underscore.js template engine.

<https://github.com/jwarzech/realtime_hn> Backend is sinatra, takes a HN story url and polls the comments in reverse chronological order.

<https://github.com/jwarzech/inspire_board> Backend is rails, polls images from dribbble and lets you save your favorites.

<comment><USER24> I like the idea of doing authentication and OpenId in a traditional web stack, while doing the "app" portion in a javascript backbonejs stack.

That lets you use existing libraries (Devise, etc.) for the authentication and once that's out of the way, you can go have "fun" with coffeescript and mustache. :)

</comment></comment></comment><comment><USER25> "We use a really excellent async library"

Does anyone happen to know if this is an open source or in house library?

<comment><USER26> It's open-source: <https://github.com/caolan/async>

<comment><USER27> Yes, I've seen it on HN before and it's the bee's knees.

</comment></comment></comment><comment><USER28> This is a great write up. We have a very similar stack for a separate team based project we're working on. The only difference is the front end stack, and we have tornado sitting back on the backend to serve http requests when needed. What's the benefit of Mustache/Backbone over just writing your own client side templating, in a practical sense? Last time I used a KVO framework, there was a performance penalty that I did not like. Does Backbone have any performance issues? Or even mustache? What's the trade off of having these frameworks versus generating custom tempting code?

</comment><comment><USER29> This post raises a few questions for me... and perhaps some one more versed in these stacks can provide answers.

\- Does the use of CoffeeScript alleviate the MC Escher-esque quality of callbacks within closures involved in working with Javascript on both the server and client (and the data store)? I can totally see the appeal of CoffeeScript's syntax. Giving the programmers something different to look (and learn) at probably provides some cognitive benefit as well.

\- Is there an acronym/name (ala LAMP) for the Node/MongoDB/Redis stack?

<comment><USER30> 1\. CoffeeScript doesn't alleviate this yet. I doubt will be much progress on this until we find a good way to implement defer/async. [https://github.com/jashkenas/coffee- script/issues/350#issuec...](https://github.com/jashkenas/coffee- script/issues/350#issuecomment-322321)

I am writing a sizable Node app myself. In the end, you just get used to the callback style.

<comment><USER29> I am assuming that CoffeeScript should be able to add new syntax where the need arises, as the number of CoffeeScript users is probably still pretty low and able to adapt to change.

From what I can tell, CoffeeScript - and the Javascript/Node world - <USER2> really benefit from something like F#'s workflow/computation-expression syntax, which will take a fairly straight-forward readable statement and behind the scenes de-sugar the hell out of it into a bunch of closures.

[http://en.wikibooks.org/wiki/F_Sharp_Programming/Computation...](http://en.wikibooks.org/wiki/F_Sharp_Programming/Computation_Expressions)

EDIT: search for "De-sugared syntax" on this page for an example.

<comment><USER18> That's correct -- because we compile to "standard" JavaScript, we're at least somewhat comfortable introducing significant changes (even to the syntax) where desirable. Even if you never get around to updating older pieces of code, all of the compiled JS continues to be fully compatible and interoperable with the newer stuff.

</comment></comment></comment></comment><comment><USER31> "Client-side MVC" is kind of unfair to backbone.js. I haven't confirmed it myself, but I've seen people explaining how it's also suited to server-side rendering, which means it's not ruled out for competent authors doing progressive enhancement.

[http://lostechies.com/derickbailey/2011/09/26/seo-and- access...](http://lostechies.com/derickbailey/2011/09/26/seo-and- accessibility-with-html5-pushstate-part-2-progressive-enhancement-with- backbone-js/)

</comment><comment><USER32> Great write up. Hopefully that ephemeral data in Redis can scale.

They could have used something like Pusher for about half of their implementation (the websockets, message pushing, polling, etc)

I bring this up because I built an OSS Pusher clone[1], for those people that want to deploy their own, built on the Play! framework.

[1] <https://github.com/danbeaulieu/PushPlay>

<comment><USER33> There's also this ruby implementation that plays nice with all the pusher libs <https://github.com/stevegraham/slanger>

<comment><USER32> Yes, I built my version almost in tandem with Steve. Pusher hosts their js lib on github under the MIT license which made building these things supremely easy.

</comment></comment></comment><comment><USER34> If you could do this over again, would you still choose socket.io for websockets, or would you go with something else like PusherApp?

<comment><USER17> Today I'd consider going with a service, but first I'd see how <https://github.com/LearnBoost/websocket.io> looks. Most of our problems have been with abstractions in the socket.io client and scaling issues with server process chatter. The chatter shouldn't be necessary with websockets-only.

<comment><USER35> With <https://github.com/LearnBoost/engine.io> the abstractions are greatly simplified, which is what I'm announcing today.

The socket.io codebase has been shrunk dramatically, and it's as a result easier to scale/maintain.

<comment><USER17> That's great!

</comment></comment></comment></comment><comment><USER36> When I first saw Trello I "felt" that something cool was going on under the hood. So I did a little poking to see what JS technologies you guys were using. The piece I was most excited to learn about was backbone. Had never seen it before and was really impressed by the space it was filling and what it's capable of. Thanks for the write up.

</comment><comment><USER37> Great writeup!

Can you guys share what (if any) test suites were used? Was curious on this point.

</comment><comment><USER38> I have read about various issues in using HAProxy and socket-io (websockets mode). I am currently working on a project that's heading towards that direction - anyone has anything to share on that front?

<comment><USER39> nginx supports http1.1 reverse proxying as of a few weeks ago <http://wiki.nginx.org/HttpProxyModule>

Mikito Takada (Zendesk) has some really helpful information regarding socket.io + haproxy specific workarounds via [http://blog.mixu.net/2011/08/13/nginx-websockets-ssl-and- soc...](http://blog.mixu.net/2011/08/13/nginx-websockets-ssl-and-socket-io- deployment)

<comment><USER38> Thanks. Looks like it's available only on the development version - I wonder if it's production ready...

Also - I took a look at the article - it does not mention using a total node approach of using something like node-http-proxy for load balancing. Any idea on such a set-up? I need HTTPS as well.

<comment><USER40> A pure-Node approach is a lot simpler: you can just use Node's HTTPS (if you don't care about load balancing) and attach Socket.io to it. Just don't put your Socket.io server behind Nginx if you want WebSockets support.

With load balancing, I would recommend going with Stud/Stunnel and HAProxy. Terminating SSL with a specialized piece of software is nicer (separate SSL overhead to another box), and using a separate load balancer allows for more flexible options, e.g. serving static assets from Nginx. There is nothing wrong with node-http-proxy, it's just that these two projects have been around for longer and are better understood from an ops perspective.

You cannot use round robin load balancing, you need to have at least IP-based stickiness with Socket.io for now. Well, you can round robin if you use the Redis store but you'll run into inefficiencies with <https://github.com/LearnBoost/socket.io/issues/686> . I'll do a bit more coverage on other SIO deployment-related issues once I get the chapter on Socket.io finished for my (free) book in a few weeks.

If I were starting now, I'd just ignore the Flash sockets transport since it makes the whole stack more complex due to not looking like HTTP to load balancers, and start with Engine.io / WebSocket.io to take advantage of their simplicity.

</comment></comment></comment></comment><comment><USER41> Great write-up... It made me want to go play with these exact components!

</comment><comment><USER42> Love the stack, already working on something similar (mine involves nginx, though.) Given we haven't deployed, it's refreshing to see a similar diagram from those already out in front.

</comment><comment><USER43> Great writeup, I wish more companies did this: be open about their tech stack and experiences.

This inspired me to write about how I think, they could've gone even further: [http://wuher-random.blogspot.com/2012/01/single-page-web- app...](http://wuher-random.blogspot.com/2012/01/single-page-web-apps-and- rest.html)

<comment><USER17> Yep! We intend to use our shiny, well-organized REST API for future development and probably port existing functionality to it, too!

</comment></comment><comment><USER44> I am wondering where and how you terminate the HTTPS connections for websockets. Can you share some details on that?

</comment><comment><USER45> " sending changes to Models down to browser clients". _Down_ to the client? He draws the stack with the client on top and the storage at the bottom, and then says 'down to the client'? I have a collegue who also does this. I never figured out why. Can anyone explain possoble reasoning behind this wording?

<comment><USER46> Because from the client downloads assets and we are more used to thinking about what happens from the client's pov.

Alternatively, it could be for the same reason people usually say they're going "up" to a big city, regardless of the direction of travel. The server is the hub and all the clients are _down_ from it. This possibly dates back to when towns/fortifications were usually located on hills.

</comment></comment><comment><USER47> Awesome job guys. I'm really digging this stack. It is giving me motivation to learn more about Backbone.js

</comment><comment><USER48> Does it just go without saying these days that jQuery is required? Trello uses it, as well as jQuery UI, date.js, and highcharts.js far as I can tell.

I mention it because they mention other parts of the client stack (Backbone.js for ex), but not said libs.

</comment><comment><USER49> It seems Socket.io has not had any contenders yet..

<comment><USER50> Socket.io is a great library. When it comes to a contender, I see Server-side Events [1] as a contender to WebSockets. I feel it's a simpler architecture which fits HTTP/REST better than WebSockets in the general case. I built a couple of small experiments/examples using CoffeeScript and Redis [2]; the middleware.coffee in particular is designed to be used together with REST, where if you send "Accept: text/event-stream" you will get updates to the resource(s) on that URL.

1: <http://dev.w3.org/html5/eventsource/> 2: <https://gist.github.com/1560654>

<comment><USER51> Unfortunately, eventsource isn't supported by any current version of IE. It'll be awhile before ES can replace your AJAX/websocket libraries.

<http://caniuse.com/#search=eventsource>

<comment><USER50> On the plus side, it's supported by Opera Mini and iOS.

For IE and others (Android, I'm looking at you!), I have yet to try it and so can't vouch for it, but there exists a polyfill: <https://github.com/Yaffle/EventSource>

</comment></comment></comment></comment><comment><USER52> Surprised that they didn't incorporate any ASP.NET MVC in there. I was under the impression they were a Microsoft shop?

</comment><comment><USER53> Pretty cool; any insight as to how they deploy code to production servers?

</comment><comment><USER54> Very cool. What do you use for search?

<comment><USER17> Right now? Just what we can get out of MongoDB. We need better search, and it should be coming soon. Vote it up! [https://trello.com/card/board/better- search/4d5ea62fd76aa113...](https://trello.com/card/board/better- search/4d5ea62fd76aa1136000000c/4e9ec6c79335096aa4658e32)

</comment></comment><comment><USER30> The UI could be less cluttered.</comment>


Australia in Crosshairs with Over 2,300 Dumped Password Hashes | NovaInfosec.com - <AUTHOR> http://www.novainfosecportal.com/2012/08/02/xxx-more-password-hashes-dumped/ <comment><USER2> Four relatively small lists of hashes (2300 total) f, of which only two are actually australian sites.

If Australia is "in Crosshairs" based on this, what kind of target was on the USA when 6 million+ LinkedIn password hashes made a break for freedom?</comment>


Apprenticeships Help Close the Skills Gap. So Why Are They in Decline? - <AUTHOR> http://online.wsj.com/news/articles/SB10001424052702303978304579473501943642612

<comment><USER2> One issue is that it's not really possible to form long term employment contracts that are legally defensible. Only executives seem to get these kinds of packages. What this means is that employers are unwilling to invest in long term training. They either dumb down the work to make use of a fungible mass of proletarians or encourage workers to train themselves and get promotions through job hopping every 18 months.

It's mad, and the workarounds are no longer sufficient. There is an alternative that does not require unionization that the western world used for > 2,000 years. The ancients had technology and engineering also, but not much ability to use fossil fuels. They were not idiots. The skilled trade system of antiquity may have been harsh, it may have reduced individual autonomy, but it has a track record of success.

>Some companies also fear that employees will leave for better-paying jobs almost as soon as they've learned their required skills. For them, an apprenticeship amounts to training workers for other companies.

Yes. It is unless penalties for breaking the contract can be enforced.</comment>


Over 12,000 MongoDB Databases Deleted by Unistellar Attackers - <AUTHOR> https://www.bleepingcomputer.com/news/security/over-12-000-mongodb-databases-deleted-by-unistellar-attackers/ <comment><USER2> Mongo only pawn in game of life.</comment>


AT&T's "5G E" network is not true 5G - sahin-boydas https://techcrunch.com/2019/01/07/att-is-lying-to-customers-with-5g-marketing/ <comment><AUTHOR> This is not the first time AT&T has pretended to have a newer network than their infrastructure actually provides.

[https://slate.com/technology/2012/03/4g-vs-3g-beware-of- the-...](https://slate.com/technology/2012/03/4g-vs-3g-beware-of-the-murkiest- most-confusing-labels-in-tech.html)

<comment><USER2> Off-topic:

Slate trying to comply with GDPR by forcing you to agree.

Nice try Slate.

</comment></comment><comment><USER3> > TechCrunch is a Verizon Media company

AT&T should also purchase some big news blog and write about lying in Verizon "Unlimited" plans...

<comment><USER4> Relevant: [https://news.ycombinator.com/item?id=18157869](https://news.ycombinator.com/item?id=18157869)

The linked article from before it was suspended: [https://web.archive.org/web/20181007202929/https://medium.co...](https://web.archive.org/web/20181007202929/https://medium.com/@dvorak/5g-got- me-fired-ce407e584c4a)

<comment><USER5> if you read a little further down on in the comments section, you'd see that the article is pretty much a clickbait/FUD piece, and the author has a history of writing clickbait/FUD articles.

</comment></comment></comment><comment><USER6> "TechCrunch is a Verizon Media company"

Regardless, I've generally come to expect nothing less from cell companies, and I guess big tech companies in general?

Cant wait until actual 5G comes out, assuming it won't be too much more expensive than current 4G plans.

<comment><USER7> Current 4G plans aren't "actual" 4G either.

</comment></comment><comment><USER8> This should serve as another strong example why mobile operating systems should be completely decoupled from carriers.

</comment><comment><USER9> Is this going to happen each and every time a new standard comes by.

Did this also not happen with 4G LTE?

I suppose the difference being that the use of the "E" wasn't approved by the regulating body like the use of LTE was?

<comment><USER10> We call it "4G LTE" instead of just "4G" ... because T-Mobile and AT&T labeled 3G HSPA+ as "4G" ... because Sprint's WiMAX was a fourth-generation technology but wasn't very good, so it was slower than decent 3G.

Similarly, when 5G is first available, it won't be very good, and really good 4G LTE will actually be faster. So AT&T is pre-emptively calling good 4G "5G".

It sure would be nice if consumers would understand that "5G" is not always better than "4G", it's just different, and one day will probably be better, but not today, not even this year. But marketing won't risk it, they'll just re-number and re-name, and make sure everyone is properly confused.

<comment><USER11> Was Sprint's Wimax, which they called 4G before they rolled out 4G/LTE, better than or inferior to 3G HSPA+?

<comment><USER12> WiMax was slower than HSPA+ (both under ideal conditions, of course).

</comment></comment></comment></comment><comment><USER13> Previous discussion: [https://news.ycombinator.com/item?id=18736378](https://news.ycombinator.com/item?id=18736378)

</comment><comment><USER14> this sounds like exactly the sort of thing a functioning FCC should be able to stamp out right quick.

<comment><USER15> That supposes that there is an actual definition of "5g" that anyone could be held to

<comment><USER7> 5G is a specific ITU standard, just like 4G was. [https://itu.int/dms_pub/itu-r/opb/rep/R-REP-M.2410-2017-PDF-...](https://itu.int/dms_pub/itu-r/opb/rep/R-REP-M.2410-2017-PDF-E.pdf)

</comment></comment></comment><comment><USER16> Things like this piss me off way more than I should allow them to. And now the cable industry is coming with "10G".

Everything[0] is just BS now-a-days.

[0] Obviously not "everything", but the amount of BS is staggering.

</comment><comment><USER17> This may seem like a naive question but since I'm currently in my job search (email is in my profile), I care a lot about it: is there any big company that isn't up to unethical shenanigans?

For almost every industry and/or major player in a particular industry that I look at, I get the idea that there are at least some unethical things going on at best. At worst the planet is being destroyed.

Which industries / companies are clean(ish)? I'd like to apply for those.

<comment><USER18> Tons, but they're all really boring.

For example the company that likely makes the carpet you walk on everyday if you're in A class office space is made by a company that's had a carbon neutral 2020 goal since 2007, and is in line to meet that goal now.

[https://en.wikipedia.org/wiki/Interface,_Inc](https://en.wikipedia.org/wiki/Interface,_Inc).

Patagonia is likely another one you've heard of. (Or i've fallen for their marketing hook line and sinker)

</comment></comment><comment><USER19> So what if carriers enable 5G-ish radio access, but still use a 4G core network? This is an inevitable transition strategy.

Should they still market this as some kind of progress towards 5G, or wait until "5G is finished" (which doesn't happen because there is always a next release...)?

<comment><USER12> _So what if carriers enable 5G-ish radio access, but still use a 4G core network?_

As a customer is there any way to notice? If it uses NR and delivers promised 1 Gbps speed (downhill with a tailwind) who cares what the core network is.

</comment></comment><comment><USER20> This article declares the tautology that 5G Evolution is not the same as 5G.

<comment><USER21> I mean, "5G Evolution" is an AT&T specific trademark specifically designed to mislead consumers.

</comment></comment><comment><USER22> Because of course it isn't.

</comment><comment><USER23> > Disclosure: Tech Crunch is a Verizon Media company

And just curious, who decides what "5G-e" means? Since it's supposedly a "beefed up 4G" then that implies that it's more advanced than 4G. So it is an improved system. Would "4G Plus" have made the author happier?

Disclosure: I am a reasonably happy Verizon customer so I really don't care what AT&T names their network in their marketing materials.

</comment><comment><USER24> Is it too much to expect that TechCrunch should prove in their article that the phone is still on 4G when it says it's on 5G? They are declaring a conspiracy, it's their job to provide evidence. I feel like not providing any evidence is wildly lazy journalism.

<comment><USER21> It's all over the internet, so you should be complaining more about this being the next tier above blog spam rather than taking this as TechCrunch's journalism.

That being said, AT&T isn't currently shipping any 5G capable handsets, so a 5G logo popping up on any of their phones is arguably a lie. According to AT&T's stated intentions, they're rebranding LTE with most of the latest extensions as "5Ge" and real 5G will be branded "5G+". None of this is particularly hidden information.

Additionally, none of this is new, AT&T is known for faking this information, like displaying LTE when it might be able to see a tower that's LTE capable, but you're not able to connect to LTE.

<comment><USER25> ah yes, "e" and "+", the two speeds.

Just like High Speed, and Full Speed.</comment></comment></comment>


Bitcoin laws are coming: US Senate launches virty currency probe - <AUTHOR> http://www.theregister.co.uk/2013/08/14/us_senate_bitcoin_probe/

<comment><USER2> eh, lobbyists will block this as much as possible. Unless Mastercard and Visa can make money from Bitcoin transactions this will never be any type of official currency the US and its businesses will adopt</comment>


Show HN: Using Twilio to extend the functionality of Siri - <AUTHOR> http://brianshaffer.com/using-twilio-to-extend-the-functionality-of-siri/

<comment><USER2> That's a cool idea for making a Siri API for anything, but in this application, the process is too slow to be worth it (you could open the Spotify app and type in "Lady Gaga" much quicker.</comment>


Emacs 25.2 released - <AUTHOR> https://www.gnu.org/software/emacs/news/NEWS.25.2 <comment><USER2> For people who are thinking "I like to try but i dont know how it will be useful since i use IDE XYZ all day", start with org-mode. Think org-mode as a tool to manage your todo lists, although it it much more than that.

<comment><USER3> To a developer not familiar with Emacs, I'd first recommend to start with Spacemacs so they don't have to configure anything nor learn Emacs shortcuts... then Magit (shipped as git layer in spa Emacs)... and then org- mode. I had a few developer friends of mine starting to use Emacs (Spacemacs) solely because of how good Magit is.

<comment><USER4> Projectile is a great tool too. Along with helm and perspective modes, you can switch between projects with a few keystrokes. Since projectile looks for .git by default, you can have a window in each perspective dedicated to git if you so wish.

Updating code across multiple projects and committing is so easy and frictionless.

<comment><USER5> I have been using helm and projectile to easily find files. How do you use it to make edits too?

<comment><USER4> Press C-z. Multiple edits can be achieved through marking in helm. I need to get a grips with this part myself - sometimes it's just easier to use dired.

Have you figured out how to switch between recent list and other match lists? Sometimes get stuck when a prefix matches bang on in recent and need to get to the other matches.

</comment></comment></comment></comment></comment><comment><USER6> No matter how many IDEs I try I always come back to emacs. I am spoiled for life.

<comment><USER7> What keeps you coming back?

<comment><USER8> To me, the appeal is that emacs is not so much a text editor, but a development and runtime environment for a custom programming language that happens to be designed with building text editors in mind.

For practically any given programming language I might want to use, emacs either supports it out of the box, or there is a package available that makes emacs support it.

Plus, it is kind of an integration machine in a much more comprehensive sense than most IDEs - inside emacs, I can run a shell, a web browser, a mail client (although I don't do that), a file manager, an IRC client, an audio player (including last.fm scrobbling!), read xkcd, and much more (don't get me started on org-mode, I am still just beginning to scratch the surface of that).

Being able to use it for so many purposes means that all the skill and muscle memory I acquire in using it pays of many times over across all those use cases. Any customization or extension I make is potentially available across all those use cases.

<comment><USER9> Nicely put... wish I had tried Emacs sometime. I can relate to what you are saying as I use Vim commands/shortuts everywhere from my Window manager, browser to custom key mapping of lkjh for arrow keys on my ergodox keyboards..

<comment><USER10> If you want to dip your toes in the water, I recommend Spacemacs. It uses VIM key bindings by default, and it goes to great lengths to help you find the functionality you're looking for.</comment></comment></comment></comment></comment>


Equations system with a black rectangle in LaTeX - <AUTHOR> https://gus-massa.blogspot.com/2020/06/equations-system-with-black-rectangle.html <comment><USER2> We struggled with this, especially for collaborative projects, and it was one of the motivations for WritelaTeX (now Overleaf[1]). Having more eyes on the problem (on a shared doc), especially the ability to loop in LaTeX experts, usually resolved many of these similar types of issues.

[1] [https://www.overleaf.com](https://www.overleaf.com)

Edit: for avoidance of doubt, I'm one of the founders of WriteLaTeX/Overleaf -- we've been providing LaTeX compilation-as-a-service for ~8 years now!

<comment><USER3> My colleagues and I use Overleaf for writing journal articles -- it's a really great tool. Thank you so much for writing it. I just really wish I could get my university [Oxford] to pay for it!

The bean counters don't have a problem paying Microsoft the amount of a small African nation's GDP per year, but I pay for my Overleaf pro account out of my own back pocket, and can't even claim it back on expenses...

<comment><USER4> > and can't even claim it back on expenses...

This is ridiculous, how on earth a serious university have come to that?. If you need a laptop or an external disk you need to also buy it yourself?

<comment><USER3> As far as I can tell, in my department Overleaf isn't liked because it's (a) a personal service, (b) a subscription, and (c) from a company that they don't have in a labyrinthine Oracle-provided, horrific and IE5 "preferred" online purchasing system [which typically takes ~8 months to be added to]. Of these (b) is really the biggest bugbear because my research fellowship has a fixed budget for a fixed term and monthly subscriptions are basically anathema to the way that they do accounting.

I have one piece of commercial FEM software that is only ever "licensed not sold" [annually] -- an annual license is ~£1k; a perpetual one ~£100k. It's quite clear that the company want to go in that direction. I hate it. I don't like subscriptions, and I get riled by central university finance departments every single time...

Ironically, laptops and hard drives etc are quite easy to purchase: they count as 'consumables' and I have never worked in a department that actually audits their lifecycle properly -- much like lab reagents, once they're bought, the university [or my interactions with it at any rate] doesn't seem to even care if the first thing I do is throw it in a bin. This is a Good Thing™ as far as I'm concerned. Having decent computing wherever I am is absolutely key to getting my job done.

Slightly more on topic, the other thing I would say is that quantitative departments are very good at teaching their students LaTeX, but not necessarily teaching them to it _well_. I won't exactly say that learning TeX made me a better physicist, but it definitely helped me communicate like a professional one. I interact a lot with doctors and bioscientists -- I basically work in medical imaging -- and trying to cross that divide is very hard; we forget that doing a physics degree gives you lots of transferable skills. Overleaf is _excellent_ at providing a "user-friendly face" for projects that I can share with doctors -- they don't need to understand the code, nor have the distribution installed locally; they can just contribute to a paper in progress quite easily, and in particular it's "track changes" feature is something that they like. My team and I tend to use git and % comments, but the value of a web UI is definitely there.

Debugging problems like alluded to in the original article is _definitely_ easier locally, however.

<comment><USER4> > Debugging problems like alluded to in the original article is definitely > easier locally, however.

Notice that you can have both things here. Each overleaf project is actually a git repo that you can clone and edit locally.

</comment></comment></comment></comment></comment><comment><USER5> Yeah, debugging LaTeX absolutely sucks. I always fantasize about some strongly-typed variant and even wonder if XML would make a nicer Tex language.

<comment><USER6> This is the kind of side project I should have seen 100 times on HN, but I don't remember ever seeing one. Why aren't there alternatives to LaTeX out there? [0] It's OK to have respect for Donald Knuth but still accept that language design has come a long way since 1984.

[0] Of course there are plenty of alternatives to LaTeX. Microsoft Word is just one. But I mean a language that compiles to TeX.

<comment><USER7> > Of course there are plenty of alternatives to LaTeX. Microsoft Word is just > one.

Only to the extent modelling clay is an alternative to CAD/CAM software and CNC tools.

There might be a GUI replacement to LaTeX, but it would have to be a graphical way to manipulate structure, not a WYSIWYG system. And anyway, WYSIWYG is hardly ever WYSIWIG [1], or WYSIWTG [2]. (More like WYGIWYD [3], or YAFIYGI [4].)

[1] What You See Is What I Get

[2] What You See Is What They Get

[3] What You Get Is What You Deserve

[4] You Asked For It, You Got It

<comment><USER8> > There might be a GUI replacement to LaTeX

The closest thing I know of is LyX. I actually started with it before diving into LaTeX.

[https://www.lyx.org/](https://www.lyx.org/)

</comment></comment></comment></comment><comment><USER9> What I don't quite understand is why array was used here in the first place instead of equation* or similar.

Anyway interesting post I wouldn't have guessed that carries onto the next array. Why does it only affect the first cell (or first part of the first row) though instead of the whole row?

<comment><AUTHOR> It was a third hand file with a lot of packages and the content was quite different, so perhaps there was a good reason in the original file that was lost in the minimization or perhaps the problem is that sometimes people prefer to reuse a few environments for all the possible constructions.

About the first cell: IIUC You can determine the color by the row, the column or by the individual cell, so the color is recalculated in each cell. For some reason, it fails in the first cell. I'm still curious.

</comment></comment><comment><USER10> You could base a new package on this. \usepackage{blackout}

<comment><USER11> The censor package does pretty much this: [https://ctan.org/pkg/censor](https://ctan.org/pkg/censor).</comment></comment>


Google Tags boost traffic  - <AUTHOR> http://blumenthals.com/blog/2010/07/29/google-tags-do-they-help-an-anectdotal-review-2/

<comment><USER2> What are Tags in the context used in the article?

<comment><USER3> [http://www.google.com/support/places/bin/answer.py?hl=en&...](http://www.google.com/support/places/bin/answer.py?hl=en&answer=171905)</comment></comment>


Dijkstra: CS has "invited the quacks and charlatans in . . . Software Engineering gurus." - <AUTHOR> http://www.kirit.com/Blog:/2007-12-15/Pseudo%20scientific%20Software%20Engineering <comment><USER2> Why are people influenced so much by Dijkstra's thinking?</comment>


Is it a must for every programmer to learn regular expressions? - <AUTHOR> http://programmers.stackexchange.com/questions/133968/is-it-a-must-for-every-programmer-to-learn-regular-expressions

<comment><USER2> Yes, it is, without a doubt. It's one of the most universal tricks of the trade that you'll literally never regret learning, mainly because just about any environment you'll ever work in will by necessity support regexes, and in many, it will be the primary way you interact with text.

But it's also a must that you realize that despite the fact that they're exceptionally useful and widely supported, regexes are a disgusting abomination, one that we should be absolutely mortified to be associated with. It's one of the worst syntaxes to ever be invented, and every one of us should feel the cold stink of UX failure wash over us every time we write a regex. If we ever catch ourselves writing a DSL that in any way, shape, or form resembles regular expressions, we should stop _immediately_ and ask what the fuck is wrong with us and why we're being so opaque and random. Regular expressions are quite literally one of the worst syntaxes to ever be introduced in our field.

I worry a lot when someone doesn't know regular expressions at all. But I worry far more when someone thinks they're beautiful. That person has far too high a tolerance for unintuitive syntax and code, and will cause vastly more damage to my codebase than even the rank amateur that still uses "goto" on a regular basis.

Which is not to say that we shouldn't lean on regexes heavily anyways when they're appropriate - as programmers our primary job description is that we're paid good money to work with shitty interfaces in order to express simple ideas and algorithms.

<comment><USER3> How would you design a regular expression syntax more intuitively? Personally, I find beauty and simplicity in regular expressions. Sure, they can grow to hideous atrocities, but you can achieve such disastrous feats with any language/syntax. Maybe you could back up your claim of regexes being a disgusting abomination with, at the very least, anecdotal evidence.

<comment><USER4> A typical regex looks like this:

      \b[A-Z0-9._%-]+@[A-Z0-9.-]+\.[A-Z]{2,4}\b

Which is also what happens when a cat walks across the keyboard.

<comment><USER5> I find that perfectly readable, except for the \b which I hadn't seen before. It's matching an all-uppercase email address.

<comment><USER6> There's plenty of upper case e-mail addresses that won't match that expression.

<comment><USER7> /i

<comment><USER6> There's characters missing[1] and the tld is too short[2]. And even if that's fixed, we still don't match internationalized addresses or actually validate that the e-mail address exists. You're probably better off with something like...

       if "@" in email and "." in email.split("@")[1]:
           send_verification(email)

...but you should probably also check for common misspellings like "gmial.com" etc.

[1] <http://en.wikipedia.org/wiki/Email_address#Syntax> [2] [http://en.wikipedia.org/wiki/List_of_Internet_top- level_doma...](http://en.wikipedia.org/wiki/List_of_Internet_top- level_domains)

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> I really liked eykanal's answer [1].

        > Regular expressions are a tool. It happens to be a
        > very useful tool, so many people choose to learn how
        > to use it. However, there's no "requirement" for
        > you to learn how to use this particular tool, any
        > more than there is a "requirement" for you to learn
        > anything else.

Nails it. I do think most programmers will eventually run across a problem to which the solution is 'Use Regex', but it's not an absolute "must" like boolean logic.

[1] [http://programmers.stackexchange.com/questions/133968/is- it-...](http://programmers.stackexchange.com/questions/133968/is-it-a-must- for-every-programmer-to-learn-regular-expressions#answer-133970)

<comment><USER9> good point indeed. I spent my first years programming DSP algorythms on embedded systems. Hardly any strings used, let alone there was any need for regular expressions. Learning their ins and outs back then would have been a waste of time, and would have been like a bricklayer having a fork in his toolbag.

</comment></comment><comment><USER10> Probably worth revisiting this bit of commentary from Rob Pike:

"Regular expressions are hard to write, hard to write well, and can be expensive relative to other technologies... Standard lexing and parsing techniques are so easy to write, so general, and so adaptable there's no reason to use regular expressions.

"Another way to look at it is that lexers and parsing are matching statically- defined patterns, but regular expressions' strength is that they provide a way to express patterns dynamically. They're great in text editors and search tools, but when you know at compile time whatall the things are you're looking for, regular expressions bring far more generality and flexibility than you need.

"Encouraging regular expressions as a panacea for all text processing problems is not only lazy and poor engineering, it also reinforces their use by people who shouldn't be using them at all."

[http://commandcenter.blogspot.com/2011/08/regular- expression...](http://commandcenter.blogspot.com/2011/08/regular-expressions- in-lexing-and.html)

<http://news.ycombinator.com/item?id=2915137>

Personally, I think they're pretty darn useful and too powerful to _not_ learn, but Pike's comment makes me think that maybe they're also a crutch that I've relied on too much rather than learning enough about lexing/parsing.

</comment><comment><USER11> Yes. Next question?

</comment><comment><USER12> From time to time I re-read Larry Wall's "Apocalypse 5" - <http://www.perl.com/pub/2002/06/04/apo5.html?page=6> It is still in interesting look into why regexs are the way they are.

Also if you are familiar with this quote:

      "Some people, when confronted with a problem, think
      'I know, I'll use regular expressions.'   Now they have two problems."

Read Jeffrey Friedl's research into the quote - <http://regex.info/blog/2006-09-15/247>

</comment><comment><USER13> While knowledge how to use regular expression is invaluable, I also recommend learning how they actually work under the hood. It really gives a good lesson when regular expressions are applicable, and when they're not. From my experience, while many programmers are apt in tools like regexps or grammar->parser generators, they very rarely know how it actually works, which results in people trying to parse HTML with regexps or similar things. It is also a good starting point to some very interesting theoretical stuff like the theory of computations.

<comment><USER7> Also, you see people doing simplistic string comparisons using regexes. Which is ok sometimes but is an easy target if your system has performance issues.

</comment></comment><comment><USER14> I find one the hardest aspects of using a regex is all the subtle differences between languages/environments. I use them about four times a year but I feel like I have go read a mini tutorial each time. I know the concepts but I can't remember the special char for whitespace or digits in the particular language I'm working in. Also the java implementation is so bad. Having to encode it as a string with its own escaping rules is not easy. Then the 3 line api usage is a real hassle.

</comment><comment><USER4> The ability to parse text quickly is invaluable when writing code.

I would say it's a must to take your coding skills to the next level.

</comment><comment><USER15> There's no "a must." But I don't consider a programmer to be a good one if he doesn't know regexps.

If you ask "do you personally really need regexps" I'll tell you: don't learn them. As you're asking that question at all, I understand that you're not interested to learn and that you are looking for an excuse not to learn, so do something that interests you.

</comment><comment><USER16> I regularly forget regex intricacies, so now have an app called "regexbuddy" for Windows. Very very useful, has a killer help file, has the ability to adjust and test regexes for many languages.

Surprisingly, here: <http://www.regexbuddy.com/>

The help file alone will make you want to buy it.

</comment><comment><USER17> I think they're a valuable as a simple tool e.g. using :s/^#// in vim.

However, I try to avoid using them in my code unless they improve readability. Using re.VERBOSE can help in Python.

If you find a regex online, you should definitely reference it in your code, to help provide background understanding, such as validating a UK Post Code.

</comment><comment><USER18> I taught my friend how to use regular expressions for search and replace. He's found them invaluable and he's NOT a developer. He uses then to clean up and filter lists of keywords from Adwords.

I reach for refex frequently but almost never as something I add to the code I'm writing. They're just amazing for filtering and bulk editing text.

</comment><comment><USER19> I was thinking about this question and it occurred to me. The programmers who probably don't need to know regexes[1] almost certainly already learned them. So I guess that's a yes.

[1] Thinking of embedded systems developers creating code for, say, automotive entertainment systems, or control code for scientific or medical hardware.

<comment><USER20> _[1] Thinking of embedded systems developers creating code for, say, automotive entertainment systems, or control code for scientific or medical hardware._

Don't they have log files to read?

<comment><USER19> That's what grep is for ;-)

But the point is I can't imagine someone getting to that point without previously having learned it.

<comment><USER20> Err, grep uses regular expressions.

<comment><USER19> You don't need to know regular expressions to use grep for reviewing log files though.

</comment></comment></comment></comment></comment><comment><USER5> I think every programmer should have an understanding of what Regular Expressions are, how they can be used, and where to find a cheat sheet or reference (like www.regular-expressions.info).

It helps to know the basic syntax by memory, but you can just as easily look it up if you understand how they work.

</comment><comment><USER21> I think every programmer should know them, but should try to avoid using them unless they really are the best solution to your problem. Too many people reach for them too soon, leaving hard to understand and hard to maintain code that could have been better expressed some other way.

</comment><comment><USER22> Yes! They're not that hard and there's only so much syntax to them (vs. an actual programming language). I essentially learned from this site by just entering in random strings and trying to match them. <http://rubular.com/>

</comment><comment><USER23> Regex is a good example for a bad interface. When usually half of the input needs to be escaped something must have gone fundamentally wrong. I regularly forget regular expressions and look them up again when I really need them.

</comment><comment><USER24> I think it's essential to know of their existence, but not necessarily to know their syntax inside-out. Basically, enough enough that you're not writing looping and parsing code yourself, using a regex where applicable.

</comment><comment><USER6> The problem is that regexp is often misused as things like html parsers and e-mail validators. Learning regexp syntax without learning when and how to use it, makes you a worse programmer not a better one.

</comment><comment><USER25> At least is good to know what can be solved with regular expressions, then if you need to solve something you can always look it up in a book or use some software for it (I use RegexBuddy).

</comment><comment><USER26> Only if you're doing a lot of string parsing. Just like learning about algorithms on graphs is a must only if you're working with graphs of a non- trivial size.

<comment><USER27> I don't agree. Regular expressions come in handy as well during your day to day work: you can save a lot of time if you know how to use them in your editor "search & replace" function, or just when a simple search in a directory gives you too many results and you need to switch to power tools (read: egrep).

</comment></comment><comment><USER28> Learn the 20% that you will use 80% of the time. If your not willing to do that your probably not serious about programming in the first place.

<comment><USER29> Amen to that. For reasons I can only guess at, I'm the "regex guy" at work. When people have a regex question they come to me. And the interesting thing is that the answers are not hard, well within the 20% area. I've found that most people don't care to learn how to use character classes, or even what they're for. For example, to match the single letter 'i', I've seen people write:

        [i]

Many don't know the difference nor implications of .* ? vs .*

And on and on. It amazes me that these people actually code for a living.

<comment><USER30> Learning about the nongreedy modifier is IMHO a must when it comes to regexps. It makes them much more convenient to write.

</comment></comment></comment><comment><USER31> If you wanna do it for serious. At the very least you really ought to work how grammars work (which are strictly more powerful than regexes).

</comment><comment><USER32> They are useful but non-intuitive, so I imagine most people relearn them each time they meet - unless you are stuck doing it all the time.

<comment><USER19> I wonder if Perl programmers are resented for our regex knowledge ;-)

</comment></comment><comment><USER33> I regularly tell non programmers that it is the most useful concept that they can learn (if they deal with data)

</comment><comment><USER7> You can get by as a programmer without knowing regexes. You can also get by without knowing about pointers. Heck, maybe you can even get by without knowing SQL or OOP.

But you will be limited. You will run into a lot of situations where your ignorance will create friction and limitations. Something as simple as editing an nginx configuration file, for example.

You _should_ learn regexes, you should be able to use a regex as necessary and be able to understand regexes you come across, you should understand when they are most useful, and when they are not. You should learn their limits and _your_ limits in using them as well. When used appropriately they can be a potent addition to your technical knowledge. When they should be used but are avoided the result is typically a massive inflation of the effort to solve a simple problem. And when used incorrectly they can result in an inflation of intractable complexity (much like any technology).</comment>


Google Testing Search for Television - <AUTHOR> http://online.wsj.com/article/SB10001424052748704869304575109912574043580.html?mod=WSJ_hp_mostpop_read

<comment><USER2> This announcement must be a bummer for SetJam.com</comment>


100% Guaranteed SEO Bullshit - <AUTHOR> http://wprealm.com/blog/100-guaranteed-seo-bullshit/

<comment><USER2> The term "SEO" has taken on a stigma, which is why those who actually know how to rank correctly per Google's guidelines (which the article enumerates fairly well) are using different terminology to describe what they do.

There is still "flash" SEO, but rankings can easily plummet as quickly as they were gained. Honest SEO takes longer, but has staying power for obvious reasons.</comment>


No More Promises: Realizing Your Programming Job Sucks - <AUTHOR>

Read time: 2 minutes
Somewhere in the annexes of maxims and guidance, there is a statement that reads:
"Actions speak louder than words."
When I worked for my first company out of college I began with "part-time" work as a junior Rails developer. The rate was $1000 a month for up to 30/hrs a week. Two months would then pass and they would consider an increase in salary. Why would anyone accept that offer as a junior Rails developer? Was it me being naive, hopeful, stupid, or something more? I think it was the belief in promises. Their first promise, a $90,000 salary by the end of the year.
Two months passed and the work increased. I'm given tasks with impossible deadlines. In one situation I worked 60 straight hours with 1.5 hours of sleep. Was I given a bonus? No. Was I given a week off to recover? No. More promises were given. Shares in the company, you'll have normal work again, other developers will have to go through what you went through, etc.
By the end of the year, I'm working full-time plus doing Rails work for the company. I've sharpened my skills and I feel comfortable with Rails. Expected salary: $2500 a month. Real salary: $1000ish. We are given a project that takes 3 months to complete, due within 2 weeks. Management won't budge on date with little previous consultation with developers for a deadline. What's the solution? Hire contract developers and take salaries out of current developers to pay contractors.
Don't make my mistake and trust a company with promises that once they are 'big' you will get all these great benefits. I was stupid. I learned and quit. Now hopefully I can find a Rails job with a salary I can actually live on. <comment><USER2> We have a saying here in Romania that says [roughly translated]: « The cunning promises and the fool expects. »

I feel and know exactly what you are passing through because I've been there, with every concession you make, and every promise you agree is harder and harder to renounce because you perceive to loose all.

It's called the monkey trap. [http://www.tarekcoaching.com/blog/dont-fall-in- the-monkey-tr...](http://www.tarekcoaching.com/blog/dont-fall-in-the-monkey- trap/)

And you made the wise decision in the end so I'll finish with another quote:

"If the fool would persist in his folly he would become wise." (William Blake)

</comment><comment><USER3> Just change of company, you're in the wrong place.

</comment><comment><USER4> You're not a effing peasant, don't act like one. Especially in the context of job market boom.

</comment><comment><USER5> Run don't walk. There are currently more jobs then people in our field. You will run into this type of company again but in the future you will know to tell them to fuck off. Especially when you see how easy it is to get another job.

</comment><comment><USER6> Yeah, sounds a bit like my promotion which was first supposed to be March, then June, then Sept....and now, it will "definitely" be in March...again.

Have a few interviews lined up this week.</comment>


For Haggis Purists in the U.S., the Law Is No Obstacle - <AUTHOR> https://www.nytimes.com/2020/01/24/business/haggis-smuggling.html <comment><AUTHOR> Impossible haggis?</comment>


Ask HN: Who is building something new in Python 3? - <AUTHOR>

The post today about Python 3 has me thinking about what to use for new projects. It is clear that many people are choosing to stay with Python 2.x because they have a large, active codebase that would be difficult to migrate.<p>If you have started a significant project recently in Python 3: What is your project, and how has the decision to use Python 3 affected the project? <comment><USER2> [https://github.com/rectangletangle/nlplib](https://github.com/rectangletangle/nlplib)

"What is your project" A bunch of tools for natural language processing, with an emphasis on code composability.

"How has the decision to use Python 3 affected the project?" Because my project has an entirely new code base, it made sense to aim for longevity by using the most recent version of the language. It seems like a lot of the FUD around 3 has been about third-party library support. However, I've found that everything I really needed was readily available for 3. Though there're probably some specific usecases that 3 just doesn't quite cover yet. Seeing as most of the major third-party libraries have made the switch, I think that it will take major Linux distros shipping 3 as the default for it really to catch on.

Print being a function, has been a nice addition seeing as it's handy to pass it as a logging callback; it can also now be used within a lambda. There's a bunch of nifty new syntactic sugar like extended iterable unpacking, which make life just a little bit easier. The more generalized comprehension syntax is nice, seeing as it's an extremely powerful way to make sets, dicts, or whatever. The cleanup done to the standard library, specifically urllib has been nice. I haven't found a good use for the new function annotation syntax; however, I'm sure someone will come up with a neat way to utilize it. Subgenerators for recursive flattening are nice, as well as the generator syntax as a whole.

</comment><comment><USER3> I largely stopped coding and python and have moved to node.js and go (mainly because I'm most interested in 'real time' webapps and gevent/twisted offer nothing compelling compare to either nodejs or go).

Where I have continued to use Python is statistics and helping biology PhD students do basic modeling. (To be fair, none of this really requires Python 3)</comment>


Intel DG1 Xe GPU Announced at CES and Sneaky First Benchmarks - <AUTHOR> https://youtu.be/uHQXMUJ2t2U <comment><USER2> The lag problem strongly suggests (to me, a non-expert) that this is predominantly a software/driver issue, which isn't that difficult to believe for a pre-release version of a device aimed at developers to jumpstart "an ecosystem".</comment>


Geographical Attractors: How Personality and Place Affect Personal Flourishing - <AUTHOR> https://medium.com/the-polymath-project/geographical-attractors-how-personality-and-place-affect-personal-flourishing-4611d8e363b9 <comment><AUTHOR> Here's an Outline link:

[https://outline.com/uydP7b](https://outline.com/uydP7b)</comment>