Ask HN: Where to look to learn UX/Design - <AUTHOR> I understand that this wont make me a web Picasso or anything, but I would love to be 'not horrible' when it comes to sites design and UX. What are some good places to look at, get inspiration from, and especially, learn from? <comment><USER2> My favorite UX books:

The Design of Everyday Things ([http://www.amazon.com/Design-Everyday-Things- Donald-Norman/d...](http://www.amazon.com/Design-Everyday-Things-Donald- Norman/dp/0465067107/))

The Humane Interface ([http://www.amazon.com/Humane-Interface-Directions- Designing-...](http://www.amazon.com/Humane-Interface-Directions-Designing- Interactive/dp/0201379376/))

Contextual Design ([http://www.amazon.com/Contextual-Design-Customer-Centered- In...](http://www.amazon.com/Contextual-Design-Customer-Centered-Interactive- Technologies/dp/1558604111/))

<comment><AUTHOR> I'm reading The Design of Everyday Things right now. It is an amazingly good book. The examples are awesome. It really is changing the way I look at everything.

<comment><USER3> The Design of Everyday Things is cool, indeed. Norman's a great author who keeps questioning what we take for granted.

However, if you are looking for something more pragmatic, Steve's Don't Make me Think and JJG's The Elements of User Experience should be the top of your reading list, especially on web design.

</comment></comment></comment><comment><USER4> Some previous discussions:

Ask HN: Learning Web Design — <http://news.ycombinator.com/item?id=1604915>

Ask HN: How can I get better at design? —<http://news.ycombinator.com/item?id=1474315>

I've found this online book great — <http://designingfortheweb.co.uk/book/index.php>

Experienced designers always emphasize learning about typography, and which seemed unintuitive to me until I began to study it. Bringhurst is a fantastic place to start there:

[http://www.amazon.com/Elements-Typographic-Style-Robert- Brin...](http://www.amazon.com/Elements-Typographic-Style-Robert- Bringhurst/dp/0881792055/ref=sr_1_1?ie=UTF8&qid=1286480633&sr=8-1)

<comment><USER5> Bringhurst's book is advanced and heavy. It's good. But I'd start with 'Thinking with Type'.

</comment></comment><comment><USER6> I want to do the same (not suck at it) so i started to look at web design communities a while ago. I found some interesting things that helped me be a little better:

\- <http://webtypography.net/> for typography basics

\- <http://uxmovement.com/> for general user experience advice

\- [http://uxmovement.com/resources/4-best-design-pattern- librar...](http://uxmovement.com/resources/4-best-design-pattern-libraries) (inspiration :) )

\- <http://alpha.patterntap.com/collections/Tabs> (found recently about it).

\- [http://www.blog.spoongraphics.co.uk/articles/25-classic- font...](http://www.blog.spoongraphics.co.uk/articles/25-classic-fonts-that- will-last-a-whole-design-career)

Hope it helps.

</comment><comment><USER7> If you want to understand the psychology of design you need to look for books on "human factors" and not web design books. Human Factors is a scientific discipline that is a subset of IO psychology.

While many of the books recommended on this board are good books, some of the authors learned through trial and error without the guidance of the scientific rigor that is provided by Human Factors, while others are just plain wrong.

The mind works in particular patterns, understanding those patterns will help you conceive of interfaces and test those conceptions, while verifying that it does indeed improve whatever metric you are trying to improve, not just emulate something that worked for someone.

Now I am not saying you need a PHD in Human Factors to design good interfaces, but a little grounding goes a long way. It will help you understand why something worked for someone in a book and determine if it will also work for what you are trying to apply it to.

<comment><USER8> Can you recommend any books on "human factors"?

<comment><USER9> This is an excellent book that discusses human factors at a lot of levels from city planning, to the design of a simple device. It is kind of like a mash up between "Powers of Ten" and a Malcom Gladwell book. Not web centric, but a really good read and intro to HFE.

<comment><USER10> sounds fascinating, but there's no book title in your post!

<comment><USER7> He is probably talking about the handbook of human factors and ergonomics. It has been updated since I purchased it, and is more information systems focused now. It is like the bible of usability but it is an expensive text, that is why I did not recommend it.

I guess I should have and put a disclaimer on it. When I bought it, it was around $275. If you have to buy one book and you can spare the change, this is the one. But some of the others will get you going faster.

It depends on what you want to do, This book is deeply focused and considered an academic text, many teach from it. Once the subject matter in this book is mastered you will be a HF professional for sure, but some of the other books can get you up and running faster.

At some point you should consider this book for your bookshelf but it is not a beginners book that is another reason I was hesitant to recommend it.

</comment></comment></comment></comment></comment><comment><USER11> Check out Smashing Magazine <http://www.smashingmagazine.com/>

Their posts are usually really informative and provide a lot of jumping off points and design analysis. They have a book out as well that I've only thumbed through a bit, but looked really well done.

</comment><comment><USER12> I like Luke Wroblewski's advice a lot (<http://www.lukew.com/> plus his two books on form design and web usability)

I have to worry about form design mostly, so two links I liked were

<http://www.uie.com/articles/web_forms/> and [http://www.uxmatters.com/mt/archives/2006/07/label- placement...](http://www.uxmatters.com/mt/archives/2006/07/label-placement-in- forms.php)

I also stole some ideas from here: <http://developer.fellowshipone.com/patterns/>

Those are the ones I've read and somewhat followed.

I also bookmarked this framework to try out: <http://gantry-framework.org/>

There are also several eyetracking services (some provided by Ycombinator companies I believe) - I'd love to have eyetracking studies on the company website, but the current management doesn't mind that it looks like crap since it will be outsourced anyways.

<comment><USER13> "I also stole some ideas from here: <http://developer.fellowshipone.com/patterns/>

I discovered that a couple of weeks ago. It's really, really well done.

</comment></comment><comment><USER14> I basically sent the same question in an e-mail to Amy Hoy last night. She recommended these two books, "The Universal Principles of Design (have to get a used copy) and About Face 3 for interaction design."

<comment><USER4> Why a used copy of "Universal Principles"? It's available on Amazon: [http://www.amazon.com/Universal-Principles-Design-William- Li...](http://www.amazon.com/Universal-Principles-Design-William- Lidwell/dp/1592530079)

</comment></comment><comment><USER15> I have to say of all 37Signalers, I enjoy listening to Ryan Singer the most. He seems to have a genuine interest in teaching and not in a "controversial for the sake of marketing" kind of way. He has a bunch of videos on Vimeo.

He talks more about user experience and interaction, not graphic design, that as someone else mentioned, you can find tutorials on the tuts plus network.

<comment><USER16> Agree. I sometimes like listening to Jason also.

But yea, Ryan Singer is the awesome.

</comment></comment><comment><USER17> It's worth checking out Don Norman's (the author of The Design of Everyday Things) recommended reading list:

<http://www.jnd.org/recommended_readings.html>

I have yet to choose a book from the list that didn't leave me feeling much more informed. If you click on a book in the list it jumps to a description of why he thinks someone should read it.

I desperately want to get the Handbook of Human Factors and Ergonomics, but it's just way too expensive:

<http://www.jnd.org/recommended_readings.html#000235>

</comment><comment><USER18> "About Face" by Alan Cooper is far-and-away the best book I've read on interaction design.

Don Norman's books are also worth a read (though more difficult to immediately apply to creating software).

For general visual design "The Non-Designers Design Book" by Robin Williams is an excellent tutorial to teach you how to be, as you say, "not horrible".

</comment><comment><USER19> I just started reading Cooper's _About Face_ and by chapter 2 it's already changing how I think about UI design. Granted, I sucked at it, but still...

</comment><comment><USER20> Adding to the others.

You should understand the difference between Web design and UI/UX design: it's like Monet/van Gogh versus Da Vinci.

In UIs you're crafting like an "engineer", while with general website design (eg. most of the cases done by graphic designers) you're painting. Either way practice is required (if you're a programmer you know this, you can't learn programming only by reading books :).

Also if I may, I would recommend "Semiology of Graphics" by Jacques Bertin, it's a reference to information design and visualization (if you plan to design apps, you may want to give it a look). [http://www.amazon.com/Semiology-Graphics-Diagrams- Networks-M...](http://www.amazon.com/Semiology-Graphics-Diagrams-Networks- Maps/dp/1589482611)

</comment><comment><USER21> Many YC startups have stunning design e.g. heroku, wufoo. When you come across a beautiful site, open it in Firebug to see how it was done.

<comment><USER22> Stunning design doesn't really come from having "beautiful" html code so much I feel. People who really get design are the ones who think about how a product is going to be used, who is using it, and in what context. If you are talking strictly visual design, then the CSS/HTML markup is going to be even less relevant. For that I would go to CSS sites and galleries and derive inspiration from there.

<comment><USER21> Oh I agree. We inspect elements of beautiful sites in Firebug to answer the question "How was that _specific thing_ made?" Obviously Firebug does not find inspiration for you, it won't teach you the principles of design and it isn't going to design sites for you.

</comment></comment></comment><comment><USER23> "Don't Make Me Think" by Steve Krug is the single best book to start with on UX. It is short, extremely accessible, and very useful.

</comment><comment><USER24> 1\. Rosenfeld Media has a ranking tool for UX books <http://www.rosenfeldmedia.com/zeitgeist/> and top lists provided by respected industry professionals.

2\. Whitney Hess, a respected UX professional with great PR skills, has a recommended list at [http://www.flashlightworthybooks.com/Best-Recommended- Books-...](http://www.flashlightworthybooks.com/Best-Recommended-Books-User- Experience/224)

As a person who once switched to UX design for a while, I can say that they are definitely great books, though I can't understand everyone' obsession with Steve Krug (he has EXTREMELY basic books).

Also, Whitney has two popular blog posts of So You Wanna Be a UX Designer series: [http://whitneyhess.com/blog/2009/06/30/so-you-wanna-be-a- use...](http://whitneyhess.com/blog/2009/06/30/so-you-wanna-be-a-user- experience-designer-step-1-resources/) and [http://whitneyhess.com/blog/2009/11/23/so-you-wanna-be-a- use...](http://whitneyhess.com/blog/2009/11/23/so-you-wanna-be-a-user- experience-designer-step-2-guiding-principles/) . They will provide you basics, but also overwhelm with the huge volume of recommended sources.

3\. konigi.com (ther blog and wiki) is a top source of information about ALL old and new UX tools, some UX basics and new UX trends. They also sell some very handy tools for UX designers <http://konigi.com/tools/overview>

</comment><comment><USER16> There's no "The way to design" but there is "A way to design".

I would recommend you "Defensive Design for the Web" by 37 Signals and "Designing Web Interfaces" by Bill Scott.

These books use examples of Web Interfaces but you can apply them on anything. Be it a software or hardware.

Also, I would say, when you design something, Dont focus much on visual appearance. First try to make things clean and ask your non-tech friends to use it. See what difficulties they face in using your product. You will learn a lot from these experiences.

</comment><comment><USER25> I'm not a UX guy, but I found this to be a good infographic to encapsulate what UX actually means:

<http://uxbasis.hellogroup.com/>

</comment><comment><USER26> I like the books "Don't make me think" and "Designing the obvious" for some basic web UX knowledge. They are short and to the point.

<comment><USER27> These are exactly the two books I was going to recommend. Don't Make Me Think is like the K&R of web UX books, and Designing the Obvious is like a nice web 2.0 companion for it.

Also, I'd throw in The Design of Everyday Things -- see [http://www.amazon.com/Design-Everyday-Things-Donald- Norman/d...](http://www.amazon.com/Design-Everyday-Things-Donald- Norman/dp/0385267746). It was written long before anyone ever thought of web usability; instead, it focuses on the usability of things you interact with daily in real life. Let's just say that you'll never look at teapots or door handles the same way again...

</comment></comment><comment><USER28> Here's a list of books I've put together for classes: <http://hci4.me/static/16-475-references.pdf>

MIT's OCW class on UI - [http://ocw.mit.edu/courses/electrical-engineering- and-comput...](http://ocw.mit.edu/courses/electrical-engineering-and-computer- science/6-831-user-interface-design-and-implementation-fall-2004/)

Also, look at RISD, Stanford, CMU, Berkeley, Cornell (in no particular order).

If you can, anything by Doug Engelbart is amazingly motivational (inventor of the mouse) and insightful.

If you contact me, I'd be happy to send you my slides from lectures I've done on interaction and answer any questions.

</comment><comment><USER29> tl;dr: Take a course!

Don't know about all these books recs. All good books, but reading books wont make you a better designer. Reading books about fine art wont make you a better painter; and reading books about soccer wont make you a better soccer player. You'll learn the rules, sure, but at the end of the day you need to develop the skills by training/being coached (if I can run with the metaphor), so I suggest taking a short course on graphic design fundamentals and learn by doing.

Getting drilled on the fundamentals will stay with you much longer than anything you read in a book about ux/design.

</comment><comment><USER30> I found it possible to absorb more knowledge (wisdom?) about UX by performing user tests than by any other method of studying the subject.

Grab a copy of "Rocket Surgery made Easy" by Steve Krug, a copy of Silverback for Mac, and start doing some user tests.

In time, you will start to see the same patterns coming up over and over again, and this information will begin to unwittingly inform your thinking during the design phase.

This isn't going to help you with visual design so much, but it certainly will help you with the communication aspects of the software/website, and will help you to avoid design/UX pitfalls.

</comment><comment><USER31> "don´t make me think" by steve krug (is a classic)

if you wanna learn about users, the way they think, and which problems they have on the web (eg with forms,..) just sit by your girlfriend, your mum or everybody you can watch in front of a computer.

the best is to watch them using something you created. so you get a clue of how different problems (than you think) other people have that you might think.

i was yesterday at a customer showing my progress. so he (a trainer for wordpress stuff) registered on the site and asked and did some (for me) weird things.

there´s tons of things to learn from that. :)

</comment><comment><USER32> I steal heavily from some of the best designers/design shops out there.

Here's some of my favs (look through the portfolios)

<http://www.odopod.com> <http://bigspaceship.com> <http://barbariangroup.com/> <http://codeandtheory.com> <http://akqa.com> <http://rga.com>

</comment><comment><USER33> Plenty advices on books and urls have been given but I'd like to shed some light over practice. There is no book, people or tip will move you forward faster than design-and-critique.

Design, and art in general, is not very different from coding. You have to write code to be able to appreciate and learn from other's work. Here is one good link on how to critique art: <http://bit.ly/d5Ep7p>

</comment><comment><USER34> This is an answer to your request for "good places to look" rather than learning user interfaces: I've been curating a list of "clean sites I like the look of" since '06:

<http://www.delicious.com/jashmenn/cleansitesilikethelookof>

</comment><comment><USER35> People have posted a lot of resources here, but are there any resources that talk about how to use specific tools to create webpage designs, rather than just concepts behind designs? I'm not sure which tools I should even use to go about creating a design (other than hand coding HTML and CSS)

</comment><comment><USER36> I can't believe a quick Ctrl-F didn't find him already mentioned, but look up Jakob Nielsen and his ten usability heuristics. If there's one thing doing some postgrad work in HCI taught me, it's to remember those absolutely. Always, always evaluate what you're making against them.

</comment><comment><USER37> You know how you tinker with code day after day, and you've been doing it for years? How you're connected with peers and betters who know how to code also, and who can critique your work? You know how you think about code even when you're not coding?

Do that for UX/Design.

</comment><comment><USER38> Check out Joel Spolsky's UI Interface design. <http://www.joelonsoftware.com/uibook/fog0000000249.html>

and as well as mrshoe, I love The design of everyday things.

</comment><comment><USER39> Want your UX/Web Design to make money? Try Web Design for RIO ([http://www.amazon.com/Web-Design-ROI-Browsers- Prospects/dp/0...](http://www.amazon.com/Web-Design-ROI-Browsers- Prospects/dp/0321489829))

</comment><comment><USER40> This thread on Quora provides a lot of great advice: [http://www.quora.com/How-can-I-learn-to-be-a-good-product- de...](http://www.quora.com/How-can-I-learn-to-be-a-good-product-designer)

</comment><comment><USER41> Ben Fry's Dissertation is awesome, very easy to read and FREE! Ben is the creator of the Processing language (which is awesome by itself)

<http://benfry.com/phd/>

</comment><comment><USER42> I find <http://psdtuts.com> pretty inspiring. However, their tutorials are often WAY over my head.

</comment><comment><USER43> I recommend 'The Non-Designer's Design Book' by Robin Williams <http://amzn.to/aMd1kR>

</comment><comment><USER44> No mention of Jakob Nielson?

<http://www.useit.com/alertbox/>

</comment><comment><USER16> check this out <http://news.ycombinator.com/item?id=1778880>

</comment><comment><USER45> What I learned from a Human-Computer Interaction class in college

\- Always think of the user. All tradeoffs should be made in favor of the user

\- Don't expect users to read anything; they're busy

\- Make it as simple as possible, but no simpler. Don't dumb it down, just don't complicate it

\- For each page/window, know exactly what the user wants to do there.

\- Make things clear, easy to see, and easy to learn

\- Have an Undo command, for Christ's sakes

<comment><USER45> Forgot \- Feedback. Users need to know when they have performed an action correctly.</comment></comment>


Show HN: Employify.me - We eliminate CVs, cover letters and lenghty applications - <AUTHOR> http://employify.me/

<comment><USER2> I don't want to sign up to find out if I want to sign up.

You have to tell me why, then what and then I will look at the how.

<comment><USER3> Great feedback! Sorry for the lack of clarity on the first instance. We have now changed the description to:

We eliminate resumes. Instead we increase your chances of getting hired. { Hint: You will get an actual taste of the job }

What do you think of this one?</comment></comment>


Ask HN: You're in beta. How much $ is 1,000 random users worth? - <AUTHOR>

Imagine that you have your product and it's at the point where you're ready to get some real users in -- maybe to test, maybe as actual customers or whatnot. The opportunity arises to get about 1,000 of them, give or take, to sign up, all within a 24-48 hour period, with two caveats:
1) You'd have to give them some incentive, like give one of them a free iPad or something.
2) You don't know much about the audience you'd get other than that they all have basic English literacy.
On top of the incentive, what would you be willing to spend to get those 1,000 people? <comment><USER3> Correct me if I am wrong but I am going to assume a few things:

\- A service offers 1000 users for $X

\- The service pays users $Y to join site with the potential to win an iPad/other offer as well as $Y.

If that is the case, _the value of users is negative to the startup._

Here's why:

\- Users are incentivised to join site, to win iPad thats it.

\- They are won't pay to join a service (if its a paid service)

They are not incentivised to join and actually use the product, which are incentivised by acquiring them via Word of Mouth, SEO and other forms of advertising channels. Thus the LTV of the customer is $0 - they won't use the site, they'll just signup as _it becomes more profitable for them to join 100 sites and do nothing than it is to join 1 and do something._

Similarly, the service itself is incentivised to actually cheat the startup as they will then get $X and a free iPad or something. They could easily do this by using a bot to join the services.

Even if the service won't do that, there will be users on the service who are actually prepared to do that as well using multiple payment emails etc - <USER2> they are paid for joining too!

Essentially _by paying for this service, the startup has wasted money_ by paying for 1000 users who are going to do nothing on their service and have given a free iPad (or other prize).

</comment><comment><USER4> My guess - not much. There are a lot of factors that go into how much a group of users is worth. Do they add a lot of value to your product? I would pay a _lot_ for Woz, Bill Gates, Obama and The Pope to be active users of my product, because their membership would raise recognition of my brand. But just four random people? Worth a lot less. Four random people who are signing up and will forget about my service the next day? Even less than that.

An IPad costs about $500. If you gave one away for every 1000 people, you'd be paying $2 per person to try out your product. Assuming that your product is somewhat niche, you might have a signup rate of 1% (random guess) from a pool of the general population. This would make your cost per user $200.

It would probably be a better idea to imagine who your users will be and target this subgroup specifically. As an analogy, if I'm selling calcium pills to everyone I see on the street, I probably won't sell to many - no matter how good they are. But if I go find people who are afraid of osteoporosis, I will probably be able to sell to many of them.

I would suggest that you stop thinking in terms of absolute numbers of random people (especially if they're not paying) and think more about the value that they can bring to you (whether it is word of mouth marketing, content creation, advice, money or something else). For many products, random people are not the best choice for an initial user base.

</comment><comment><USER5> There's two services here that need to be separated. Doing this as customer acquisition seems to be what most of the commenters here are focused on and does seem like a poor deal.

On the other hand, 1000 software testers for the price of an iPad are just a couple of quarters each. Plus, here's where their unreliability becomes an advantage. Bob's drunk uncle might be a lousy customer, but the aspects that make him a terrible customer make him a great QA agent. He'll forget his password and create fifty new accounts, then demand that they be merged. He'll post comments in EBCDIC. He'll declare he lives in the state of Oregon in the nation of Uganda. He's never heard of Paypal.

After launch, he's a support nightmare and you'll be happy that he disappears the instant that the service is no longer free. However, during the beta, a thousand drunk uncles will find all the edge cases that you never even thought to unit test. They'll force you to tighten down your UI to the simplest possible level of readability. They'll stress test your support infrastructure. And they'll do it for half a buck.

</comment><comment><USER6> Close to nothing, honestly. If the only reason the users are signing up is to win a free iPad (especially with the odds being only 1 in 1,000) I severely doubt that _any_ of them would convert to paid customers.

Now, if you could send 1,000 paid customers my way in that time, I'd be extremely happy to offer you 65% of the initial revenue generated.

</comment><comment><USER7> I'd offer them some kind of premium features and/or discounts later down the product (time)line.

<comment><AUTHOR> Sorry, I revised the question -- I'm basically asking, what you'd pay on top of the premium. To boil it down, I want to know what you are willing to pay to acquire a random user.

<comment><USER8> Sorry, wouldn't pay a dime. This is a little akin to buying 1000 twitter followers. Sure they'll give you apparent social proof, but you'll have a hard time selling to them. What you are really asking is how much is an unqualified lead worth. You can buy random traffic all day, but if none of them purchase/upgrade its worthless to you.

You have to ask yourself if you really think that you can convert any of these users. There is a world of difference between targeted and untargeted traffic. You probably wouldn't even get any usable metrics out of the deal.

You are also talking about an investment of 2 bucks per user, and you could almost do better with a basic cpc campaign. You'd end up spending a bit more, but you be much more targeted.

</comment></comment></comment><comment><USER9> If you would get value from people signing up, just testing your services you'd probably be better taking the money you would use to buy the ipad and posting on Mechanical Turk.

You'd then get a good number of people to sign up to use your service, they could even do a screencast of their interactions, how they navigate your services and common places where users get stuck.

Asking and paying for direct feedback will be far more useful than buying users with no interaction requirements and hoping for some meaning to arise from your stats.

Least after the testing is done you can purge your system of the testing accounts and then you know who your genuine users are.

</comment><comment><USER10> "maybe to test, maybe as actual customers"

There's a huge difference between the 2. You do know you can talk to some mommy bloggers and hold a giveaway, and give 1 iPad in exchange for a few hundred users...

</comment><comment><USER11> A random user who is coming to my site to win something who is probably not even in my demographic. And you want me to pay for this?

</comment><comment><USER12> Literally nothing, or less. Sounds like a worthless headache.

</comment><comment><USER13> I'd offer a commission to the producer of the traffic, based on conversion after beta. Anyone can generate traffic, and even signups to a free beta.

Getting users that are qualified AND interested that will convert to paid accounts is very difficult, and if whomever is referring users isn't gaming the system, they should have faith that their referrals will convert to paid accounts.</comment>


14 Years of SQL Injection and still the most dangerous vulnerability - <AUTHOR> https://www.mavitunasecurity.com/blog/sql-injection-vulnerability-history/ SQL injection has been around for 14 years and it is still the most dangerous and widely exploited web application vulnerability. Ever wondered why? <comment><USER2> TL;DR: "Because you aren't using our snazzy new product; try it today!"

Also:

        This is primarily because of the most obvious problem:
        We are still using relational SQL databases.
        Were we to use NoSQL database systems such as MongoDB
        or CouchDB, none of these attacks would ever happen

Yeah, and motorbikes are safer than cars because the third and fourth wheels never fall off...

<comment><USER3> Make perfect sense, since there is no SQL in NoSQL its impossible to have a SQL injection, problem solved...

<comment><USER4> In MongoDB you could theoretically have a JavaScript injection, which I'd argue is pretty bad.

<comment><USER3> Sorry the sarcasm didn't transfer to my post, you are correct.

</comment></comment></comment></comment><comment><USER5> When combining software components, you have to gateway between them -- this is done by escaping inputs and validating outputs. It's often non-trivial to grok where the junctures between components are, and how to properly connect them. This requires thought, experience, and diligence.

Why it's so common is that you can easily connect components incorrectly, still have it "work" enough to do a demo, and, there is lot of pressure to put things that appear to work into production without auditing them.

Addendum: Shish2k observes this article claims NoSQL databases would somehow solve the problem, rather than seeing NoSQL as just a different kind of component with it's own escaping/validation needs. Classic -- someone writing on this very topic is blinded to the phenomenon he discusses; the article itself demonstrates why we have SQL injections.

<comment><USER6> "this is done by escaping inputs"

Something about that sounds wrong. It is as though you are suggesting that we use in-band signaling with a bunch of notch filters to ensure that Cap'n Crunch whistles cannot be used to get free calls. The right answer is out-of- band signaling -- in other words, not constructing queries / command strings / etc. from user inputs.

Major SQL databases all support prepared statements; this seems like a far more robust approach than trying to sanitize inputs.

<comment><USER7> I think that comment is a succinct summary of why SQL injections are still common, anyway. The answer: because people think "escaping" is the answer.

Imagine constructing a function in some other language this way:

        function square(x) {
            code = x + "*" + x;
            return eval(code);
        }

This is obviously wrong. Now, let's say I suggest fixing it with escaping:

        function square(x) {
            code = escape(x) + "*" + escape(x);
            return eval(code);
        }

Any sane programming community would string me up by my thumbs for doing this, and rightly so.

Yet, do the exact same thing in SQL and it's just business as usual. _Occasionally_ people talk about prepared statements and parameterized queries and such, but usually people just talk about escaping. As if the problem is just that you need more backslashes, not that you're glomming strings together at runtime and then calling eval() on them.

It's so bizarre. People would think you're crazy for doing it in one environment, but doing it in a different environment where the exact same problems are present is just fine! I can't fathom how we got to this place.

<comment><USER5> I should have used properly "construct input" (as suggested by dragon-writer) rather than "escape". If possible, you'd use query parameters or some other reusable and verified mechanism to encode user content. If you're embedding a data stream of one type within another, you still have to manage boundaries to encode content properly.

I'd note for database languages, query construction is often two phase, where you build your query template based on selection/filtering needs, and then, use parameter substitution with user-provided content. I've seen experienced programmers do both at the same time and resort to manual escaping of user content rather than seeing them as distinct phases.

I think it's about managing the movement of information between contexts... one process' data is another process' code. The most important context switch happening between untrusted inputs and the rest of your system.

I apologize for deleting my other comment; I was trying to shorten it.

<comment><USER7> I'll move my reply here, then.

You make good points, and I'm trying to figure out exactly how your points and mine hook together.

I _think_ what it ultimately comes down to is distinguishing between data and code. More specifically, data can contain code, but code should never contain (foreign) data. So escaping in general is fine, but escaping _in this context_ , where you're building a string of code, is no good.

That, I think, is why I'm comfortable with e.g. the escaping done to a string when emitting JSON, but not with escaping a string to put it into an SQL query.

With parameterized queries, you're still ultimately passing the code and data over to the database over some sort of stream (assuming an out-of-process database server), but _hopefully_ that stream is designed as data containing both the parameterized SQL code and the parameters, rather than just SQL code with escaped parameters.

<comment><USER8> Doesn't the database server just escape the data in the parametrized query? So if you used the database's escape functions directly wouldn't it be the same thing? I guess if you are arguing that developers are more likely to mess it up than the database server then I see your point.

<comment> vertex-four Not usually. What usually happens is something like this:

        SELECT * FROM bar_table WHERE x = 123;

Is converted into a datastructure like this in the database engine (really simplified):

        {
          "querytype": "select",
          "fields": ["bar_table.x", "bar_table.y", "bar_table.z"],
          "tables": ["bar_table"],
          "where_filters": ["x", "=", 123]
        }

Which is then converted into a list of things to do to run that query.

Now, if you give it a prepared statement:

        SELECT * from bar_table WHERE x = ?;

The database engine can convert that into a closure, something like this:

        function(arg1){
          return {
            "querytype": "select",
            "fields": ["bar_table.x", "bar_table.y", "bar_table.z"],
            "tables": ["bar_table"],
            "where_filters": ["x", "=", arg1]
          };
        }

Then when you run the prepared query, it can simply call the closure, putting the right value in the datastructure without escaping anything, and continuing on from there.

Of course, the realities of it are more complicated than that for optimization reasons, but that's the basis of it.

<comment><USER8> Makes sense. Thanks for the explanation.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> Unrealistic deadlines and sales reps promising features to clients is a large part of the problem. Get the changes in, test the usability, and ship it. Get it out to the client pronto. We have an earnings report due by X date and this will help the bottom line.

It's not that devs and DBAs don't care, it's that in many cases (not all) they aren't given the time to carefully test the system to see how it might be abused.

And also, many companies view security testing as an afterthought or an audit requirement. If they do not have to do it (time is money) they won't. And when they are forced to do it, they rush through it.

In some ways this is a philosophical fight between a group that wants to ship software and make a profit and a group that wants to design and test systems to make sure they are correct. Suits vs technologists.

<comment><USER10> Deadlines are not an excuse for not doing basic pen/vuln/fuzz testing prior to a release. Even the most basic of tools can detect SQL injections.

<comment><USER9> I understand this. When security is not part of the culture of the company, and management just wants to ship software, devs won't have time to test. MS used to be this way (back before XP SP2). They stopped and made security a focus and allowed devs to work on securing the software. Today, they are much better off because of that. Not all companies think about security. Many only think about the bottom line.

</comment></comment></comment><comment><USER11> Because there are developers and project managers out there who still don't understand it very well, or (dare I say) don't care enough to understand it. There will always be people who will just get the work done to a level that will reasonably satisfy their bosses, so that they can take their paycheck, and go home.

This is not software engineering alone, by the way.

</comment><comment><USER12> SQL injection is so prevalent because developers overwhelmingly think about their data in terms of its representation, rather than its meaning.

User input and SQL statements are two completely separate things, but it just so happens that they can both be represented in memory using the same layout. Hence many developers stop thinking in meaningful terms like "user input" and "SQL query", and instead start thinking in terms of implementation details, like "strings".

The solution to code injection is not to jump on the bandwagon du jour, but to maintain and enforce distinctions between meaningful datatypes. See, for instance, [http://blog.moertel.com/posts/2006-10-18-a-type-based- soluti...](http://blog.moertel.com/posts/2006-10-18-a-type-based-solution-to- the-strings-problem.html)

"Systems Hungarian" has the same cause; Hungarian notation means prefixing variable names with useful information about the contents, for example "dAge" could be the difference between two ages and "cUsers" could be a count of how many users there are. When this practice became enforced at, for example Microsoft, it quickly degraded into the meaningless "Systems Hungarian" which just gives unhelpful implementation details, eg. "iAge" and "iUsers" to indicate that they're both integers. Widespread use of Systems Hungarian, enforced by clueless management, lead to many developers hating Hungarian notation completely, despite never knowing that it has nothing to do with machine representations.

</comment><comment><USER13> SQL is hard to get rid of injections and here's why I think it is so.

How would an ideal injection-free application look? I'd imagine a set of unmutable precompiled SQL statements (the code) each controlled by a set of parameters (the data.) No gluing of statements from strings at runtime and the parameters are obviously passed out-of band. You can't forget to escape user- provided data because in this setup the SQL code and the user data never mix, so escaping is not necessary.

Unfortunately is not feasible to use the above design with SQL. A simple filter with a parameter that can be unspecified (as in "don't care") would require 2 different precompiled statements ("select ... from foo" and "select ... from foo where param=<placeholder>".) The number grows exponentially with the complexity of the filter, so the runtime construction of SQL statements is inevitable.

You can go about it in a smarter or a dumber way but the danger remains.

<comment><USER14> > The number grows exponentially with the complexity of the filter, so the > runtime construction of SQL statements is inevitable.

You can still construct at runtime. Just let your typesystem handle the distinction between SQL code and data.

<comment><USER13> Good point, but this would require composable SQL in the typesystem, right? Maybe libraries like SQLAlchemy come close to that (or maybe are already there?) but look how long it took.

<comment><USER12> Type systems tell you when your code is wrong. You don't have to do any SQL manipulation _in_ your type system, you just make sure that SQL and non-SQL data have different types, and carry on as normal. If your code is joining functions together in an unsafe way, the type system will reject it and you have to go back and change it. If it's safe, they type system accepts it and then the code can be compiled into some untyped* language (assembly, Javascript, JVM bytecode, whatever)

* By 'untyped' I mean that the type-system that you were using is no longer enforcing anything. There may be another type system in the compiler target.

<comment><USER12> It would actually be possible to use a type system to compose SQL; we could use this to guarantee there are no syntax errors. For example we could have a types "SqlTable", "SqlQueryType", "SqlWhereCondition", "SqlComparisonOperator", etc.

However, we don't need to do anything nearly so elaborate to stop SQL injection. We just need a type "SqlQuery", since that will be instantly unusable by all string concatenation functions. We then make a concatenation function for SqlQuery values and a "stringToSqlQuery" function (or ".toSqlQuery" method, if you prefer) which converts strings to SqlQuery values by escaping them.

This way, we've turned SQL escaping into a type coercion, so we can only pass the type checker by escaping every string we put in our queries. Note also that it solves the double-escaping problem: since escaped strings have a different type to unescaped strings, we can't send them back through the escape function; ie. "stringToSqlQuery(stringToSqlQuery(foo))" is a type error.

</comment></comment></comment></comment></comment><comment><USER3> Isn't this more about using strings to build executing code from untrusted input?

This kind of stuff always seems to surface when one language is embedded in another and this tends to be database query languages and one of the most popular is SQL, but it's not limited to it, just look at javascript injection from parsing JSON using eval.

I am not seeing injection going away until we stop sending bits of code between runtimes as strings, which will be hard to avoid so long as databases and applications use different languages. It's a lot of work to abstract one language into the other (See LINQ to SQL), where as string concatenation is understandably the easy way out.

Ideally we could all agree on a good binary data format(something like Go GOBs) and a AST structure to serialize to it then we would have the common building blocks to send dynamically built code between runtimes/languages without using strings.

<comment><USER15> > Isn't this more about using strings to build executing code from untrusted > input?

No.

Its about lack of care in building executing code from untrusted input. If you took data from the outside world, converted it into x86 machine code, and sent to it another system, you could still have the same problem if you didn't handle the input properly.

SQL-by-string-interpolation/concatenation is an easy target because its a common enough mistake that the same style knowledge of how to exploit the holes in it can be used against lots of targets, not because the transfer format from the poorly-built system generating the query to the critical backend system handling the query uses strings.

<comment><USER3> My point is it's much easier to lack care when using strings. If for example SQL where only exposed as an AST object model to the client language injection would be much harder to accidentally allow. Also you could have another layer of your app actually examine the AST for security issues (no DML statements allowed) or certain tables restricted etc.

We actually use a component in our app the builds an AST from SQL which can then be verified, this is not trivial. Also LINQ builds an AST which is then transformed to SQL statements in LINQ to SQL, this AST can also be examined before executed and prevents injection.

</comment></comment></comment><comment><USER16> Because it's easy to do things the wrong way and a site built the wrong way _still works_ , it's just insecure.

<comment><USER15> i.e., the problem is that "properly secures access to data" is all too often not part of the definition of "works" applied to systems.

<comment><USER16> Sure, probably true. But I would assume most people who write insecure code do so because they don't know how to do it the right way, not because "must be secure" wasn't in the requirements doc.

<comment><USER15> If security (operationalized properly) was part of the requirements against which it was evaluated prior to release, insecure code wouldn't be released.

("Must be secure" is a much higher level requirement than anything that is testable, but a high level requirement is meaningless except to the extent its operationalized into lower-level requirements that are testable -- or analytically provable, but that's even harder.)

</comment></comment></comment></comment><comment><USER17> Why?

Because most of the products are developed by the lowest bidder.

</comment><comment><USER18> blogspam. just one big advertisement for their product</comment>


Widening Russia Money Laundering Scandal Hits European Banks - <AUTHOR> https://www.bloomberg.com/news/articles/2019-03-05/dirty-money-scandal-widens-with-reports-on-nordea-and-lithuania <comment><AUTHOR> I'm following this story more to see how countries answer the question of...

How do you punish a big firm that your country deeply depends on?

Canada is going through this now with a major construction company SNC Lavlin, that seems to be guilty of using bribes in Lybia.

The Canadian government seems to have interfered with their court case for what could be seen as "good intentions". If proven guilty SNC could n o longer bid on Canadian government contracts and this would hurt alot of large infrastructure projects in Canada, as well as the 8000+ jobs in Canada that had nothing to do with the bribes.

Germany is doing something some what with Deutsche Bank, a bank plagued by scandal and fines but one that is a critical part of the German financial infrastructure.

So how do you punish these banks and not hurt the 10,000+ workers who had no part in this while or harming your economy?

Seems like one answer is to go after the C-Suite executives.

<comment><USER2> If the company is big enough, I'd suggest splitting the company (as if as the result of an anti-trust proceeding), taking the fragment of the resulting company that the govermnent depends on for services and _nationalizing_ just that part; and then throwing the _rest_ of the company—the part that the government now explicitly doesn't depend on—to the wolves, by allowing the case against it to proceed.

Basically, rather than protecting the whole just because some part of it's necessary for you, just pluck that part out and keep it for yourself.

<comment><USER3> This policy would greatly incentivize companies to behave.

<comment><USER4> The danger is that it would also greatly incentivise governments to misbehave though, right?

<comment><USER5> Perhaps we should look at the people rather than the companies or governments. Narrowing down the blame would prevent future executives (and rogue engineers /s) from being too comfortable under the cloak of "the company" when taking such initiatives.

Volkswagen didn't cheat on emissions tests, specific people knowingly lied and cheated to get to achieve their goals. And while the company should be allowed to pay back all damages in a fashion that does't cripple it entirely, those people responsible for the decisions should not be given this privilege. I'm sure there are equitable options in this regard. Of course none of them would ever apply to those people since surely they put all that money to "good use".

</comment></comment></comment></comment></comment><comment><USER6> What's funny is that all these banks have mandatory anti-money laundering training courses that are enforced down the hierarchy to developers, IT staff and even call center operators, while most of the fraud is done at the top tier with the consent of high ranking managers.

<comment><USER7> I always felt like those were done for two reasons:

1) tell your clients/press that you have them

2) if you get caught tell you "we've told you not to do it. we're at no fault here"

From my personal experience they are not being taken serious if the client is important. Or are being taken serious in a way that you have people who help you out with that kind of shady business stuff.

<comment><USER8> The second part of item 2 is that you then set up the incentives in such a way that some of your minions will choose to break the law (and hide it from you) without being explicitly instructed to do so.

<comment><USER9> Plausible deniability is the first step to creating a criminal empire where you don't get caught. Anyone who has ever seen The Wire already knows where this ends up.

</comment></comment></comment></comment><comment><USER10> For anyone interested in how (dirty) big money moves around the world in our current financial and legal framework, the "Billion Dollar Whale" book provides a very interesting insight.

A fascinating read of fraud at the highest level.

[https://www.goodreads.com/book/show/38743564-billion- dollar-...](https://www.goodreads.com/book/show/38743564-billion-dollar-whale)

</comment><comment><USER11> Imagine if the mafia ran an entire country.

<comment><USER12> A state is not all that different than a mafia (monopoly power). Try not paying your dues.

<comment><USER13> This is a false equivalence.

To paraphrase Erik Naggum, you are emphasizing the things that states and criminal enterprises have in common that are not important, while ignoring the ways in which they differ that are important.

It's true that a state has a certain monopoly on governance. But it is not true that states and the mafia are not all that different.

<comment><USER14> It's literally called "the worst argument in the world"!

[https://www.lesswrong.com/posts/yCWPkLi8wJvewPbEp/the- noncen...](https://www.lesswrong.com/posts/yCWPkLi8wJvewPbEp/the-noncentral- fallacy-the-worst-argument-in-the-world)

<comment><USER13> Please accept all of my upvotes, this is the first time I am learning about "The Noncentral Fallacy."

Thank you.

</comment></comment></comment></comment></comment><comment><USER15> It always baffles me how the people who can't stop critiquing Western European civilisation end up hiding their money here. Must be doing something right. Patriotism isn't about waving the flag, beating up foreigners and singing the national anthem- its about paying your taxes. Which amusingly none of the Russian elite seems particularly interested in.

<comment><USER16> > It always baffles me how the people who can't stop critiquing Western > European civilisation end up hiding their money here.

It's about as baffling as when governments that constantly criticize others for violating human rights have no problem selling weapons to those exact same parties, and letting their oligarchs/royal family/well-connected goons buy up expensive properties inside of their own borders.

</comment></comment><comment><USER17> I was just wondering if a whistleblower program that rewards credible whisteblowing with a portion of the fine exists and it does!

[https://www.whistleblowersinternational.com/what-is- whistleb...](https://www.whistleblowersinternational.com/what-is- whistleblowing/rewards/)

There aren't many famously-rewarded whistleblowers, and of them, banking seems to be a rarity (except for Deutsche Bank), but it's certainly an interesting concept. Effective? Time will tell.

</comment><comment><USER18> This is the way modern Russia works. Troika is probably one of a cleaner financial institutions there, but they have to service the ruling class just like everybody else

</comment><comment><USER19> question: say Russians want to launder $10B. Banks help them. What is the % today to launder money? And how much would banks keep?

I am surprised execs would risk jail, fines, lifetime job loss unless they would profit handsomely. Maybe they can get paid on the side for such deals? I get the bonus payments from the bank but still...

<comment><USER20> But they don't risk any of those things. Aside from Iceland can you mention a bank where their execs did jail time in the past 10 years for the shenanigans that have come to light? HSBC was caught red handed laundering money for drug cartels and I think maybe there were some fines? None of them went to jail.

<comment><USER19> still, how much will the bank get on fees and what % does trickle to C-level execs? I mean still the reward can't be that great (relative to their existing wealth /salaries

</comment></comment></comment><comment><USER21> In reality, a liberal bribing policy to get business done, to protect profits and increase margins, along with a solid policy to push wages down and reduce the workforce is what's best for every country... at least for the ones who pull the strings in said country.</comment>


Kubernetes founders launch Heptio to help bring containers to the enterprise - <AUTHOR> https://techcrunch.com/2016/11/17/kubernetes-founders-launch-heptio-with-8-5m-in-funding-to-help-bring-containers-to-the-enterprise/ <comment><USER2> [https://news.ycombinator.com/item?id=12984668](https://news.ycombinator.com/item?id=12984668)</comment>


ThePirateBay.Org Is Immune from SOPA - <AUTHOR> http://www.techdirt.com/articles/20120109/04205617341/if-sopas-main-target-is-pirate-bay-its-worth-pointing-out-that-thepiratebayorg-is-immune-sopa.shtml

<comment><USER2> I think the solution to the following is for us in the rest of the world to just block all US based IP's until you guys get your political act together... ;)

From the Wikipedia article on SOPA:

Art Bordsky of advocacy group Public Knowledge similarly stated that "The definitions written in the bill are so broad that any US consumer who uses a website overseas immediately gives the US jurisdiction the power to potentially take action against it."[44]

<http://en.wikipedia.org/wiki/Stop_Online_Piracy_Act>

</comment><comment><USER3> The mere fact that SOPA uses DNS to identify particular web sites and their "origin" shows how little these lawmakers know about internet and its technologies.

<comment><USER4> Only, they don't have to know shit.

If what they do re: blocking doesn't fit their purposes, they can adjust it so it does.

If you have the power to pass laws to a specific objective, the technical details are irrelevant.

</comment></comment><comment><USER5> But not from ICE.

Honestly, I'm surprised they haven't taken it yet. I never go there, so I didn't realize that TPB still had any US-based domains left.

<comment><AUTHOR> ICE is illegally taking down those websites. They have no authority to take them down. This is why they are only doing it to random no-name websites that people haven't heard of, thinking the vast majority of them will not even respond. If what they are doing was legal, they would've taken down TPB by now.

It's just one more case of doing something illegally, and then using the excuse that "we were doing it before anyway" to convince politicians and pass it into law later on. I just hope that if SOPA and PIPA are rejected, someone will try to go after ICE for their abuses.

<comment><USER5> None of that has stopped them in the past. Apparently, they know a judge who doesn't mind sealing the docket in such a way as to deny the defendant even the opportunity to see them in court.

<comment><USER6> An example Internet Domain Freedom bill (opposite of SOPA) should counter this in making it illegal to take down the domain without judgement, so it would be excepted from seizures until after due process.

At the moment, they can initially seize pseudo-legally. Then it's a question of being illegal since they can't hold it without cause.

<comment><AUTHOR> I'm for Internet Freedom laws, too, but I just worry that once we give the Government the power to control the Internet, we're only 2 or 3 Congress cycles away before someone starts tweaking that law in their favor, and start ripping the "Internet liberties" away from the law.

I would be much more confident if it was a Constitution amendment, but not done right away, because I don't trust the current generation of politicians with it, but maybe sometime by the end of the decade.

<comment><USER6> The government has seen SOPA, DMCA, PROTECT-IP, COICA, CEST, OPENA, ACTA and more. The anti-Internet groups are already giving government and corporations the power to control the Internet since 00s. I guess you could say that they are beyond tweaking the law in their favour, they're introducing laws in their favour.

If they won't stop demanding control of Internet, then we have to demand Internet Freedom bills. Those Internet Freedom bills are to replace SOPA in outlawing the opposite; outlawing attempts to seize domain names without judgement being passed.

Edit: The general aim of the bill is to make initial seizures illegal. It can be argued that it's hard to run off with the domain name during court case for example.

<comment><USER7> _The anti-Internet groups_

Which groups are you referring to?

<comment><USER8> In context: Whoever lobbied for this list of laws.

<comment><USER7> That's about as fair as calling those who oppose them "pro-piracy" or "pro- counterfeiting".

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> Reminds me of what is going on in The Netherlands (such a liberal country too, so kind of weird).

2 ISP's (Ziggo & XS4ALL) have to block TPB within 9 days, or face charges of 10000 euros a day. Kind of unfair, since their competition haven't been affected by this, yet. Can't find any English news articles so will blog about it later. Brein, the dutch anti-piracy police will also be able to provide these 2 providers with other domains that they'll have to block.

<comment><USER10> In Belgium, a court order forced the ISP's to block TPB. They complied immediately.

The same day TPB registered a new domain name in Belgium, to prove the uselessness of such action.

What a waste of public money.

</comment></comment><comment><USER11> .com/.org/.net is in a sort of limbo about whether it's an international or US based domain. A lot of the control is in the USA, and laws like SOPA seem to view it as an American possession.

However in many non-USA countries, .com is quite commonly used in the internet.

I wonder what would happen if courts in the USA ordered a .com removed and the original .com went to local courts who then ordered ISPs to resolve it? Could there be a split in the .com namespace then?

<comment><USER2> I've just realised I know less than I thought on this... So SOPA wouldn't be able to block .ie or .co or .tk names?

<comment><USER11> Block it where? SOPA, being a law from the United States is only valid in the USA. Different countries have different laws.

As an example, if China bans a site, is it banned in the USA? So if the USA blocks a site, is it blocked in the EU?

However it gets interesting with domain names. No-one really knows what would happen if the USA courts tried to delete a .ie domain. They wouldn't have the authority to do it. USA courts may have to result in ordering all ISPs to block that site, but people in other countries would still be able to do it. After all, the USA doesn't own the internet.

But, like I said, who owns .com? What could happen then?

<comment><USER12> In response to hessenwolf and rmc:

SOPA does not 'delete' any domains. SOPA says that it can tell ISPs and Domain servers to not resolve certain domain names (within the US, the rest of the world doesn't, or shouldn't use US ISPs or domain servers). And it only affects domain names registered outside of the US. As others have stated here, the US already has existing measures to address US domains via ICE.

One of the reasons behind the claim: "SOPA will break the internet", is that many people in the US may start using non US domain servers to circumvent SOPA, and this would make our current routing and caching mechanisms inefficient.

<comment><USER11> _SOPA does not 'delete' any domains_

Gotcha. I don't follow the minutiæ that closely.

<comment><USER12> I totally understand, but I encourage those that feel strongly about this subject to learn those kinds of details. There's a lot of Pro- and Anti-SOPA FUD out there.

</comment></comment></comment></comment></comment></comment><comment><USER13> This is going to just force US citizens to start taking their domain and hosting business outside of the US, until they pass another law stating you cannot do that.

</comment><comment><USER14> I am beginning to believe that the only thing that will convince the politicians is to go ahead and pass SOPA, then everyone just have every large commercial site shut down constantly with SOPA complaints, after all there are no repercussion for ruining someones livelihood with SOPA.

</comment><comment><USER15> No, it's not. It might be immune from the DNS blocking but that is only part of the bill. The other part provides a process for content owners to issue "takedown" orders to ad and payment processors who do business with companies like TPB.</comment>


How I built a Real-time Collaborative Editor with Web Sockets, Node.js & Redis - <AUTHOR> http://www.web2media.net/laktek/2010/05/25/real-time-collaborative-editing-with-websockets-node-js-redis/ <comment><USER2>
      > You may think this is more or less similar to what Etherpad had - yes, it is!

Huh he says nothing about implementing operational transform so I can't imagine that this anything like Etherpad at all.

      > However, on some occasions these two actions needs to be executed concurrently

Some occasions? Using diffs? What about conflicts?

</comment><comment><USER3> Is it possible in this context to replace Node.js with Event Machine without any kind of loss of expressiveness, abstraction, performances? Or, in other words, what are the advantages of node compared to an event programming library for another language like Ruby or Python?

Since Ruby supports fibers, it should actually be simpler to take the state across events firing, at least in theory, but since I'm everything but a node.js expert I would love to hear your opinion on the matter.

<comment><AUTHOR> As I mentioned in the post real bottleneck occurs when it's used with the other supporting libraries (which would be the practical case). Most libraries in Ruby (or Python) are implemented as blocking processes. This defeats the purpose of using an evented programming library.

In contrast, most of the Node.js' libraries are written in a non-blocking manner. This may be because of the initial path the eco-system took. Also JavaScript, being naturally event-driven language encourages writing non- blocking code.

<comment><USER3> Hello laktek, I really agree about the fact that node.js being from the ground up an evented environment got most libraries written in the same paradigm.

On the other side I fail seeing how Javascript is a more event driven language compared to Ruby. In the context it's often used (the web browser) it supports callbacks, but not with some special semantic. Just callbacks, anonymous functions.

For this point of view I think that languages with coroutines are probably more powerful as event-driven programming languages, because it's possible to save the state across calls in a much more high level way, leading to code resembling a lot erlang code for instance.

Thanks!

</comment></comment></comment><comment><USER4> > V8, on which Node is built, supports only IA-32 and ARM processors.

What a pity it doesn't support 64 bit yet, I'd definitely play around with this otherwise.

<comment><USER5> 64 bit is supported on Linux <http://code.google.com/p/v8/issues/detail?id=330#c7>

<comment><USER4> They should really update that page then, anybody reading it that plans to deploy on a 64 bit system is going to be turned away.

</comment></comment></comment><comment><USER6> The author states that a difference between Realie and Etherpad is that he users Web Sockets for client/server comm. I'm fairly certain that Etherpad uses Web Sockets too. Am I mistaken?

<comment><USER7> Etherpad had 3 modes of transport between client and server: short-polling, long-polling, and streaming, each with varying support for different browser/OS/proxy settings. The client starts the connection with the most widely supported method (short-polling), and then attempts to "upgrade" the connection by testing out faster connection types.

I think the messages from client to server always had to do an HTTP post, so we did incur the overhead of all the HTTP headers with each post from client to server, which happened at most twice per second.

You can view the client-side JS code for handling these 3 connection types here:

[http://code.google.com/p/etherpad/source/browse/trunk/infras...](http://code.google.com/p/etherpad/source/browse/trunk/infrastructure/net.appjet.ajstdlib/streaming- client.js)

I didn't write this, but that's my understanding of how it works.

You can also get around the overhead of HTTP headers by using a Flash shim, but we never got that to work well through weird proxy configurations and misbehaving ISPs.

</comment></comment><comment><USER8> Good impressive and nice work. Congratz

</comment><comment><USER9> Awesome! Keep up the good works!

</comment><comment><USER10> Right tools for the job indeed.

</comment><comment><USER5> Demo looks impressive, congrats.</comment>


Ask HN: Will Google penalize my site if I don't use Google Analytics? - <AUTHOR> I'm starting a new niche site, and would like to preserve my visitors' privacy for those to whom that's an important issue.  For that reason, I'd like to leave out as many third-party tracking links as possible.
According to one of the big SEO firms[1], the top three ranking factors that Google uses are:
1) direct website visits 2) time on site 3) pages per session
To measure these, it seems to me a site would either have to ping back to Google on every page hit, or else the user's browser (i.e., Chrome) would have to do it for them.
Chrome stores it's users browser history on Google's servers[2], so they know this data for all logged-in users.  But it can't track non-logged-in users, nor users of other web browsers.
If I leave off my GA code, Google won't be able to measure these three metrics for many visitors.  Won't that hurt my web site ranking?
[1] - https://www.semrush.com/ranking-factors [2] - https://www.google.com/chrome/browser/privacy/index.html#browser-modes <comment><USER2> I do not use google analytics</comment>


How Cloudflare Helps Serve Up Hate on the Web - <AUTHOR> https://www.propublica.org/article/how-cloudflare-helps-serve-up-hate-on-the-web <comment><USER2> I'm the last person to defend Cloudflare. They do a fair bit of crappy things (treat TOR poorly amongst them).

But, we do have freedom of speech in the US. And while that doesn't mean everyone has to give you a platform for that speech, it does mean a content provider needs to decide between being a judge or allowing everything.

Sure, Neo-Nazis seem like an easy case - nobody likes Nazis. But, what about people calling for a boycott of Israel? Or what about critical political speech? Should Cloudflare (or anyone) be in the business of making judgements about those kinds of content and if they're allowed or not?

<comment><USER3> I think the more important issue in here is that Cloudflare forward the personal information of anyone who complains about the site to the site owner. Other major providers don't do this, and it's not clear that people filing complaints know what they're getting into.

<comment><USER2> So, if you go to Cloudflare's abuse form ([https://www.cloudflare.com/abuse/form](https://www.cloudflare.com/abuse/form)) they do state:

> CloudFlare will notify the site owner and, where appropriate, the web > hosting provider for the site in question.

That being said, it's in body sized font, in a huge block of text, and if I wasn't looking for it, I wouldn't see it.

So, it's "stated" but probably not very clear that abuse reports may be forwarded. As a webmaster/previous ISP, I expect (again, prior knowledge here) that abuse complaints will be forwarded. But, does the average person? Probably not.

My $0.02 - Cloudflare states it, but they need to make it much more clear if they intend to keep the policy, or possibly revise the policy. However, bad UI seems a weak reason to vilify them.

\----

Edit: Actually, they state it twice - including right before the submit button in larger font size. It's pretty hard to miss.

</comment></comment></comment><comment><USER4> The use of improper SSL termination and re-captchas that take sometimes up to 20 minutes to complete (for one CAPTCHA, fuck you Google!) are much more atrocious abuses than the existence of some hate speech on sites they host. I'd much rather see them address those actual problems than read about some bitchiness that they won't censor their content. I actually agree with Cloudflare on this. Yes, the Daily Stormer might be an easy site to assess and block (I don't know, I've never visited), but once you're in the business of censorship, the censorship wing of the organization will eat up tons of resources and is guaranteed to turn away customers. See any major Internet company for plenty of examples of this. It seems to me that by not engaging with this stupidity, they are making a wise business decision.

</comment><comment><USER5> Complain to their web host then, Cloudflare has nothing to do with this unless you're arguing there's a right to DDoS them.

<comment><USER6> The only way to get information about their web host is to submit to Cloudflare's abuse form, which, well...

</comment></comment><comment><USER7> Why is the story hidden?

Ever since Cloudbleed effected so many sites and exposed TLS protected data to everyone incl search engine spiders, we should have moved on.

[https://news.ycombinator.com/item?id=13721452](https://news.ycombinator.com/item?id=13721452)

</comment><comment><USER8> Cloudflare is one of the worst companies there is. They weaken the security of the web. They block Tor users from using many popular websites. They host websites that offer the same DDOS services they will charge you for protection from. The day Cloudflare dies, which should have been the day Cloudbleed was discovered, will become an annual holiday for the Internet.

<comment><USER9> That seems like quite a bit of an exaggeration. CloudFlare is great for a lot of reasons, and they have their faults just like every other company out there.

<comment><USER8> Do you have any thoughts on my post rather than "no?"

</comment></comment></comment><comment><USER10> _Cloudflare also has an added appeal to sites such as The Daily Stormer. It turns over to the hate sites the personal information of people who criticize their content. For instance, when a reader figures out that Cloudflare is the internet company serving sites like The Daily Stormer, they sometimes write to the company to protest. Cloudflare, per its policy, then relays the name and email address of the person complaining to the hate site, often to the surprise and regret of those complaining._

 _This has led to campaigns of harassment against those writing in to protest the offensive material. People have been threatened and harassed._

Really, Cloudflare? It's one thing to provide a CDN service for these these sites; turning over this information is completely different.

<comment><USER11> cloudflare is _legally obligated_ , like all ISPs, to forward DMCA complaints. do you think copyright claims should be done anonymously?

<comment><USER12> Complaints about sites people don't like very likely are not DMCA complaints.</comment></comment></comment>


Lyft IPO Is Said to Be Oversubscribed Ahead of Listing Next Week - <AUTHOR> https://www.bloomberg.com/news/articles/2019-03-19/lyft-ipo-is-said-to-be-oversubscribed-ahead-of-listing-next-week <comment><USER2> People looking to make a quick buck and then the stock will crash.</comment>


A new apparel start-up makes the greatest hoodie ever known to man - <AUTHOR> http://www.slate.com/articles/technology/technology/2012/12/american_giant_hoodie_this_is_the_greatest_sweatshirt

<comment><USER2> [http://www.slate.com/articles/technology/technology/2012/12/...](http://www.slate.com/articles/technology/technology/2012/12/american_giant_hoodie_this_is_the_greatest_sweatshirt_known_to_man.html)</comment>


Liberty Global to buy Virgin Media for $23.3bn - <AUTHOR> http://www.bbc.co.uk/news/business-21347814

<comment><AUTHOR> Can any HN folk tell me if this would mean that UK Virgin Media customers are now some how subject to US law? I'm imagining that a situation could occur where "my" packets could end up some how in a US based cache or proxy with out my knowledge or permission, and then become the basis for some prosecution.

Yes, this is a slightly paranoid exaggeration, but given recent stories regarding US prosecutors and the reach of the US, it has to be a concern.

<comment><USER2> It's probably worth noting that Virginmedia uses Google for e-mail, so that part of your traffic may already be subject to US law.</comment></comment>