Falsehoods Programmers Believe About Search - <AUTHOR> https://opensourceconnections.com/blog/2019/05/29/falsehoods-programmers-believe-about-search/ <comment><USER3> I've implemented search engines for small to relatively large organizations. Even at the companies where nobody knew anything about how search hardly any of these falsehoods were believed.

Also this doesn't work as a good Falsehoods Programmers believe thing subject because Falsehoods programmers believe are not about technologies but about non-technological things that are commonly needing to be handled in programs - hence Falsehoods programmers believe about:

Names, Phone Numbers (sort of technical but it's not falsehoods about how phones work, but rather about how phone numbers are structured and what they 'mean'), Credit Cards, Addresses

Good possible future Falsehoods programmers believe about:

Sleep patterns, Personal identifiers, Genders

In fact I am currently dealing with a falsehoods programmers believe about versioning of laws and standards at work.

<comment><USER4> Eric Myers needs to write an article called "Falsehoods developers believe about writing falsehoods developers believe articles".

<comment><USER5> "Falsehoods programmers believe considered harmful"

<comment><USER6> Falsehoods programmers... you wouldn't believe what comes next?!!

</comment></comment></comment></comment><comment><USER7> Falsehoods Google, Microsoft, JIRA, and others seem to believe about search:

That, when searching for a string, I don't want exact matches to appear in the results.

If your search ever DOESN'T return exact matches (barring common misspelling correction), you're doing something seriously wrong.

<comment><USER8> I'm not sure I understand you.

If I search for "restaurants", I want search results that ARE restaurants, not search results which have the word "restaurants" in them.

What do you want to have happen?

<comment><USER9> It depends what you're looking for. If it's an error message, you probably want that exact string, not results that are errors but don't include the string.

It used to be that searching literally "restaurants" (i.e. with quotes) would search for an exact match (particularly useful for multi-word searches in those days), but no more. It's taken as a 'hint' or something, I believe, but not an absolute instruction.

<comment><USER10> And this is how it began the AI takeover.

</comment></comment></comment></comment><comment><USER11> There's also:

That we want well known standards like CTRL + F in a browser to be hijacked and replaced by default with a custom search experience that's a lot worse than a browser's search.

Try CTRL + F'ing on Stripe's documentation: [https://stripe.com/docs/api/plans](https://stripe.com/docs/api/plans)

<comment><USER12> On Chrome, you can hit CTRL + G, which does the same as CTRL + F, but is not hookable by web sites

<comment><USER13> Thanks for that, I didn't know it! It seems that also F3 works, and in fact CTRL+G is alias of F3, and both work in Firefox as well.

The only issue is that in Firefox, it is only equivalent for the first search; once you close the bottom bar, subsequent F3/CTRL-G just do "find next occurrence" and do not display the bar anymore. Chrome always displays the search input on the other hand.

Edit: since talking shortcuts, in Firefox ' (apostrophe) is like CTRL-F but searches only hyperlinks (and you can cycle through in case of multiple matches with F3/CTRL-G) which is _extremely useful_ for quickly navigating pages via keyboard only.

</comment></comment></comment><comment><USER14> * When you find the boolean operator 'OR', you always know it doesn't mean Oregon

One of my favorite sets of local search bugs involve interpreting "near me" as "near maine".

<comment><USER15> Trying to fix every single problem in the search module/layer/service is an anti-pattern by and of itself.

There's an anecdote[1] from early days of Google Search where a certain domain was ranking 1st for an unrelated query (i.e., a false positive). The managers refused to move ahead before that got fixed, but the bug/edge case proved a head scratcher for several weeks on end.

Lastly one of the engineers solved the problem - by buying the domain and taking it offline.

Point being, if you can fix the problem outside of the code domain, do just that.

[1] sadly can't seem to find it - mostly getting spam articles related to SEO

<comment><USER16> I'm pretty sure I've read a similar story in the book "I'm Feeling Lucky". It goes like this:

In the early days of Froogle, a shopping search engine made by Google, searching for "sneakers" always yielded a garden gnome wearing sneakers, one unit on sale, as the top result. This was considered bad, as someone searching for "sneakers" probably wanted to buy sneakers, not garden gnomes. The whole team tried to fix it, but they didn't want to just hardcode an exception. It eluded them for a while. Finally, it was not there anymore. They asked around for who had solved it, no one answered. Finally, one colleague arrived late - and placed the gnome on their desk.

<comment><USER17> "Buy the Gnome" should become a saying, like "eat the frog".

</comment></comment></comment></comment><comment><AUTHOR> Howdy. Author here. Really cool to see so much good discussion on this. I want to turn several of them into blog posts on their own with explanations/stories/what-have-you. Taking votes for what you'd like to see first. For the record, my fave is "Languages don't change".

<comment><USER15> Thank you for the thorough and practical write-up.

About the only thing I would add to it is i18n concerns.

A few quick ones off of the top of my head:

      - Words are separated by whitespace or dashes.
      - Customers only ever enter ASCII.
      - Customers only ever enter accented characters with/without accents.
      - A "Unicode-capable" system will happily take in any valid unicode.
      - A "Unicode-capable" system will pass through any valid unicode undisturbed.
      - Software systems perform Unicode normalization.
      - WinNT API is UTF-16.
      - There is 1-to-1 mapping between uppercase and lowercase.
      - Unicode collation algorithm is optimal for every single language.
      - Unicode collation algorithm is optimal for multi-language document sets.
      - Distinguishing/coalescing plural and singular forms of words is easy.
      - There are separate plural/singular forms of words.
      - Words have stem and optional suffixes, but not prefixes.
      - Soundex etc. works for every language.

<comment><USER18> > There are separate plural/singular forms of words.

Or that there are just two plural/singular forms (1 and many) for translating strings, or that which form to pick is clear.

While English has one form for 1, and one form for 0/many:

\- French pluralises 0 the same way as 1,

\- Czech has a form for exactly 2-4 items,

\- Irish has forms for exactly 3-6 and 7-10 items,

\- Polish has a form for all numbers that end in 2-4,

\- Russian has a form for all numbers that end in 1,

\- Arabic has forms for exactly 0 and 2 items, ending in 03-10, and many more.

A strings table will need at least 10+ variants if you want to translate strings referring to number of items.

</comment></comment></comment><comment><USER19> I think this article is setting up a pretty high bar for search. For small datasets, you can very well just add an automatically generated "description" column in your database, and then do a SQL LIKE query: it's a simple substring matching.

It's by no means smart, doesn't handle misspellings or anything, but it works reasonably fast and predictably. This is basically how almost every desktop app with a search bar works. This is how word processors and editors work when users search within the document.

<comment><AUTHOR> Sorry but that's going to result in a pretty horrible search experience. If you are putting a search bar on your page and that's your search backend - <USER2> might as well skip search entirely because it's only going to cause you and your customers pain.

The difference with find on page is that it's obvious and transparent what is being searched and the expectations of the interface. Trust me when I say that a search bar to a layperson on your site is them thinking "oooh I can google"

<comment><USER19> Sound like you need to redesign your search bar on the page to convey to users the expectation that this is a simple search.

<comment><USER20> Do you have an example of that that simple search bar could look like?

<comment><USER19> I'm not a professional UI/UX designer but here's a guess. It should not be the prominent thing on your page. Prominent search bars like the one on Google's home page convey to users the idea that this is the primary way to navigate and use this website, and therefore be loaded with higher expectations. So don't make the search bar look prominent.

Next, make it context-specific. Don't put a search bar at the top of the page suggesting that this bar can search for everything. If you use a simple implementation like a SQL LIKE to implement search, put the search bar right next to the thing that is displaying results from the table. Make it look like it's filtering the table.

Finally, label the search bar using words like "Keywords," which also suggest to users that they should be typing keywords instead of a more complicated natural language phrase.

<comment><USER20> Those are interesting ideas thanks for the thoughts. FWIW I've seen users try to use even non-prominent search boxes like those as if they can do more than SQL LIKE. Most users have no idea how any of this works and just want answers.

Mostly I think this whole thread demonstrates the point of the original article, but I appreciate your response.

</comment></comment></comment></comment></comment></comment><comment><USER21> This is both awesome and so so discouraging. Does anyone have some direction on how to produce good search systems??

<comment><USER22> Focus on measuring search quality and methodology first. Be a scientist. Great search teams obsess about methodology. Treat everything you try as a hypothesis, not guaranteed to work. Create a feedback loop that improves the pace of experimentation.

Other than that, the solution space is just as wide open as regular programming. It's just in many ways more frustrating because nobody knows what they really want from search, they just "know it when they see it" and no two users really can agree on what a good result is! :)

<comment><USER23> This is a very, very insightful point. I would add: never expect a singular "perfect" algorithm, but rather build a framework that lets you blend (and evaluate/weight) the signals from various hacks, workarounds, heuristics, and "proper" algorithms.

</comment></comment></comment><comment><USER24> Search engines work like databases - Too vague but arguably yes in the abstract.

Search can be considered an additional feature just like any other - Yes? How do you falsify this?

Search can be added as a well performing feature to your existing product quickly - Yes if you're using a CMS with search already there like Drupal, or you can use that thing where your search uses/directs to Google.

<comment><USER25> > Search can be added as a well performing feature to your existing product > quickly - Yes if you're using a CMS with search already there like Drupal

Adding a feature by using a product that already has that feature is not "adding a feature to a product". It's "doing nothing since there's nothing to do". ;-)

Using Google search for pages might work for simple sites that mostly host text content, but not for things like "find all foos that are between 20 and 30 kg".

<comment><USER24> If it's just a few things like "find all foos that are between 20 and 30 kg" then that might be nothing but building a simple query out of a few criteria. Not all searches need to be or even aspire to be a super-general search like Google. The ebay search probably isn't all that complicated (relatively) for example. If you're trying to make another Google for some strange reason then the article applies more.

</comment></comment></comment><comment><USER26> * All customers may see the same data

God, how I hate that authorization woes find a way to make everything else 5x more complicated.

<comment><USER27> I wonder if that's aimed at permissions-based stuff or, like, search bubbling?

<comment><USER26> I'm talking about permission-based stuff.

</comment></comment></comment><comment><USER28> On misspellings (since there are quite a few lines here dedicated to them), I had the fun responsibility of learning / knowing too much about how our search worked (we were/are using an old version of Solr), and started telling people that there's a way to at least do _something_ about misspellings.

After conversations with two or three product managers, it became clear that the best course of action was to do nothing at all. I'm definitely not an expert on search or human behavior, and running through all the possible interpretations of how to handle misspelled words and what the customer wants was way more work than I was prepared to do.

I'll even point out that my initial suggestion was, "Let's just copy Google and do, 'Did you mean to type _______?'" Even that was met with, "what if the customers X" "what if the customers Y" etc. etc. Wasn't worth the time (at the time).

<comment><USER29> You could call it related searches and only display the suggestion when all words are either in the products catalog or in the dictionary, also checking if the query returns something with a phrase search. That can help with typos without ever being to weird

</comment></comment><comment><USER30> A list of postulations without examples or explanations is not useful.

<comment><USER31> Quite true!

Or even enough context to interpret:

> Search can be considered an additional feature just like any other

Is that a falsehood? - what does it even mean?

<comment><USER32> Almost nothing. I guarantee that for any non-trivial feature, you could just say:

"<non-trivial feature F> can be considered an additional feature just like any other"

And everyone will agree that's probably false. They could have written "search is almost never a trivial feature, and you should take your time to consider complications", but I suppose that wouldn't sound as a cute as a "Falsehoods Programmers Believe" list.

</comment></comment></comment><comment><USER33> Related to "Customers who know what they are looking for will search for it in the way you expect", many people don't understand that a search engine works by matching text strings (albeit in an often sophisticated way). They see it as sending commands that the search engine understands, and will then find results for...

<comment><USER34> I know VSCode had an issue where people would type whole sentences into the settings search bar. They got around it by incorporating some of Bing's NLP logic. Goes to show, even amongst the "technically inclined" (those who not just use VSCode, but also try modify things in it), this still holds.

</comment></comment><comment><USER35> Search interfaces should have a configuration for smart users:

      [ ] Disable fuzzy parsing hacks (reject my queries if they have bad syntax).
      [ ] Don't search for sound-alikes; assume I spelt everything rite.
      [ ] Respect the non-alphanumeric characters in my query, which I put there for a reason.

</comment><comment><USER36> > A customer using the same query twice expects the same results for both > searches

Really, this is a falsehood? Like, I want the same query to give the same results given the same dataset always. When do you not want that?

<comment><USER15> _> >> A customer using the same query twice expects the same results for both searches_

Of course this is false; please consider:

\- customers expect to see in search results whatever new information they added/updated in the system (this is related to "Customers don't expect near real time updates");

\- customers expect "personalized" search results; having built up a history of searches centered around particular subjects (say, programming), you'll expect much different results for "string" than the general population gets;

\- customers expect new/more results having logged in, or having gained new permissions/roles;

\- customers running "knowledge" or "command" queries ("what is the weather?" "password 16") expect varying results

<comment><USER37> Or, for a short query string, the user may have a different intent without realizing they've put the same query in.

I might dash off a search for "sneakers" when I am researching footwear. A week later, I might be thinking about movies and enter the same query string, expecting IMDB results.

</comment></comment></comment><comment><USER38> That I actually want Sublime Text to stop responding for 5 mins while searching for a single space character across my entire project.

</comment><comment><USER39> I'm just waiting for the inevitable article titled "Falsehoods Programmers Believe Lists Considered Harmful."

</comment><comment><USER40> Is everything we believe about everything wrong?!

<comment><USER15> "All models are wrong, but some are useful" (generally attributed to the statistician George Box).

A belief, or a system of beliefs, is but a model. It's virtually guaranteed to be wrong. It also may very well serve the important function of being _simple enough_ to handle in-core, while at the same time being _close enough_ to substitute for the real thing.

<comment><USER41> > virtually guaranteed to be wrong

I would go a step further and say all formal models are _proven_ to be wrong. After all, that's what Gödel and Turing kept going on about.

We can't prove any non-trivial program ever halts or does not halt. In fact, we can't (or don't) prove much about our programs we run anywhere.

All programs are a collection of assumptions. To bring this back to the topic at hand, if all of our search assumptions are _useful_ to some meaningful number of people then it really doesn't matter how many "falsehoods" we trip over. Those falsehoods fall away, becoming mere insignificant edge-cases. Satisfying all people all the time in all cases is a fool's errand.

Articles like this are good at letting you know your blindspots so you can _choose_ your blindspots rather than succumb to them. But don't let it become dogma.

<comment><USER15> _> all formal models are proven to be wrong_

Your point certainly holds true for any _physical entity_ as far as we know - probabilistic quantum effects, Heisenberg's Uncertainty, chaotic systems, and all that.

However if you were to model a _theoretical entity_ , and given a few more constraints (like strict computability, which precludes a turing-complete systems), you can indeed have correct models. Alas, in practice this is a rather rare example.

</comment></comment></comment></comment><comment><USER42> I've implemented a fair bit of search engines. Usually the problems are with non technical people in a project. I've had to coach a fair bit of product owners and UX designers on the basics of search. There are two issues I tend to have with them: 1) they avoid things that they think are hard that just aren't 2) they are unaware of features that e.g. Elasticsearch would support that are highly relevant to their project and therefore don't plan for using those.

A UX person thinks of search as a text box "like google". However, a lot of search UIs have a lot going on when you start typing and when you get results back to refine search results, DYM corrections, breakdowns/aggregations, suggestions, etc. A lot of these features require careful planning and design and are not necessarily easy to bolt on if you don't.

I've also had to do basic things like patiently explaining the difference between sorting and ranking and humbly suggesting that, maybe, having a multi column layout with sortable columns isn't necessarily the right thing for presenting search results where the output is a list of stuff in order of relevance.

Engineers are easier to deal with once you sit them down and talk them through how stuff works.

</comment><comment><USER27> As with many of the other commentors here, I wonder how many programmers truly believe these things. Maybe as recently as the 90s or 2000s. Maybe developers who are fresh out of school.

But we've had search engines as a major part of our lives for about two decades now. Most of us use one at least daily. We're familiar with the complexities of search engines and how they differ from simply searching a document for an exact string or even a regular expression. Many programmers like me work with tools like analytics and log aggregators that expose the complexities of search to us in a way that's more intimate than the veneers of Google and Amazon.

Maybe I'm just lucky in that my experiences have dispelled these notions of search being easy or simple. But I hope I'm not alone.

Also, there's a disparity between what search is and what your users expect. Technically, I could make a really simplistic "search engine" that amounts to a SQL LIKE query. It may not be good or what users might expect coming from Google/Amazon/etc, but it would be a search engine. (Oops. Looks like my pedant hat slipped back on when I wasn't looking.)

</comment><comment><USER43> I don't know, the first third of the list contained about ten things I don't think any programmer believes about search, so I gave up at that point.

</comment><comment><USER44> _cough_ "setup" is a noun, "set up" is a verb </pedant>

</comment><comment><USER45> I would add to the list of falsehoods:

\- customers are always searching for a specific item, rather than an entire category

\- customers know that a search engine for one kind of item (e.g. products for sale) won't also search the entire rest of your website

</comment><comment><USER46> One major annoyance,hard to search for any topic related to c programming online, one has to wade through mountains of results on C++ and C#.

</comment><comment><USER28> > Choosing the correct search engine is easy and you will always be happy with > your decision

I laughed, but I don't think this is a correct representation of something many programmers genuinely believe. It's worded in such a way that it's clear this is a joke. Not sure if I should read the full list if it's just going to be jokes like this one.

<comment><USER47> How many genuinely unique search engines are there really to choose from? (Not counting those based on the same underlying libraries)

<comment><AUTHOR> Quite alot, actually: [https://en.wikipedia.org/wiki/List_of_search_engines](https://en.wikipedia.org/wiki/List_of_search_engines)

</comment></comment></comment><comment><USER48> My favorite is "languages don't matter and I can just throw text in there"

</comment><comment><USER49> "Once setup, search will work the same way forever" \- I don't know a single programmer who believes this about any software.

</comment><comment><USER50> > Regular Expressions have minimal performance impact

REs and FSMs equivalent.

<comment><USER21> for real? Is RegEx actually a FSM behind the scenes? or are you trying to say something else

<comment><USER15> Yes and no. The theoretical "regular expressions" are indeed Type-3 grammars in Chomsky's hierarchy.

In practice, the common "RegEx" implementation implement a lot of extras, that break the theoretical backing, and also exhibit highly non-linear behaviors. Cf. this excellent paper by Russ Cox: [https://swtch.com/~rsc/regexp/regexp1.html](https://swtch.com/~rsc/regexp/regexp1.html)

<comment><USER50> Thank you for this reference!

</comment></comment></comment></comment><comment><USER51> __

<comment><USER21> Algolia looks good, but are there any OSS alternatives for those of us trying to bootstrap a search system

<comment><USER52> For a blog/static website, Tipue Search [1], or maybe Datasette [2]. There are Pelican [3]/Jekyll [4] plugins for the former.

[1] [http://www.tipue.com/search/](http://www.tipue.com/search/)

[2] [https://24ways.org/2018/fast-autocomplete-search-for-your- we...](https://24ways.org/2018/fast-autocomplete-search-for-your-website/)

[3] [https://github.com/getpelican/pelican- plugins/tree/master/ti...](https://github.com/getpelican/pelican- plugins/tree/master/tipue_search)

[4] [https://github.com/jekylltools/jekyll-tipue- search](https://github.com/jekylltools/jekyll-tipue-search)

</comment></comment></comment><comment><USER53> Pretty much none of these are things programmers believe about search.

Putting limited effort into creating a mediocre search feature doesn't mean that you believe these falsehoods; it just means that you're too resource constrained to put serious investment into creating and improving a high quality search feature.</comment>


Show HN: Upload e-mails from Outlook using only the browser - <AUTHOR> https://mailee.herokuapp.com

<comment><USER2> Looks interesting. What are the typical use cases for this? Why would companies running on Outlook use this?

<comment><AUTHOR> This could be of interest if companies do not want to or for any reason cannot use an Outlook add-in to upload e-mails to apps in the cloud. With this solution you can simply use your browser after adjusting the security settings.

One use case could be the lead or opportunity management in a CRM related scenario. The sales man could simply attach the e-mail in the CRM and afterwards it would be available for all of his colleagues. As the meta data of the e-mail (e.g. subject, body) is also transferred you could easily search through all your and your colleague's mails that are only relevant for the CRM.</comment></comment>


PodTech Sells For Less Than $500k - <AUTHOR> http://www.techcrunch.com/2008/07/17/podtech-sells-for-less-than-500k/ <comment><USER2> Then why sell now? Were they losing money? I am not to familiar with podtech.net but with 7.5 million in venture funding, a 500 grand sale is a huge hit.

<comment><USER3> They were obviously losing money. You don't sell out for 15 times less than what your VCs have put in if you're banking.</comment></comment>


MLHelper: Algorithms and utils for Machine Learning in JavaScript - <AUTHOR> https://github.com/laoqiren/mlhelper <comment><USER2> I want to point you to [https://github.com/stdlib- js/stdlib](https://github.com/stdlib-js/stdlib). You can use it for the lower level utils. There are almost more matrix utils in the js world then there are devs. Let's stand each others shoulders.

Let's standardize a little more than what lodash/underscore does.

</comment><comment><USER3> I just don't understand: why? Beyond the novelty, why are people spending time doing this in JS?

I code things in JS because I spend a lot of time with JS. So a quick CLI tool others would write in Go, I might do in JS just because it's so familiar. But I also spend a lot of time in Java. I'd never choose JS over Java for something like this. I feel like the only reason you would ever do this in JavaScript is because you literally don't know any other language.

<comment><USER4> There's a huge demand for explanatory material for ML, and running small-sized demos in the browser is a great tool for that purpose.

I don't know how many people are using JS to train particularly large models, or models they'd use in production, however.

Finally, I wouldn't be surprised if we see something like "ML tools for Brainfuck" sometime on HN, because everyone wants to learn ML and implementation is a good way to learn and everyone wants to write something new and "fun" (for various definitions of fun).

<comment><USER5> Yes, making some wheels in your language is a good way to learn

</comment></comment></comment><comment><USER6> Can anyone comment on which JavaScript ML libraries are most popular or most commonly used? I've just started using ml.js and it does what I need, but I feel like the ecosystem is so huge I could be missing out on some real gems

</comment><comment><USER5> Thanks for your votes!

</comment><comment><USER7> Could be interesting to see something like this integrated with DeepLearning.js</comment>


What does your OS X dock look like? - <AUTHOR>

I have just switched back from a Linux (Arch) to OS X, and I was wondering what applications you usually have sitting in the dock. <comment><USER2> Dock is usually hidden, use Quicksilver [<http://qsapp.com/>] to launch programs and run common operations on files.

Common software used - iTerm/DTerm, Textmate, assorted browsers, Zend Studio/Rubymine/Netbeans, Adium, Colloquy, zsh, CS5, Caffeine, Dropbox, GitX, Hex Fiend, Skitch, NetNewsWire, Parallels Desktop...

<comment><USER3> I'd like to recommend Alfred App [<http://www.alfredapp.com/>] as an alternative for Quicksilver.

<comment><USER2> I've seen it before, but haven't tried it. What do you like better about it than QS?

<comment><USER3> I found the general user experience more pleasant and Alfred easier to configure for my needs. QS did (and probably still does) a good job, but I wouldn't switch back anymore.

</comment></comment></comment></comment><comment><USER4> I actually paid $10 to get rid off the dock, most of the common ways to get rid of it kept breaking in various ways.

I feel kinda used about paying for that

</comment><comment><USER5> <http://i.imgur.com/bzGxO.png>

Minecraft, WoW, Steam - you know, the essentials.

</comment><comment><USER6> i'm a south paw myself, but not for long. ditching my macbook pro, buying a thinkpad x220, and going back to ubuntu.

<http://imgur.com/FnSyK></comment>


A Dismal Guide to Concurrency - <AUTHOR> http://www.facebook.com/note.php?note_id=379717628919 <comment><USER3> Nice article.

While I have high respect for the exposition in this article, and by extension Mr. Vogels work that the author makes a reference to, I think it's a tad too early declare eventual consistency a clear winner. In fact, I think it's too early to declare that a there is a contest in the first place. While there seems to be some level of agreement that partition tolerance is required for real world systems consistency vs. availability is more of a continuum instead of two discrete choices. I find that in day to day work some data requires a paxos level of consistency (say a configuration file containing your inter- cluster topology) and some data can be highly inconsistent (like activity data such as 'read' status of emails).

As a side-note, I would argue that all eventual consistent systems rely on some domain knowledge about the data that makes it possible for the system to reliably merge conflicting data. Such algorithms usually have the property that they only move state in one direction, emails only move from unread to read for example.

What I'm trying hard to figure out is how Consistent, Available and Partition Tolerance map onto Intelligence, Emotional Stability and Good Looks. Do we always require Good Looks and thus end up choosing between Intelligence and Emotional Stability or ...

<comment><USER4> I happen to agree, funnily enough. I only talked about the extreme ends of the spectrum. I suspect that C/A systems have a natural place, say, up to datacenter-sized systems. After that they founder on latency and partitions.

The big thing about Dynamo in particular isn't eventual consistency but _configurable_ consistency. You can specify how much consistency you want and how long you will wait for it.

</comment></comment><comment><USER5> Great footnote. I'm undecided if I should tackle Clojure or Erlang next. Each one is excellent and interesting in its own way.

> A lot of writeups repeat a "nine nines", ie 99.9999999% reliability claim > for Erlang-based Ericsson telephone switches owned by British Telecoms. This > works out to 31 milliseconds of downtime per year, which hovers near the > edge of measurability, not to say plausibility. I was present at a talk > Armstrong gave in early 2010 during which he was asked about this. There was > a little foot shuffling as he qualified it: it was actually 6 or so seconds > of downtime in one device during a code update. Since BT had X devices over > Y years, they calculated it as 31ms of average downtime per device per year. > Or something like that. Either way it's an impressive feat.

<comment><USER6> what's up with

\- erlang: R14 and somehting like Memory management threads

[http://groups.google.com/group/ceug/browse_frm/thread/28375b...](http://groups.google.com/group/ceug/browse_frm/thread/28375bca0277a16b)

\- scala: lots: Akka, scalaz, lift threads, STM, Bagwell's data structures

\- clojure: special purpose mutability: transients will be old hat, Hickey's working on "cells", (lately he's been coding reify/deftype/defrecord)

\- F#: lots, I haven't been tracking, and i just saw the cheapest Vis studio 2010 is $800(?!)

\- haskell (GHC) CHP's, STM, lots other stuff

(Suppose i oughter blog about)

<comment><USER6> OK, let me try to be more helpful. Here's good backgrounders

<http://www.tbray.org/ongoing/What/Technology/Concurrency/>

[http://www.sauria.com/blog/2009/10/05/the-cambrian-period- of...](http://www.sauria.com/blog/2009/10/05/the-cambrian-period-of- concurrency/)

[http://groups.google.com/group/clojure/browse_thread/thread/...](http://groups.google.com/group/clojure/browse_thread/thread/5c7a962cc72c1fe7?hl=en)

[http://blogs.azulsystems.com/cliff/2008/09/jvm-language- su.h...](http://blogs.azulsystems.com/cliff/2008/09/jvm-language-su.html)

<http://wiki.jvmlangsummit.com/pdf/36_Click_fastbcs.pdf>

\</comment></comment></comment><comment></comment><comment></comment><comment>---

also recommended: Herlihy/Shavit book, blogs by Sutter and Duffy,

[http://www.amazon.com/Art-Multiprocessor-Programming- Maurice...](http://www.amazon.com/Art-Multiprocessor-Programming-Maurice- Herlihy/dp/0123705916)

<http://herbsutter.com/>

<http://www.bluebytesoftware.com/blog/Default.aspx>

</comment><comment><USER7> > A Consistent/Available system means that reading and writing always works > the way you expect, but if one node fails the whole system stops working.

I've seen this mistaken statement several times now and it really bugs me. A consistent/available system can't tolerate partitions, i.e. a split in the network. However, it can certainly tolerate a single failure and remain consistent and available. That's the whole point. A trivial system with these properties is one that replicates every update to all available nodes and requires reads to get matching results from at least half of the nodes. Paxos is another example.

<comment><USER4> You've got a good point, and I'm not satisfied with that section. Can you think of a real-world example of a network partition? Maybe a parade that bisects the city?

<comment><USER7> A washed out bridge.

<comment><USER4> I like it. We have a winner. Thank you!

"Think of a parliment that must have more than half of members present in order to hold a vote. If too many can't make it, say because a flood washes out the bridge, a quorum can't be formed and business can't proceed."

</comment></comment></comment></comment><comment><USER8> Relevant complementary reading (Widefinder implemented in Clojure): <http://meshy.org/2009/12/13/widefinder-2-with-clojure.html>

</comment><comment><USER9> This author is really good at analogies.

<comment><USER10> Yes, my favourite:

"In essence, transactional memory allows threads to ask for forgiveness instead of permission."

<comment><USER11> Good article. I'm still waiting to hear back so I can have y shot. I wanted to give STM a try too.

</comment></comment></comment><comment> j-g-faustus Thanks for a nice overview. I have by and large managed to bypass concurrency issues so far, but these days an i7 CPU runs 8 threads and a run-of-the-mill dual socket Xeon could have 24 threads, so I suppose I'll just have to bite the bullet in order to get anywhere near the performance even a single box can supply.

Where would goroutines, from Google's Go (<http://golang.org/> ), fit in here?

My understanding is that goroutines are pretty much the same as Erlang message passing, except that Erlang messages can work across multiple computers, while goroutines are restricted to one computer (same memory space). Is this correct?

<comment><USER4> I don't know much about Google Go (or coroutines, for that matter), but from the description it sounds like you are correct.

</comment></comment><comment><USER12> Its not only languages, but data structures that need to change. C.f. this video (<http://vimeo.com/6624203>) from ICFP '09

</comment><comment><USER13> Can someone mirror it outside Facebook?

<comment><USER4> [http://carlos.bueno.org/2010/04/dismal-guide-to- concurrency....](http://carlos.bueno.org/2010/04/dismal-guide-to- concurrency.html)

The original link is viewable without needing to be logged-in to FB, as far as I can tell.

</comment></comment><comment><USER14> Will this change with quantum computing and entanglement?

 _You can't avoid it by using electrons instead of street urchins. It's impossible for an event happening in one place (eg data changing inside one computer or process) to affect any other place (eg other computers or processes) until the information has had time to travel between them._

<comment><USER15> No. Causality can only propagate at the speed of light, with or without quantum mechanics.

<comment><USER16> My understanding is that 'entanglement' is like having two urchins each draw a ball from a bag with a black/white pair. One urchin goes East, the other goes West.

Assuming they arrive at A and B, respectively, this may be useful somehow. But it cannot really communicate _new_ information from A to B.

(Unless the balls are carried _unobserved_ , and one of the recipients - <USER2> staring _real_ hard ? - can somehow choose the color of the ball he is unwrapping.)

<comment><USER15> Your analogy is correct inasmuch as the urchins cannot communicate for the same reason the Alice and Bob, each with one particle of an entangle pair, cannot communicate. But it can be shown by very clever experiments that the future statistical observations of Alice and Bob _cannot_ be explained by assuming each had a particle with properties fixed when the two parted ways. Entanglement is very subtle.

</comment></comment></comment></comment><comment><USER17> tl;dr

<comment><USER18> you missed out, this was worth reading

<comment><USER19> And while it's ok not to read stuff, he shouldn't add noise to the discussion. We don't care.

</comment></comment></comment><comment><USER20> Does...this article have e thesis? Or is this just sort of an overview of stuff...</comment>


TSA Doesn't Care That Its Luggage Locks Have Been Hacked - <AUTHOR> https://theintercept.com/2015/09/17/tsa-doesnt-really-care-luggage-locks-hacked/ <comment><USER2> The law pretty much mandates that unless you're carrying a firearm, your luggage cannot be secure. Anybody can pick these locks, even without the "master keys", and trying for anything else will just get your luggage cut. Anything which needs security needs to travel on your person.

So, given that "Secure" is out, I've opted for "obvious when tampered with". I just use boring zip ties that I mark with a sharpie to make them unique (and show if they've been replaced).

I'll also toss a handful of unmarked ties onto the "top" of the luggage, so the TSA can zip up when they've had their way with my luggage. I've occasionally sent a piece of luggage through, and found it re-zip-tied with a new tie, with a TSA love note inside.</comment>


The Online Marketplace That's a Portal to the Future of Capitalism - <AUTHOR> https://www.nytimes.com/2017/05/03/magazine/the-online-marketplace-thats-a-portal-to-the-future-of-capitalism.html <comment><USER2> It is kind BS that it's cheaper to ship something from china to across the street.</comment>


Coinbase is Down - <AUTHOR> https://www.coinbase.com/

<comment><USER2> Appears to be back.</comment>


Ask HN: Should Learn/switch to JavaScript Programming (a Java Developer) - <AUTHOR> I've been a Java developer for the past 4 years. Recently I've been reading and learning a lot more about JavaScript also.Some, people say soon Java could get replaced by JavaScript. Would this ever happen? Are experienced people at HN hearing less of 'Java' projects than say 5 years ago? Will I get a good job if I switch to JavaScript or will I keep getting good opportunities if I continue in Java, esp as Java 8 has managed to include some concepts of functional programming. What will keep Java alive and moving forward in the coming years, other than legacy support codes and stable community contributions. <comment><USER2> What pushes Java and keeps it thriving today are big data and Android. It also has an enormous library and framework ecosystem.

As for language progress, think JVM, not Java. I program exclusively in Groovy. You also have Scala, JRuby, Clojure, and others that run great on the JVM and are more feature-rich than JS (even ES6).

Job market is entirely dependent on your locality. I'm an expert on California's tech job market, and even LA and Orange counties have very different job markets, different still from SV.

</comment><comment><USER3> I see two orthogonal questions:

1\. Learning Javascript => Yes, you're interested and it can't hurt.

2\. Switching to a Javasript job => Depends on the market and what you like to do.

Good luck.</comment>


AppFog Wants To Do For Developer Platforms What Google Did For EMail - <AUTHOR> http://techcrunch.com/2012/07/25/appfog-wants-to-do-for-developer-platforms-what-google-did-for-email/ <comment><USER2> Im really not sure about the pricing model. Here are a couple of points:

There are tons of people looking for $5 vps that have like 500mb on ram. Your pricing structure will be like a magnet for those type of people. I dont think may of those people will upgrade to a $100 a month plan though.

As with the point above, you are relying to heavily on the people who have bought into the system at the free plan and now have to migrate. If I were to go onto one of the paid plans I would feel bad knowing most of the money im spending is going to provide a free service tier.

50Gb data transfer limit 0_0. Its 2012, bandwidth is dirt cheap and 50GB really is not that much. Especially if the person is paying $100+ per month. If someone is using more than 2GB of ram they are most likely using more than 50GB of incoming/outgoing data.

Edit: I signed up to test it out and while it says unlimited services (free plan) once you signup it limits you to 10.

Edit2: Turns out you need to have some ruby knowledge to get started (even if they offer a ton of languages). Since I dont I will have to postpone playing with it till the weekend where I can dedicate some time to playing with ruby.

<comment><USER3> FYI, it's $0.15/GB for additional data transfer: [https://groups.google.com/d/topic/appfog- users/UtTmKrxSTYA/d...](https://groups.google.com/d/topic/appfog- users/UtTmKrxSTYA/discussion)

<comment><USER2> It doesn't matter, even if the bandwidth was free. It is not stated anywhere so a potential customer will see that number and be less likely to buy the plan even if it is the smallest limitation.

</comment></comment></comment><comment><USER4> I'm seeing a lot of comments talking about "just get a {VPS, dedicate box, Amazon, etc...}". I think people are missing the point. AppFog is trying to take all the "hard" work out of SysOps. Think Chef on crack, you are paying for their pre-configured infrastructure from the top down.

Their idea is provide a powerful, scaleable platform quickly and easily. Their mentality is called NoOps (<http://blog.appfog.com/what-is-noops-anyhow/>). It's a cheesy marketing term, and pissed a lot of people off, but it's pretty spot on for what their service offers.

</comment><comment><USER5> <https://console.appfog.com/pricing>

2GB RAM FREE | 4GB RAM $100/month | 16GB RAM $380/month | 32GB RAM $720/month | More if you contact AppFog

Scaling your app is free, choosing multiple infrastructures is free, custom domains are free, fastest available servers in the infrastructures

<comment><USER6> on the other hand, ovh offers dedicated servers for:

16GB RAM £70/month

24GB RAM £100/month

24GB RAM £180/month

64GB RAM £260/month

<http://www.ovh.co.uk/dedicated_servers/>

<comment><USER7> That's so far from a like for like comparison that I can only think you've not read what AppFog is offering...

</comment></comment></comment><comment><USER8> _AppFog's service extends across different infrastructure providers_

Dropping into the IaaS world for a minute, is this as common as it seems me to be?

Personally, I run servers over number of cloud providers, and then have a few VPSs around too, plus some servers at home, and lots of people I know are the same.

Is this a common thing people do?

I've been thinking about doing a dashboard type app to display/manage servers across providers.

What tools are people using at the moment for this kind of thing?

(I'm actually running a very short survey on this at the moment. I'd love it if people would do it - happy to share the results here: <http://www.surveymonkey.com/s/LDNZFG3>)

<comment><USER9> It may be a common thing individual developers that play with everything do. I doubt it's common for businesses to deploy production apps across multiple clouds. Unfortunately I don't think that survey will be able to answer that either way.

<comment><USER8> It's interesting.

I've been running that survey for a little while, with a non-HN audience. More than 50% use more than one provider, and people managing more than 10 servers (who I presume are doing it for a business) don't seem any less likely.

(Thanks for all the answers BTW. I especially liked the OrionVM vs Ninefold comparison someone did in a comment - I'm from Australia, and for various reasons are reasonably familiar with the offering available. Alex from OrionVM is on HN, too)

</comment></comment></comment><comment><USER10> "Page Could Not Be Loaded

We're very sorry, but the page could not be loaded properly. This should be fixed very soon, and we apologize for any inconvenience.

Debug Info:

Status: 503 Response: Service Unavailable XID: 1889032339

AppFog"

Uhh, scaling issues?

</comment><comment><USER11> Just curious - How is this different than buying a server from Amazon, and sticking Ruby (or whatever it is that I need) on it and using it? Does the incremental value provided by Appfog lies in the fact it abstracts this get- meself-a-server-from-amazon part? Genuinely curious and hence asking.

<comment><USER5> A thousand words, or 30 seconds start to finish of deploying a Ruby app yourself on AppFog - <https://console.appfog.com/signup>

<comment><USER11> I registered as you suggested. I agree that getting productive in 10-15 minutes is great, but the setup time, at least for me, even if it takes a whole day, is a negligible overhead in my life as a programmer. Currently I run multiple servers on Amazon and as someone that programs in more than two languages, I drop an instance of whatever I am working on (Ruby, Java, Django as examples) and they really do not take much time at all. I am sure this post shows my age, but I am trying to understand the real value a service like Appfog provides.

Also may I ask how do I setup ssh trust and how do I access the command prompt.

<comment><USER7> AppFog handle things like software updates, scaling beyond one machine, DDoS attacks, all those kind of things.

You ask them to deploy a (for example) MongoDB service, they do it, and they are responsible for the operation of it.

The idea is to reduce the amount of day to day (or more likely month to month) work involved in doing things like security patches, upgrading infrastructure, etc, that suck up a lot of time, especially for people who don't have experience in building load balancers and so on.

<comment><USER12> That is true, but what appfog (and the other PaaS providers) do not give is the ability to tweak service[1] configs even to a limited degree. Not needing to worry about Mongo or MySql setup is great, but for all of my non trivial use cases, I have had to adjust configs at least a little.

[1] Cloud Foundry(and by implication AppFog) differentiate apps from services, where services are backend infra like databases.

</comment></comment></comment></comment></comment><comment><USER13> I think AppFog needs to figure out their branding, with regards to how AppFog and PHPfog are perceived.

Is AppFog the parent company and PHPfog is a product of that company? Are they two separate brands/companies?

We're using PHPfog for our MVP of matchist (matchist.com). Does the new AppFog free plan apply to PHPfog too? I sure hope so, otherwise we just got screwed by this.

I think they need to be a bit more clear about this.

<comment><USER14> AppFog is the parent company. PHP Fog and AppFog are the two PaaS products that AppFog the company make and sell.

</comment></comment><comment><USER15> Does this give me one VPS that I can resize up to 2GB without paying for?

Can I launch multiple VPSs that are up to 2GB?

If you are trying to say its not a VPS, I don't believe you. I bet it is at least built on some kind of virtualization. What is it exactly?

<comment><USER7> It's not in itself a VPS no, it's an "application environment", that's self contained and runs inside the Linux operating system. You can have multiple applications running over multiple instances, and Appfog handle all the deployment, scaling, and back-end management.

Those Linux instances are hosted on one of multiple providers (your choice), for example Amazon, Rackspace, HP Cloud, etc.

<comment><USER15> I didn't ask if it was in itself a VPS. You know what I asked.

Thank you for giving me a little bit of an idea.

So basically, AppFrog launches VPSs with their own images on Rackspace or AWS for you, which you don't access directly. So back to my question.

Is there a button I can press to say that I want another 'instance' (= VPS)? If I press that button twice, can I get two 2GB instances? Or is there no way to ask for another instance directly, instead I deploy another app, and it will always just resize the instance up to 32GB at which point then it would finally deploy a new VPS?

I understand that hand-waving and magic is the way that you sell a 'cloud service'. I need more details about the mechanics of using it and also specific information about the implementation in order to evaluate it, however.

I mean not to be a dick, I think the idea of starting with 2GB of RAM for free is freaking amazing and the right direction to go.

<comment><USER7> Yeah you can explicity launch multiple instances at your own control, each one uses the same amount of memory - so you could launch 10 instances with 200MB of RAM, or 1 instance with 2GB of RAM.

It's definitely worth signing up for, I've been using it for a couple of months and overall I've been impressed, though they still have too many glitches for me to be entirely happy deploying an important application to it.

</comment></comment></comment></comment><comment><USER16> I tried phpfog free for a while, I had very basic codeigniter setup and noticed the mysql queries where much slower than a regular shared hosting. I guess the free account works as a dashboard demo?, not sure.

</comment><comment><USER17> This is nice, but i hope FREE lasts....

<comment><USER18> They're going to be explaining the business model tomorrow on their blog: <https://twitter.com/appfog/status/228380656407826432>

<comment><USER14> Right here: [http://blog.appfog.com/if-paas-is-expensive-and-slow-why- not...](http://blog.appfog.com/if-paas-is-expensive-and-slow-why-not-use-a- vps/)

</comment></comment></comment><comment><USER19> This is going to change the PaaS game!

</comment><comment><USER20> They claim to support .NET. But after sign up and creating an app in "Step 1: Choose an application" i can not select any .NET "Application".

So their .NET Support claim is essentially a lie ?

<comment><USER9> It seems, from their past blog posts, that some advertised features are in "private beta" still.

</comment></comment><comment><USER21> AppFog wants to make developer platforms load slowly and be so disorganized and haphazard that you can't tell what's going on... great.

<comment><USER22> People need to spend some time understanding their architecture before just blindly moving into a PaaS provider. There are pitfalls and I bet we see some startups go the PaaS route then pivot later on.</comment></comment>


Breaking np-complete with memcomputing. Is it real? - <AUTHOR> http://meanderful.blogspot.com/2015/07/breaking-np-completeness-without-quantum.html

<comment><USER2> > If we solve NP-complete efficiently then we may break TSP and all sorts of > magical optimisation problems joyously drop out of the sky at our feet. It > is not clear if this would also break integer factorisation and thus RSA, > but perhaps it might.

Not true. Factorization is NP, so having a fast way of solving NP-complete problem would yield a way to solve factorization as well. On the contrary it is not known to be NP-complete (and is believed to not be), so solving factorization in polynomial time would not help with other NP problems.

<comment><AUTHOR> Thanks.

</comment></comment><comment><USER3> No, for the reason outlined by Aaronson in [0] - I don't see anything in the update which addresses this.

[0] [http://www.scottaaronson.com/blog/?p=2212](http://www.scottaaronson.com/blog/?p=2212)

<comment><AUTHOR> Interesting. I'm inclined to agree but I pause when I read that Scott Aaronson still seems to hold the line on quantum computing not doing anything too useful for NP-complete or a useful subset. The weight of opinion seems against him on quantum though he remains empirically correct for the foreseeable future.

<comment><USER4> "The weight of opinion seems against him on quantum though he remains empirically correct for the foreseeable future."

Then you're misjudging the weights "for" and "against" him by putting _way_ too much weight on people who don't know what they're talking about and way too little on those who do. It is likely that your "for" group are still operating under the idea that "quantum" works by "trying all possible answers then returning the correct one". This is empirically, mathematically-provably wrong. Anyone operating under this idea deserves for you to weight them at _zero_.

Despite the fact we still can't build a very big quantum computer, we actually do know quite a bit about what they can do and not do. And as Scott Aaronson points out _very_ frequently, if in fact they prove either able to do something our current theories say they can not or unable to do something that our current theories say they can, either way that will be very interesting, _precisely_ because it will imply that there is something wrong with quantum mechanics, which for all its "woo woo" reputation is one of the most solid math-to-reality theories we have ever had in the history of mankind.

Scott Aaronson isn't "holding" the line... he and his fellow-travelers are _drawing_ the line.

Edit: I'm also unsure on why you think Aaronson believes quantum "won't work"... he's on the optimist side that quantum computers can be made practical. If you mean that he doesn't think "quantum" can solve NP- completeness, well, of course he doesn't... he understands the _mathematical proof_ that it doesn't, so that's hardly surprising.

Edit edit: A positive followup to this negative message: Consider reading Quantum Computing Since Democritus [1], or if you don't want to spend the dough, read through the class notes that turned into that book [2].

[1]: [http://www.amazon.com/Quantum-Computing-since-Democritus- Aar...](http://www.amazon.com/Quantum-Computing-since-Democritus- Aaronson/dp/0521199565/ref=sr_1_1?ie=UTF8&qid=1436446995&sr=8-1&keywords=quantum+computing+since)

[2]: [http://www.scottaaronson.com/democritus/](http://www.scottaaronson.com/democritus/) , see "Lecture Notes" section.

<comment><USER5> >If you mean that he doesn't think "quantum" can solve NP-completeness, well, of course he doesn't... he understands the mathematical proof that it doesn't, so that's hardly surprising.

There's no proof BQP ≠ NP (as well as no proof it does).

Scott thinks it doesn't, but doesn't have a proof (if he did, he'd also have a proof of P≠NP (and showing something implies a resolution of P versus NP is a great proxy for "it's not easy and hasn't been solved yet")).

</comment></comment></comment></comment><comment><USER6> This article says nothing. It waffles vaguely about a number of unrelated topics, adding nothing of value to anything that's been said previously, and critically adding nothing of value to the paper it links to.

I suspect it is on the front page of Hacker News only because no-one else wants to appear ignorant of the topics in question by pointing out that it makes no sense.

<comment><AUTHOR> That's harsh but perhaps fair. My little blog post does represent a fair amount of ignorance on my part and I too wonder why it got to the HN front page.

I do appreciate the education I received :-)

</comment></comment><comment><USER7> No. Grab a copy of Ja Ja's book. There are some cool things you can do with a PRAM machine. [http://www.amazon.com/Introduction-Parallel-Algorithms- Josep...](http://www.amazon.com/Introduction-Parallel-Algorithms-Joseph- JaJa/dp/0201548569)</comment>


Our Superficial Scholars - <AUTHOR> http://www.washingtonpost.com/wp-dyn/content/article/2011/01/21/AR2011012104554.html

<comment><USER2> The article rings true to me. In college I met a lot of eager kids who worked hard and were good at... being students. They executed assignments diligently and knew how to get straight A's.

However, many of these kids lacked basic critical thinking skills. They didn't read the news or have much of an idea of what was happening in the world around them. They largely did what they were told and didn't have the guts to question professors.

I'll share an anecdote:

At the very first meeting of a statistics course, we were informed that our assigned professor was sick and a new professor would have to fill in. A kid sitting next to me said, "I already looked him up - he gives 60% A's. The last guy gave 65% A's." I asked the kid if the new professor was a good instructor. He gave me a dumbfounded look and repeated, "He gives 60% A's."

</comment><comment><USER3> I believe a different question should be asked.

Instead of: "Do universities fail to educate students on the underlying importance of the problems they are training to solve?"

We should question: "Is our system of education more than just a game?"

My thesis is basically thus: While the issue presented in the article reflects accurately on many students, the root is not grounded in university education, but in our system of education as a whole.

HNer adrianparsons notes (1) that many of his former classmates were excellent at being students, but their interest was not in their learning, but rather in passing the classes. Where, then, did this way of thinking originate? Not in universities, as the author suggests, but much earlier. Students are trained from a young age to participate in the game that is schooling; you are foisted into generic classes and made to take surface-level standardized tests (measuring nothing but the rote memorization seemly criticized by the author). Very rarely are you taught by a Professor in elementary, middle, or high school. Rather, "education professionals" prepare you for what is to come later. Just as often as not these teachers are uninterested in your learning (2), simply seeking a paycheck. Both teacher and pupil are playing the same game, one in which neither benefits from going above and beyond. Is it any small wonder that students enter college as excellent students, but poor learned citizens?

The article is far too narrow; The problem, much more broad. It's not the universities failing students, but the institutions that placed them there. We're not going to college intellectually and socially curious, only to have that squashed out of us; We're going to college expecting more of the same.

(1) <http://news.ycombinator.com/item?id=2133712>

(2) This is not to say that great primary education teachers do not exist. I was lucky to have been influenced by a number of excellent teachers at a young age, though in hindsight I find that they were far and few between.

*Disclaimer: I am a first-year student at American University, and clearly hold a grudge to a system that, I believe, has wasted so much of my time.

</comment><comment><USER4> When I was on staff at the US Naval academy, I had the interesting experience of interacting with the bottom 2% and top 2%. There's a line in the US Code that says we have to identify midshipmen who lack the aptitude for commissioned service. Like porn, it's hard to define, but you know it when you see it.

So the job also was in charge of identifying the best of the best and we had all sorts of metrics to assess the whole spectrum of midshipmen.

Anyway, so I got to work with a lot of the Rhodes, Fitzgerald, etc scholars. And I was doing my med school pre-reqs, so I sat next to several of them in class. During the summers they'd intern for me. I've had a bizare number of academic elite as my direct reports. Then, in med school, I interviewed applicants who had made it to the interview process, who are from the same end of the gene pool.

Before all of this exposure, interviewing the best of the best, I had been a division officer on a Navy frigate, and then an air craft carrier. Those jobs had some real leadership responsibility. Literally, "go this way, not that way", "do this, not that", "Yes, I said don't shoot" leadership. So, that added a different flavor to the experience of interviewing these kids. Frankly, my reaction was about like the author's: dismay. Despair. Disbelief.

Conversely, I knew a few of the Rhodes kids when I was a midshipman too. We weren't surprised when they got the nod.

Now, what I was surprised about was how they got the applications. At least at Canoe U. They'd set up a booth. In the Poli Sci hall. Which was further from the dorm than all the science and engineering halls. Basically, if you weren't a Poli Sci major (and there were plenty), some one had to, still has to, advise you to stop by.

I thought that was an interesting twist in the dynamic.

</comment><comment><USER5> I think what the author is looking for and what most top tier students are shooting for are two different things. She's looking for someone who is good at, what my uncle called, dinner conversation. Any dinner topic my uncle could talk for hours on. He had a reasonably well thought out position on anything one was likely to ask at the dinner table.

But most people who excel in college are exceling at going deep on something. Being the world's foremost expert at something. Dinner conversation ability is probably largely unrelated to this. In fact it might be inversely related as I could imagine that many who want to go deep, view the dinner conversation as pointless -- people's minds are made up before they come to the dinner table, and don't change.

If they're really looking for people who can answer those questions, don't look for traditional scholars, look at debaters.

<comment><USER6> "But most people who excel in college are exceling at going deep on something. "

I disagree. Undergraduate education is not about becoming the "world's expert at something". That's for grad school and a doctorate. Undergraduate education should be well-rounded with a broad exposure to many different areas. Then, if the student chooses s/he can specialize and become the "world's expert" at something they came across as an undergrad.

<comment><USER5> For most undergrads, you're right (and note, I'm not saying that this is how undergrad should or shouldn't be). But the ones that are typically celebrated are those that are doing graduate/professor level research (or public service) as undergrads, and exceling. It's the Barton Reids and Wendy Kopps of the world. Those are the people that college professors heap praise on and recommend for prestigious fellowships.

Someone who is all-around good, but doesn't go deep, is likely to go unrecognized at the Rhodes-level. You must excel at something for your department or professor to take notice.

There's an old saying about Harvard... Harvard doesn't want well-rounded students. It wants a well-rounded student body. That is it doesn't want a bunch of good students. It wants the best math student, the best physics student, the best chess player, the best violinist, the programming prodigy, etc...

My point is that we've created a system where this is the metric. And when this is, you can't be surprised that when you try to take the cream of the crop, using this metric you may not get the results you expect.

Like I said, if she picked members of the top debate teams I think she'd be very _impressed_ that they have answers for all those questions.

</comment></comment></comment><comment><USER7> Oddly enough I've always thought that one of the big problems with the US university system is that it's _too_ broad.

When I went to university in Australia I studied four subjects in my first year (Physics, Chemistry, Maths, Computer Science), two subjects (Physics and Chemistry) in my second year, two subjects in my third year, and only physics in my fourth year. On the other hand, I get the impression that the average American university student wastes at least a third of his time doing random- ass subjects with no real relevance to his major, right?

In my opinion, "broad education" is what you should be getting in high school.

<comment><USER8> I disagree. My undergraduate degree was in biology, but I earned it from a liberal arts college. What did that mean? It meant that I took an awful lot of biology classes, but I was also forced[1] to take a lot of classes outside of the science building. The college's degree requirements specified that you had to take a certain number of credits from such-and-such department, a certain number from so-and-so, and so on, no matter what your major happened to be. As a science major, it was sometimes very challenging to fit all the requirements in, but I'm really glad I did. Having a broad base of knowledge outside one's speciality is extremely liberating personally, and it can be professionally empowering, as well.

It's easy, as a scientist, to lose perspective about what we do- both in terms of "why are we doing this at all" to "why do we do this in this specific way." A good liberal arts education helps you a) recognize situations in which those sorts of questions are relevant, and b) can help you develop a useful set intellectual tools for answering those questions. Also, a lot of the biology I learned is already obsolete, ten years later; while learning it was certainly worthwhile, it is of relatively little practical value to me today (and not just because I didn't end up going into biology).

Also, having to take some humanities classes helped me become a better writer, and also helped me learn how to effectively discuss my work and why it is important. Those have proved to be _extremely_ useful skills in life, both in academia as well as in the "real world".

Now, I would also argue that, had I been a humanities major of some sort, taking some hard-core science classes would have been quite beneficial- it would have taught me about a different way to engage with and think about reality, which can only be beneficial to one's ability to think about literature, or music, or whatever.

Oh, BTW- in addition to biology classes, I took a fair number of CS classes (almost enough for a minor), and I feel like my programming abilities over the years have definitely benefited from a liberal arts background. Furthermore, when I was doing hiring at my pre-grad-school job, I found that the best candidates weren't necessarily the ones with impeccable CS credentials. Often, the ones that worked out best in the end were often the ones that had broader academic horizons. They tended to have an easier time dealing with ambiguous situations and instructions, and were better able to "dive in" and "JFDI" when confronted with new and unfamiliar codebases, technologies, or projects. That's not to say that CS classes weren't important, or anything like that- just that there's a synergistic effect between good CS and the humanities.

[1]: "Forced" isn't quite the right word, here, since getting a broad education was the whole reason I chose a liberal arts school. "Required," maybe?

<comment><USER7> I dunno. Personally I found that I could never enjoy things like literature and history until they _stopped_ being educational requirements.

</comment></comment></comment><comment><USER9> If you want to see more students who are broadly informed, create an incentive. Don't expect people to shell out tens of thousands of dollars for a broad university education that will land them a mediocre, 9-to-5 $10 an hour job.

</comment><comment><USER10> My experience of seeing people selected for the Rhodes scholarship in Australia was that there were two major criteria for selection:

1\. You attend Sydney University.

2\. You live in a residential college at Sydney University.

Folks outside those circles found it much harder to be selected. To be sure, those who were selected were exceptional scholars, sportspersons and active in University life. But such exceptional people exist all over Australia, not just in one subset of one prestigious, well-connected university.

<comment><USER11> There is, perhaps, a larger question...There appears to be a large mismatch between public perception of the weight of certain academic badges, and the actual substance of the badge.

In your example, the Rhodes selection committee selects only people from a particular elite slice of Australian society, yet it may have nothing at all to do with their actual intellectual qualifications, "such exceptional people exist all over Australia".

Yet the public perception is that a Rhodes scholar is somebody who is not only extremely bright, but the top of the tippy top of bright people. They are the medium from which light itself emanates.

You can find this kind of mismatch elsewhere when it comes to similar comparisons of actual intelligence, vs. perceived intelligence a particular badge bestows on a person. "I graduated from Harvard" "I went to Oxford", "The best school is Seoul University" etc. Appears to have only slightly more substance to a person's intelligence and worth than the brand of socks they are wearing. Yet the public will grant significant weight to these things.

Articles like this work to slowly peck away at this often faulty public perception. It's not that Rhodes Scholars, or graduates of top schools aren't smart -- just that similarly smart people exist elsewhere. It's only a matter of time before "Rhodes Scholar" means only slightly more than "Dean's List" in terms of perception.

I'm afraid more and more that badges that _should_ signify significant intellectual achievement are becoming more and more worthless. In order to improve the value of these badges, it behooves the selection committees to cast a wider net and find people from many other places who fit their requirements...doing the hard word of actually finding people than the lazy work of fishing in only well known ponds.

<comment><USER10> Here's where I contradict myself.

Rhodes Scholarships are valuable because they are very rare. According to Wikipedia, the USA got 32 places in 2006, Australia got 9.

But given that in a given year, tens of thousands of Australian students could apply in the first instance (having good scores, plays a sport, involved in some other activity). At a statistical level, the distinction between the top 100, the top 50, and the top 10, is going to be nigh on invisible.

So in a sense it is just as reasonable to select students from the University of Sydney (prestigious) as from the University of Western Sydney (considered to be a dump).

It's just that Sydney University happens to be wired into the Sydney establishment, especially through its residential colleges. And the Sydney establishment is the dominant fraction of the Australian establishment. And the Australian establishment provides the selection committee for the Rhodes scholarships.

Being at Sydney means that you have an opportunity to meet some of the selectors, and to get to know them, in advance. That's more or less how our Rhodes Scholar opposition leader, Tony Abbott, got a leg up.

<comment><USER7> _So in a sense it is just as reasonable to select students from the University of Sydney (prestigious) as from the University of Western Sydney (considered to be a dump)._

But there's also self-selection factors.

1\. The best students are far more likely to go to Sydney than UWS, because as you mentioned, UWS is a bit of a dump. Of course in fairness UNSW is pretty good, but...

2\. The sort of people who are interested in Rhodes Scholarships are exactly the sort of people who are likely to choose Sydney over UNSW. Why? Because Rhodes Scholars are the sort of people who think Oxford is the awesomest place on Earth, and Sydney is basically Oxford Lite.

(Declaration of bias: I went to Sydney.)

<comment><USER10> Sure, you might select Sydney over UWS. But students from other Group of 8 universities are pretty bright self-selectors too. Where are the equal number of Rhodes scholars from the Universities of Queensland, Melbourne, Western Australia, New South Wales and Adelaide, the ANU, and Monash?

(I went to Sydney too, for 2 years, before depression wrecked my life. Spent 1 year at St Andrews college.)

<comment><USER7> Melbourne I'd expect, the others not so much due to the self-selection factors I mentioned ("If you like Oxford's original 11th century gothic architecture you'll love our 19th century copies of Oxford's 11th century gothic architecture!). If there's significantly more from Sydney than Melbourne then, yes, I'd say something is screwy.

<comment><USER10> I got curious and decided to test my assertion.

Taking this list as an input: [http://en.wikipedia.org/wiki/Category:Australian_Rhodes_scho...](http://en.wikipedia.org/wiki/Category:Australian_Rhodes_scholars)

70 entries in total. I excluded 4 entries where the original university is unknown, and 3 where the person did not attend university in Australia, leaving a sample of 63 scholars.

Of the remainders:

Sydney: 13

Melbourne: 17

Adelaide: 15

Queensland: 5

UWA: 8

Tasmania: 5

While this is not a representative sample (it depends on Wikipedian "notability"), it may suggest that I am full of shite.

<comment><USER7> Well congrats, it takes a special kind of effort to go to that much trouble to disprove yourself.

I'm interested in the complete lack of UNSW and Monash. I think it supports my idea that if you're the kind of person who really wants a Rhodes Scholarship so you can go to Oxford then you're the kind of person who'll choose the ivy- covered alternative over the just-as-good 20th century one.

<comment><USER10> Well it's more embarrassing when someone does it to you. And I have a suitably fragile and inflated ego, which is why I picked Sydney out of the offers I received way back in the day.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> Hmm, that's kind of strange. There's so many important socio-political dilemmas for young people to grapple with these days, why would the author fault them for not being ambivalent toward nukes and armed conflict? Shouldn't young people be forgiven for holding the unequivocal yet naive belief that we shouldn't kill each other, or even threaten to? I mean, if they're equivocating on nukes when they're a fresh-faced 22, what will they be equivocating on when they're a grizzled 55 and at the reins of some real power?

 _get's to bottom of article, reads that author represented NM in the house...back up to the byline..._

Fucking Heather Wilson. Should have known...

<comment><USER13> Here's what the article said: _A student who started a chapter of Global Zero at his university hasn't really thought about whether a world in which great powers have divested themselves of nuclear weapons would be more stable or less so, or whether nuclear deterrence can ever be moral._

Since when does thought equal equivocation? Thoughtless action is irresponsible no matter what motivates it. The author also faulted a student who signed up to kill for the government without wondering whether there was anything worth killing for.

The only thing I disagree with in the article is the one-sided concentration on the students who are just eager to do what they're told. What about the ones who are more thoughtful? Who are they, where do they come from, and why are they different from their Ritalin-addled robot achiever peers?

<comment><USER14> She offers no evidence that the student hasn't thought long and hard about nuclear weapons. I wonder if she reached that conclusion because he disagrees with her. As if saying "If he supports Global Zero then he couldn't have considered these things."

<comment><USER7> I assume this is based on a real student whom she interviewed, and that he was unable to satisfactorily justify his belief system when challenged on it.

This kind of thing doesn't surprise me -- the students with the most strongly held political opinions are often those who have thought about the issues the least.

<comment><USER10> That's because when we talk about "strongly held beliefs" we're talking about an _emotional_ and not _rational_ attachment to the idea. Emotionally-held ideas are not examined as they are tied up in self-image.</comment></comment></comment></comment></comment>


"Pink is the first ICO to offer an equity position with ROI based on revenues" - <AUTHOR> https://pinkapp.io/ <comment><USER2> Oh yes... this lunatic, the cyberphunk/cyberpimp who decided that since they're already breaking the law, why not offer an unregistered security.

Prison.</comment>


A Collection of Covid-19 Company Policies - <AUTHOR> https://github.com/kuiro5/covid-19-company-policies <comment><USER2> I was looking forward... maybe qualify: USA based, Hi-Tech, etc...</comment>


UX Weekly, Take 2 - <AUTHOR> http://uxwkly.com/

<comment><AUTHOR> Here's some of my reasoning for the newsletter, and what's going to happen in the future:

[http://piranhabay.co.uk/post/65912038650/ux-weekly- take-2](http://piranhabay.co.uk/post/65912038650/ux-weekly-take-2)</comment>


Reasons to Use Inspiring Tools and Brands - <AUTHOR> http://particletree.com/notebook/reasons-to-use-inspiring-tools-and-brands/ <comment><USER2> I have always found that the aesthetic and inherent 'rightness' of mechanical tools makes a difference in how efficiently I am able to work with them. Working with Snap-On or MAC ratchet sets just feels _better_ than using the equivalent taiwanese tools. I had not realized until just now that my feelings about developing using a Macintosh were probably related to the same sense of wonder/awe/respect :-)</comment>


CMS Pipelines - <AUTHOR> https://en.wikipedia.org/wiki/CMS_Pipelines <comment><AUTHOR> 1\. Unix pipes are 1 dimensional

grep "^#" dat1 | foobar > dat2

whereas the key thing about this idea is that real pipes can be multidimensional

grep "^#" dat1 | foobar

| |

| | snafu | stitch > dat2

2\. This system treats ">" as a program in itself and verifies a "program descriptor" before it trashes the next one in the pipe

3\. Unix pipes give you the error code of the first stopped component in the pipeline whereas in this model you can hook up instrumentation/statistics to any stage of the pipe

<comment><USER3> unix pipes are multidimensional via fifo. I have done like follows when I want a poor mans "GROUP BY"

mkfifo p1 cat data | tee p1 | awk '{print $5}' | sort | uniq -c | join -1 5 -2 1 -o 1.1,2,2.1,1.2,2.2,2.3 - <USER2>

</comment></comment><comment><USER3> I have dealt with cms pipelines. It is a lot like everything ibm vs unix, over-specified, verbose and awkward. The unix pipeline is designed around making the normal case easy and straight forward. Remember unix is a game.

</comment><comment><USER4> On Debian, you can use pipexec to create and manage a directed graph of processes and pipes.

[https://github.com/flonatel/pipexec](https://github.com/flonatel/pipexec)

</comment><comment><USER5> TIL Rexx wasn't something OS/2 specific :)

<comment><USER6> My only exposure to Rexx was when it (ARexx) started shipping with AmigaOS, as the preferred scripting language. It was very nice for the time, and I still kinda pine for a truly universal scripting language that has that much interop with the OS and a large number of applications for the system. It kinda provided a means to use desktop GUI apps in a manner similar to CLI programs with pipes.

</comment></comment><comment><USER7> We took "Far better than Unix Pipes with multistream+multilevel dispatch" out of the title. That's editorializing, which is not allowed in HN titles.

If you want to say what you think is important about an article, you're welcome to do so by posting a comment about that to the thread. Then your opinion is on the same level as everyone else's.

<comment><AUTHOR> Got it. Thank You. I didnt know about this policy but basically trying to answer the question as to why one should read it/look at it. I was trying to stave off the question that this is for 'mainframe' computing and that it is dated etc

<comment><USER7> By all means please make the case in the thread about why this is interesting. And thanks for posting so many high-quality stories to Hacker News! Historical material, like this one, is particularly welcome.</comment></comment></comment>


WePay Drops Prices, Rolls Out White Label Payments API - <AUTHOR> http://techcrunch.com/2012/08/14/paypal-competitor-wepay-adds-white-label-api-drops-prices

<comment><USER2> That is great that they listened to customer feedback. I started using other providers because of handling everything through the iFrames and callbacks. They have a lot to offer, especially with the ability to create customer accounts. It is worth checking out.

</comment><comment><USER3> how does this compare to stripe?

<comment><AUTHOR> * Full support for platform use case including the easiest possible merchant onboarding and ability to charge "app fee" for services.

* Support for all experience levels from hosted pages to payment buttons to iFrame solution (full PCI compliance) to Credit Card Tokenization (PCI compliance required).

* Simpler Credit Card Tokenization API that allows you to immediately charge the user and store the credit card for later use.

<comment><USER4> _Full support for platform use case including the easiest possible merchant onboarding and ability to charge "app fee" for services._

IMO, this is the most interesting and differentiated part of the service as compared to, say, Stripe (which I use and love).

Who else offers turnkey support for building your own marketplace, not just taking payments from customers?

<comment><USER5> Paypal, Amazon and WePay are the only solutions I've found for marketplace- style payments.

<comment><USER6> Is Paypal considered a marketplace-style payment system?

Two other newcomers are BalancedPayments.com (Gittip.com which has been getting some attention on HN is using them now) and BancBox.com . I have not used either but they are on a shortlist for a project I'm working on.

</comment></comment></comment></comment></comment><comment><USER7> It seems like WePay is still trying to figure out its place in the industry. It's an inferior offering to PayPal and was rapidly passed over by Stripe.

</comment><comment><USER8> Sad that you have to be a USA resident :/

</comment><comment><USER9> Is it possible to accept wepay for merchants from another countries, e.g Russia?

<comment><USER9> Seems like it is not the case: > You must be a US resident to accept payments.

</comment></comment><comment><USER10> Isn't there some link between techcrunch, y-combo, and HN?

<comment><USER7> Ycombinator operates HN (as should be apparent by the URL). Neither are related to TechCrunch beyond all having an interest in the tech industry.

Downvote.

<comment><USER10> Thank you for the answer, and the downvote.

After you replied I remembered that TC is Michael Arington, arch-nemesis of Leo LaPorte.</comment></comment></comment>


Idea: Better filtering for HN/social news - <AUTHOR>

I think I should be able to view HN/reddit-like sites with a mode that hides all stories from me until the second derivative of the number of comments gets close to 0.
I don't necessarily want to see the newest or the best, I just want to see stories once they've matured/developed a little bit without losing them.
I often find myself checking back on stories after a while to see new comments, but not necessarily on popular stories, and it's not really something I'd want to bookmark.
news.ycombinator.org/best would be cool otherwise.
Noprocrast is good, but I still end up viewing the same thing repeatedly. Maybe I should just check the site once in the evenings or something.
What do you guys think? <comment><USER2> Noprocrast?

<comment><USER3> GIYF, "hacker news noprocrast"

<comment><USER2> Ah, thanks. Everyone noobs out at times. For some reason, because he said '/best', I assumed it was '/noprocrast', but that 404d.</comment></comment></comment>


Make Your Site Feel Faster by Changing Colors - <AUTHOR> http://blogs.adobe.com/creativecloud/make-your-site-feel-faster-by-changing-colors/

<comment><USER2> what could be more jarring the yc's color scheme and yet i never sense, much less experience, a lag nor do i quit the site in frustration cuz i'm waiting too long for it to load. just the opposite in fact. there is so much good stuff here that i can always count on having my paradigms shifted. i can only imagine how much more time i would be spending here if they were decked in soothing shades of blue.

i rather agree with radware, in that, "[a] combination of page bloat, missed opportunities, and site owners struggling with web performance best practices" might have more to do with manipulating user perceptions.

but hey, color me skeptic.</comment>


Walmart reportedly won't bid for Whole Foods after Amazon's huge offer - <AUTHOR> https://techcrunch.com/2017/06/23/walmart-reportedly-wont-bid-for-whole-foods-after-amazons-huge-offer/ <comment><USER2> It should, as a defensive and existential move. Walmart is having a hard time competing now so they need to slow Amazon down. If they don't Amazon will have a strong real world presence which they will use to over power Walmart.</comment>