An ant colony has memories its individual members don't have (2019) - <AUTHOR> https://aeon.co/ideas/an-ant-colony-has-memories-that-its-individual-members-dont-have <comment><USER2> We are all colonies of creatures, really interesting book written 100 years ago called the "Soul of the Ant" by [https://en.wikipedia.org/wiki/Eug%C3%A8ne_Marais](https://en.wikipedia.org/wiki/Eug%C3%A8ne_Marais)

</comment><comment><USER3> Working in Industrial Automation has given me alot of respect to ants. While I was interning at Tesla a few years back, it amazed me how 'no one person' understood the massive operations of building the model 3, but together (along with our 1000's of suppliers) we were able to make extremely advanced technology.

Essentially, humans are just a more advanced version of ants. No one understands the vast amount of knowledge we've gathered, but this knowledge has allowed us to be able to sustain our vastly growing population numbers. Without this 'specialization of knowledge' or given some apocalyptic scenario, our ability to sustain our numbers would drastically decrease.

</comment><comment><USER4> > _Foraging in a harvester ant colony requires some individual ant memory. The > ants search for scattered seeds and do not use pheromone signals [...]_

This was surprising to me. Until now, I was under the impression that all ants used pheromones, which leads to coordination not with each other but through the environment [1].

[1] [https://en.wikipedia.org/wiki/Stigmergy](https://en.wikipedia.org/wiki/Stigmergy)

</comment><comment><USER5> More impressive is the slime molds, since they are not even animated as ants

[https://www.scientificamerican.com/article/brainless- slime-m...](https://www.scientificamerican.com/article/brainless-slime-molds/)

</comment><comment><USER6> This reminds me a lot of multi agent systems in computer science. A very exciting concept which is aiming to provide a framework for distributed artificial intelligence:

[https://www.sciencedirect.com/topics/chemical- engineering/mu...](https://www.sciencedirect.com/topics/chemical- engineering/multi-agent-systems)

</comment><comment><USER7> Kurzgesagt has a couple of amazing short videos on ants: [https://www.youtube.com/watch?v=7_e0CA_nhaE](https://www.youtube.com/watch?v=7_e0CA_nhaE)

</comment><comment><USER8> « Colonies live for 20-30 years, the lifetime of the single queen who produces all the ants, but individual ants live at most a year. »

Individual ants... except for the queen. How likely is it that the _queen_ is acting as the memory of the colony?

(I don't think this is actually the answer—I agree that it's more likely that the memories are being held in the collective—but it has to be ruled out somehow.)

</comment><comment><USER9> > It searches until it finds a seed, then goes back to the trail, maybe using > the angle of the sunlight as a guide, to return to the nest, following the > stream of outgoing foragers.

I remember reading that ants counted steps to find their way home. Perhaps I'm remembering incorrectly or maybe it was false?

Either way, cool article. Emergence is a cool property that shows up everywhere!

</comment><comment> awinter-py see also this 2014 article on consciousness in 'rather dumb group entities' including 'antheads'

[https://faculty.ucr.edu/~eschwitz/SchwitzPapers/USAconscious...](https://faculty.ucr.edu/~eschwitz/SchwitzPapers/USAconscious-140721.pdf)

</comment><comment><USER10> Another nail in the coffin of the Chinese Room. I hope we can finally bury that misconceived argument soon.

</comment><comment><USER11> Eugène N. Marais - The Soul of the White Ant (1937) [http://journeytoforever.org/farm_library/Marais1/whiteantToC...](http://journeytoforever.org/farm_library/Marais1/whiteantToC.html)

</comment><comment><USER12> For a few years now I am wondering if ant colonies do something like portfolio optimizing, as in how many ants to send to which food location, depending on risk and reward. I'd guess they do. But I haven't figured out a way to prove/show that yet.

</comment><comment><USER13> I wonder if individual members ever make decisions against the ant colony. For example if the ant colony as a whole discriminates a certain group of ants, will those ants simply obey? Basically are there rebel ants?

<comment><USER14> An interesting aspect to me is that a collective intelligence can easily create a diffusion of responsibility. It's a lot easier to kill or give into base impulses when the blame is shared by the whole. Bees and ants don't tolerate nonconformity, they'll kill a queen if she's not filling her role.

I didn't enjoy To Kill a Mockingbird, but I think it's good for kids to read it.

</comment></comment><comment><USER15> Comparing this to ant colony algorithms and considering ant colony as a network of actors; Can we analytically measure how much more information the network has compared to the aggregation of the individual entities in the network?

</comment><comment><USER16> Just like your individual neurons dont understand Chinese :)

</comment><comment><USER17> I feel like this is tangentially related: [https://wiki.c2.com/?TheFiveMonkeys](https://wiki.c2.com/?TheFiveMonkeys)

</comment><comment><USER18> The novel "Children of time" by Adrian Tchaikovsky imho greatly picks up this notion among others, highly recommended read

</comment><comment><USER19> probably the same analogies can be drawn for human society

</comment><comment><USER20> But are ants Turing complete ?

<comment><USER21> A simplified model of ants are Turing complete (the colony, not the ants) [https://link.springer.com/chapter/10.1007/3-540-59496-5_343](https://link.springer.com/chapter/10.1007/3-540-59496-5_343)

</comment></comment><comment><USER22> The title is extremely sensationalistic, Occam's razor points to undetectable(by humans) scent trails that lead the ants to the same harvesting locations every year.

There are no revelations about memory or collective consciousness to be found in this article. Every Occam in the world would infer human-undetectable scent trails from the evidence presented here, not some cosmic revelation about "how memory works" like the title and first paragraphs of the article heavily imply.

@pg @dang this title and article in general is outrageously misleading.

<comment><USER23> This reminds me of some conversations I've been having with a friend of mine who is skeptical of "emergence" (or at least the way it is often described). After going over the problem with him for a while I eventually was convinced that emergence is not "more than" the sum of the parts, emergence is _precisely_ the sum of its parts.

Ants produce these higher level patterns not because there's some magical thing that "emerges", but because they are precisely evolved to coordinate with each other to create those patterns.

</comment></comment><comment><USER24> the internet has paths that its individual routers don't understand ;-)

</comment><comment><USER25> For a sci-fi take on this, Children of Time and Children of Ruin, Adrian Tchaikovsky

<comment><USER26> Just to elaborate (mild spoilers):

At some point in the _Children of Time_ book, ant colonies become domesticated and are cultivated into general computation "devices".

I definitely recommend the first book, but have yet to finish the second one.

<comment><USER27> Second one has a different vibe, but I also enjoyed it. The author has a great way of helping you understand the different senses and intelligence of the different species.

</comment></comment></comment><comment><USER28> wikipedia.org/wiki/Ant_colony_optimization_algorithms

</comment><comment><USER29> I recommend looking up Donald Hoffman's theory of conscious agents.

He proposes that all sentient beings are networks of conscious agents, the simplest conscious agent being binary (it has a world, and can only act in two ways).

Any composition of conscious agents is itself a conscious agent.

Interestingly, "the world" of each conscious agent might be only other conscious agents.

<comment><USER30> This notion of the 'self' is "standard" in Dharmic traditions, and has been for over two millennia.

Then again it remains very fashionable to steal ancient ideas from East and market them as "brand new" genius inventions of Westerners. The amount of uncited plagiarism that occurs in this manner is quite simply astonishing.

<comment><USER31> The tone and the emotional message of your post though is not at all in the spirit of Dharmic traditions.

What is "stealing" in a world where everything is one and the conscious universe is trying to understand itself in the best way? Why would it assign negative connotation to copying information, if that information is in fact truthful!?

</comment></comment></comment><comment><USER32> Doesn't Hofstadter have a rap about ant colony consciousness?

</comment><comment><USER33> read GEB

</comment><comment><USER34> Reminds me of the "Tradition is smarter than you are" article that was posted here a few days ago.

[http://scholars-stage.blogspot.com/2018/08/tradition-is- smar...](http://scholars-stage.blogspot.com/2018/08/tradition-is-smarter-than- you-are.html)

<comment><USER35> Both this article and the one you linked reminded me also of what Nassim Nicholas Taleb described in "The Most Intolerant Wins: The Dictatorship of the Small Minority" [0].

 _> The main idea behind complex systems is that the ensemble behaves in way not predicted by the components. The interactions matter more than the nature of the units. Studying individual ants will never (one can safely say never for most such situations), never give us an idea on how the ant colony operates. For that, one needs to understand an ant colony as an ant colony, no less, no more, not a collection of ants. This is called an "emergent" property of the whole, by which parts and whole differ because what matters is the interactions between such parts. And interactions can obey very simple rules. The rule we discuss in this chapter is the minority rule._

 _> The best example I know that gives insights into the functioning of a complex system is with the following situation. It suffices for an intransigent minority –a certain type of intransigent minorities –to reach a minutely small level, say three or four percent of the total population, for the entire population to have to submit to their preferences. Further, an optical illusion comes with the dominance of the minority: a naive observer would be under the impression that the choices and preferences are those of the majority. If it seems absurd, it is because our scientific intuitions aren't calibrated for that (fughedabout scientific and academic intuitions and snap judgments; they don't work and your standard intellectualization fails with complex systems, though not your grandmothers' wisdom)._

Of course in this context the grandmothers' wisdom is tradition in some way, as this is passed down by generations. Same as many practices in religion, some of which that might have been useful at some time (like not eating pig meat, because one would get sick quicker as pigs might eat anything they would find).

I live in Thailand and my girlfriend is Buddhist. Often I just go with the flow with regards to Buddhist practices, even as a non-believer, cause there might be some real use for these practices that I don't understand as a non- believer. At the very least it will make the Thai people in our village accept me more whenever they see me doing the same actions as my girlfriend at our local temple (burn incense, "pray" to some statue, etc...).

\---

[0]: [https://medium.com/incerto/the-most-intolerant-wins-the- dict...](https://medium.com/incerto/the-most-intolerant-wins-the-dictatorship- of-the-small-minority-3f1f83ce4e15)

<comment><USER36> >The interactions matter more than the nature of the units. Studying individual ants will never (one can safely say never for most such situations), never give us an idea on how the ant colony operates. For that, one needs to understand an ant colony as an ant colony, no less, no more, not a collection of ants. This is called an "emergent" property of the whole, by which parts and whole differ because what matters is the interactions between such parts. And interactions can obey very simple rules.

I wish Taleb would give more on identifying & learning on a systems wide approach. For abstractions and "less than obvious" spheres this becomes difficult to separate the forest from the tree, or is the forest the system, or the genera of the plant in question, or its bordering systems, etc... behaviors and patterns which are emergent only at the individual level make '10,000 foot views' harder to perceive, let alone examine and extrapolate from "obey very simple rules"

<comment><USER37> I really appreciate Taleb's ideas in general, but this one strikes me as emotionally-driven, vastly more "intuitive" than substantiated.

It's a like a world chess master or NBA player telling others "play _better! "_ — what Taleb means here, imho, is that too many scientists fail to propose models that fits his mathematical perception, his world view, but if it were that simple, he'd have a book called "system thinking".

He touches a lot on how he views things, so you can infer a lot of his mental framework from reading e.g. the Black Swan or Antifragile — both great in their own respect. But simple rules on this topic, that would/will be groundbreaking.

I honestly pride myself as a "transdisciplinary" mind (which comes with a lot of "imposter-of-all-trades" syndrom, but meh, it's also humbling to realize the path to knowledge may not be the most rewarding short-term path). Taleb is one of those relatively "wide" minds, he's able to speak with substance on a lot of domains, but like many abstract thinkers I think he displays a lot of the casualness towards the difficulty of actual implementation.

It's great to talk about systems but the reality is often about refactoring horrible codebases and if it works you'd rather spend more money on the actual mission that making things and concepts prettier. Even, especially at the edge.

My 2 cts obviously. TL;DR, I wouldn't look much into it. It's one of those things we only hear because who says it is famous, not because there's so much velocity to the idea.

</comment></comment></comment></comment><comment><USER38> I think something similar has been suggested in the book called Godel,Escher,Bach.

<comment><USER39> Was thinking the same thing, that is the character 'Aunt Hillary', which is an ant hill. In the Dutch translation (which I read years ago) she's called 'Myra Hoop'. Funny how those translation still make sense, same as in the Harry Potter universe where the names even have to be anagrams.

<comment><USER40> It's not an accident: Hofstadter and the translators put a lot of work into preserving the wordplay across translations. [https://en.m.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach#Tra...](https://en.m.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach#Translation)

</comment></comment></comment><comment><USER41> I wonder if this could be used as an argument for Panpsychism?

<comment><USER42> I'm trying to figure out a way to incorporate this notion of hierarchical life into panpsychism and the "global brain" hypothesis. (Disclaimer: I'm already sold on and a proponent of the theory of panpsychism.)

It seems more and more to me like this hierarchy goes all the way up (to the single superorganism that is the universe) and all the way down (to the presence or absence of fermions in particular states, inducing a duality and thus a basis of computation via the Pauli exclusion principle). If this hierarchy is consistent across all scales, then we can conclude that if consciousness exists at one level then it exists at all levels. Sentience/awareness is a different question, mind you, and "memories" are associated with awareness of past events.

I'm also starting to believe that "consciousness" in terms of directed will doesn't truly exist, and that only "experience" exists. The rest (wants, desires, opinions, will) are electrochemical reactions which respond to local changes in the environment, although we experience them as much more than that for ultimately self- (and macrosystem-)serving reasons. These electrochemical reactions are present because they have over time become more important in the processes necessary for the propagation of whatever they're supporting. This is all very vague and hand-wavy but this article on the thermodynamic theory of life might be clearer [1].

In the discussion yesterday about this topic on HN I brought up the example of ant colonies [2] in an attempt to spur discussion in this direction.

[1] [https://www.quantamagazine.org/a-new-thermodynamics- theory-o...](https://www.quantamagazine.org/a-new-thermodynamics-theory-of- the-origin-of-life-20140122)

[2] [https://news.ycombinator.com/item?id=22047653](https://news.ycombinator.com/item?id=22047653)

<comment><USER41> Oh wow, thanks for the great reply. I didn't think anyone would respond. I'm going to dig into the links.

Do you have any other suggested reading?

<comment><USER42> I'd suggest, if you're already receptive to these concepts, to delve into Buddhist/Zen Buddhist/Taoist literature in addition to the more cerebral (pun?!) stuff out there. They're basically saying the same thing, albeit with much different language and framing. In particular the notions of interdependence, sunyata (my personal favorite idea/concept ever), and duality(ies).

Beyond that, unfortunately most of my exposure to the ideas related to panpsychism come in fits and bursts, and usually pieces which aren't about panpsychism inspire my ponderance more. Subjects include: animal consciousness/experience; the apparent intelligence of complex systems, whether man-made or independently-arising; autonomic, pre-conscious behavior in humans; computational theory, especially in physical systems; emergence and complexity writ large; complex adaptive systems in general.

Unfortunately I haven't done a lot of seeking out books on this topic. Nautilus and Aeon magazines (the latter is linked in OP) have thought- provoking stuff which touch on these topics more often than you'd think.

</comment></comment></comment></comment><comment><USER43> The safe existence of corporations suggests to me that the popular fear of AGI taking over and destroying humans is unrealistic. Corporations are like giant powerful AGI machines with the single purpose of making money for their shareholders no matter the consequences. They do become a dangerous threat to humans if left unchecked but we've developed systems to keep them under control.

<comment><USER44> >Corporations are like giant powerful AGI machines with the single purpose of making money for their shareholders no matter the consequences.

That is patently wrong. A corporation exists to distribute risk, and acommplish a task, ideally in a way that creates value for backers, but it is not a given that all profit creating avenues of behavior are desired or worth the egregious cost in negative externalities, or even that a corporation _must_ generate profit.

And unfortunately those control mechanisms you mention seem to be failing with alarming regularity due to regulatory capture.

<comment><USER45> > it is not a given that [...] a corporation must generate profit

I think that you are technically right, in that there can exist not-for-profit corporations. But for-profit corporations, which is what most people think about when they say "corporation" are generally legally required to put profit above any other value, assuming they are operating legally.

I completely agree with you that this is not a necessary way of organizing human society, and we are seeing more and more that the current for-profit system is disastrous for the environment and for society in general - especially given the inefficiency of regulation that you also mention.

<comment><USER46> > But for-profit corporations, which is what most people think about when they > say "corporation" are generally legally required to put profit above any > other value, assuming they are operating legally.

No they aren't. They do have a fiduciary duty to act in the interests of shareholders, which means not taking actions which are unexpected and obviously harmful to other shareholders like paying all the company's revenues to another company wholly owned by the CEO. But that duty to shareholders actually even _obligates_ them to take into account factors other than profit, whether that's mitigating risks or abiding by a shareholder resolution to follow a 'socially responsible' business practice that costs them a lot of profit, and management absolutely also has enough discretion to choose to design and follow its own 'socially responsible' business practices or decline to enter a profitable sector they don't want to involve themselves in. No executive has ever been penalised for not putting profit above any other value.

Companies pursuit of profit is much less driven by legal obligation and much more driven by the fact that greater profitability tends to generate greater returns to the management as well as the shareholders.

</comment></comment></comment></comment><comment><USER47> Yesterday there was a link in HN regarding artificial intelligence, and a user raised an interesting question: if a ML algorithm can be considered conscious, could a group of people doing the equivalent calculations by hand be the same?

I think that ant example answers that question with a strong "yes".

<comment><USER48> Here is an even more interesting question: if an ML algorithm can be considered conscious, do you even need to "run" it (whether on a computer or by a "group of people doing calculations by hand") for that consciousness to exist? An algorithm is fully deterministic... it will always produce the same result, always "think the same thoughts" given the same input. So where is the consciousness, in the execution, or in the algorithm?

Think of Conway's Game of Life, which has been shown to be turing-complete. The various "creatures" that exist in its worlds exist even without the program being run. We run the program only to observe them... the execution is for the external observer's benefit, the creatures exist in the "mathematical space" of the cellular automaton, not on the computer where the simulation is run. If an algorithm can be conscious, then so can one of this game's creatures, and its consciousness will be observing and interacting with the Game's mathematical reality, which will seem quite physical to that creature even without ever being simulated on any "physical" (to us) hardware.

Maybe all of existence is like that, no? There are people who say that maybe our Universe is a simulation. I say sure, but it doesn't need to be "running" on anything... it just exists because the rules it follows exist mathematically. It does get simulated, to an approximation however... inside our conscious observation of it! The Universe "exists" mathematically, but a subset of it "runs" in the brains of the observers to which it gives rise... it is the ultimate strange loop, forever eating its own tail.

<comment><USER49> > An algorithm is fully deterministic... it will always produce the same > result, always "think the same thoughts" given the same input.

Well, these are not just algorithms, they are actual agents. So they are embodied and embedded in the environment. Each action or movement can change the information that this algorithm learns from. So different previous experiences mean different agents. Also, the learning process is reliant on noise, and this can cause different outcomes. You would have to reproduce the whole environment in order to get to a situation that an agent will produce the same results given the same input.

Also, laws of physics are like a fixed algorithm our brains run on.

> So where is the consciousness, in the execution, or in the algorithm?

Consciousness is in the triad formed of environment, agent and reward signals. It's a continuous loop of perception, judgement and action, followed by observing the reward signals. The purpose of this loop, for biological agents, is self reproduction - so it is a self reliant ultimate purpose, it needs no external purpose except this one.

</comment></comment></comment><comment><USER50> > Ants use the rate at which they meet and smell other ants, or the chemicals > deposited by other ants, to decide what to do next. A neuron uses the rate > at which it is stimulated by other neurons to decide whether to fire. In > both cases, memory arises from changes in how ants or neurons connect and > stimulate each other.

Main takeaway, very interesting analogy. Ant colonies are great examples of complex systems with emergent large-scale behavior. Indeed the same could be said about networks of neurons. Interesting to think of an ant colony as the sum of oscillations of signals.

> Every morning, the shape of the colony's foraging area changes, like an > amoeba that expands and contracts.

Sounds like an emergent macroscopic "heartbeat" of the colony.

> In an older, larger colony, each ant has more ants to meet than in a > younger, smaller one, and the outcome is a more stable dynamic.

It makes sense that small perturbations would temporarily morph the heartbeat, but would probably snap back into the default oscillation pretty quickly. It would be interesting to see if a small colony is equally resilient to small perturbations as a large colony is to large perturbations, keeping some adjusted ratio of the perturbationSize/colonySize constant.

> individual ants live at most a year.

This comes as a surprise to me.

<comment><USER51> > Main takeaway, very interesting analogy. Ant colonies are great examples of > complex systems with emergent large-scale behavior. Indeed the same could be > said about networks of neurons. Interesting to think of an ant colony as the > sum of oscillations of signals.

Indeed. I've long thought that an ant colony should be seen as a single individual, rather than a group. One part which can procreate, like the reproductive system. Another which can fight off invaders, like white blood cells, or perhaps muscles. The anthill, in turn, is like a body; constructed by the cells and neurons, and protecting the system as a whole.

<comment><USER52> I was going to say you can also look at a country like this, with roads as veins, military as white blood cells, scientists/universities for brains, etc.

But my theory falls apart with the reproduction system. We don't really reproduce other 'countries'. While with ant colonies, a single entity produces all the "cells", and also all the "embryos" to start their own ant colonies.

In that sense an ant colony uses asexual reproduction if looked at as a whole.

Thanks for the new insights! :)

<comment><USER53> > But my theory falls apart with the reproduction system. We don't really > reproduce other 'countries'.

Sure we do. The UK has a whole lot of offspring, for instance.

<comment><USER54> Offspring, perhaps. But kidnapping is not reproduction.

<comment><USER55> I'd say it's reproduction, it's just not voluntary reproduction. Every time a culture invades another, you can think of it as creating a new culture that has the "genes" of the previous two. Examples include the influence of the moors on Spain, the changes to the English language due to the Norman conquest, and the modern unique Afrikaner culture of South Africa.

<comment><USER56> I'd say it is a mix.

Partly reproduction, partly conquering. As usual the place is not empty.</comment></comment></comment></comment></comment></comment></comment>


Google's decreasingly useful, spam-filled web search - <AUTHOR> http://www.marco.org/2617546197 <comment><USER2> I think people, possibly including me, get irked with Demand Media et al more because they're more successful than we think they _deserve_ to be rather than because they actually decrease the value of the SERPs. For SERPS where DM ranks well, the results prior to DM existing generally pretty much sucked. Maybe that is a Google issue, maybe that is an Internet issue (memo to Internet: middle aged women exist, please write for them, kthxbye), but for whatever reason, if you routinely Googled for [how do i make a blueberry pie] every week for the last ten years I don't think you ever had an awesome search experience.

DM pages are adequate for much of what they rank for, in much the way that USA Today is an adequate newspaper, your local state school provides adequate degrees in history, etc etc. They're adequate in a scalable manner, though, and they understand Google much better than the average publisher, which means they get visibility in excess of what some people might expect.

P.S.

Demand Media: <http://www.ehow.com/how_2933_make-blueberry-pie.html>

Virtuous publishers on the Internet: <http://www.pickyourown.org/blueberrypie.php>

If I wanted to bake a blueberry pie, I'd go for that second page every day of the week, but it is highly non-obvious to me that it is a _better_ result qua search engine result than the DM page. I love this example because I think Google fundamentally doesn't think [how do i make a blueberry pie] is looking for a blueberry pie recipe. Most searches will _not actually convert_ to pies. For the 98% of searchers who merely want to satisfy their pie voyeurism need, the DM content may well be _better_.

<comment><USER3> > _[how do i make a blueberry pie]_

You know very well that this is not how search works. You don't "ask a question" to the search engine like you would ask your grandmother.

You type in words that you expect to be in the pages you're looking for, and the search engine lists pages _that actually contain ALL of those words_.

One of the main improvements of Google in the very early days was that it used the AND operand by default, whereas competing search engines used OR by default, resulting in an incredible amount of noise.

In essence, searching for "how do i make a blueberry pie" (with quotes) should return _only spam_ , because only spammy and SEO optimized sites would contain the phrase as such. A real recipe would maybe contain the phrase "how TO make a blueberry pie" but not "how do i..."

\- - -

I think your point was that "middle aged women" don't know any of this.

It would be arguable (probably wrong, but still) that people who don't know this, who didn't make the effort to understand a little how all of this works, deserve the spam they get.

There is a good way to discriminate between good and bad content, and that is to _know a little about what you're searching_ in order to search for words that will be present in good quality content and NOT in spammy pages.

For example, it's reasonnable to expect a good recipe to give instructions in metric system as well as imperial; if you add "celsius" to the search then the second (informative) recipe arrives first:

[http://www.google.com/search?q=how+to+make+blueberry+pie+cel...](http://www.google.com/search?q=how+to+make+blueberry+pie+celsius)

<comment><USER2> _I think your point was that "middle aged women" don't know any of this._

No. That is overbroad, untrue, and would be very injurious to my professional reputation. I said that the Internet is skewed away from producing content responsive to their needs, which is about as controversial as saying that they are slightly underrepresented on HN relative to, I don't know, twenty- something males.

Non-technical users frequently use natural language search. The experience for natural language search is fairly poor. There are many classes of search which offer poor experience, but it is the one which leaps to my head first because I deal with non-technical users every day.

 _people who don't know this, who didn't make the effort to understand a little how all of this works, deserve the spam they get._

Words cannot express the depth of my distaste for this position. I will accept that "Google screwed up" or "I screwed up" if one of my users has a suboptimal Internet experience (which starts at Google because _Google is the Internet_ and ideally ends at my site), but I cannot accept that she is responsible if she has a poor user experience. We've got the teams of PhDs, the highly paid SEO consultants, and the lifetime of building an accurate mental model of how the devil box works. She wants to teach kids to read, not learn magic incantations. It should -- "should" in the sense of "would be optimal for the business", "would be optimal for society", and "as a moral imperative for computing professionals" -- just work for her.

<comment><USER3> (I really don't understand your first/second sentence (what's injurious?) Also, I'm 40.)

> _I will accept that "Google screwed up" (...) if one of my users has a > suboptimal Internet experience_

I respect that, and from a business point of view you're very right.

The problem is, how do you improve her experience without screwing up mine? Why can't I search for pages that actually contain all the words I'm looking for, as I typed them, and not words "that were present in the page linking to this page" or words Google think I want although I didn't type them in?

From a "moral" point of view (which you brought up), if she "wants to teach kids to read" maybe she could start by learning how to spell?

<comment><USER3> I don't know why the above comment is being downvoted, but I'm guessing it's because it sounds "elitist" (which is apparently a very great crime).

To elaborate, then: I agree with the parent comment that it's Google's job to make everyone's experience optimal (and not the user's), and it's certainly in the best interests of Google (or any business) to cater to the needs of as many of its customers as possible (although in the case of Google, as has been pointed out many times before, users are in fact the product).

But I would argue that the real elitists are people who think "middle aged women" shouldn't be expected to actually learn how to use machines.

"Middle aged women" (why single them out?) use machines all the time, whether at work or at home. They're expected to know how to use a spreadsheet, a word processor, a food processor. And they do. But somehow this expectation is lifted for "the Internet". Why?

A search engine is not a person; it's certainly not a mind reader. A search engine is just a machine.

<comment><USER4> The objective is Clarke's third law:

"Any sufficiently advanced technology is indistinguishable from magic."

The winner in search will be the one who strives for that.

If _I_ were presented with "How do I cook a blueberry Pie" - I would immediately do the following search:

[http://www.foodnetwork.com/search/delegate.do?fnSearchString...](http://www.foodnetwork.com/search/delegate.do?fnSearchString=Blueberry+Pie&fnSearchType=site)

Plus a few other pre-eminent and trusted food-networks (Allrecipes) - searching for "Blueberry Pie" on each one, I'd then scan the quality of the comments - looking for insight into other chefs (cross checking their history to see if they, in turn, can be trusted) who clearly have tried out the recipes, and have made relevant comments. I would then identify the recipe that looked most likely to work for me.

I would expect no less from a sufficiently advanced search engine in this, and all other domains.

<comment><USER3> The Food Network? Really? The home of "semi-homemade"...? ;-)

About Clarke's law, here's an observation by George Bernard Shaw: _"Build a system that even a fool can use, and only a fool will want to use it."_

Quotations aside, the process you're describing is certainly excellent; it's probably what Blekko is trying to pull, in a scalable way. It'll be interesting to watch how it plays out.

<comment><USER4> The second iteration, of course, is to engage with every (valid, trusted, revenue generating, etc...) customer who searched, determine the quality of the results, and then feed _that_ information back into the algorithms. You could then bias based on domain experts (world class chef's feedback on BlueBerry Pies more important than an anonymous user)

It may be the case that (AllRecipes, TheFoodNetwork, Etc...) are NOT the best place to search for a recipe, and that, indeed, "<http://pickyourown.org> is knocking it out of the park this week.

There is a lot of room for search to improve - I think the company that beats google (if it's not google that does so first), will be the one that manages to start creating the search<\-->Consumer<\-->Search feedback quality looop.

PageRank was just the beginning.

<comment><USER5> Google may already collect enough data to do this. They track clicks on the search results, so they can see whether you liked the results, whether you went back to a different result after visiting your first, and whether you modify your search terms for another search because the first one did not work out.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> We're working on it (as always.) There is a big improvement inspired by the stackoverflow post on its way shortly.

If people want to help out, _the best_ thing to do is to post examples of specific queries. Those become the "fixed points" around which we can tune until we get it right. The more example queries the better, and I'll make sure they get to the right people.

A good way to get example queries is to look through your search history, which if turned on can be found here: <http://www.google.com/searchhistory>

<comment><USER7> What happened to exact search queries:

For instance, if I search for "a-r" I receive results for "ar".

I hate this. It makes it impossible to filter irrelevant results.

Or try this query: "a-c" -"ac"

This will return 0 results.

<comment><USER8> And this is where the downfall of Google begins. I also hate that some exact queries are being broad matched without my consent.

</comment></comment></comment><comment><USER9> In a lot of the comments around this lately, people have been saying that this is something Google can fix, or needs to fix.

I would suggest that the content farms' success in gaming specifically Google's algorithm was an inevitability (whatever the current state of the arms race) and the only thing that will weaken the effectiveness of their techniques is to expose their business model to a greater range of algorithms. If you have three or four search engines all working on slightly different principles, it becomes a lot harder to game them all with the same content, even if gaming any one of them would be trivial. In other words, competition in the SE space at the algorithmic level is something we sorely need to see.

In parallel, my suggestion for one new search engine to add to the mix: a crawler for unsubsidised content. That is, the results consist solely of pages that don't carry advertising of any kind. This wouldn't exclude ecommerce sites but would exclude most kinds of affiliate marketing. Subscriber-only sites could pay to be indexed at a flat rate, though guaranteeing that this fee wouldn't affect rankings might be tricky. Alternatively a journal-access style of subscription model could see the SE paying the content site owner when one of its paying users consumes their information.

<comment><USER10> _a crawler for unsubsidized content_

Note that this doesn't require an all-new crawl/engine: just for an existing engine to offer an advanced operator that filters ad-drenched pages from results. Even just an operator that eliminated AdSense sites would be a big win for some queries.

<comment><USER9> While technically feasible, this would rule out a large number of high quality content sites which are currently ad-supported. For example, specialist community forums often carry ads just to cover costs. You need to allow for a different funding model for sites like that, such as subscriptions.

</comment></comment></comment><comment><USER11> Neal Stephenson's novel _Anathem_ has a section that talks about how the 'reticulum' (the internet, in the book's fictional world) was overrun with false copies of documents with slight changes made to them. 99.99% of all of the information on the internet was spam.

A huge industry of commercialized systems connected to the internet for the sole purpose of filling it with spam, and then the corporations would sell back filters and knowledge of which documents _weren't_ spam to customers. Eventually, the algorithms used to modify documents developed a malicious edge, so that the thousands of spam copies of an original document would be deceptive in ways that would harm people (e.g., in Marco's electrical plug wiring example, the document would have been modified so that it could get you killed by telling you to touch the wrong wire or something.)

Inevitably, it spiraled out of control, and a sophisticated system of social trust and ranking was put in place by IT workers and systems administrators, which are a caste and race of people in the fictional world.

Good book. Prescient, even.

<comment><USER12> It wouldn't be the first prescient thing Neal Stephenson has put in a book. Whenever I see Google Earth, I think of Snow Crash.

<comment><USER13> Since Google Earth was inspired by Snow Crash I'm not sure you can classify it as prescient in the same way.

</comment></comment></comment><comment><USER14> This is much ado about nothing. Google has a few search problems, and they always have, and they always improve.

Also, if Marco is going to list some problems, how about listing some problem searches? I search for what he lists, and the top result is fine in most cases, and debatable in others.

You folks think Google sucks? I don't. It's awesome, and I rely on it more everyday.

<comment><USER15> It really depends on what you're searching for. I purchased an handset that runs android in september and was hoping to find out when it would be upgraded to a newer version of android. The first 2 pages were almost completely filled with the same article.

to be fair though, I tried the same search on duckduckgo and bing and got mostly the same results.

I was just expecting google to be better than that.

<comment><USER14> Sounds like an obscure search, and if you aren't going to list the search terms you tried, it's really hard to judge whether it's user error or Google error. Google is only psychic to a point.

</comment></comment></comment><comment><USER16> Sites like Demand Media see a gap for particular content and churn out cheap crap.

Blogs see an idea in the public consciousness and jump on the bandwagon with derivative posts.

Anyone see the parallel? ActualLy there is a difference: the DM writer got paid.

Product searches have been screwed for years. I've often wished I could filter out any price search engines and/or retailers from results. What's worse is that all these sites have places for reviews (of which there are never any) but hey the review keyword is there.

But as for this post there's nothing new here. It's a rehash of a bunch of other posts from the last month.

I can still find what I want with ease on Google. Am I just some kind of gifted searcher? I seriously doubt it.

It's like these posts are all making slippery slope arguments ("there are two content farm results in the first page. If this trend continues there will be 7000 content farm resets") rather than complaining sbout the actuality.

The other mistake made here is to assume Google's algorithm is static. This is false. It's a rapidly moving target.

Like another comment says: such noise (spam) isn't unique to Google so is the "problem" with Google's index or the Web itself?

If nothing else these posts all make the case that Google's index is algorithmic. I say this because at different times you'll see conspiracy theories about Google promoting certain properties over others.

Here's a question: if Google started blacklisting sites, how soon would the complaints of censorship or favoritism take?

<comment><USER17> I agree with most of your points, but I would truly value Google providing me with the capability of blacklisting websites for myself.

Such a thing is already possible with just browser plugins, but I'd like that blacklist to follow me around and grow algorithmically (based on preferences of similar users), and hacking around a product's deficiencies is really not "voting with your wallet".

And in such a case Google couldn't be accused of censorship / favoritism.

</comment></comment><comment><USER18> Wonder how many duplicate topic and mostly duplicate content articles we're going to see about how Google provides duplicate content and duplicate topic answers to searches?

My irony meter is pegging.

<comment><USER19> It is a public debate whose nature is similar discourse that focuses in on an approximate truth. Your comment is especially unfair when directed at such a consistently high quality blog.

<comment><USER20> Could the OP be referring more specifically to HN? HN is akin to a curated system, and of course there are plenty of duplicate submissions on HN.

I'm not expressing an opinion one way or the other. Rather, that was my take on his comment, which amused me.

</comment></comment></comment><comment><USER21> What really irks me in the last few month is that google increasingly doesn't actually answer my questions. More often than not, none of the results on the first page contain all of my search terms, and most of the time it is the most specific term that it is missing everywhere. Or the big G has replaced that term with something completely unrelated. I have to prefix every search term with a + if I want to get a result quality that is even remotely similar to what use to be the default.

<comment><USER22> Example searches? That's the most constructive way to help us improve.

<comment> muuh-gnu Isn't it telling that you, if youre speaking on behalf of google i.e. working there on search, dont even know where your weaknesses are dont know where to even begin? If you have to rely on community input to even spot a direction for improvement you probably are in deeper trouble than all these articles suggest. Would Google even be where it is today if they hadn't spotted a weakness and improved it a decade (or more) ago?

I've personally mostly given up google for basically _any_ kind of product search, because it reminded me of going through my spam-riddled email inbox before my provider hat spam filters (or before I switched to gmail). Before that, I've had to give up the Google Groups Usenet interface because it was so cluttered with spam, that it wasnt even funny any more how useless it was, and Google _still_ kept associating its name with it.

If you have to, kick the spammers out manually, to prevent users jumping ship, until you eventually sort it out algorithmically.

Edit: When you downvote a posting I put time in to write, you could at least tell me what downvoteworthy I actually did, so I can refrain from doing it in future, since it is in our common interest to avoid downvoting and being downvoted. So, can you please elaborate in hindsight?

<comment><USER22> I know plenty of weaknesses in Google, and we work hard on the problems that we think matter the most (e.g. in 2010, we worked a lot on hacked sites so that regular people wouldn't stumble into an awful experience).

But it's very helpful to get independent, outside examples. It moves the conversation past "Google sucks" to "Google sucks because of query X." Sometimes those queries are new, but often what's just as useful is hearing what people dislike about the current results for the search X.

<comment><USER23> I find it very hard to believe that Google's Matt Cutts would come down from the ivory tower to answer comments on a website.

Google suspended my Google Checkout account because two words in the title of my book (make money editing from home) sent up a flag. No matter how many times I offered to give them a copy of my manuscript, they declined.

The eventual apology was appreciated, as was the full reinstatement, but it doesn't change the fact that I was wrongly accused, wrongly convicted, and had no court of appeal to go to except the very people who shut me down.

I no longer use Gmail, Google Toolbar, Chrome, Google AdSense, Google Checkout, Picasa, Blogger, Feedburner, Google Webmaster Tools, Google as a search engine, or whatever else I can think of. Google violated my terms of service.

Oh, and the nastygrams looked outsourced, and the apology had typos and bad grammar. You guys should hire an editor.

<comment><USER23> Actually, I'm not done. I didn't do anything wrong. Google admitted this. It was their mistake, not mine. They admitted this in their apology, which I did appreciate.

For my account to be randomly suspended reminds me how much an Act of Google resembles an Act of God. That's what annoyed me so much. Google did evil that day.

And that, to bring it back to the point "Matt Cutts" made, is WHY Google sucks. It can't be fixed.

</comment></comment></comment></comment></comment></comment><comment><USER3> It's easy to filter out spam once we identify it; so the question is: "what is spam"?

Some argue that content farms such as Demand Media aren't spammers, because the content they produce actually satisfies _better_ the casual searcher than elaborate, savant exposés on the same subject. Casual content for casual searchers.

Others consider that content farms-issued pages are the epitome of spam: spam that doesn't look like spam, and that ends up cluttering search results. Spam is not irrelevance: _spam is clutter_.

A corollary to "what is spam" is: "who should make the call"?

Originally, Google tasked itself with making this call, and it did a pretty good job at it.

But why not me? It should be possible for Google to make a difference between "casual" and serious content, and then let the user decide which they prefer.

Well actually, that is already possible: it's called "reading level" and it's accessible in the advanced page.

Searching for "how to wire an outlet" gives ~12 M results, the first of which comes from about.com.

When filtering the search to display only "advanced reading level" results, there are only 264,000 results left, the first two coming from Wikipedia (and the 3rd and 4th still coming from ehow.com).

So Google _already_ knows what is "casual content" and _already_ lets users filter it out.

Maybe a simple solution would be to add the filter directly in the search results page instead of having it buried in the advanced options.

</comment><comment><USER24> One thing that I notice about the spam sites and scraper sites is that they often have very similar content and/or layout. What if Google was able to determine how similar certain sites where and consolidate those into a single result, like they do with Google News?

Then when I search for AMD Bulldozer news and there are 20 sites all with the same article, from the same date, I don't have to change my search parameters to show just the last month. Instead, it would determine that the content was similar, smash into a single result, and leave room for 9 more less-similar results that may better include what I want.

</comment><comment><USER25> Decreasingly? This has been a rollercoaster for years. I was more of a Webmasterworld regular a few years ago than now but around 2005-2006 a lot of people thought Google had gone to pot.

[http://www.theregister.co.uk/2006/05/04/google_bigdaddy_chao...](http://www.theregister.co.uk/2006/05/04/google_bigdaddy_chaos/) <http://www.webmasterworld.com/google/3040496.htm> <http://www.seo- news.com/archives/2006/apr/6.html> <http://www.webmasterworld.com/forum30/34407.htm> <http://www.mattcutts.com/blog/feedback-webspam/>

Plus ça change..

</comment><comment><USER19> Could it possibly be at google is in the middle of an innovator's dilemma?

Twitter, hacker news, tumblr, and quora are all really shitty google replacements. But I use them to get certain kinds of information. It isn't enough to justify a radical change at google -- especially if they are even slightly focused on maintaining revenue.

There must an an opportunity for a more curated experience where the browsing behavior of a few thousand selected people can be used to juice authority. I don't think the human editors need to know they are doing that job. Maybe they should use chrome data for this.

</comment><comment><USER26> A first step would be to hide the queries data (especially trending queries). It was an interesting curio but its major consumers now are spammers.

</comment><comment><USER27> Am I using a different Google than him?

I type in "how to wire an outlet" and all the top results look useful. Sure there are some ads embedded on the pages and the top hit is about.com with a 10-page slideshow, but every hit looks like it explains exactly how to wire an outlet.

<http://www.google.com/search?q=how+to+wire+an+outlet>

Even when I try the spammiest searches, it looks like they're returning pretty relevant results:

<http://www.google.com/search?q=best+price+on+viagra> <http://www.google.com/search?q=wrist+watch+deals>

<comment><USER28> i ignored these "google sucks because of spam" articles until this one. i tried his first worse example [large sensor compact camera]. almost all the results on the first page are good. #3 is suite101 which is one of these farms, but it actually contains good content too. so i will go back to ignoring these "google sucks because of spam" articles, unless one shows up that has some quantitative results.

</comment></comment><comment><USER29> My guess for the near-to-mid term is celebrity curation.

I keep thinking about how Roger Ebert, after decades of movie reviews, started branching out into political (anti-Tea-Party) commentary and other articles. If you knew that a trusted brand (for many) like Ebert was curating home TVs, or projectors, or blank DVD media in an unbiased way, wouldn't you want to see what he had to say?

Or Thomas Dolby on audio equipment, Sting on Tantric books, and so on. They'd make money through affiliate links or even subscriptions.

</comment><comment><USER3> Isn't it possible that all of this recent bad press about Google would be a consequence of "Instant"?

Here's my thinking:

\- to get good results, one needs to type as many relevant words as possible

\- Instant encourages people to type _less and less_ words (not even words: a few keystrokes and you're on)

But if you type very few words, or if you search for "frequent" queries (generated by Instant in response to your few keystrokes) then all you get is spam.

Spam is optimized for frequent queries, not very specific ones. Instant should be renamed _Instant spam_.

<comment><USER30> That's not really the case, plenty of spammers optimize for long tail searches because 1. they're easy to rank for. 2. they're easy to create ambiguous, autogenerated content for

<comment><USER3> > _plenty of spammers optimize for long tail searches_

But how do they do it? By nature, there are many more long tail searches than frequent ones, and each one is rarer (or unique).

How do spammers find them?

<comment><USER8> Through keyword generating tools such as the Google Adwords keyword generator and a host of free online ones (of questionable quality).

</comment></comment></comment></comment><comment><USER31> I started writing out a comment on the somewhat heretical notion that biasing search results _against_ AdSense click throughs would probably be a strong predicter for spam detection, but the comment got long enough that I folded it into a blog post:

<http://news.ycombinator.com/item?id=2074621>

</comment><comment><USER32> _It's impossible to do any meaningful product research with Google._

Right now, I often start my product research within Amazon. However, that's only a start, as Amazon isn't great for everything. For large appliances, Consumer Reports is a good starting place. I guess I'm an example of the switch from search engines to "expert" sites.

<comment><USER3> Amazon is great but search on Amazon doesn't work too well; what works very well is to use Google and restrict the search to Amazon:

 _site:amazon.com some product_

</comment></comment><comment><USER33> I have to say there's some truth to this. Why is it that I increasingly must search through the search results just to find the site that originally published the string returned in the first 3 to 8 results?

I don't want to patronize all these sites repackaging content created by others, yet they continually appear before the creator.

</comment><comment><USER34> One wonders if Google is becoming the new Yahoo. If so, a big opportunity for the likes of DuckDuckGo and other nimble searchers. Today's upstarts can also run on the cloud, sidestepping the need to build Google-scale data centers (at least initially).

</comment><comment><USER35> This kind of worries me.

On the one hand, Google isn't the best web search tool. I've switched to DuckDuckGo, and so has everyone who's seen me use it. But, I think Google still provides a valuable public service: indexing the entire web and handling that much traffic is not an easy task, and a lot of other things (like DDG) depend on that humongous cluster.

So on the one hand I want to see the best search engine win, but on the other hand if Google goes out of business (or more likely, starts losing money and canceling projects) then I'm afraid it'll take a lot of things out with it, with no clear replacement.

<comment><USER36> Indeed perhaps time to give duckduckgo a try. They seem to actively filter out all ad sites. I've been going nuts in the past couple months with google searching for technical solutions to problems, command clicking on links to open in tabs and discovering that most are clone ad sites from a question on stackoverflow.

<comment><USER35> It was these that won me over: <http://duckduckgo.com/bang.html> . Most of the time I know what general sort of thing I'm looking for, and I'm happy to give the search engine hints if it'll help. Even if I don't, it'll essentially flat-out ask me what sort of thing I want, and then give me more specific results: <http://duckduckgo.com/?q=ruby>

But, I mean, I'm doing all this by typing it into Chrome's address bar. And a lot of the time, DDG is just returning results from Google's API. I want to _use_ DDG, but I want to make sure Google _still exists_ because I need it even if I don't use it.

<comment><USER17> I just use the browser, instead of the !bang feature in Duck Duck Go.

Chrome / Firefox have the option of adding a search shortcut.

For instance I type "py package-name" for searching inside Python's index, or "am product" for searching in Amazon, or "w something" for searching Wikipedia, or "t some words" for doing a google translate, or "hn something" for searching Hacker News.

Chrome even does these shortcuts automatically, so you can just "amazon.com android" and it will do a search on amazon; although in Firefox it is easier to add your own.

<comment> T-hawk FWIW, Opera has this too, and in fact had it first, by about 2005.

</comment></comment></comment></comment></comment><comment><USER37> Isn't this how Facebook topples Google and completely dominates the internet? By incorporating your social graph into your search results, your relationships can influence what is returned by the search.

Suppose you could create some sort of "friend" list with HN users and that were used to prioritize your search results. If you get a result you don't like, click that you don't like it and the software will reduce the weights of the parts of your social graph which caused that result to be highly scored.

</comment><comment><USER38> I set up a second, filtered search using Google Custom Search and added it to my browser. I don't always use it, but it's easier to switch to when I encounter spammy topics (like code look-ups). It's pretty easy to blacklist fakes... and even useless SEO-heavy sites like experts-exchange, bigresource, etc.

Here's how, if interested: <http://radleymarx.com/blog/better-search-results/>

<comment><USER39> I am beginning to use this mode to Google search more, cut out a lot of the spams.

</comment></comment><comment><USER40> Is there a huge incentive for Google to improve if a lot of the content farms are monetised by AdSense and actually return Google money?

You could argue that they might lose their spot as the default search engine for a lot of people, but Microsoft has presumably thrown a huge amount of money and expertise at the problem and hardly dominated. I suspect this is not going to be a significant problem for Google in a hurry.

</comment><comment><USER41> I'm not sure if prioritizing links over keywords is really going to help matters.

I know a lot of 'little guys' who know something about a topic and can write prolifically, but who suffer under the delusion that 'If I build it, they will come.' Success in SEO is largely possible because 95% of webmasters have no idea how to promote content.

I've also developed 'digital libraries' for major academic organizations and a common thread there is a complete lack of interest in indexability. There's a lot of fantastic content trapped in the ivory tower because nobody considered the 'unwritten standards' for how the web worked.

A big part of the problem is that it's very hard to get legitimate links these days. You used to be able to get into the Yahoo directory for free, but now you have to pay a $300 a year bribe. Before 2000, it was common for people to create large collections of links they liked. Today, major players like Engadget have a policy of not wasting their PageRank on other sites. Afraid of spam, many blogs and forums are on a hair trigger to stop people from dropping links in comments, relevant or not.

If legitimate links are harder to get, that 'lowers the bar' for spammers.

A real answer to spam would be to strengthen the signal so it can break through the noise. It might be helpful to be able to get more feedback from web users about the quality of pages, but this is tough. The horrible truth is that there are more pages on the web than there are viewers, so even if you could get feedback from 10% of viewers, many pages would be badly undersampled. Spammers would also target any feedback channels that exist, and with low response and sampling rates, it might be easier to overload the feedback chanel than it is to create link noise.

Another answer is to beat Demand Media at their own game, the same way that Stack Overflow has beaten the spam sites that dominated programming questions two years ago.

</comment><comment><USER42> I think search quality would go up if Google gave me the option of blocking domains from SERPs. I never want to see results from a content mill (eHow, Mahalo, etc) in addition to all the made for Adsense sites I come across less frequently. They could also use the collective blocking data to help tweak the spam filter.

</comment><comment><USER43> I think the integration of social media is a possible solution. Recommendations and likes (from Facebook) or other places are hard to artificially jack up and can also offer great results if it ties them to your friends. I like the direction Bing is going with this. It is the only way I can see to get large human edited results of the web.

Unless Google develops highly advanced AI (which is a possibility) computer algorithms can be gamed. Humans can be gamed as well but because we are all so different I don't think there is a single approach that would fool a large segment of the population at once.

<comment><USER44> "Recommendations and likes (from Facebook) or other places are hard to artificially jack up"

If recommendations and likes are added to Google's algorithm, people will find ways to artificially jack them up. For example, marketers have aligned networks of Digg users to increase the amount of Diggs. I find it hard to believe that Demand Media and others will not be able to artificially inflate Facebook likes.

</comment></comment><comment><USER45> I'm not quite sure how it will happen, but at some point I think it will be beneficial to commoditize the underlying crawl and index data, so that there can be more domain specific focus and more diverse sources of innovation applied to solving this and other search problems. One or two sources trying to be all things to all people and all problems isn't going to scale.

Blekko's slashtags are a good start, but it needs to go much further.

</comment><comment><USER46> These bad search results are not an IT problem, they are the results of top Google executive policies: act like you want to be good and do better for the Google users but keep on serving up the same old stuff because it is tied to the revenue cow and the paying clients. This problem would never have existed if Google considered the end user experience more important than the advertisers.

</comment><comment><USER47> Why does Google no longer offer the option to permanently remove a specific domain from your search results? My personal search quality would be dramatically improved if I could specify even a short blacklist.

In fact, dear lazyweb: is there a browser extension or greasemonkey script that makes Google return 100 results at a time and then filters out the best 10 based on a blacklist?

<comment><USER48> Set Google to return 100 results and then use a userscript or extension to filter them. E.g. [https://chrome.google.com/extensions/detail/ddgjlkmkllmpdheg...](https://chrome.google.com/extensions/detail/ddgjlkmkllmpdhegaliddgplookikmjf)

</comment></comment><comment><USER49> Couldn't part of this problem be solved with an algorithm that identifies when several pages have roughly the same content (ie. original wikipedia article + 5 copies of it elsewhere on the web) and then giving the oldest occurance in the index a much higher rank?

That would kill incentive to create these spam-sites and give the user the result s/he was looking for.

</comment><comment><USER50> Maybe Google could scrape DDG on the fly for each search, then do a diff, and filter out any results that arn't in DDG... that would be the fastest way to remove spam ;)

</comment><comment><USER51> People just can't come up with right search queries and guilt search engine. I always find what i want via google.

</comment><comment><USER23> I was on the Internet long before Google showed up, and I'll be here long after Google is dead and forgotten.

</comment><comment><USER52> Google, can't you solve these problems with money?

Pay army of users pressing ham/spam buttons, mechanical turk style.

</comment><comment><USER53> Please, not a rehash of what we've been reading for the last couple weeks.

<comment><USER54> I do find it interesting how this has become a meme, but I think that Marco has added something interesting to the discussion. I like his categories of searches, and the decision he outlines google as needing to make.

But, really, if you don't want to talk about this subject anymore, close the tab.

<comment><USER53> Apologies. I skimmed through the first few paragraphs and thought that it was a summary. I was attempting to help HN cut back on multiple posts on the same old content. Ah well, I guess I won't be doing that any more.

</comment></comment></comment><comment><USER55> I don't think this is surprising - the top management seems more interested in building OSs and social networks. Search doesn't seem like their highest priority anymore.

<comment><USER22> There's more people working on search quality than ever in our history. The PR team for search quality does a great job pitching stories about search quality, how it's hard, etc., but lots of reporters prefer to write about the shiny things (or more tangible things--you can hold a phone in your hand) rather than improving search quality.

But Google continues to work hard on improving search every day, even if that doesn't always get covered.

</comment></comment><comment><USER56> _One solution may be for Google to radically change their algorithms and policies for web search to de-emphasize phrase-matching and more strongly prioritize inbound links and credibility._

Inbound links and the calculated "credibility" from the same are what killed the web the first time around. There was once a democratized web era when that actually worked -- when millions of people had their little Goecities pages and were linking the cool stuff -- but in the modern era it's 99% consumers who cast no votes, and the last 1% is extraordinarily incestuous circular link love: Marcos links to Coding Horror who links to Daring Fireball who links to Scoble who links to Marcos, etc.

People with neither information or authority end up being the credible authority on matters they have aren't authorities on. Scoble a few years back pointed out the fact that according to search engines he was the most important Robert in the world. That is a frightening concept.

We will move from an era of search engines to an era of expert engines. Many of the questions I used to "ask" Google I now ask of Wolfram Alpha, and its approach has turned out to be quite useful. Expand that computer knowledge more broadly, and improve the human syntax parsing. and we'll have a winner. Several such systems are built around computer learning of the wikipedia corpus.

<comment><USER57> _Several such systems are built around computer learning of the wikipedia corpus._

Any ones in particular that you've found work well?

<comment><USER58> Powerset worked well enough that Microsoft bought them for $100 million.</comment></comment></comment>


Apple approves "Notch Remover" app for iPhone X - <AUTHOR> https://www.macrumors.com/2017/11/13/apple-approves-notch-remover-app-iphone-x/ <comment><AUTHOR> Honestly I stopped noticing that notch after the first day of use. It's weird at first but then it just disappears once you start using the phone a lot. And actually it even starts feeling weird looking back at the regular iPhone screen with a clock in the middle.</comment>


Are dating websites past their prime? - <AUTHOR> http://www.washingtonpost.com/wp-dyn/content/article/2010/06/26/AR2010062600189.html <comment><USER3> I married my ex-wife from eharmony. (thanks for the compatibility match guys!) I married my current wife, and the mother of my two children, using yahoo personals. I also dated using match.com

In my opinion, dating sites are still extremely efficient ways of meeting people. I exclude eharmony, both from personal bias, and from the fact that they don't allow you to really search the database.

After you leave college, real life meetings become much more complicated. The bar scene is a lottery. How likely is it that you'll actually coincide in time and space with somebody who you would be happy with, and then how likely is it that you'll even get to talk to them if they are there when you're there? Grocery stores and other non-traditional meeting places, I personally avoided bothering women in, assuming there should be some neutral ground where a woman might wish to be left alone.

For me, I wanted college educated women who had opinions about politics and whose opinions I shared. I wanted women with no children, who lived within 50 miles of me and didn't smoke or do drugs, but who did drink socially, wanted children, had traveled the world some, and were not strongly religious. I wanted women who enjoyed outdoor sports such as scuba, skydiving, and rock climbing. I wanted women who didn't own small yap dogs.

Can you imagine bumping into women like this randomly? But with a dating site, it became so much easier.

I still went out on a lot of dates that were bad matches, but I definitely skewed my chances of finding somebody I could respect and have fun with by taking the random chance aspect out of the equation.

EDIT: The article doesn't actually disagree with this. It's saying that paid dating sites will lose out to unpaid. I was responding to the HN title which derides all dating sites.

<comment><USER4> > Grocery stores and other non-traditional meeting places, I personally > avoided bothering women in, assuming there should be some neutral ground > where a woman might wish to be left alone.

So many of the single women in my social circle complain they can never meet decent guys and would kill to be talked to be a decent, intelligent guy at the grocery store.

Worst-case scenario is a 15 second disturbance of minor flattery following by each of you carrying on with your day. Really, that's the worst case scenario. Really.

<comment><USER5> I am guessing you haven't approached women much? Your "worst case scenario" only applies if they actually think you are a decent guy up to their standards. Otherwise they might be offended that you even dare to talk to them (best case scenario: they just pretend they didn't see or hear you).

Not that I advise against approaching women, just saying.

<comment><USER6> > I am guessing you haven't approached women much?

That's not really called for! The GP is right though; you're unlikely to be, say, given a slap or yelled at! This assumes, of course, your not being leery or crude and not asking them on a date straight off. etc. (where you will likely get a slap).

But there is _nothing_ wrong with saying hi and having casual conversation with someone in the grocery store. 99 times out of 100 it will just fizzle out after a couple of seconds - one time in a hundred you might hit on a common interest and the rest, as they say, is history. I've met a number of great girlfriends in this way.

It's also a great way to get over shyness.

Also; you _should_ talk to random people anyway. It brightens up their day and yours in a perfectly harmless way.

There is this social convention that we all must be silent and look dead ahead when queuing, for example. Everyone I know thinks it is silly but they don't want to be the first to broach the convention - just in case....

<comment><USER5> "That's not really called for"

Didn't mean it in a nasty way. And yes, actual physical attacks are unlikely as a response. Unfriendliness is rather likely, though. But if you have a hardened "shell", no problem. Or if you have become very adept at chatting.

Also, if those single girls at grocery stores are so desperate, why don't they make a move themselves?

<comment><USER6> > Unfriendliness is rather likely, though.

I'd say that is highly unlikely also. At worst you'll just get ignored :) but generally anyone will talk to you if you look and speak normal/friendly.

> Also, if those single girls at grocery stores are so desperate, why don't > they make a move themselves?

Social convention. Also; this is the beginnings of a strawman because it's not like that at all :) the GP is just suggesting we talk to people more - <USER2> is always a good thing!

<comment><USER5> He said he knows scores of single women who are dying to be talked to.

<comment><USER6> Yep; which is why your point is something of a straw man - because it is the extreme interpretation of his point that doesn't stand up.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> I strongly recommend okcupid.com. In my opinion it is the best dating site, especially for hackers - they even let you A/B test your photos. I'm a complete introvert/nerd and I've used their site with a lot of success. It is by far the best overall user experience out of all of the various dating sites - and it's free. Their blog is great, too: <http://blog.okcupid.com>

<comment><USER8> I have a date lined up from OKCupid. One word of caution, but this goes for online dating in general. It's taken me 6 months to land a first date. In real life (err I guess offline) it's easier for me to land a date than it is online. So, I suggest using online dating to augment finding dates offline in a more traditional sense. It is far easier for people to be picky online than it is offline.

<comment><USER9> I'm not trying to be harsh, but you're doing it wrong.

I'm not particularly gifted in the looks department, but I can get about 1 date a week. How? Testing and speed. Here are a few tips..this advice isn't really making any assumptions about you..it mostly is a result of mistakes I've made and have seen other guys make.

Testing:

-Photos: ask friends to help pick the best ones, take some new ones, use OKCupid's photo-testing service, etc.

-Profile: I probably went through 5 complete rewrites, followed by a lot of constant tweaking. I think the biggest mistake most guys make is being way too serious in their profile. You've really got to keep it light and playful. Sure, be authentic and show who you are, but wait until after a few dates before getting all deep. Just have fun.

-Messages: Play around with the type of message you sent. Try some serious, try some playful, try some ridiculous/absurd. You'll see what tends to work more. I'm assuming you're doing more than just saying "hey what's up," that will just get you ignored.

Speed:

Don't write long messages implying you think you two are soulmates. The worst thing you can do is get hung up on hoping this one person with a great profile will reply. Getting into this kind of pattern will make you think way too hard about what to write to them for your first message. Look for interesting profiles, write a quick message, move on. You've got to get a good volume of messages out there, don't spend too much time on a particular profile.

EDIT: Forgot to mention one more thing, take it offline as soon as possible. Don't let the messages drag on..if someone replies to me I usually ask to meet up for a drink in the 2nd message I send.

<comment><USER8> Or, I might just be incredibly picky. See my reply here <http://news.ycombinator.com/item?id=1466098>. I guess that brings up another point. Some people might not have found someone offline because they're just picky. So don't presume that people "lower their standards" when they date online.

I have 16 conversations dating back to May in my inbox. Let me know if that's par for the course. The problem is, of these 16, 8 were girls that initiated contact with me and that I didn't find very attractive. Of the remaining 8, I had a date lined up with 1 that fell through, and a date currently lined up with another.

EDIT: Another stat. I sent out 39 messages from May until now. Of the 39 messages, 8 got replies. So that's about 1 in 5.

<comment><USER10> > The problem is, of these 16, 8 were girls that initiated contact with me and > that I didn't find very attractive.

Do you mean that it's unattractive for a girl to initiate contact with you?

<comment><USER8> Not at all. The most annoying thing on OKCupid is seeing a repeat visitor that will not strike up a conversation with me until I initiate it. I actually find take-charge girls really attractive. What I meant is that I didn't find the girls who initiated contact with me so far, to be attractive. A few had very poor self esteem that I could tell just from the message they gave me. Many would say they have mutual hobbies/interests but when I probed further I realized they were lying and were the kinds of people that actually don't do much, which is also a turn off. Somewhere along the line I learned to overcome my introvert tendencies for periods of time. And I want to enjoy life with somebody who has similar interests. I go to the gym, play sports, have an active social life. I don't necessarily want somebody who is the same as me, but I also don't want somebody who's going to loaf around on the couch at home for an entire weekend. So far, girls like that seem to be initiating contact :-/

</comment></comment></comment></comment></comment></comment><comment><USER11> Speaking as someone who just recently started using match.com (after the end of a string of long relationships), it seems to me that:

1) dating sites are the perfect example of the benefit of pseudonymity for allowing people to choose how much to reveal about themselves while still being attractive

2) dating sites rely on geographical favorability. I live in DC so as a single guy it works EXTREMELY well. But I hear it's not as great for DC women. But DC is known for having far more highly-educated single women than any other city. But would a dating site work as well in smaller communities where people marry out of high school and generally keep their social circle their whole lives? Is NYC too much of a social cauldron for dating sites to work?

3) Facebook hook-ups seem to work mostly if you are being hooked up with a friend of a friend, so the initial relationship is being built upon referral instead of semi-blind dates.

<comment><USER12> I would be very wary of spending money on paid dating sites. There are just too many stories about fraud: <http://forums.plentyoffish.com/datingPosts2493442.aspx> (Google will find you many more)

okcupid and plentyoffish are much much better options.

And Yahoo Personals and Match.com definitely aren't the only ones. I've heard of other paid dating sites doing the same. Of course these are only rumors which is why it never gets very far in court but it's enough for me to never want to spend my money on paid sites. The incentives to move into fraud in paid dating are very very strong.

<comment><USER13> Methinks the free sites doth protest too much. I've used Plentyoffish, Okcupid, and Match.com. Only Match.com was a fraud-free experience. Sure, a paywall induces a motivation for 1st-party fraud, but it drastically reduces the motivation for 3rd-party fraud. Someone who solves both will become a rich individual.

For the record, I met my beautiful girlfriend of two years on Match, so count me as biased. :)

<comment><USER14> I've used okcupid for maybe a year and a half, and have gotten maybe two or three fraudulent messages,

<comment><USER13> I recieved a chat request from a women 30 minutes after I signed up, while I was editing my profile. She looked like a model and her profile picture was her in a bikini. It took her 5-10 minutes to respond to my IMs, which were mostly me asking her if she was a legit and "her" saying "yes, I'm real." uh huh.

<comment><USER14> Like I said, I've seen a couple fakes. They've always been really easy to spot, though, and disappear quickly.

I'm in the Boston/Cambridge/Brookline area, though. I imagine the real:fake ratio gets worse as you get into more sparsely populated areas.

</comment></comment></comment></comment></comment></comment><comment><USER15> This article asks some interesting questions, but provides few numbers. The big question isn't "are paid dating sites dying". I think the answer is clearly yes, but quite slowly.

The real question is about free sites like okcupid.com and plentyoffish.com , sadly, they have no numbers about these sites.

</comment><comment><USER16> It sounds contradictory, because I think dating sites are yet to reach their prime, let alone having past it already.

<comment><USER17> are you talking about the current dating sites? really?

<comment><USER16> I mean anything.. current/old.. the sites which targetted "dating" in specific were never able to succeed, unless the social networking came into picture.I see "dating" and "social network" as 2 different things. How many people actually log in onto a dating site with a hope that they will actually get a genuine date!.I think the number is pretty less. Actually speaking "social netorking" sites generate more traction in the dating world than the actual "dating" sites.

<comment><USER18> "Actually speaking "social netorking" sites generate more traction in the dating world"

I am not so sure about that. I don't really see people hooking up with friends of friends

</comment></comment></comment></comment><comment><USER19> tl;dr: The boring old corporate behemoths aren't growing anymore, therefore online dating is dead.

They might as well have written an article about how TV is dead because nobody buys tube TVs anymore.

Match.com is an awful, awful user experience. It's a wonder that they're still able to fleece so much money out of people.

</comment><comment><USER18> I really didn't know dating sites were this successful...hmm time to jump in..off to make a profile :)

</comment><comment><USER20> The analogy with peak oil is somewhat incorrect, because new people are being born (and enter dating scene subsequently) all the time. Dating sites will not run out of people.

(As contrast, new oil is not being "made". Once it's all pumped dry, it's over.)

</comment><comment><USER21> If you haven't read the OKCupid's blog post, "Why You Should Never Pay For Online Dating," I _highly_ recommend doing so. If you're contemplating online dating, that is.

[http://blog.okcupid.com/index.php/2010/04/07/why-you- should-...](http://blog.okcupid.com/index.php/2010/04/07/why-you-should-never- pay-for-online-dating/)

Interesting points: * "you are 12.4 times more likely to get married this year if you don't subscribe to Match.com"

* the severe male/female imbalance ("sausage fest") leads to a "desperation feedback loop" where men spam more and more women, with fewer and fewer responses.

In my opinion, with the rise in social networking platforms, dating sites are dead or dying - at least for 'mainstream' use. Even OKCupid, I think, will be pointless as people learn to leverage Facebook for dating.

<comment><USER22> Facebook is certainly useful in facilitating dating with people you have already met irl, but I dont see it used for _generating_ dates with people you have not already met.

Although I am curious if anybody has counter-experience on this :D

<comment><USER23> Yeah, the social mores on FaceBook seem to frown upon random contact with lots of strangers. It's good for maintaining relationships, not so good for starting them.

I've noticed a bunch of friends either getting married or beginning serious LTRs after meeting online, though. They usually fall into two categories.

1\. A set of mutual friends of them say "Hey, you two would be perfect for each other," introduce them, and then the relationship blooms through FaceBook or old-fashioned e-mail.

2\. They're both participants at a "niche" online forum or social networking site, and then end up meeting in person eventually and take it from there.

I think the latter situation is way, way underreported. We pay attention to the big social sites like FaceBook and Twitter and the big dating sites like eHarmony and Match, but there's an enormous long-tail of niche sites that toss up a website and a forum and build a community. You can find a fansite for virtually every celebrity, every TV show, every sport, many of the more popular books, and even niche professional interests like startups and programming language design. Many of these grow very close, tight-knit communities, and when those communities are of mixed gender, it's very common for some dating to occur.

Heck, if HN weren't overwhelming male, I bet we'd see some Hacker News couples. There've already been a bunch of "Founder Dating" sites spun off from it.

<comment><USER10> How is the gender split on HN, roughly?

<comment><USER23> I have no idea, but judging from comments, it seems to be <10% female. That might be an underestimate, though, since many women who post on predominantly male forums often don't reveal their gender.

</comment></comment></comment></comment></comment><comment><USER24> wondering why would someone ever go to a dating website, seriously, can't imagine that in my wildest dream. And do people pay for it? Seems it is a thing for the past generation. We aren't convinced.

<comment><USER25> Apparently your wildest dreams aren't very wild.</comment></comment>


Simplicity and collaboration - <AUTHOR> http://dave.cheney.net/2015/03/08/simplicity-and-collaboration

<comment><USER2> _"Simplicity is the ultimate sophistication" — Leonardo da Vinci_

Leonardo didn't say that—it was William Gaddis in his massive novel about art forgery called _The Recognitions_ , sometimes thought of as the first postmodern novel. In Leonardo's time "sophistication", had the concept existed, would have seemed like an obviously bad thing.

I believe it was Steve Jobs who popularized this misattribution, as well as the pseudo-Picasso one about how great artists steal (which really came from T.S. Eliot).</comment>


Vanilla–hide Mac menu bar icons for free - <AUTHOR> http://matthewpalmer.net/vanilla/ <comment><USER2> Any particular reason this isn't FOSS? Seems like a nice alternative to Bartender, but making it "free as in speech" rather than "free as in beer" would make this much more attractive.</comment>


Court Says Constitution Protects "God Hates Fags" Picketers at Military Funerals - <AUTHOR> http://www.nytimes.com/2011/03/03/us/03scotus.html?pagewanted=all <comment><USER2> I'm disappointed this wasn't a 9-0 decision. I'm also disappointed it had to go to the supreme court. These protesters are clearly assholes, but freedom of speech is absolute.

<comment><USER3> Legally, freedom of speech is not absolute. Alito invoked the fighting words doctrine, which has been upheld (though in narrowing interpretation) by the Supreme Court on numerous occasions. I'm not saying it would quite apply in this case, or that I disagree with your sentiment, but there are limits established by precedent to the freedoms defined in the Constitution. Whether we agree they are valid is another matter.

<comment><USER4> It seems clearly wrong that a funeral should be interrupted by anyone for any reason. Don't families have the right to bury their loved ones in privacy? The court should not defend the rights of one group if by doing so it strips another of theirs.

<comment><USER2> The families have a right to a certain level of privacy, but this took place in public and protests were given restrictions on how close they could be by local authorities.</comment></comment></comment></comment>


Show HN: HNStats – track stats for HN posts - <AUTHOR> https://hnstats.info/ <comment><AUTHOR> Creator here. Porter.io used to have a service where you could track some basic metrics of Hacker News posts, but was either shut down or has not been working for while. So, I wanted to create a simple clone of the service.

HNStats stats for this post: [https://hnstats.info/news.ycombinator.com/item?id=13541433](https://hnstats.info/news.ycombinator.com/item?id=13541433)</comment>


Show HN: Node Package for Apple Push Notification Service via HTTP/2 - <AUTHOR> https://www.npmjs.com/package/apns2 <comment><AUTHOR> Github repo: [https://github.com/AndrewBarba/apns2](https://github.com/AndrewBarba/apns2)

</comment><comment><USER2> Looks great! I think this is the first open implementation of the new http2 api (?)

Looking forward to trying it out.</comment>


DDoS Attack Against Default DNS System v2 - <AUTHOR> http://status.namecheap.com/archives/17727

<comment><USER3> This is a pretty big and complex attack. We have a ton of DDoS mitigation in place but it has taken some time for the filtering to become effective.

We are mitigating this as fast as we possibly can. I and we share your pain and frustration at the inconvenience this causes.

<comment><USER4> For those who don't know Matthew Russell is part of NameCheap.

<comment><USER3> I am, yes.

I'll check HN as we restore service and update will now go out on our status page every 20 minutes.

My apologies once again for the inconvenience this causes.

</comment></comment></comment><comment><USER3> We have mitigated over 75% of the attack at this time. Most servers are performing well.

I appreciate my responses here are a little obscure but we do practice security through obscurity so I am not going to get into specifics that can be used against us in any future attack.

I will say we employ a range of technologies, internal and external, a ddos defense/mitigation. This does include CloudFlare

</comment><comment><USER5> It seems like DDoS attacks are becoming more prevalent. Are there forums where attacked companies can share data about the attacks they see and what they had to do to mitigate them? I feel like there should be public community resources around DDoS attacks like there are around other security vulnerabilities.

<comment><USER6> I manage a private combination of communications bulletins and attack monitoring for a couple of smaller communities / organisations that get attacked on the regular (mostly hosting providers and gaming organisations). I've been considering letting it go invitation-only public.

<comment><USER5> Wow, I think it would be awesome to publish some analysis of lessons learned + recovery playbook.

<comment><USER6> I'll look into cleaning everything up a bit and formalising reporting and perhaps release it properly.

A lessons-learned whitepaper would be radical, I'll chat to a few of the people that I run this with and see what they think.

</comment></comment></comment></comment><comment><USER7> In the age of DNS providers with APIs, you really should have at least two. At YC we've just moved everything to Route53 and will be adding more - <USER2> CloudFlare and Google Cloud DNS.

All of it will be managed by configuration files in version control, and won't require anything antiquated like zone transfers.

If the code is clean enough I'll release it with pluggable registrar and DNS provider modules.

<comment><USER8> Please. The zone transfer thing is what's been the most burdensome. We'd love to have a github repo that we could create a post commit hook to tell a bot to update the syncing.

<comment><USER9> [https://dns-api.com/](https://dns-api.com/) is a Git based Route 53 reseller ([https://dns-api.com/docs/](https://dns-api.com/docs/)). Netflix's denominator is also a great project for managing multi-provider pushes: [https://github.com/Netflix/denominator](https://github.com/Netflix/denominator) . It even supports advanced record types, such as geo and weighted sets.

</comment></comment></comment><comment><USER10> This would be made easier in Namecheap offered Secondary nameserver support. For example, DNS host at Namecheap, but use Linnode as a secondary.

</comment><comment><USER5> If folks are looking for an alternative DNS host, there is Google Cloud DNS, which I have been using for a while via Google Domains and it's pretty awesome. Conceptually similar to Route53. Can use Google Cloud DNS without using other Google Cloud Platform services. [https://cloud.google.com/dns/docs](https://cloud.google.com/dns/docs)

<comment><USER11> >If folks are looking for an alternative DNS host, there is Google Cloud DNS, which I have been using for a while via Google Domains and it's pretty awesome.

It's pretty easy to setup too. [1] For my purposes cheaper than Route53.

[1] [https://www.zeitgeist.se/2014/05/01/google-cloud-dns-how- to/](https://www.zeitgeist.se/2014/05/01/google-cloud-dns-how-to/)

<comment><USER12> I'm curious what you're doing that's substantially cheaper with Google Cloud DNS. The base pricing looks the same, except Google being $0.2/month and AWS $0.25/month per hosted zone. Also keep in mind that queries for "intra AWS"/ALIAS record queries are free on Route 53.

</comment></comment></comment><comment><USER13> Last time this happened, we had a pretty bad time. NS and SOA TTL being set at 24 hours made switching to Route53 rather difficult. Hopefully, they'll recover soon.

Anyone know the details of how these things happen?

<comment><USER9> FYI: You can invalidate Public DNS's and OpenDNS's caches manually:

[https://developers.google.com/speed/public- dns/cache](https://developers.google.com/speed/public-dns/cache) [http://cachecheck.opendns.com/](http://cachecheck.opendns.com/)

</comment></comment><comment><USER14> I'm going to move some things over to Route 53. I like Namecheap but this is the second DNS issue they've had in a month that has affected several of my sites.

<comment><USER4> Why not use cloudflare? It is free and has ipv6 baked in.

<comment><USER15> Anything to keep in mind when switching to cloudflare? Is it completely free?

<comment><USER4> Nope, it works well. You can just use their DNS service.

<comment><USER16> People have claimed (on HK) that their free tier slows down your site, any truth to those claims in your experience?

<comment><USER17> I've had that experience (with Cloudflare in general, not DNS). My average response times as measured by Google webmaster tools went from 30ms to 300ms, if I remember correctly. This way ~18 months ago though, so I don't know if it has changed.

And it's to be expected to be slightly slower, at least for dynamic requests, since it's a reverse proxy. I doubt it's limited to the free tier.

I've used the DNS for a few years without any issues whatsoever, so I'd recommend them for that.

</comment></comment></comment></comment></comment></comment><comment><USER18> Wondered why my site was spotty. I love Namecheap as a registrar, but I'm questioning now whether I should be using them as my DNS provider.

Any suggestions?

<comment><USER19> I can fully recommend Point ([https://pointhq.com/](https://pointhq.com/)), haven't had a single problem yet and support thus far has been very good. The web interface is easy to use and they offer both an API as well as the ability to export zone files.

My only gripe would be that the claimed support response times as well as the 'call-me-back' button don't work as advertised.

<comment><USER20> There doesn't seem to be any sort of information about their network, is it multicast? Geo distributed across the world? They mention "Access distributed nameservers across the UK and United States.", does that mean they have 1 VPS in each country serving DNS and that's it?

Seems like they have a nice UI but the lack of technical details would make me a bit wary of it.

<comment><USER19> I don't know why they still have that information there... The nameservers I use of them are in: Mountain View, US Dallas, US Amsterdam, The Netherlands Maidenhead, UK Asia Singapore Frankfurt, Germany Paris, France

And I know they have at least 5 more.</comment></comment></comment></comment>


How realtime vehicle classification can improve traveling with public transport - <AUTHOR> https://www.bastiangeneugelijk.com/ovassistant.html <comment><AUTHOR> Writer here. As someone who spends a lot of time in public transport I am always disappointed in current public transport planners, since they are 'static' and can't handle well disruptions.

I created the public transport assistant concept to explore the possible advantages of live location data within public transport and to investigate the possibility of creating a more dynamic route planner. For example, once we know in which train a passenger is, we can check when this train will arrive at a certain station and act upon it.

My next step will be to create a minimum viable product that encapsulates a basic route planning system (e.g. OpenTripplanner) and the described vehicle classification system.

Let me know what you think, feedback is always welcome!</comment>