Building high-level features using large scale unsupervised learning - <AUTHOR> http://research.google.com/pubs/pub38115.html <comment><USER4> Can we perhaps edit "singularity is near" out of the title? This sounds impressive, but having a bunch of racks able to classify the outline of a face is vastly disconnected from machine and humanity merging.

<comment><AUTHOR> It does this for 20,000 different objects categories - this is getting close to matching human visual ability (and there are huge societal implications if computer reach that standard).

This is the most powerful AI experiment yet conducted (publicly known).

<comment><USER5> > It does this for 20,000 different objects categories

With 15.8% accuracy.

> This is the most powerful AI experiment yet conducted (publicly known).

It's only powerful because they threw more cores at it than anyone else has previously attempted. From a quick skimming of the paper, there does not appear to be a lot of novel algorithmic contribution here. It's the same basic autoencoder that Hinton proposed years ago. They just added in some speed ups for many cores.

It's a great experiment though. You shouldn't detract from its legitimate contributions by making outlandish claims.

<comment><USER6> To a computer vision researcher, 15.8% on 20k categories is phenomenal.

</comment></comment></comment></comment><comment><USER7> People are missing the point here.

Yes, 15% accuracy doesn't seem great.

BUT the detector built its own categories(!). It managed to find 20,000 different categories of objects in Youtube videos, and one of these categories corresponded to human faces, and another to cats.

Once the experimenters found the "face detection neuron" and used it to test faces THAT neuron managed 81.7% detection rate(!).

Forget the singularity, and just think about how amazing that is. The system trained itself - without human labelling - to distinguish human faces correctly over 80% of the time.

<comment><USER8> > BUT the detector built its own categories(!).

It's not revolutionary. Clustering algorithms and neural nets are plenty.

Really, what differentiates this network is its scale.

<comment><USER9> And, thanks to scale, that some of the clusters corresponds to high level concepts. According to the article, earlier attempts have mostly resulted in low level concepts like "edge" or "blob" to be detected.

Also, it was (again, from the article) plausible but not a given that high level concepts could be found from unlabeled data.

That "cat" is one of the high level concept you get from using random Youtube videos as raw data is both impressive, and slightly amusing.

<comment><USER8> Exactly. Regarding your remark about edge detectors: such self-organizing neural nets are organized into hierarchical layers, and early layers' units are going to learn to become detectors of statistically common components of the input image, in the same way as the initial layers of the visual system perform blob and edge detection (retina, lateral geniculate nucleus, V1). In mathematical terms, these early units learn the conditional principal components of the inputs. The layers that are built upon these detectors, if correctly organized, are going to build upon this initial abstraction and learn more complex features: for instance to find these these edges in relative positions (to each other). Eventually, up the abstraction chain, units detect such statistically frequent features as the shape of cat's ears (common in youtube videos, I imagine), etc...

(Sorry I wrote that fast, I hope it's understandable)

</comment></comment></comment></comment><comment><USER10> There was an interesting discussion on Quora about this recently[0]

The most relevant quote being perhaps:

"The magic of the brain is not the number of neurons, but how the circuits are wired and how they function dynamically. If you put 1 billion transistors together, you don't get a functioning CPU. And if you put 100 billion neurons together, you don't get an intelligent brain."

0\. [http://www.quora.com/How-big-is-the-largest-feedforward- neur...](http://www.quora.com/How-big-is-the-largest-feedforward-neural- network-ever-trained-and-what-for)

<comment><USER11> That's an interesting discussion, but this experiment suggests exactly the opposite (perhaps that's why you included the discussion). Who knows, if we put 1 billion cores together, and fed it a massive amount of data (akin to what a baby receives as he/she matures), perhaps we would get a brain we would consider "intelligent". The fact that this system was able to pick out high- level features like "face" and "cat" without any prior training -- and with only 1000 cores, not 1 billion -- is quite suggestive that they're on to something.

EDIT: Mistyped number of cores. 1000, not 100.

<comment><USER12> "That's an interesting discussion, but this experiment suggests exactly the opposite (perhaps that's why you included the discussion)."

It absolutely does not. This experiment supports that position strongly.

What this experiment shows is that said meaningful structure can be progressively, automatically discovered.

"and with only 1000 cores, not 1 billion"

Comparing CPU cores to individual neurons is more than slightly disingenuous.

</comment></comment></comment><comment><USER13> 16,000 cores sounds impressive until you realize it's just five to ten modern GPUs. For Google, it's easier to just run a 1,000 machine job than requisition some GPUs.

See: <http://www.nvidia.com/object/tesla-servers.html> (4.5 teraflops in _one card_ )

Reminder: GPUs will destroy the world.

<comment><USER14> What a GPU calls a "core" doesn't at all correspond to what a CPU calls a "core". Going by the CPU definition (something like "something that can schedule memory accesses") a high end GPU will only have 60 or so cores. And going by the the GPU definition (An execution unit) a high end CPU will tend to have 30-something cores.

GPUs do have fundamentally more execution resources, but that comes at a price and not every algorithm will be capable of running faster on a GPU than on a CPU. If neural networks just involve multiplying lots of matrices together with little branching they might be well suited to GPUs, but most AI code isn't like that.

<comment><USER12> "What a GPU calls a "core" doesn't at all correspond to what a CPU calls a "core"."

They aren't as different as you imagine. They're general purpose programmable arithmetic units with processing rates on the order of 20-30% of CPUs, provided the limitation that they're all doing roughly the same thing.

For most machine learning tasks, that's exactly what you're doing anyway. Oh no, your neural network engine has to be parallel? C'est damage!

<comment><USER14> A CPU core isn't a general purpose programmable arithmetic unit, though. In fact what you call a "core" when you're talking about CPUs is composed of _multiple_ such general purpose programmable, as well as less general purpose memory load/store units that can still be used for basic arithmetic and a instruction fetch and scheduling system. So a core in you Intel iFOO processor is structurally equivalent to what NVidia calls an SM. Now, and NVidia SM has 48 execution units to Intels 6, but it operates at a lower frequency and doesn't have the bypass network, branch predictor, memory prefetcher, etc that you could find in an Intel core. So there are some tasks where the Intel core will be much faster than the NVidia SM, and some tasks where the NVidia SM will be much faster. And the case here does seem like one where the GPU has an advantage. But saying that the NVidia GPU has 1526 "cores" is just dishonest.

<comment><USER12> "In fact what you call a "core" when you're talking about CPUs is composed of multiple such general purpose programmable, as well as less general purpose memory load/store units"

So are GPU cores.

"But saying that the NVidia GPU has 1526 "cores" is just dishonest."

No, it isn't. You can run 1536 things in parallel at speeds that would have qualified as full cpu speeds several years prior.

Something isn't any less a core merely because it does less juggling magic, and that juggling magic is actually undesirable for a heavily parallelized task.

"So there are some tasks where the Intel core will be much faster than the NVidia SM, and some tasks where the NVidia SM will be much faster."

This conversation already has a context. Arguments which ignore that context completely miss the point.

If you don't understand how I achieved the amount of processing I did, that's fine. Playing games with the semantics of a "core" somehow magically requiring all the features of current Intel-strategy chips, though, are not going to convince me.

There is more to Heaven and Earth, Horatio, than is dreamt of in Intel's philosophy. This sort of attitude towards what constitutes the no true scotsman "a real core" is why Arm is in the process of eating Intel alive, and why Tilera stands a decent chance of doing the same thing to ARM.

This is merely extreme RISC. I realize it's sort of a tradition for the modern VLIW movement to suggest that if you can't double-backflip through a flaming hoop made out of predictive NAND gates it somehow doesn't count.

But, if you actually look, the rate of modern supercomputing going to video cards is rising dramatically.

So obviously they count as cores to _somebody_.

You also seem to have missed the point. It's not the core scale that we're discussing here. It's the dataset scale. The number of cores you throw at a problem is not terribly important; 20 years ago it would have been breathtaking to throw 32 cores at a problem, and now that's two CPUs.

What makes an experiment cutting edge is the nature of the experiment, not the volume of hardware that you throw at it. I was talking about the /data/ and the /problem/ . Predicting movie ratings is a hell of a lot harder than feature detection.

</comment></comment></comment></comment></comment><comment><USER12> The singularity is a poorly constructed myth. It is built around the presumption that intelligence is a linear function of CPU power, and that surely as CPU power rises, so shall intelligence; the problem is, that prediction was made in the 1970s, since which CPU power has risen ten decimal orders of magnitude, and we still don't have much better speech recognition than we did back then, let alone anything even approaching simple reasoning.

The ability to detect faces is not a signal that general intelligence is right around the corner.

<comment><USER14> Well, maybe. There are a whole lot of very different things called "The Singularity" and some of them are much more reasonable than others.

There's the Cambpellian Singularity, which says that we won't be able to predict what will happen next. Pretty non-controversial as far as it goes.

There's the Vingean Singularity, which says that if we ever develop AIs that can think as fast and as well as humans then due to Moore's Law they'll be thinking twice as fast as humans after 2 years, so they'll start designing chips and the period of Moore's law will fall to 1 year, and so on with us reaching infinite computing power in finite time. I think this vision is flawed.

Relatedly, there's the Intelligence Explosion Singularity (associated with Yudkowsky), which says that as soon as its AIs designing AIs, smarter AIs will relativly quickly be able to make even smarter AIs and we'll get a "fwoosh" effect, though not to infinity in finite time. I find this unlikely, but can't rule it out.

There's one I don't have a handy name for, but lets call it the AI Revolution viewpoint, which is that AIs will cause civilization to switch to a faster mode of progress, just like the Agricultural Revolution and Industrial Revolution did. This one will only look like a singularity in hindsight, and might seem gradual to the people living through it. I think this one is pretty credible.

There's the Kurzweilian Singularity, where thanks to Accelerating Change we'll someday pass a point which will arbitrarily be called the Singularity. As far as I can tell this is Kurzweil appropriating the hot word of the moment for his ideas a la Javascript.

Then there's the Naive Singularity, which equates processing power with intelligence and then concludes that computers must be getting smarter. This is indeed totally naive and not something we should worry about. I guess the linked paper is evidence that you can substitute a faster computer for smarter AI researchers to some extent, but probably not a very large one.

<comment><USER12> "There are a whole lot of very different things called "The Singularity" "

If the singularity was a legitimate concept with anything approaching experimental evidence, then this could not be true. This observation of yours - with which I agree - suggests to me that The Singularity needs a pope hat.

It is instructive to notice that all of "the singularities" are the products of science fiction authors, and in the case of the original, a particularly bad one.

There is a delightful level of schadenfreude involved in observing the multiplicity of "The Singularities." In two different ways its name says "there's only one," and yet they still can't agree on topics that are critical and fundamental to the concept itself, like the definition of intelligence, or whether or not to circumcise.

Pass the sacramental chalice, please?

<comment><USER15> Vinge is a retired math professor, I.J. Good was an accomplished mathematician, Kurzweil makes hard to swallow predictions but is still an accomplished technologist and I happen to very much enjoy Vinge's writing. Regardless, the idea is worth considering independent of who is saying it.

I am also sure you know that words - such as variety and polymorphism - <USER2> different context specific meanings. Singularity in this case as in the kind of thing you can find on a variety but not on a manifold.

The idea of infinite recursive Moore's law fueled intelligence explosions leading to super human intellects by 2030 is something I assign a low probability to. I don't find it hard to believe that there is some point in the future - say 2131 - such that if anyone alive today or previously were transported there, they would never be able to understand what was going on and _everyone_ from that time would think circles around them.

<comment><USER12> Maybe you didn't realize this, but the science fiction author bit wasn't meant as a slander. Many high quality people, and also Kurtzweil, are science fiction authors.

What I was getting at was "you realize they're writing books to make people happy for money, not doing legitimate science on that day, right?"

.

"words - such as variety and polymorphism - have different context specific meanings."

Sure. All handwaving about the rules of language notwithstanding, though, none of The Singularities have merit or underlying measurement, even if you want to talk syntax and grammar to create a seeming of academia by proxy.

.

"The idea of infinite recursive Moore's law"

... is nonsense. What would "recursion" be in the context of Moore's law? Have you even thought this over?

What, Moore's Law solves itself by going deeper into itself until the datastructure is exhausted?

.

"fueled intelligence explosions"

The science fiction part. I mean, you might as well say "fuelled by warp drives," because there's no evidence they're going to happen either. Or unicorns.

.

"is something I assign a low probability to."

This suggests that you don't know what probabilities are. Probabilities are either frequentist, which cannot happen here because we have no knowledge of the rates here (this would be like calculating the frequentist probability of alien life - it's just making numbers up,) or Bayesian, where you draw probabilities from observed events, at which point the probability is exactly zero.

So, is it undefined or zero that you're promoting?

.

"I don't find it hard to believe that there is some point in the future - <USER3> 2131"

(rolls eyes)

.

"they would never be able to understand what was going on and everyone from that time would think circles around them."

It seems you don't even need to be transported into the future for that.

<comment><USER15> 1.) Okay.

2.) Singularity as in breakdown not as in single. You purposely muddled the meaning to make your quip work.

3.) Moore's law fueled as in AI gets interest on their intelligence. Recursive as in AI makes smarter AI makes smarter AI...

4.) Science fiction or not i find it unlikely.

5.) Bayesian. Look up prior.

6.) 2131 was tongue in cheek.

7.) Thanks ;) You actually never address my main point though.

<comment><USER12> "2.) Singularity as in breakdown not as in single. You purposely muddled the meaning to make your quip work."

This is a blatant falsehood. I have solely and exclusively used it as a title for Kurtzweil's concept. It has no meaning; it's a name. I have muddled nothing. It is inappropriate for you to make accusations like this without evidence.

.

"3.) Moore's law fueled as in AI gets interest on their intelligence"

Yes, that's what I said at the outset: this whole thing is driven by the false belief that intelligence is a function of CPU time. There is no experimental evidence in history to support this, and there are 65 years of counter- examples.

Repeating it won't make it less wrong.

.

"Recursive as in AI makes smarter AI makes smarter AI..."

Oh.

This gets to a different false presumption, namely that the ability to create an intelligence, as well as that the power of the intelligence created, is a linear function of the prior intelligence.

This whole treating everything like it's a score, like it's a number you tweak upwards? It's crap.

You can't make an AI with an IQ of 106 just because you have a 104, and the guy who made the 104 had a 102.

This is numerology, not computer science.

.

"4.) Science fiction or not i find it unlikely."

I can't even tell what noun you're attached to, at this point. What do you find unlikely?

.

"5.) Bayesian. Look up prior."

What about bayesian, sir? I don't need to look up prior; I used it, correctly, in what I said to you. You're just telling me to look things up to pretend that there is an error there, so that you can take the position of being correct without actually having done the work.

There are zero priors of alien life, sir. That was my point, in bringing up what you're now blandly one-word repeating at me, in your effort to gin up a falsehood where none actually exists.

.

"7.) Thanks ;) You actually never address my main point though."

You don't appear to have one.

Maybe you've forgotten that you were replying to someone else, who already said that to you?

</comment></comment></comment></comment></comment></comment></comment><comment><USER16> The only originality in this work is the processing power used. The principle in it self is not original.

For a possible resemblance with the real cortical neural network working principle and face or object recognition, this is just a farce.

Regarding getting closer to the presumed singularity, this is like saying that cutting flint is close to making diamonds.

The authors didn't claim that, but the abusive use of "neural network" for such kinds of applications is just doing that. It is a dishonest abuse of people who can't make the difference.

The true problem is that significant quality work toward modeling real cortical neural network is drown in the sea of such faker crap.

<comment><USER17> I think much of this is overhyped as well, but I disagree that whether modeling human brain structure is relevant. The term "neural network" has historical baggage (responsible for some of the hype), but these days refers to a class of mathematical approaches with only historical connection to "neurons". Those can be interesting on their own for AI purposes, and imo accurate modeling of the human brian, while interesting for neuroscience research, is not necessarily the way forward for AI research.

<comment><USER16> My use of modeling was indeed inappropriate. What I meant is the working principle of cortical neural networks. The artificial model would then be a kind of proof of concept.

Regarding you other point, it is a matter of research strategy. I think that the path trying understanding the working principle of real cortical neural network is the shortest path to AI. My impression is that the other path which is to play around with artificial neural networks is too hazardous.

We can make a parallel with learning to fly. We are in a similar situation regarding how the brain works and AI. Understanding how birds fly require a true research. People seems to simply focus on flapping while this is not the real working principle of flight.

I see there a strong analogy with artificial neural network. The most relevant properties of cortical neural networks are ignored.

With flight the proof condition of mastering it was obvious. With AI, it is less obvious. I would be glad to hear suggestions. Face recognition is the most difficult condition because this process is the end product of many prior processes like 3D perception and feature extractions. My current impression is that talk decoding would be a much better candidate. Siri shows the potential impact of such AI product. At least the turing test would be a direct match.

<comment><USER17> I guess I take the opposite suggestion from the flight example: we succeeded in flying once we started focusing more on the physical/mathematical research of aerodynamics and lift, and less on attempting to mimic biology, in copying the biomechanics of bird wings. We ended up producing something that flies, but not in exactly the way that birds fly. That's what I tend to view as the better route for AI as well: instead of trying to copy the details of how a brain works, focus more on first-principles mathematical/logical principles of inference, whether they're symbolic ones (e.g. theorem-proving) or statistical ones (Bayesian networks, etc.).

Admittedly this is a big area of disagreement both within and outside the field.

</comment></comment></comment></comment><comment><USER18> You'd be surprised at how little precision this much computing power gets you, even on very basic classifiers. If anything, working on this kind of stuff has given me an appreciation for just how far away the singularity may be.

</comment><comment><USER19> 15.8% accuracy != singularity is near.

No more supplements-eating Kurzweil, walking Terminators and Skynet-like BS please.

<comment><USER20> I wonder what accuracy would human get if you trained him/her only with 10 million static 200x200 px images in complete silence.

<comment><USER19> I don't think this was "complete silence" in this case. They actually trained the computer (meaning they provided data as input).

<comment><USER21> scotty79 means to showing a human _only_ the 10 million 200x200 images. We probably see a lot more than that, in 3D from different perspectives with continuity in motion, and in a lot more detail and possibilities for filtering.

<comment><USER19> > scotty79 means to showing a human only the 10 million 200x200 images

Then, what kind of thing are you measuring? Recognizing patterns ? We know humans are very good at that. We can see thousands of different people everyday but we can recognize in the blink of an eye a familiar face. A computer or computer network is very, very, very far from being able to do that yet.

<comment><USER22> But why are humans are very good at that? Could it be because humans have gotten tons of high-resolution, 3D imagery with binaural audio? Whereas this computer got a handful of low-res still images. Is the solution to just throw more horsepower at the problem, or is there really some inherent quality of the brain that's different?

edit: this quote puts things into perspective a bit "It is worth noting that our network is still tiny compared to the human visual cortex, which is 1,000,000 times larger in terms of the number of neurons and synapses."

<comment><USER21> Also, consider training time. I doubt babies are recognizing 20k objects at 15% success rate after just a few days. Though to really compare that, the speed of the brain vs the speed of the supercomputer has to be normalized for, and training data and network size has to be similar as well of course for any _real_ comparison.

</comment></comment></comment></comment></comment></comment></comment><comment><USER23> Maybe I'm missing something here, but how exactly is it "unlabeled" data if they're specifically feeding it millions of pictures of "faces"? I mean, if you make a specific selection of the type of images you train the network on, isn't that basically equivalent to labeling them?

<comment><USER24> The aim of the paper was to produce an unsupervised system that would generate high level features from noisy data. These high level features could then be used in supervised systems where labelled data is added.

Thus, the paper is about using an unsupervised system to help a later supervised system. An advantage of this is that, as the unsupervised system isn't trained to recognise object X, it instead learns features that are discriminative. This same network could be used to recognise arbitrary objects (which is what they do later on in the paper with ImageNet).

<comment><USER25> In other words: imagine a baby. She sees 100k "images" of faces. Thanks to the statistical regularity of the world, she now has a "subsystem" that recognizes a face _in the absence of her knowing what it's called_. Then, when she is told "this is a face" she pins this thing pointed-to to the existing, unnamed representation.

</comment></comment></comment><comment><USER26> While I'm excited about progress, 15.8% accuracy is not exactly "Singularity is near"

<comment><AUTHOR> They just have to scale it up - more computers, more days, and the accuracy level should increase accordingly (that is my intuition and hope on this, though i could be wrong).

<comment><USER27> Haha, no.

\--- last company was in computer vision.

<comment><USER28> "last company was in computer vision."

Did you sell, leave or did it fail? Why? I have some ideas that I think are novel applications of computer vision, and just within the range of what's feasible, but it seems that most computer vision applications look like that at first, and then after 90% done find out that the second 90% is exponentially harder and, realistically, infeasible. How could I test my ideas against that? Or am I asking from wrong premises?

<comment><USER27> Company was doing well and had a good idea. Product worked great, we did our job. The problem is, management didn't.

I left after all the other engineers did.

</comment></comment></comment></comment></comment><comment><USER29> This isn't singularity material. While this may not be a bog standard neural network, it has no feedback. It cannot think, because thinking requires reflection. It is trained by adjusting the weights of the connections after the fact using an equation.

Is it cool, and perhaps even useful? Yes. But don't confuse this research project for a precursor to skynet.

<comment><AUTHOR> I put the singularity bit in to make it relevant to people who would otherwise not get the significance of this (which is that large scale neural nets can work - something people have been trying and failing at for decades).

<comment><USER30> Or you put it in as a cheap link-bait tactic.

<comment><AUTHOR> The site is google research - I have no ads on it. This paper's been out for weeks but no mentions anywhere - thought i would give it a deserving push.

</comment></comment></comment></comment><comment><USER31> "We also find that the same network is sensitive to other high-level concepts such as cat faces..."

"Our training dataset is constructed by sampling frames from 10 million YouTube videos."

</comment><comment><USER32> If anyone is interested to read more on this topic, there is another recent, closely related and perhaps slightly more accessible paper ( "High-Level Invariant Features with Scalable Clustering Algorithms" <http://bit.ly/KDuN04> ) from Stanford that also learns face neurons from unsupervised collection of images (disclaimer: I'm co-author). It uses a slightly different model based on layers of k-means clustering and linking, but the computation in the end is very similar.

I'm familiar with both models so I can also try to answer any questions.

<comment><USER33> I think your comment <http://news.ycombinator.com/item?id=3838971> is very relevant to this discussion.

</comment></comment><comment><USER34> I'm seriously considering quitting my job and studying ML for a few months in a desperate attempt to get work in projects like this. I feel like I'm missing out but too dumb for traditional grad school.

<comment><USER35> That's what I did a few months ago - quit my job and decided to go to a grad school to study AI (with focus on neural nets and ML).

<comment><USER36> What did you learn so far about neural nets? I recently looked into machine learning and naively thought I could find at least one practical fun tutorial "Here is a neural network API in C, you have to do that and this to let a simulated robot evade obstacles or learn to play Asteroids". Instead my (extremely superficial) search did find that neural nets are pretty arcane, genetic algorithms trapped in local minima and you are faster and better of coding logic yourself, developing a mathematical model to calculate results, instead searching for patterns in vast sets of data.

<comment><USER37> SVMs are a relatively easy to use (but not to understand) method that yield impressive results for beginners. See the libsvm website, there are plenty of good material there (<http://www.csie.ntu.edu.tw/~cjlin/libsvm/>). But as stated somewhere else here, game AI is a whole different story.

</comment></comment></comment></comment><comment><USER38> Fascinating, exciting times we live in. I'll upvote the link as soon as the "singularity" silliness is taken out of the title.

</comment><comment><USER39> From the article: "It is worth noting that our network is still tiny compared to the human visual cortex, which is 1,000,000 times larger in terms of the number of neurons and synapses."

</comment><comment><USER40> > _the dataset has 10 million 200x200 pixel images downloaded from the > Internet_

They take the frames from YouTube. It is weird to me that YouTube, (derided as a way of sharing funny cat videos) is able to contribute something actually useful to the world.

<comment><USER41> Youtube contains a lot of educational material you would not find otherwise. If I wanted to learn sewing on a sewing machine, I would just watch some video tutorials - try that with a book on sewing. Same thing with instructions on how to play instruments. Heck you can even watch videos on how to fix problems with your car engine. Many procedural instructions can't be transported properly via papers or books. It also allows asynchronous video messaging for laymen asking experts stuff which is difficult to get via written text. I bet that youtube will contribute very much to knowledge preservation and distribution in the long term.

<comment><USER40> I hope that Google look after it a bit better than they've looked after their Usenet archives. Google groups search is a particularly frustrating experience.

I agree that there is some great content on Youtube. Interesting that you mention sewing machines, because that's something I've used and they are particularly helpful. (See also all those other crafting videos; latch-hooking etc.)

</comment></comment></comment><comment><USER42> As far as I can tell, this is let's train a huge number of models and then cherry-pick few that works well on a test set, so an overfitted junk. What have I missed?

<comment><USER43> It seems like they are cherry picking the one that works well on the training set no? Where did you get the sense that they were doing it on test?

<comment><USER42> The train is not labeled, so it is not possible; and they do not mention that the labeled set was split or used in validation -- it is just called "test".

<comment><USER43> "We followed the experimental protocols specified by (Deng et al., 2010; Sanchez & Perronnin, 2011), in which, the datasets are randomly split into two halves for training and validation. We report the performance on the validation set and compare against state-of-theart baselines in Table 2. Note that the splits are not identical to previous work but validation set performances vary slightly across different splits."

<comment><USER42> As I understand, this is only about this side experiment with ImageNet data which uses logistic regression on those neurons in some cryptic way; I was trying to comprehend the core work (faces) before that.

<comment><USER43> Well, that's the record breaking bit.

</comment></comment></comment></comment></comment></comment><comment><USER44> We need to harness this neural net to improve kittydar:

<https://news.ycombinator.com/item?id=4116990>

</comment><comment><USER45> The singularity is already here. There are black holes in our universe. This submission title really annoys me.

<comment><USER46> Even if you are trolling, I think I'll leave this here. <http://en.wikipedia.org/wiki/Technological_singularity>

<comment><USER45> I'm not trolling, I've read Keurzweil's book... It's extrapolating. If I was a risk assesor, I'd err on the side of a mass extinction event over us consuming the entire universe as data.

<comment><USER47> There's already been 4 extinction events, it's pretty safe to bet that it will happen again.

But they happen 50-100 million years between each other and usually take thousands of years to take full effect once they begin.

Even if technological singularity takes an extra 100-200 years to really happen, if any significant 'AI' is achieved, a lot could happen in a thousand years, let alone a million.

<comment><USER12> For what it's worth, your ability to post that is based on that they were actually near-extinction events.

</comment></comment></comment></comment></comment><comment><USER48> Is accuracy referring to recall, precision, or some other measure?

<comment><USER49> When there is no possible difference between recall & precision, you report one figure, accuracy.

</comment></comment><comment><AUTHOR> Google Research and Stanford researchers test out a 1 billion connection, 9 layer, 16,000 core deep learning neural network (see geoff hinton and andrew ng talks on youtube) to recognize 20,000 different objects in images (with low accuracy but huge improvement over previous approaches).

This was done no with no pre-labeled images (except for fine tuning)! A brain that learned from raw images. The same algorithm can be applied to any data type (financial data, text, audio, images/video) without any human involvement (except gatheting of unlabelled data and running the system). Pretty much the artificial intelligence holy grail!

<comment><USER50> "Singularity is near"? "Holy grail"? You may be getting a little carried away here.

The outcome shows a very nice improvement on an unsupervised classification and feature detection task, but it also highlights that unsupervised machine learning still has a long way to go. 16% accuracy from a network with 1bn connections and 100m inputs using (if my math is right) 1.15m hours of CPU time. Which of these would be the easiest way to continue making gains: investing more time/hardware, increasing the complexity of the model, or developing a new and improved algorithm altogether? All of these sound pretty intensive to me.

<comment><AUTHOR> If the algorithm keeps increasing in accuracy as you scale up computation and add more unlabeled data that is pretty amazing. You might get something that matches human performance on vision/speech recognition etc.

<comment><USER51> If you extrapolate that way you'd conclude that naive Bayes is the solution to AI. Improvements tend to tail off fairly quickly as you add more data and computation, unfortunately.

</comment></comment></comment></comment><comment><USER52> The added singularity is near part is silly. In ten years when that's a 500,000 (or 5 million) core neural net, the singularity is near part still won't apply.

</comment><comment><USER53> I thought that the best so far was simulating 1/2 a mouse brain compextity at 1/2 speed.

<comment><USER54> Where can I find this study?

<comment><USER21> He might be referring to blue brain, though I recall that simulated 100k neurons from the neocortex (not quite half a mouse brain). The thing is that they simulated the physical processes in the synapses and more for a more accurate representation, so it's actually a lot cooler than it sounds!

Here's a link I googled up: <http://bluebrain.epfl.ch/cms/lang/en/pid/56882>

</comment></comment></comment><comment><USER55> "we trained our network to obtain 15.8% accuracy"

Film at 11

<comment><USER56> Look at this way: for a 20000 category classification problem, guessing randomly will give you 0.005% accuracy. Compared to chance, this is pretty good.</comment></comment>


Why is it difficult for developers to learn another programming language? [pdf] - <AUTHOR> http://nischalshrestha.me/docs/cross_language_interference.pdf <comment><USER2> I find the opposite is true. Learning the first programming language was hard, the second was easier, the third and fourth easier still. At some point you develop intuition about what computers can and cannot do in a universal sense and then you realize every language is just different nuances and emphases on those capabilities, and it becomes pretty easy to pick up a new one very quickly.

Edit: after reading the paper I understand the emphasis is on conceptual mistakes people make when coming from one language into another - you assume things work like A and make mistakes when B works differently under the hood and you don't realize it until banging your head against it for a while.

That experience does ring true, but I stand by my original comment, this is mostly only an issue for language #2, and with each language it gets easier as you begin to think more in universal terms and have fewer assumptions and gaps where you've been accustomed to something that "just works" in the environment you're most familiar with.

<comment><USER3> My biggest problem learning a second language is that it's so much less useful than the first.

Ten years ago I learned C# and since then I've dabbled in other languages like Haskell or Lisp which were a little fun, but I keep getting drawn back to learning more about my stack. There's still decades of knowledge I lack about Azure/Entity Framework/Asp.Net/Xamarin/Sql Server/WPF/C# CMS/C# CRMS/C# e-commerce/Roslyn the opensource compiler/Blazor/Typescript/React/CSS. (Not to mention all of the useful soft skills, like sales/ marketing/negotiation/management)

And I'd rather be able to build out a desktop app, a web app, or a mobile app than be able to build out a web app in js-node/C#-mvc/ruby-rails/python- django.

Basically when I learn I want to be able build I couldn't before (like a mobile app) or get better at building(learn how to better optimize t-sql queries). And when I've spent time learning new languages I felt like it wasn't as useful at delivering on these two goals.

<comment><USER4> Maybe look at F# for something .net compatible. Personally its use in some contexts, particularly those using the F# type providers has served me well.

I also prefer to use FSX scripts over CSX in contexts where that is useful. I find my deeper knowledge of the .net infrastructure easier to leverage than having to learn powershell but FSX syntax also cleaner for scripting.

<comment><USER3> I've used f#. Type providers were really cool. And I like how terse it is but I feel like the gap is narrowing, and I couldn't ever use it on a real project. I'd feel guilty building a project for a client that is supportable by a tenth the number of developers without very good reason.

</comment></comment></comment></comment><comment><USER5> Is it? My personal experience makes me think that it isn't. I've programmed professionally in eight languages if you count significant line changes, ten if you count any change. At this point changing editors is harder than changing programming languages.

<comment><USER6> Your first language is hard because you're learning a ton of things all at once, only one of which is the language itself.

Your second language is hard because it's not the first, even though it's probably pretty similar. But the differences will catch you out. A second language challenges the mental models you built up with the first language -- you have to tear some of your understanding down and come up with a more generalized perspective.

Languages similar to the first two become easier to pick up after this point, because you're mostly looking for a handful of differences, not a total paradigm shift. But languages that are sufficiently different, like C and Haskell, require a lot more of the ground-up type of work you thought you'd left behind by now.

It gets easier and easier as you cover more of the conceptual landscape. But I'd be willing to bet that many developers haven't had to journey far from home.

<comment><USER5> I'd agree that similar languages are easier to learn, but once you've learned how to learn a programming language, new ones are easier, even if they're dissimilar from your prior languages.

For the record, my professional work includes languages as dissimilar as PHP, Java, and Common Lisp, while my hobby hour work includes Rust and Haskell. That's a pretty wide range of the language space that exists today, IMO.

<comment><USER6> My language gamut is similar to yours -- I don't disagree. I usually find it straightforward to pick up something new too. But there are still languages I know are going to take a lot more time for me to grok, like Coq. (I tried Coq a few years ago, and even though I already knew a fair bit of Haskell, the differences were enough to completely throw me off.)

</comment></comment></comment></comment><comment><USER7> It's easy to learn new languages, hard/slow to learn _the ecosystem..._

We should get better at creating mixed-ecosystems and cross-language libraries... the JVM was a step in the right direction but most new development _completely forgot about that!_

<comment><USER8> Agreed. There is also another problem: For example, I know that I can code with C# but I do not know what is the traditional workflow for designing a micro-service in .NET

You just do not know how to "Do things in the right way"

</comment></comment><comment><USER9> The pure language features itself, is not hard to learn. This is just the tip of the iceberg.

It's all the frameworks, and their undocumented features, that sometimes doesn't make sense, but it's the way it is done, which is what makes learning new languages hard to master.

<comment><USER10> On top of that there is also understanding the philosophy of a language. Learning python as my 3rd productive language was 3 full-time weeks, "Doing things the pythonic way" took years.

<comment><USER11> That is also why getting into frameworks is a bit hard for people who are used to do things without frameworks. Like people who stick with JQuery and find Angular hard, because you have to do stuff "Angular way". You can write a lot of javascript and switch to typescript but "Angular way" is yet another thing to learn.

I can write python and understand the code, know some quirks but for large code base I would probably be lost because I don't know "python ways".

</comment></comment></comment><comment><USER12> I think the title should have been: When it's difficult to learn a new language, why is that?

Some paths are easier than others so it really depends on the new language and where someone is coming from. I started with PHP/JS on a self taught basis. Then I started to develop interest in other languages. Python was a no- brainer, C++ looked quirky but okay, in Java most thing looks familiar. But functional languages get me. Rust is another beast, but totally looks tamable.

Haven't finished the article yet, but it has some great insights.

</comment><comment><USER13> My anecdotal data appears to point to the opposite. My first language (JS) involved learning:

\- What is even programming? What is a function? an argument? What is state? What are types? \- JS itself \- How browsers work \- JQuery (do people still use that?) \- NodeJs \- How backends work \- How databases work

Then, the subsequent language, Java, was only different in patterns, paradigms, keywords, and frameworks (at least, at the amateur level I was going at). Args were still args, strings were strings, sure we had classes instead of functions here and there, but for a beginner it was close enough. Java, the second language, was far easier to learn. And so forth, with each language becoming progressively simpler.

<comment><USER14> JS is a really horrible language to learn from the perspective of a teacher.

So many concepts dumped on you all at once. Before you even know what a function is, you have the entire DOM API dumped on you and is available in the global scope.

If you try to learn OOP in it, you have the choice of trying to grapple with the leaky ES6 class keyword or you get prototypal inheritance and `this` will come up.

That said, from a learner's perspective it's great: you have the entire DOM available to you instead of having to learn how to take the car of the cdr of a list and wondering how that has anything to do with building the next Airbnb/Uber of this world...

At least with Python/Ruby they commit hard to terseness / fluent interfaces so you have less syntactic noise to deal with, and Haskell commits hard on equational reasoning so you can pick up computation if you've come from a mathsy background. Or Scheme which is as close as you can get to "no syntax" as you can (IMO)

<comment><USER12> I used to teach programming with JS. It was not really a blocker for me at any level. When we hit some quirks I stopped to explain why this is and how it's different in other languages.

Maybe because I like JS made me enjoy teaching it as well :)

<comment><USER14> I actually quite like JS, but I almost feel bad for people who discover it as their first foray into programming.

I mean it's great because it makes programming more accessible to non devs. But man is it unfortunate that JS was the language that ended up being the language of the web.

The browser is the ultimate interactive development environment. You can literally play with the page in situ. Akin to CAD and Excel (...or Emacs[0])

[0][https://www.gnu.org/gnu/rms-lisp.html](https://www.gnu.org/gnu/rms- lisp.html)

<comment><USER12> Don't feel bad, my first language I've ever written programs in was C-64 basic :)

I don't know what would have been a better choice, but JS is not bad for a browser language. It had a rough childhood, but turned out to be a decent adult.

I still remember VB Script supported in old IE, but then there was JS and I just focused on that. It really made no difference to me since I was able to build cool stuff I wanted to. That brought me joy of creation and that's what mattered, not "that's not OOP like god set in stone".

Whatever enables people to be more involved in programming, I consider it as a good thing.

</comment></comment></comment></comment></comment><comment><USER15> Can we just call "cross-language interference" an "accent"?

When I code VBA macros, I pile up all of the "End If" and other compiler noise in column 80, with comments.

Because I'd really rather be coding Python.

</comment><comment><USER16> Title is a little misleading as it seems to imply that programmers find it difficult to learn another language. But there is no such implication in the study. The study really seems to be about specific language features that can contribute to making it harder/easier to pick up.

</comment><comment><USER17> What i struggle the most when coming to a new language is the tooling that comes with it. And how do I structure my code in this new language. Sure some things are universal, but some things are just hard to find out. The js ecosystem is an especially tough example for this.

</comment><comment><USER18> My CS college program made me take a programming language fundamentals class. I still remember it. The class was one of the most difficult classes. We had to learn 4 programming language paradigms: imperative, declarative, object- oriented, and functional. In each paradigm, we learned to write programs in a chosen language.

I find the experience very useful for my software career. It helps me learn new programming languages quickly. When I see functional operators, I remember Haskell.

I'd recommend to master one language in different paradigms. It pays dividends.

[https://en.wikipedia.org/wiki/Comparison_of_programming_para...](https://en.wikipedia.org/wiki/Comparison_of_programming_paradigms)

<comment><USER19> I had a class that was even more intense, CSC324H at the University of Toronto. We learned Scheme emphasizing on higher-order functions, ML emphasizing on static typing, Haskell emphasizing on lazy evaluation, and Prolog emphasizing on {Horn clauses, unification, backtracking}. All in the span of a 12-week semester.

</comment></comment><comment><USER20> Circumstances and expectations might be different. When starting on your first language, was different than the second, and different from your third, etc.

Your first language, you know you don't know much. So you try out and play with it. If you're unlucky, you get stuck, don't get the books, software or help that would make you proceed (ie. machine code for C64), so would need something else or give up altogether. For some experiences, you just need to know the right people, be in the right circumstances.

On your second language, you probably already seen program listings in several languages already. You expect to get up and running after a few hours. This might happen if the paradigms are the same. For a completely different paradigm, you could need to grind it over time or start over as a beginner in that paradigm again.

Also, for many, learning new languages, frameworks etc. are not the fun part anymore, so get less emphasis on playfulness and spending time with it. When preference is on being productive, one might miss many big opportunities.

</comment><comment><USER21> I think this depends a lot on the developer.

Some Devs I work with want to dabble with many tools and solve problems by picking (or creating) a new tool rather than learning how to solve problem with existing tool. They learn, and are talking about (and trying to get team to adopt) other languages all the time.

Personally, I want to totally master a small number of tools. (perhaps just my inability to master many). So, it's not that I find it hard to learn new lang. It's I'm unwilling to waste effort on it unless there is advantage. And to overcome the mastery I have attained after 25 years of Python that advantage has to be large. [btw Past langs include VisualBasic, Pascal, PHP, C, C++, and PERL. But only PERL did I use long enough for mastery]

Neither is obviously better/correct. Like I said, it depends on how the developer best works.

</comment><comment><USER22> My anecdotal evidence is exactly opposite to the paper. I would expect the title of the paper to appear as a headline trotted out in some mass media publication with the article written by a journo that dabbled in BASIC in high school and decided that programming and software development wasn't for him. Maybe I've forgotten what it is like to not be able to program, in the same way I don't understand how to not swim, or how to not ride a bike, or how to not read and write, but learning a new language, learning it well, to be facile in the language and the standard APIs that come with the language (ignoring the tens of thousands of obscure little npm packages in javascript for instance) is not a huge amount of work.

</comment><comment><USER23> What is really important when learning a new language is just forget any idioms you learned the last time! Know how computers work, and have a general knowledge about memory management, data structures, algorithms and clean code, but try to understand the new language's idioms. You will have an easier time and make less mistakes.

For example, there are typical mistakes when a C or Java programmer uses Python for the first time, I can tell somebody's history just by looking at what idioms they try to force on Python which shouldn't.

Example for C programmer writing Python:

        for i in range(len(mylist)):
             print(mylist[i])

Example for Java programmer writing Python: Starting hello world with "class HelloWorld:". LOL You don't need to start with classes and objects in Python!

</comment><comment><USER24> Not really true, it gets easier the more languages you learn. I do know of developers who spent their whole working lives using only one language and they are very good at that one language. (I am thinking of MUMPS and the developers I met when working in the medical s/w field) Should we just learn one language and do we gain by knowing many and why are there so many languages that do the same thing? If there were to be one language what would it look like? [https://en.wikipedia.org/wiki/MUMPS](https://en.wikipedia.org/wiki/MUMPS)

</comment><comment><USER25> I write JavaScript and python daily. I'm fairly fluent in many languages including C++.

The only language/platform I'm having difficulty with is Swift and MacOS (Cocoa)! It's a pile of ancient junk and 5 versions of a new language with absolutely turd documentation from the platform and a gigantic pile of "meh works for me". It feels like trying to run in moving sands with your eyes closed and your hands tied.

I'm only saying this to make a point: sometimes it's not the developer having a hard time as much as the ecosystem making it a living hell.

</comment><comment><USER26> In terms of your career, it's better to become an expert at one language than a jack of all trades. I can quickly move from language to language, but make the most money as a consultant for languages in which I'm an expert.

I also think many developers use a single language for work and don't have much time after work to learn another one.

</comment><comment><USER27> It isn't in my experience. A list of languages I have used professionally: C, C++, Java, C#, Javascript, Typescript, Assembler (PowerPC), SQL, ... And for fun: Basic, Pascal, Haskell, Lisp, Scheme, Idris, Assembler (Z80 and I86), ... Only Haskell was hard to learn (but most definitely worth it).

</comment><comment><USER28> In my opinion it is all down to muscle memory. When you don't even need to think about whether this class will behave one way or another, you just know that if you type it this way it will work without a doubt.

</comment><comment><USER29> I mostly get stuck on the function, func, fun, void, def, etc.

Most other stuff is very similar.</comment>


French Artist Creates Trompe-L'oeil Illusions - <AUTHOR> http://designyoutrust.com/2016/07/french-artist-creates-mind-blowing-trompe-loeil-illusions/ <comment><USER2> When one designs self-driving cars without LIDAR, one should remember that illusions like these exist.

<comment><USER3> So true... [http://imgur.com/gallery/mOTHgnf](http://imgur.com/gallery/mOTHgnf)

<comment><USER4> Nasty. That's totally irresponsible. At night in an unfamiliar neighborhood that could easily cause a fatality.

<comment><USER5> Then consider this ... :-/ [http://s-media-cache- ak0.pinimg.com/736x/fd/c3/02/fdc3028d49...](http://s-media-cache- ak0.pinimg.com/736x/fd/c3/02/fdc3028d4946bef8357eadad6db2fe8c.jpg)

</comment></comment></comment></comment><comment><USER3> There is a nice Trompe-L'oeil in Madrid called "Los Pacos" where two former neighbours are represented on the collapsed side of a building: [http://madridfotoafoto.blogspot.com.es/2008/06/trampantojo-e...](http://madridfotoafoto.blogspot.com.es/2008/06/trampantojo- en-la-plaza-de-los-carros.html) (Spanish)

This website features more examples of this kind of works under the label "trampantojos".

</comment><comment><USER6> A commenter pointed out that one or two of these images appear to be photoshopped, most likely mock ups of a scene that the artist intended to create. The commenter took a lot of heat, but does appear to be correct. The fifth-to-last image (the before) and fourth-to-last (the after) show it pretty clearly if you check out the clouds.

<comment><USER7> Possibly, but also maybe the other way around. Perhaps they didn't have a before picture of the scene from the right angle, so the before one is photoshopped?

<comment><USER6> True, I hadn't thought of that!

<comment><USER8> Although the first hypothesis looks more plausible.

</comment></comment></comment></comment><comment><USER9> So much nicer than filling the space up with billboards. It's a bit of a shame that the default artwork for a lot of outside space is paid advertising.

<comment><USER10> There are a couple countries where it is common to have intricate art on the outside of certain large vehicles. Like private buses in Haiti[1] or trailer trucks in Pakistan[2]. I don't know if there would be any possibility of renting out the surface area to advertisers instead, but I did hear a podcast once analyzing the economics of paying to have the private busses in Haiti decorated. If I recall, it basically came down to a feeling in the population that it was more prestigious to go through the city in something beautiful. Perhaps proof of the "value" of art that works even within a first-year economics class.

[1] [https://www.google.com/search?q=painted+bus+in+haiti&source=...](https://www.google.com/search?q=painted+bus+in+haiti&source=lnms&tbm=isch&sa=X&ved=0ahUKEwiy6Ln7zPrNAhVJ9h4KHdIfBfwQ_AUICCgB&biw=1339&bih=778)

[2] [https://www.google.com/search?q=pakistan+truck&source=lnms&t...](https://www.google.com/search?q=pakistan+truck&source=lnms&tbm=isch&sa=X&ved=0ahUKEwi1qqTfzPrNAhXFrB4KHRYLC3wQ_AUICCgB&biw=1339&bih=778)

<comment><USER11> In Argentina it was usual that the bus had "fileteado". It was a 10 or 20 years ago, and the reason was that the bus lines were something like a cooperative and each bus driver was the owner of his bus. But in the recent times, the bus lines have a more usual company structure and the buses are owned by the company, so each bus has a standard color. Anyway, they still have some internal custom decoration, so in many cases you can recognize which bus you take if you use the line frequently. Also it's usual now to have advertising outside the bus.

Wikipedia: [https://en.wikipedia.org/wiki/Fileteado](https://en.wikipedia.org/wiki/Fileteado)

Autotranslation of the Spanish article with more images: [https://translate.google.com.ar/translate?hl=es-419&sl=es&tl...](https://translate.google.com.ar/translate?hl=es-419&sl=es&tl=en&u=https%3A%2F%2Fes.wikipedia.org%2Fwiki%2FFileteado)

Even more images: [https://www.google.com/search?q=fileteado+colectivo&tbm=isch](https://www.google.com/search?q=fileteado+colectivo&tbm=isch)

</comment></comment></comment><comment><USER12> Trompe-L'oeil works great indoors, too. You can inexpensively add all kinds of detail, such as coffered ceilings, without having to pay a carpenter! A favorite is making the ceiling look like it is open to the sky. I've seen it used to repair badly damaged marble, it was so good you had to look really close to distinguish the paint from the marble.

</comment><comment><USER13> There is a well known Trompe-L'oeil in Chamonix as well: [https://gryllosblog.files.wordpress.com/2015/06/img_7624-pai...](https://gryllosblog.files.wordpress.com/2015/06/img_7624-painting- on-house-chamonix.jpg)

<comment><USER14> That's a different angle on the last one in the article.

</comment></comment><comment><USER15> Wow, that is really wonderful. Really makes the city look more lively.

I really like the Trompe-L'oeil in Brussels, when I visited their once. They had famous characters like Tin-tin. It made the city express that it had culture, just even during the first glance.

</comment><comment><USER16> Really Wonderful !

</comment><comment><USER17> Wow!</comment>


Aussie coffee entrepreneurs continue to spread - <AUTHOR> http://internationalbs.wordpress.com/2011/02/07/more-on-the-international-aussie-coffee-influence/

<comment><USER2> Usually ordering a "flat white" while abroad results in either blank stares, or laughter. It'll be nice to just get a coffee.

<comment><USER3> "Long black" almost works though.</comment></comment>


Show HN: Delivering Image Advertisements in Minecraft - Built with Meteor.js - <AUTHOR> https://www.adventurize.com <comment><AUTHOR> Hi,

Founder here, happy to answer any questions on our Minecraft In-game Advertising.

Thanks for having a look!</comment>


What Do Blind People Actually See? - <AUTHOR> http://nautil.us/blog/what-do-blind-people-actually-see <comment><USER2> As someone who has been blind since birth this is accurate. I know that people who go blind later in life continue to dream in color since they have the memory of color to reference while I do not. I wonder if someone who goes blind later in life experiences things differently, possibly by automatically filling in what they assume they would see based on what they know of there surroundings from there other sences.

<comment><USER3> I am not blind, but I do occasionally get migraines with visual auras, as do my wife & mother. Exactly once in my life that I recall, I had a migraine that caused total perceptual blindness (oddly, this has also happened exactly once to my wife, before we knew each other). It's extremely odd- one's visual field (at least in our cases) steadily contracts, until there's just nothing left. It's not an annulus of darkness closing in, it's juts nothing- the field you can see is smaller, and there's still nothing outside of it, just like there always isn't. Like what the back of your head looks like, as the article says.

In the end, you literally see Nothing. It's truly bizarre[1]. It's not an experience I particularly want to repeat, but I'm kinda glad it happened just once, for the empathy value. It's not really something that can be effectively described if you haven't (not) seen it yourself; even now, I have trouble re- imagining the experience, 'cause my memory wants to fill in blackness or grayness or whatever. But at least in theory I now know exactly what the Nothing from _The Neverending Story_ (doesn't) look(s) like. :)

[1] What makes it even more bizarre is that migraine blindness is a strictly neurological effect- there's no damage to your eyes or optic nerves- which means you can get temporary blindsight. In my wife's case, this meant she managed to walk a couple blocks home from a friend's house, crossing suburban streets and so forth, totally normally, while all the while convinced that she was completely blind. Which, as far as conscious perception goes, was the truth!

<comment><USER4> _It 's extremely odd- one's visual field (at least in our cases) steadily contracts, until there's just nothing left. It's not an annulus of darkness closing in, it's juts nothing- the field you can see is smaller, and there's still nothing outside of it, just like there always isn't._

This is the first time I've heard of anyone else experiencing this phenomenon besides myself. Have there ever been any long-term effects on you?

Mine is slightly different. I don't have migranes, for one. Also, in addition to experiencing exactly what you described (minus the migrane), sometimes I experience the inverse: I can no longer see whatever I look at, but I retain peripheral vision. For example, if I were to look at the "o" in the word "for," I wouldn't be able to see the entire word. I could still read, with difficulty, until the effect became so strong that reading was impossible. (I still had peripheral vision, sort of, but eventually paragraphs became unseeable rather than individual words.)

Whenever any of these things occurred, I was completely normal again within two hours. It was bizarre, and it's mystified myself and two eye doctors. It's only happened less than a dozen times, but it's been happening since I was very young, and my most recent was less than a year ago.

The "blindness" was exactly as you describe: not a blackness, but a nothingness.

Do you happen to have any more info about the condition? It's very elusive.

<comment><USER5> I suffered something similar, just once, about twenty years ago, in high school, and never since. Although, instead of an impression of nothingness, it felt more like a Photoshop content aware fill, similar to the optic nerve blind spot, but moved to the fovea.

I was walking the hall to my first period, mainly looking at the bland floor tiles, and didn't realize anything was abnormal. When I happened to look up at the faces of fellow students, I noticed they had no faces, just skin colored smoothness. My hands, and text on the posters on the walls appeared in my peripheral vision, but as soon as I looked directly at anything, it seemed to disappear into an average of it's surrounding colors. I didn't freak out, but calmly walked to my first period, where I was scheduled to take a test. I made way to my desk, and marveled at this oddness, but didn't say anything, until the test was passed out and I could tell it was a page of text, but, of course, the words disappeared upon inspection. I was really impressed how useless my peripheral vision was in dealing with text or faces.

I had an awkward conversation with the teacher, who I feared thought I'd invented a disorder to get out of the test, before being allowed to visit the nurse.

The condition lasted maybe an hour. The ophthalmologist I saw shortly after found nothing wrong with my retinas or eye pressure, and declared it a visual migraine and let me go, with a newfound empathy for those with macular degeneration.

<comment><USER6> The content-aware fill is what I get as well -- I have some permanent blind areas in my right eye due to optic nerve damage; I lost a lot of peripheral vision, but also the normal optic nerve blind spot is quite a bit larger in that eye -- about 1/4 of the height of my visual field, and half as wide -- fortunately still not affecting the central point of focus.

Obviously no black patch or anything like that; I first noticed the missing section (before I had visual field tests done to detect it properly) while driving, stopped at a light; I rubbed my other eye momentarily, and the traffic light vanished.

</comment></comment></comment></comment></comment><comment><USER7> First of all, be aware that the group that's actually totally blind is relatively small. Most people who are legally blind have some form of light perception, or can see something. I've been totally blind since birth.

Blind people can have a good or bad sense of direction, just like their sighted peers. I think the device described in the article might be more useful on short distances and less relevant for knowing where your home is while you're far away from it. This because blind people don't have the visual queues to determine if they're walking in a straight line for example. Getting immediate feedback could help with such skills and learn them how to verify the signals from the device with other senses.

Sensory substitution, aka how to replace input from one sense with input from another is a quite interesting topic.

<comment><USER8> > I've been totally blind since birth.

Please forgive me for the naive question, I'm probably full of misconceptions about blindness, but out of pure curiosity, how do you read and write so well without seeing? Is the technology so advanced today? Are you using a screen reader and a regular keyboard? (e.g. typing and than playing it via the reader) how do you know if you made a typo? (e.g. who's and whose sound the same)

You really didn't have any typos, and as well your indentation and styling are perfect (much better than me but I'm not a native English speaker)

Can you read and write code without seeing? Are there any blind programmers that you know of.

again asking out of pure naivety

tl;dr, I'm impressed with your reading and writing considering that you said your are totally blind, and curious how do you do it. (I always was afraid of going blind for some reason, and afraid I will never be able to write code again)

<comment><USER7> I'm not a native English speaker either.

> Is the technology so advanced today? Are you using a screen reader and a > regular keyboard? (e.g. typing and than playing it via the reader) how do > you know if you made a typo? (e.g. who's and whose sound the same)

Yes, I'm using a regular keyboard and a screenreader. Besides speech output I use a device called a refreshable braille display which can display up to 40 or 80 characters at a time depending on the model being used. I'm sure you will find some pictures if you Google it. However, those devices are quite expensive and many blind people just use speech output. Even with just speech there are options for reading letter by letter, word by word etc. However, I see some people make much spelling mistakes because they just listen and not rreally read how a word is spelled.

> Can you read and write code without seeing? Are there any blind programmers > that you know of.

Sure you can. I make a living by programming and doing web accessibility consultancy. Another nice example is the free and open source NVDA screenreader[0], which is developed by two blind guys.

[0]: [http://www.nvaccess.org/](http://www.nvaccess.org/)

<comment><USER9> Fascinating, thanks.

At what speed do you replay text->audio (ie realtime, 1.5x,2x,...)?

Lastly, if you disregard price is the braille 'refreshable display' a better experience?

<comment><USER7> > At what speed do you replay text->audio (ie realtime, 1.5x,2x,...)?

Approximately the speed that the guy at the beginning of this video uses: [https://www.youtube.com/watch?v=92pM6hJG6Wo](https://www.youtube.com/watch?v=92pM6hJG6Wo)

> Lastly, if you disregard price is the braille 'refreshable display' a better > experience?

It just what you're used to I guess, I know lots of people who can get by without one. I especially like it when doing coding and doing stuff in a terminal. Easier to see indentation, parenthesis, quotes etc.

</comment></comment></comment></comment></comment><comment><USER10> A great article. I recently visited the "Blind Exhibition" in Warsaw. They show you some items that blind and visually impaired people use on a daily basis and then you have to guess what they are for. I'd never contemplated how blind people know when a coffee cup is full, or pair their socks.

They then take you into a series of rooms (bathroom, the street, art gallery, kitchen etc.) which are completely black. Not a shred of light. You are blind. I can't describe how it felt other than terrifying. I didn't know if my eyes were open or not. It wasn't the black that I saw when closing my eyes, or am sleeping in a dark room. It was this empty hollow of nothingness.

As someone who has always wondered what it's like to be blind then closed my eyes for a few minutes and stumbled around my home - it's nothing like that at all. I left having found a new sense of understanding for the struggles that blind and visually impaired people go through on a daily basis.

An interesting but harrowing statistic they mentioned at the end of our trip was that the majority of people go blind, instead of are born blind. And the main cause is diabetes. It's not uncommon that those with diabetes lose their sight and then their sense of touch. Imagine not being able to _feel_ your way around after going blind.

I highly recommend the exhibition if you're ever in Warsaw, and I think they have it in a couple of other cities: [http://niewidzialna.pl/en/](http://niewidzialna.pl/en/)

</comment><comment><USER11> I still want that vibrating compass belt.

<comment><USER12> I wonder if it could be done more subtly, perhaps with a few vibrating motors embedded in the skin around one ankle.

<comment><USER13> There is something similar to this that is already reality: people use a magnet on one finger to be able to perceive EM fields.

We have the ability to turn our senses off when we are going to sleep, so any "sixth sense" prosthetics should be switchable or at least not strong enough to disturb sleep.

</comment></comment></comment><comment><USER14> > "When the belt experiment was over, Wächter felt lost"

It seems slightly cruel to me to give someone a device to augment their senses without some provision for them to continue using it if the experiment is successful.

I suppose it's also possible that Wächter didn't want to continue using / being reliant upon the belt despite the loss of the spatial sense it had provided.

<comment><USER13> I think the experiment was still in progress when they took it away, since they observed his reactions to this. Later they might have given him the device.

</comment></comment><comment><USER15> This made me realize that while it is instinctively difficult to imagine a lack of sight or touch, a lack of sound, smell, or taste, seems relatively easy to imagine. Sight vs sound is especially interesting; we tend to think of lack of sight as blackness, as the article describes, but there doesn't seem to be an instinctive parallel for lack of sound (at least in my mind). I guess it would be "silence", but that's much more akin to lack of sound than the concept of blackness is to lack of sight (even if blackness is technically lack of light).

Although having written that, I now wonder whether a person who _loses_ their hearing might be plagued by phantom hums or such things, as can sometimes happen to hearing people when exposed to extended silence.

</comment><comment><USER16> For me this raises a question on a slightly different topic: if you're born deaf, what languages do you think in? I can consciously think in English and French, but if you have never heard a spoken language before, what would your thoughts be like?

</comment><comment><USER17> I'm mostly blind in one eye. --The center of the pupil was destroyed by a parasite when I was very young, and I have no memories of normal sight. I do have some peripheral vision in that eye, but not enough to read or perform normal activities, especially because I have to turn my head to see, such as it is - I can make out certain objects and get a sense of colors, but nothing is clear - reading would be out of the question, for example.

Describing my lack of sight is difficult, because few people have the necessary frame of reference to understand. The best way I've found to describe it is to say that I can only see one side of my nose. Now, I know the other side is there, obviously, because I can look in a mirror and see it with my good eye. But without a mirror, that side of my nose doesn't exist to my vision. --There's not a dark blob or anything like that - it simply does not exist.

It also means I have no real depth perception. I have to try and estimate the distance of an object based on what I know of it's true size - when it's moving towards me, I don't get a real sense that it's actually coming at me, just that it's getting larger at some rate. Needless to say, that makes playing games like baseball very diffciult as there is not necessarily other objects around that make it easier for me to recognize the rate at which the ball is flying towards me.

<comment><USER18> Can't you move your head side to side to judge depth by parallax movement? At least a little bit?

</comment></comment><comment><USER19> This reminded me of the language of the Guugu Yimithirr [1] who always describe left/right/forward/back with cardinal directions. Thus, they always have a great sense of direction, and know where north is, even indoors. In that case, it's like the vibrating belt is built into the language.

[1] [http://www.nytimes.com/2010/08/29/magazine/29language-t.html...](http://www.nytimes.com/2010/08/29/magazine/29language-t.html?pagewanted=all)

</comment><comment><USER20> There's an analogy I like to bring up. I don't remember where I heard it, but It would be great to find a better articulation, but here it goes.

Gregor Mendel is called "father of modern genetics" for his work on plant hybridization and discoveries in the area of hereditary traits. He's called that because he described "invisible factors" producing "traits." We would now say that he discovered genes and their expression.

Around the same time Charles Darwin produced his great works on evolution. He was also essentially relying on invisible factors inherited from previous generations producing discernible traits in subsequent generations. He went beyond that and hypothesized about how constant sexual and natural selection would allow certain traits to be more common in the species as a whole.

Both of these men were essentially studying and describing genes, geneplexes & DNA. But, DNA was unknown to them. They knew that traits are handed down from one generation to another. But, if you asked them what the physical form of these traits was, they would have no answer.

DNA & Genes were not observable to them. They had placeholders in their mind. I guess they new that the seed produced by the parent plant contained the traits, but they might have also used "spirit" or "essence" as their placeholder.

We are at this stage when it comes to cognition. We don't know what a thought, memory or emotion looks like. We don't know what their physical form is. We know it has something to do with neurons and the brain, but we don't really know the how and what.

There is an obvious hole in our knowledge. It' missing in discussions like this.

</comment><comment><USER21> This story reminded me of the guy that had a magnet implanted in his pinky finger, who was then able to sense electromagnetic fields and other stuff

[http://gizmodo.com/5895555/i-have-a-magnet-implant-in-my- fin...](http://gizmodo.com/5895555/i-have-a-magnet-implant-in-my-finger)

<comment><USER22> Implant seems extreme. One could do well with, say, compass armbad with a strong magnet and a pokey attacmhent.

</comment></comment><comment><USER23> A couple of years ago I read a non-fiction book called Crashing Through[1] - about a guy who was blinded by some chemical at a very young age (<1y) and was able to have his vision restored in his 40s via a new procedure.

It went on about some difficulties he had processing the new data. His brain had to "learn" how to see. It was so difficult that he would sometimes close his eyes and and rely on his echolocation skills to navigate.

Very interesting book if you want a first hand account of what it is like to go from being blind to being able to see. (and also about being blind)

[1] [http://www.amazon.com/Crashing-Through-Extraordinary- Story-D...](http://www.amazon.com/Crashing-Through-Extraordinary-Story- Dared/dp/0812973682)

</comment><comment><USER24> I was born (40 some odd years back) with an underdeveloped optic nerve in my left eye. An early surgery for Amblyopia corrected my gaze, but those first years of wearing a patch over "my good eye" resulted in nothing more than a lot of stress for my folks. And while I attempted to explain what I could see (are they shadows?), I never really had an answer as to why I could count "how many fingers" someone was holding up. Curiously, it was only within the past few years that I realized my left eye renders what one perceives peripherally. Hold your hand to the side of your head -- it's there, but do you really _see_ it? I can't imagine putting blindness into words.

<comment><USER25> Wow, I think you might have a very similar vision issue as I do!

Left eye as well, although my eye is healthy, it was closed when I was a baby so my vision never developed properly. Eye patching was tried on me as a kid, but just resulted in a cranky toddler me. It took years and years for me being able to verbalize what I could see out of my left eye--I basically have some limited peripheral vision and no central vision. My eye doctor explained that part of the issue is that peripheral vision primarily detects motion while central vision is for detail--so while I can detect motion and light, it's hard for me to put an image together.

I have so many memories of trying to read that big stupid E with my left eye and not being able to explain what I saw. (Um, there's a light on, right?)

</comment></comment><comment><USER26> A very interesting take on the perceptions of blind people. I don't think I am acquainted with anyone who has been entirely blind from birth (which is a rare condition of life). I know a professor of psychology (who indeed does much of his research on human visual perception)[1] who has very limited vision, and thus is "legally blind." He began life, I think, with a bit more vision than he has now, but has long been visually disabled. In a personal conversation, he related that his originally intended research field was astrophysics, which of course can be studied by analyzing instrument readings from radio telescopes and other devices that don't necessarily have visual outputs. My wife is his piano teacher, and she has had to adapt her teaching to her first student who cannot read a printed musical score.

A much younger person I know who has very limited vision (and the prospect of declining vision as she grows up) attends summer mathematics programs with children running around playing soccer and Frisbee and seems to handle that with aplomb. To not even be able to recognize shapes or moving human beings, something that the blind people I know best are still able to do, would be especially challenging.

Aside: Have you all noticed that people who have acquired profound deafness that begins in adulthood have much less understandable speech than people with normal hearing? Apparently we all rely on feedback from our own senses to keep our speech behavior within the phonologically normal range of whatever language we speak as a native language, and habit alone can't maintain the fine tolerances necessary for readily understandable speech.

AFTER EDIT: Of course anyone can experience total lack of sight simply by going into a totally unlighted place. The human eye doesn't emit vision rays, after all (even though the ancient Greeks seemed to think otherwise), so if you are where there is no light, you see nothing with your eyes.[2] Studies on the human diurnal behavior cycle are sometimes done in deep caves with no source of artificial light.

[1] [http://legge.psych.umn.edu/](http://legge.psych.umn.edu/)

[2] [http://www.spring.org.uk/2008/03/50-of-college-students- thin...](http://www.spring.org.uk/2008/03/50-of-college-students-think-we- see.php)

[http://www.npr.org/blogs/health/2013/10/30/241906619/seeing-...](http://www.npr.org/blogs/health/2013/10/30/241906619/seeing- in-the-pitch-dark-is-all-in-your-head)

<comment><USER27> Originally from SF area, I have been living in Tokyo for over a year now, and since moving I rarely speak with native English speakers anymore. I've been told by friends and family on occasion my accent and grammar is slowly deteriorating.

<comment><USER26> The Americans I knew when I lived overseas for three years (on each of two stays that long) definitely had trouble with vocabulary retrieval, the more trouble the longer they had been away from native speakers of English. Correspondingly, Chinese people I know who have been away from places with lots of Chinese printed text often have a lot of trouble writing Chinese characters that everyone learns in school. Even "overlearned" skills that become second nature can degrade through lack of continued practice.

</comment></comment></comment><comment><USER28> Herzog's _Land of Silence and Darkness_ [0], which follows some elderly members of the German deaf-blind community, is one of the most fascinating documentaries I've ever seen. One quote in particular has stuck with me since I saw it years ago:

"People think that deafness means silence, but they are wrong. It is a constant noise that ranges from a gentle whisper going through some cracks to a constant buzz, which is worse."

[0]: [http://en.wikipedia.org/wiki/Land_of_Silence_and_Darkness](http://en.wikipedia.org/wiki/Land_of_Silence_and_Darkness)

</comment><comment><USER29> Having congenital Glaucoma, I've always had to live with the possibility of one day being blind. My left eye is already pretty much there.

What has always fascinated me about sight is how our brains augment and outright invent things you think you see with your eyes. It's not at all about believing what you see, but about seeing what you believe.

The Glaucoma has steadily been eating away at my retina and optic nerves over the many years. Causing blind spots to form all over my visual field. In daily life, I can't see those spots. As in, there are not actually black holes in the images I perceive. The brain somehow manages to fill in those gaps with visual information directly surrounding those gaps and combine it with what my experiences/memories tell me should be there.

It's only when I start concentrating on really small details, that these holes become apparent. Particularly when looking at small LED lights in a dark environment. The LED keeps disappearing and reappearing as I slowly turn my head in various directions. Everybody has a single blind spot like this in the center of their visual field which behaves in the same way. Imagine this, but multiplied over 60 - 80% of your visual field.

Additionally, my almost-blind left eye has caused me to lose depth perception all together. This means those fancy stereoscopic 3D things are pointless to me and one would expect I would have a hard time in traffic. Not being able to judge the distance to an oncoming car can be deadly. But again, the brain seems to draw on its memories and years of experience and somehow manages to account for the lack of depth perception. It's not perfect, but enough to cope in daily life and safely move from A to B. At least on foot, that is. I am not allowed to drive a car for obvious reasons. A moped is technically permitted, but I don't. It moves too fast for me to accurately judge my surroundings in time. The same even goes for a bicycle. I only ride those in daylight. Not at night.

As far as blindness goes, I've had this once. As a kid, I fell out of a tree and landed flat on my back. For the following 45 minutes I was completely blind. It freaked me out to no end, as I was terrified it would not go away. Luckily it did. Not looking forward to that again!

</comment><comment><USER22> Strange that the article doesn't mention the blind spot that everyone as.

[http://en.wikipedia.org/wiki/Blind_spot_(vision)](http://en.wikipedia.org/wiki/Blind_spot_\(vision\))

<comment><USER30> Mind Blown. Not by the fact we have a blind spot, but how, like in the article it's not a black zone, but really the absence of any input that the brain tries to fill in. Now I'm curious to test how the brain interpolate on this area, like trying to put different background color or texture on each side of the blindspot.

<comment><USER31> Having tested it, it seems to do a linear interpolation from the edges, often retaining the surrounding texture. Kinda like the smart fill from Photoshop

</comment></comment></comment><comment><USER32> I have exotropia on one eye and completely lack binocular vision.

My right eye is my "main" eye (95%) and my left eye just submits the missing parts from the left that my right can't see because of the nose being in- between.

I always have the right side of my nose in my field of view, except that at the same time it's somehow not there. Like 50% opacity. The left side of my nose isn't visible.

When I "hide" a finger behind my nose for the right eye and look in it's direction it's gone. When I stare straight forward, it appears again.

(My) vision is weird :).

</comment><comment><USER33> Katawa Shoujo actually addresses this in-game. One of the game's characters, a blind woman, describes her dreams including sensations from her other senses: smell, taste, touch, hearing. As something I have wondered about, I was really pleased that the game's authors thought of this.

</comment><comment><USER34> A blind person once explained it to me:

> I see what sighted people describe as "white". When I ask a sighted person > what they see out of their elbow they typically get it. They see "nothing", > but if pressed will usually say "static" or "white".

</comment><comment><USER35> I suspect you experience true blindness (briefly) during laser eye surgery, when the flap is cut in the cornea. I remember how odd it felt - the feeling of seeing nothing at all, not even darkness.

<comment><USER36> I could see after the flap was lifted. This was 2005 so I don't know the procedure anymore, but I was supposed to keep my eye trained on a flashing dot while the laser ablated my cornea.

<comment><USER35> I meant when the blade goes into the eye. There were a few seconds after that before vision came back.

</comment></comment></comment><comment><USER37> Tommy Edison, the Blind Film Critic's take: [https://www.youtube.com/watch?v=ZDHJRCtv0WY](https://www.youtube.com/watch?v=ZDHJRCtv0WY)

</comment><comment><USER38> They see what you see out of your elbow.

</comment><comment><USER39> Many years ago, blind people might put their hands on your face to help them get a sense of you. A blind friend of mine, Jim, got in the car with us, one night, and sat next to a friend he did not know. He put his hands on the other guy's face, then paused, then said, "Man, you are uuuuuugly!".</comment>


Petition to open source code from healthcare.gov, including code by CGI Federal - <AUTHOR> https://petitions.whitehouse.gov/petition/release-open-source-community-source-code-healthcaregov-specifically-all-code-written-cgi-federal/XvBXgZWx <comment><USER2> The administration has has taken several steps backwards in terms of transparency. How serious are they regarding open gov?

I hope this happens, but I'm doubtful.

</comment><comment><USER3> Can't this be done with a Freedom of Information Act request?</comment>


Stephen Hawking hacked by Syrian Electronic Army - <AUTHOR> http://www.panarabiaenquirer.com/wordpress/stephen-hawking-hacked-by-syrian-electronic-army/

<comment><USER2> Just in case you were wondering:

From [http://www.panarabiaenquirer.com/about/](http://www.panarabiaenquirer.com/about/)

 _IS IT REAL?

No. It's made up. All of it. Yup, even that story. As the Middle East's premier source of SATIRE (much like The Onion, The Daily Mash and Emirates 24/7), The Pan-Arabia Enquirer is not intented, in any way whatsoever, to be take as factual. It's just here to amuse and any similarities in names (well, except for Ahmadinejad) are purely coincidental. So please don't take offence to anything we write, OK, because it's just not worth it._

<comment><USER3> Yeah I was just about to type that. It's the arab version of The Onion. It's pretty new so I still see a lot of links to it in my facebook feed.</comment></comment>