Descriptive Camera: A camera that prints a description, not an image - <AUTHOR> http://mattrichardson.com/Descriptive-Camera/?src=twitter <comment><USER2>
      </comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>-----
      | |                             | |
      | |    This is a picture of     | |
      | |  30-60% off top brand name  | |
      | |    shoes like Nike, Vans,   | |
      | |    Adidas, Toms, and more.  | |
      | |   Call 1-800-TURK-GAMER to  | |
      | |   find your perfect pair.   | |
      | |                             | |
      | </comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>- |
      |                                 |
      |                                 |
      </comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>-----

<comment><USER3> Why does the human race have to have griefers? Why can't we all just get along and have cool sci-fi tech without people trying to game it?

<comment><USER4> Well, let's draw a distinction here. The above isn't griefing, it's simple capitalism. It exists for some reason other than aggravating people and/or wasting their time. You might also have griefers who would put out text- photographs like:

         ________________________________
        /                                \
        |+</comment></comment></comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>+|
        ||  This is a picture of how I  ||
        ||  just want to tell you how   ||
        ||  I'm feeling, gotta make you ||
        ||  understand, never gonna     ||
        ||  give you up, never gonna    ||
        ||  let you down, never gonna   ||
        ||  turn around and desert you, ||
        ||  never gonna make you cry,   ||
        ||  never gonna say goodbye,    ||
        ||  never gonna tell a lie and  ||
        ||  hurt you.                   ||
        |+</comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>+|
        |                                |
        \________________________________/

<comment><USER5> Except that might not be used by 'griefers' instead it could be a love note between two lovers.

'Greifers' is just a subjective term depending on which chair you're sitting in to watch the performance.

</comment></comment><comment><USER6> Amazing - the meta concepts behind the "Descriptive Camera" are limited only by the imagination, the resolution of the sensing devices, the ubiquity/bandwidth of the data pipe, and the ability to manage/train the backend human workforce and maintain a consistent level of quality. I'm pretty certain, that within 3-5 years, a billion dollar company will emerge using the basic concepts captured by the "descriptive camera." That is, a remote sensing device feeding data into a human-backed work-management queue, and returning some type of structured higher-level processing of the remote-sensing device's data in close to real-time.

Just off the top of my head:

      o Flora/Fauna identification
      o Scene analysis (Car Crashes/Building Wreckages)
      o Medical analysis
      o Substance analysis
      o Structural integrity review
      o Online Translators.  Today.

I'm pretty certain companies would be more than happy to pay $1000+ for 3-4 hours of a real-time translator than worked through a smartphone for business trips. Ubiquitous high bandwidth wireless (LTE)/High Resolution Mics/Cameras/smartphones are the enabling technologies which trigger this class of application.

<comment><USER7> _Medical analysis_

I don't think I would rely on a description from a Mechanical Turk human for describing what they see in a pictue.

 _Is that a tibia or a fibula?!?_

<comment><USER6> Right - you don't send to Mechanical Turk. You instead send it to a highly trained backend workforce, presumably highly skilled in the vertical that you are doing analysis on. And, it's not just a picture, but video/sound/other sensing. Who knows what the next advances in remote telemetry will be.

<comment><USER8> Indeed, we do this kind of work today in MobileWorks, using exactly this fact.

Once your workforce is skilled all kinds of new opportunities of this nature open up.

<comment><USER6> Nice. I guess we don't need to look too far to see that soon-to-be-billion dollar company then. Have you considered doing real-time feedback to a device that has remote telemetry (video/sound/pictures/etc...)?

<comment><USER8> We've done some work in this vein already, as a matter of fact: real-time communication between the crowd and cameras / robots. We're currently supporting a few fast camera-driven smartphone applications that developers have built on the platform.

I think what you're after, though, is more comprehensive integration: a remote telemetry system that has crowd intelligence baked into its circuits along several media, rather than one: analyzing simultaneous audio and video. I don't think this would be difficult for a developer to build, and it's a great idea.

</comment></comment></comment></comment></comment></comment><comment><USER9> I was hoping that the description would be derived algorithmically rather than via human intervention. I guess that really was a bit optimistic though :(

<comment><USER10> Agreed, the concept's great but really you're just taking a picture then asking someone to describe it & printing that; that tech's been around for a while.

A more practical approach may be to develop a camera with access to the web. When pictures have been taken and the camera can get a 3g/4g/wifi network connection it uploads the pics to the cloud (optionally removing or retaining to save space on the camera's local memory or allow the images to be accessed on the camera in offline mode). You can then put in some serious server power to run through all uploaded images to describe the contents (including tagging people in the photos and using GPS data to help identify content based on location context) and have this data automatically sent back to the camera (if desired) as well as stored against the image online. Randomly picked images, or those with a low accuracy rating (e.g. the algorithm has rated its description as tenuous) could be sent to a service such as mechanical turk so that the results can be verified / improved upon / to ensure the algorithm has feedback to learn from.

<comment><USER11> Heh you're describing the whole implementation except for that pesky but critical computer vision part.

</comment></comment></comment><comment><USER4> It is a cute idea but unfortunately (or fortunately?) this particular implementation of the camera comes with (a) opinions, (b) typos, and (c) probably trolls, if they know that their text will automatically be accepted and printed out.

I guess these factors could probably be reduced with more money? I don't mean paying more for the job: I mean using the API to ask a second human to verify the results of the first human and/or fix typos.

<comment><USER12> I think you're missing the point of it.

<comment><USER4> Am I? Would the point somehow be served if you took a self-portrait and the response back was "haha ur so ugly"...?

(I should add that I know that Amazon offers a review process, but I am not sure how it integrates with the above camera, and I am thus assuming that the camera automatically has to accept a description as correct before printing it out. This might not be the case; and I don't know.)

<comment><USER13> You are. This isn't ment to be a completely realistic project ready to be shipped, it's a glimpse of the future, or maybe an art project. I think it's wonderful to see out-of-the-box stuff like this appearing on Hacker News.

<comment><USER4> My apologies; I suck at communicating.

I think I agree. This camera is cute but this particular implementation has flaws -- that's what I said before and what I hoped would be my take-home message, and I guess I didn't highlight that enough. I certainly didn't intend, "oh, fix this before it goes into production!" or anything like that.

</comment></comment></comment></comment></comment><comment><USER14> Add speech synthesis and this would be an awesome camera for the blind.

<comment><USER15> Even with a six-minute latency?

<comment><USER16> Absolutely. for example, a big challenge for blind people is interacting with devices in the environment that report status via visual signals. For example, a blind coffee fiend I know can't tell what state his fancy espresso machine is in ("brewing", "out of beans") without sighted help to read the little display. He would happily wait a few minutes to get someone to tell him - it's quicker than waiting for his wife to come home.

<comment><USER4> Thanks for this. Now I'm trying to think of ways that you could actuate thin metal rods, so that you could transform a greyscale picture into a sort of texture-photograph, so that you could "see" details by placing a hand atop it. If the rods are needle-thin you could probably get a reasonable resolution on it, and even have some brightness/contrast sliders on the side to help -- but it seems like the real problem is just moving all of those little rods automatically.

<comment><USER17> If you built it, I think you'd find the real problem is having enough resolution in your fingertips to work out what the hell the picture is about.

Lots of people who become visually impaired later in life can't read Braille as their fingertips aren't sensitive enough, especially if they did manual work earlier in their life. Working out a picture is even harder.

As a side note about this, William Moon invented 'Moon script' which was a kind of simplified alphabet in embossed writing to help visually impaired people read. It was invented a little before Braille invented the raised dot patterns, and even after Braille became popular, Moon still had it's niche for those who didn't have the resolution for Braille. As Moon was based on the Roman alphabet, it was also easier to learn for people who'd been able to read then lost their sight.

</comment></comment></comment></comment></comment><comment><USER18> I'm disappointed. I was thinking some one had optimized a template matching algorithm and implemented in an ASIC that you could slot into a camera.

</comment><comment><USER19> Let me see if I understand this technology correctly: you take a picture of something, mail the picture to someone, somewhere, they write a quick summary of the picture and mail that back to be printed out on what appears to be a thermal receipt printer.

Wat?

<comment><USER20> Yep, and if you can internalize why that makes the front page of HN you can get a sense of what Web 3.0 is going to look like.

</comment></comment><comment><USER21>
      It is pitch black. You are likely to be eaten by a grue.

Ooops, lens cap is on.

</comment><comment><USER22>
      Here is an idea, translate the text into a picture.  An API will take text and generate a representation of the description using images from the web via google image search.

<comment><USER23> There's been some interesting efforts at that already. eg:

WordsEye constructs 3d images from text descriptions <http://www.wordseye.com/>

Sketch2Photo takes a crude annotated sketch, and creates a composite image: <http://cg.cs.tsinghua.edu.cn/montage/main.htm>

The output of wordseye isn't great - it looks a bit 90's POV-Ray. Sketch2Photo does a nicer job - more like automated photoshopping - but needs more assistance on the placement of objects in the scene.

I'm sure there's others.

<comment><USER24> The Wordseye renderer actually happens to be a 90s vintage ray tracer.

</comment></comment></comment><comment><USER25> Maybe not too practical as presented here, but if you start with mechanical turks doing the recognition, the pairs of images / description could be an invaluable source when developing and training automatic algorithms that does this kind of image analysis.

Thare are already systems that can analyze wounds automatically by analyzing pictures of it.

</comment><comment><USER26> My first thought was "oh great, another silly art project", but I've been wanting something like the Mechanical Turk part of it for awhile.

My least favorite part of photography is organizing and finding the best photos. I'd happily pay to have a bunch of people rate all my photos (if the results are meaningful enough)

You'd probably need to show each photo to multiple Turkers to get good data. You could run analysis on the votes to kick out the Turkers who give ratings that deviate significantly from other Turkers. Does MTurk have anything like that built in?

</comment><comment><USER27> And then comes pictureless pinterest, oh wait here it is: <http://twitter.com/picturelesspins>

<comment><USER28> The source of their content are submissions from their website <http://picturelesspinterest.tumblr.com>

</comment></comment><comment><USER29> Like reverse Dwarf Fortress?

</comment><comment><USER30> This is like the DARPA Mind's Eye project, except the DARPA project does this computationally. [http://www.wired.com/dangerroom/2011/01/beyond- surveillance-...](http://www.wired.com/dangerroom/2011/01/beyond-surveillance- darpa-wants-a-thinking-camera/)

</comment><comment><USER31> > Here is an idea, translate the text into a picture. An API ...

This one uses a combination of computer vision and optional human vision. <http://www.iqengines.com/omoby/>. There is an API allowing for computer vision training.

</comment><comment><USER12> This is brilliant. It reminds me of Sascha Pohflepp's Blinks & Buttons project, which swaps your photo with someone else's taken at the same time. There's even an iPhone app.

<http://www.blinksandbuttons.net/>

<comment><USER32> Reminds me of Swapshot.

</comment></comment><comment><USER33> A camera for the blind - if the printer is adapted to print Braille.

</comment><comment><USER34> Oh god, the Searle Prophesy is coming true!

</comment><comment><USER35> its too bad tineye doesn't do a better job of matching similar, you could feed the picture in, find the closest match, and then scrape the description off the corresponding pages where it was found.

</comment><comment><USER36> Of course, these should be saved in JTEG format.

[http://whaleycopter.blogspot.com/2007/09/project-jteg- compre...](http://whaleycopter.blogspot.com/2007/09/project-jteg-compression- format-for-ink.html)

</comment><comment><USER37> Now I know how Thicknesse felt.

</comment><comment><USER38> Can someone build a version of this that prints snarky descriptions of certain pictures? Give it a libertarian political slant and a vegan diet. Only good things can come of this.

</comment><comment><USER39> I wonder how it holds up against a human being. They sould run a test on that.

<comment><USER40> Did you read the article? It is being done by a human being, trough Amazon's Mechanical Turk service.

<comment><USER39> Yes I did, Mr. serious face.

</comment></comment></comment><comment><USER41> My camera gives me descriptions too, but they're in terms of quantized frequency coefficients of 8x8 squares.</comment>


Storm Clouds: Gmail Failure Reinforces Danger Of Becoming Too Cloud-Dependent - <AUTHOR> http://techcrunch.com/2011/02/28/storm-clouds-gmail-failure-reinforces-danger-of-becoming-too-cloud-dependent/ <comment><USER2> US Airline Fatalities in 2010: 0

US Motor Vehicle Deaths: ~32,000*

What are people more afraid of? Planes. Why? There's a few theories, but one is people are afraid of dying due to somebody else's mistake, the same with surgery: powerless. Even in a car accident, there's a bit of victim-blaming: the insurance company _always_ says a person could have been more wary.

A cloud failure is much, much more rare, but more scary because we're not accountable. When it happens, we're "more helpless."

*Estimated from a drop from 2009. Can't find the actual number.

<comment><USER3> > What are people more afraid of? Planes. Why?

Well, for me one reason is that I've watched countless hours of "Seconds from Disaster" and "Air Emergency" on the National Geographic Channel.

Seriously, though, I think the control thing is big for me. I'm pretty sure that if I was wealthy, I'd be a private pilot--and still would not like flying commercially.

I don't like buses all that much, or cars when I'm a passenger, so that's more evidence in support of the control theory.

I'm OK with reasonable sized boats (e.g., the ferries in Washington State).

Ambivalent about trains--I was OK with them until I took Amtrak from the Seattle area to Beaumont, Texas, and back (yeah, patent lawsuit), and saw just how poorly maintained the tracks are. Fortunately when my train derailed, we were only going about 5 miles an hour so no one was injured. It just caused a several hour delay.

<comment><USER4> Agreed. Also I believe our subconscience (or primal instincts) plays a role here.

Flying in a plane is more or less akin to walking in the dark. Our conscious mind sees it ordinary, but our animal half still is behind the times, and feels afraid.

</comment></comment></comment><comment><USER5> No problems in being cloud-dependent. The problem is not backing up stuff because you think that Google is 100% safe. You could use POP3 and get your disk wiped out for any reason, so local-dependent isn't safe either.

</comment><comment><USER6> You would rather risk losing your data more keeping with yourself. This is a stupid post. Clouds solutions generally implement good backup solutions and with almost 100% availability to the data. Google has anyway restored more than 50% of the lost data.

<comment><USER7> I can't put a number on the amount of people I know who don't regularly back up their files themselves. One in particular lost gigs of precious photos during a laptop hard drive failure.

A company whose mandate is to provide continuity of service is going to be more reliable than 'oops I forgot to copy my files to this cheap crappy consumer drive I bought'.

Still, this incident should serve to remind that the so-called 'cloud' isn't invincible and we should try to cover our bases through multiple backups.

<comment><USER6> I suppose a cloud app is needed which would help us to backup data across multiple/popular cloud services. Chances of data being lost reduce drastically (almost to zero) if multiple clouds are keeping a backup of the same data.

</comment></comment></comment><comment><USER8> Contrast Google's 0.08% fail rate here vs. the real world 3% hard drive fail rate - by that measure Google's cloud is 37x more reliable than your local storage.

Feels like cloud-bashing is becoming a regular sport - for techies we know that redundant back-ups are important, for average consumer the cloud drastically INCREASES reliability and the chance that items you store are going to be available when you need them.

ref for 5 year hd fail rate <http://lwn.net/Articles/237924/>

<comment><USER9> It doesn't make sense to compare the percentage of Gmail accounts that lost data in a single event to the annual failure rate of a hard drive. It also doesn't make sense to assume my local storage is a single hard drive in a world where a RAID array fits in a Macbook.

</comment></comment><comment><USER10> gmail is more than "good enough" for the vast majority users, any valid criticism of it as a "could-based" email service (whatever that means) would have to compare/contrast it with the alternative "non-cloud" email service.

More importantly, it probably is not a good idea to use email (of any form) as long term storage. Much better to "zero" your inbox periodically and only save what really needs to be saved outside of the email system.

Finally, there's are HUGE distinctions between losing some/all of one's emails, temporarily being locked out of email, and whatever other problems actually occurred. It is not at all clear what the severity of the problem actually was and how permanent it was. Perhaps it is premature to create a scare story about using gmail at this point?

<comment><USER11> > it probably is not a good idea to use email (of any form) as long term > storage.

For hotmail or yahoo mail, I agree--but one of Gmail's founding principles is never having to delete an email--they say so in as many words. If it's not a good idea to use them for long-term storage of emails, they've failed (although having anything truly important in only one place does seem like an egg basket failure).

</comment></comment><comment><USER12> The fact that it's only 0.02% of users affected doesn't mean the situation should be taken lightly. I'm amongst those lucky users and trust me, it freaking sucks.

So far, it doesn't seem like Google cares much about us or is doing anything to restore our accounts. I hope I'm wrong.

</comment><comment><USER13> Am I the only one who thinks it's nuts to describe streaming video or music as "using the cloud"? Doug Engelbart demonstrated video conferencing back in 1968, was that using the cloud as well? Sheesh.

</comment><comment><USER14> To be fair at 0.02% affected (latest revised number) it's not even a 'micro- failure' it's a rounding error, not sure how it made the news.

It might give some opponents some ammo, but the chances of this type of thing happening on you're local machine is actually higher, and in this case all the data should be restored.

<comment><USER15>
      > To be fair at 0.02% affected (latest revised number) it's not even
      > a 'micro-failure' it's a rounding error

Would this fact placate you if you had just lost access to all of your email?

<comment><USER14> The user will be upset either way, but the chances of this happening on the local machine is higher, and at least in this case someone else will be charged with fixing it.

<comment><USER15> As far as "how upset will people get", people get _much_ more upset at factors that they perceive to be out of their control rather than factors they perceive to be in their control.

If my laptop hard drive dies tomorrow and I realize my backups aren't bootable, that sucks but it's my fault. I just have to get better at backups.

If Google missteps and I lose access to my email, it's completely out of my hands. There's nothing I can do, and there's nothing I could have done. It's much more upsetting than something that can be internalized.

</comment></comment></comment></comment><comment><USER16> I would be extremely careful with any service that claims to be on the "cloud". Backup your Email (no, not Gmail but Email) using Thunderbird or Outlook or any other old-fashioned email program. There's no better solution!. Use and keep thumb drive in your key-chain (everyone still carry keys even though we carry modern cellphones and what not). There's no better way to be sure than to keep and carry your important data to yourself.

It is a huge human fail when we treat Google and Facebook as Internet/Social life. They are not! Mind your own email, pictures, data, contacts, URL yourself.</comment>


Vipassana: 10 days of solitude and insights gained from it - <AUTHOR> http://www.curiousjuice.com/blog-0/bid/141396/Vipassana-10-days-of-solitary-confinement-and-insights-gained-from-it <comment><USER2> I completed my 4th course in December. Vipassana meditation is the single greatest human experience that I know of. The experience of "wisdom arising by objectively observing sensations without reaction" is easily the most profound experience I have ever had and feels like a miracle every time it happens. The answers we are all looking for to fulfill ourselves and become truly happy certainly are inside of us. We just have to be willing to sit and do nothing and watch as the "law of nature" (Dhamma) takes over and we act perfectly in accordance with it from moment to moment, as it is, not as we want it to be.

In every retreat, I experience the most mind-numbing boredom, severe burning fiery pain, full body bliss the likes of which I cannot even recollect properly and a huge gamut of sensations in between all of these extremes. I've misinterpreted the extreme experiences at given times and each retreat learn a little bit more about their true nature. I can then apply what I learn to my life when I experience boredom, pain and extreme joy and learn to experience each "properly".

The trick is finding the ardent awareness that allows you to stay there, on the path, without distraction. And then to learn how to stay that ardently aware as you go through life. Hopefully the retreat doesn't stay an isolated 10 day experience but its import, particular the technique itself, is something you can cultivate throughout your life, while sitting, walking, standing and lying down.

<comment><USER3> Where can I take this course? I live in the Bay Area.

<comment><USER4> In the Goenka tradition Dhamma Manda[1] in Lake County is the closest to the Bay Area, the center in North Fork[2] is the largest. Here is map of locations worldwide: <http://courses.dhamma.org/en-US/maps/ca.us>

[1] <http://www.manda.dhamma.org/>

[2] <http://www.mahavana.dhamma.org/>

</comment></comment></comment><comment><USER5> My take on Vipassana, in a nutshell (having gone through one 10-day course):

An experience gets memorized based on its emotional strength, so, for example, a strong negative experience (a trauma) will get memorized quite well. Later on, in a sort-of realed situation one gets to recall and relive such stored experience - the brain sort of "believes" it's important to remember. If the re-living is also strongly emotional, the memory gets "hardened", but if the reliving is less emotional the memory gets "softened". As you go through life, most negative experiences gets softer and softer with each recall until each one disappears altogether for lack of emotional response, and thus subjective importance. However a small number of such memories become immortal - each re- living of one such memory is as emotionally strong as the previous one, and so the memory of the pain never weakens. Such stored memory is called "Sarkana". The problem with carrying around too many Sarkanas is that eventually any event in your life brings back a negative memory from the past, so you spend your life reacting to something that happened long time ago and bears only superficial resemblance to the present situation.

The immediate purpose of the Viapssana meditation is to learn the ability to direct your attention where you want it to go, not where the pain pulls you, to learn dispassionate observation of your own suffering. Bi learning to ignore an itch on your nose, you also learn to ignore an itch in your soul. The less emotional response you generate, the softer the memory becomes, until it evaporates altogether. When completely through (a multi-year process), you only react to the present situation, not to any ghosts of the past.

You can make a surprising amount of progress in 10 days, but you have to keep at it to maintain the gains. I found it impossibly hard to keep meditating on my own.

<comment><USER6> 12 year meditator here. You describe well the process of cleansing/polishing the memory.

> I found it impossibly hard to keep meditating on my own

Couple things to keep in mind:

1\. Daily practice in some form is the real key. Just get up and do it.

2\. There will be ups and downs, high and lows in meditation.

3\. Vipassana is a pretty austere form of meditation. You are directly confronting the mind with little outside aid or warmup. Other forms of meditation leverage breathing techniques, concentration techniques and other practices that gradually bring the mind into submission, making the no-thought part somewhat more easy to achieve. Search around or shoot me an email if you want to know some other approaches.

<comment><USER7> I couldn't find an email to shoot you a message, so I'm responding here.

I meditated for a few months back in 2006 and it was fantastic. I, too, have trouble maintaining it. What other sorts of practices would you recommend?

<comment><USER8> Vipassana is a great practice — in my experience, finding other people to sit with consistently has helped better than anything else. Like weekly or even a few times per week.

</comment></comment></comment></comment><comment><USER9> I completed my first 10 day Vipassana course recently, and genuinely believe that a slightly modified version of the course should be an essential component of any person's psychological maturation/development. The course I did was the 'Goenka course' (dhamma.org), and I would recommend it to anyone interested in learning more about meditation. It is free of charge to attend, and you can actually only donate to the organization if you complete the full 10 days.

The course is almost entirely rational and in the spirit of honest scientific inquiry. The rules seem arbitrary and silly but they are very helpful in quieting your mind, and enabling you to focus entirely on the practice at hand. It took me until Day 3 to let myself relax and enjoy the absence of tweets, posts and text notifications rather than feel anxiety about being disconnected. Assuming one is there for 240 hours, I only have an issue with ~5 of those hours being a bit 'woo-woo' as The Amazing Randi would say. I think of Goenka (the videos used in the evening lectures are available on youtube) as a kindly grandparent who just wants the best for you but occasionally tangents into bullshit. Despite his continual admonishments to only believe what is rational, he seems to genuinely believe the magical stories surrounding the Buddha, and it was distracting to have to segue from 'this is a genuine insight into how the mind generates its own suffering' to 'oh wait he is talking BS'. If one replaces some of his more magical terms that are almost Freudian (ie Sakarnas) with 'Maladaptive neuronal connections' they actually make sense.

The sheer number of hours one is able to practice in this setting really accelerates the development of concentration/mindfulness. I think of the ~110 hours spent meditating during the course as equivalent to 220 days worth of 30minutes/day practice, and feel like I have 'time traveled' past 60% of a year of training. Happy to answer specific questions if anyone is considering attending a course.

<comment><USER5> >'Maladaptive neuronal connections'

That's a mouthful, and just as obscure for most people. Might as well go with "Sarkana" - it's easier to pronounce. Besides, the script worked well for a few thousand years now, there is hardly a reason to change it now.

But yeah, if you're scientifically inclined and the ancient Indian mythology is rubbing you the wrong way, you can safely skip it, and it still makes perfect sense in the end.

<comment><USER10> (following up on uberuberuber's reply)

> Besides, the script worked well for a few thousand years now, there is > hardly a reason to change it now.

Imo, that is a big fallacy. I believe religious beliefs can no longer provide sound ground for adjustment in today's world. It is harmful to have within your semantic environment both "spiritual" factors such as the notion of "aetheric body", and scientific factors such as the notion of "space-time" (which currently shape our civilisation). This is because the two belong to inconsistent sets, so they conflict.

Edit: For a more relevant example, consider the implicit assumption of many "spiritual" doctrines that there is a "soul", which is separated from a "mind", which is separated from a "body". This contrasts with our modern scientific assumptions, that there is just an organism-as-a-whole-in-an- environment, which functions as such. The word "organism" is more recent than "Sankharas"! As such, it doesn't carry with itself the older metaphysics.

Also, consider our modern scientific perspective that we don't drop into this world from a vacuum – we come from an environment, the uterus. This, again, stands in sharp contradiction with many "spiritual" doctrines.

<comment><USER11> Your point of view is certainly not unique, but neither is it anything close to universal. Many people myself included comfortably reconcile belief in a "soul" and a "higher power" with the demonstrated accomplishments of scientific research and study.

</comment></comment></comment></comment><comment><USER12> While I'm glad the author got useful insights into conventional reality, the article sets a counterproductive tone about the point of the practice.

Vipassana is about directly seeing impermanence, dissatisfaction, and lack of self in all sensations. The point of practice is to elevate your skill in seeing those. Because direct comprehension of the so-called three characteristics can reboot the mind in a way that is very analogous to how we reboot the computer and remove malware.

Vipassana is not about helping you deal with your psychological issues at the level of content. Its not a therapist-substitute. In fact it will often make matters (temporarily) worse, if you actually follow the instructions and do the practice.

Spending your meditation time pondering your "issues" is the biggest rookie mistake you can make. People can do this for years, and never get anywhere in their practice.

If you want to spend some time alone and contemplate your life, thats great and quite possibly useful, but don't call it Vipassana.

<comment><USER13> _> Spending your meditation time pondering your "issues" is the biggest rookie mistake you can make_

This is worth repeating. This is a silent practice, and while thoughts may arise (and very interesting ones, too!) learning to let them go, and remain (mentally) silent was perhaps the biggest obstacle to progress for me personally. Not sure if mental silence is a cause or effect of meditation, but it's the closest thing to real happiness I've experienced.

</comment></comment><comment><USER14> i have recently done the 10 day course and it was one of the best things i have ever experienced. have continued meditating (near) every day because it keeps me returning to a sense of deep calm and focus and enjoyment – feelings prior far more rare even tho i would have described myself as happy and digging life. the new digging is at a deeper level and much of what i used to perseverate upon seems reduced in import by orders of magnitude. practically the whole notion of "intellect figuring out and resolving issues", seems now completely a misguided process. returning to the state of calm fertilizes better thought which crowds out the worries. worry process becomes reduced and unmasked when it arises. the whole physical reaction cycle different. awareness that how we feel (and happiness) is in our bodies – we know the value of physical exercise. this is similar. a rigorous training that just starts doing what it's advertised to do. veritably mind blowing, yep.

<comment><USER2> I'm happy for you!

>> practically the whole notion of "intellect figuring out and resolving issues", seems now completely a misguided process"

Makes me feel like all the book learning I ever did was all for naught (aside from technical learning that helps me do my job).

<comment><USER14> thanks! ...but let me clarify. book learning is utterly awesome!! when i say "resolving issues"... i mean purely the personal, subjective, 'worrisome' kind... the endless spinning narratives we struggle within. the attachment to solving issues these through out-thinking the anxiety: this process got deprecated ;)

<comment><USER2> Well, as you keep meditating, it becomes more and more clear that all the answers you are burning to understand are found within.

</comment></comment></comment></comment><comment><USER2> Guys, what you are calling "Sarkanas", the Goenka tradition is calling "Sankhara".

<http://en.wikipedia.org/wiki/Sa%E1%B9%85kh%C4%81ra>

Nothing woo-woo about "conditioned phenomena". And it's not really a "Vipassana" term as much as a philosophical term in general. That's just the Pali name for it.

<comment><USER13> Well, to be fair Goenka uses some rather charged terms as synonyms for sankharas, such as "defilements", "impurities", etc.

</comment></comment><comment><USER15> I once read a book called adrift [1], a true story about a man adrift at sea, alone, for 76 days. The main takeaway that I recall was the profound loneliness and the desire for human contact that he experienced. I sometimes think of this I'm with friends and family, to try to really appreciate the moments that I'm with them. Luckily, I don't feel the need to impose 10 days of solitude on myself to obtain this realization (nor the other realizations the gained by the author of the post).

[1] <http://en.wikipedia.org/wiki/Adrift:_76_Days_Lost_At_Sea> and <http://en.wikipedia.org/wiki/Steven_Callahan>

<comment><USER14> that the author and so many others here are describing fantastic learning experiences suggest to me the word "impose" might be a little off.

even though you don't talk to or engage with people, the experience does not ring of aloneness actually. i felt incredibly connected to other participants and at the end when you begin talking again, there's a wonderful sense of camaraderie. made a bunch of new friends.

</comment></comment><comment><USER16> I loved about the course: \- Amazing food \- Donation/karma-driven sustainability. Sort of Open Source Meditation :) \- Silence (no bullshit talking and no bullshit listening for 10 days) \- Heightened sense of creativity achieved \- Heightened sense of humor achieved \- Great attention to needs to everyone from organizers. \- It's funny - I always gave funny nicknames to everyone.

I didn't mind: \- Goenka parables.

I didn't like: \- After suddenly switching to raw/vegetarian food - you'll be running to the bathroom for an explosive, nasty dragon fart breaks day and night. \- No clear plan to maintain gains. They suggest 1-2 hours daily meditation afterwards which is impossible to maintain.

Overall: five stars (Quebec mountains location)

<comment><USER17> A lot of people discussing the 1 hour in the morning, 1 hour before bed requirement. I also found this impractical, but lately I have been pondering something that Goenka said on this subject: If you don't have enough time, sleep less. When I heard this, it sounded ridiculous, since sleep is so important for the body.

A theory I have been developing is that perhaps meditation that achieves a no- mind state for a long period of time, may allow the practitioner's brain and body to enter the same kind of restorative repair cycle that it normally only enters during unconscious sleep (the only time a normal person's mind is truly shut-off).

A variety of experiences have lead me to suspect this, most profoundly being the night AFTER my 10 day retreat ended, when I was quite tired from a long day, having stayed up past midnight, despite having gotten up at 4:30 am that day for the final day of the retreat.

When I lay down to sleep, exhausted, I fell into a deep meditative state. I got up and opened my eyes, took a walk around the block, went back to try and sleep, and immediately was back in a meditative state. Gave up and just went along with experiencing being in that state all night, since I was totally aware, yet in the morning felt very rested despite having not slept a wink.

<comment><USER2> Yes, there is no doubt that if you meditate two hours a day, you won't need to sleep as much. You can basically assume you can lose about 1 hour of sleep, safely.

</comment></comment></comment><comment><USER18> I went to one of these with a decade of moderate meditation under my belt and just over a year of 'wanting to'.

I got extremely sick from a combination of rhinovirus and detox (I had been smoking regularly); and left on the 5th full day.

My experiences with meditation had been primarily pleasant. I learned accidentally following rote instructions at 18; and meditated regularly on and off (mostly off as years went by) until I heard about this course (which is free, including room and board), and it took just over a year until I sorted out the opportunity to go.

I honestly feel/felt scared/respectful of the whole endeavor. Meditation is what it would be like to learn to breathe underwater, if it were possible.

Our bodies know intrinsically we can't breathe water in; so when you are under water you don't breathe. When oxygen levels are high its not really a thought, but even with oxygen levels are low there is an inherent _aversion_ to sucking/breathing in the water.

Meditation is like that, in the sense our commitment to sit for ~60 minus (or 10 hours a day) is akin to committing to keep our heads under water. The difference is with meditation, the only thing limiting how long you can sit is your will.

By sitting and not reacting to our thoughts we are eventually shown this "layer" where we are unconsciously "holding our breath" in the same way we do when we are under water with high levels of oxygen. Meditation effectively "reduces the oxygen" thereby increasing the signal on the "i need oxygen" circuit, so in the same way that being under water when oxygen is running low, what was previously so subtle as to not be noticed consciously there is now this new, apparent phenomena we can't help but be made aware of.

This is the mechanism, in my estimation, that meditation gives its benefit. Essentially the 'secret' learned while meditation is that "you can breathe underwater" and all this aversion you were feeling was a total waste of energy; energy you didn't even now you were expending.

</comment><comment><USER19> If you have that secret smile from your vipassana practice, I want to share another practice that complements it. It is written in a book called, "Feeding Your Demons", authored by Tsultrim Allione.

In some ways, this method works well to prepare people for vipassana. Yes, yes, I know, you can always note any experience out; but some people are so habituated to their trauma and were never put into a situation where they dedicate themselves to doing nothing and observing everything. They will stare at you incredulously as we babble on how awesome vipassana is.

</comment><comment><USER20> My take on vipassana:

[http://andyjiang.posterous.com/vipassana-meditation-ten- long...](http://andyjiang.posterous.com/vipassana-meditation-ten-long-days- and-a-few)

Humorous account of the ten days.

</comment><comment><USER21> Here is an interesting blog post on the principle behind vipassana:

"He sees the impermanent form thus "impermanent".This is his right view.Seeing correctly he becomes disenchanted.With the waning of delight wanes passion. With the waning of passion wanes delight. With the destruction of delight and passion the mind is de-tached. "Fully de-tached" thus it is said. (Nandikkhayasuttaṃ, Samyutta Nikaya)

<http://tinyurl.com/vipassana-quote>

</comment><comment><USER22> I came across this article on Longreads. The author had a more irreverent take on his 10 day course in India, but in the end, he said it helped him deal with stress better. [http://www.mensjournal.com/magazine/print-view/the-quiet- hel...](http://www.mensjournal.com/magazine/print-view/the-quiet-hell-of- extreme-meditation-20120821?src=longreads)

</comment><comment><USER23> Oh wow, funny to see this on here as I just an hour ago applied to attend a session. Thanks for sharing your experiences everyone!

</comment><comment><USER24> Nice, very nice. I have them bookmarked for a long time but I have yet to fulfill this desire to one day just get to it.

</comment><comment><USER25> I've been to a 10 day Vipassana retreat. I appreciate people say it gave them great benefits, but it did not sit well with me, so I share my story in the interests of balance.

At first, things were fine. I adjusted to the silence quickly enough, the food was simple but nutritious, and didn't mind the meditation (other than the pain in my legs and back, but my request to sit on the back wall was granted). However, mid way through (I think the 5th day), I went to sleep and experienced a sensation I had never (to my memory) had before... as I closed my eyes, I'd experience bursts of not too pleasant images (some of a sexual nature, some more grotesque, etc...). Each image would last roughly a couple of seconds, before moving onto the next. When I opened my eyes, the images were gone, when I closed my eyes the images reappeared straight away.

At this point, I freaked out. I was worried that I'd done damage to my brain. Unable to sleep, and feeling like I had to make conscious effort to breathe, I contemplated what I could do. I didn't want to turn on the light (I had a roommate), nor did I want to spoil the course for anyone else by making too much noise, so I just stayed in bed, awake, for as long as I could (I calmed down a little in this period). I eventually struck up the courage to get up and speak to one of the course supervisors. Speaking to him helped, he advised me to try to rest and he'd arrange for me to speak to the course teacher in the morning... eventually I did get some sleep.

I spoke to the course teacher. He reassured me this was all part of the process (such visions are not unheard of, but of course I can't say for sure if you'll get them too), the way its sold is that its part of your body letting go of these thoughts. I was not convinced. Furthermore, whilst I saw the people running the course (including Goenka) were kind hearted people, I felt isolated, unable to talk to those with a more neutral perspective.

Later on, I also took unkindly to the late night teachings, which again were well intentioned and in some ways comforting, but occasionally said subtly hurtful things. For example, love between couples was described as a business arrangement (I'll love you if you love me). If you weren't paying attention such comments would slip by, but at this point I was trying to fend for my own sanity so I was paying more attention to such things.

That being said, I do believe meditation can be beneficial, just not in the way the course was led (this is just my experience, other people found the course beneficial). If they had taught the technique alone, and asked you to practice it (without trying to tell you what it all means, about your ego slipping away, etc...), then I think I would've taken to it more kindly. Buddha was supposed to have emphasised experience over verbal teaching, so I personally believe teaching the technique with simpler guidance would be appropriate. I also believe meditation is a good way to come to terms with the eventuality of death.

Again, I cannot tell you what you'll experience, and as you can see from comments here it worked for others, but just in case you try it and it doesn't work for you, please know you are not alone.

<comment><USER13> _> At this point, I freaked out. I was worried that I'd done damage to my brain._

That freakout, and that worry (which you apparently have since discovered was unfounded, I hope :) is precisely what the meditation helps you deal with, among other things.

And yes, it kind of sucks that the teachers can't speak more at liberty with you. But there's a lesson there too: in the end, the answer to any trouble you might have is to simply find the craving and the aversion. Find the ignorance in your misery. You were safe and taken care of, and you felt you were at risk. You were wrong.

Blaise Pascal famously claimed that all of man's problems stems from the inability to sit quietly in a room, alone. And we are _not_ able to do it, either. Vipassanna is the rather upsetting path toward learning to be able to sit quietly in a room, alone. And it's very simple, but not at all easy.

<comment><USER25> "You were wrong." In what way was I wrong? Perhaps you don't understand that whilst the panic attack was the 'straw that broke the camels back' so to speak, it was the teachings and approach that I disagreed with most in the end, the panic attack just woke me up to paying more attention to them.

"Vipassanna is the rather upsetting path toward learning to be able to sit quietly in a room, alone." If that's all it was, I would've been happier with it. Being told you create misery, that your love (unless you follow the path) is shallow, that your ego will disappear, being told that Buddhism is 'scientific' and then being told the smallest particles in the universe are 'kalapas' of earth, air, fire, etc... wasn't exactly reassuring. I also disliked the chanting (you don't have to do it, but I personally found it off- putting to listen to).

I actually enjoyed practising anapana and vipassana (less so the third type, can't remember the name), though of course you're not meant to enjoy it. Becoming aware of the extra sensation you have of your body was pretty interesting.

If the path you've chosen works for you, great, but please open your mind to accept it might not work great for everybody. One of the saving graces of the vipassana course for me is they didn't expect you to believe what they were saying, but rather believe in your own experience. That sort of honesty I can respect.

<comment><USER13> _> In what way was I wrong?_

"I was worried that I'd done damage to my brain" implies that you believed your brain was at risk from meditation. You continued through the course, and yet your brain still functions. You assessed the risk to your brain as "high", you came out of it (presumably) without brain damage, ergo your threat assessment was wrong.

To put it another way, you were describing your mind state in a point in time which was incorrectly assessing itself. It turned out that you did not do damage to your brain. You thought your brain was at risk, and your own experience showed that you were wrong to assess the risk so highly.

Yeah, I have (strong) disagreements with the discourses - not just the kalapas thing, but the reference to higher- and lower existences, not to mention the dogma of rebirth. It's mentioned in passing, but it contradicts earlier claims of "non-sectarianism". In fact, it left such a bad taste in my mouth I left my first course (5th night). I went back and almost left again, but mastered my anger. The third time I was less angry. Now I just laugh - the Buddha (and Goenka) are just products of their time, communicating as best they can with the tools they have. The core observations, the core practice, doesn't require any of it.

</comment></comment></comment></comment><comment><USER26> If anyone's looking to find a similar course, I run a website called Connectedtrips thats geared towards just that.

Here's a link to a list of our Meditation courses in the USA:

<http://tinyurl.com/b6zumq2>

</comment><comment><USER27> DISCLAIMER: This post focuses more on the Goenka courses than Vipassana meditation per se.

I'm quite surprised that no dissenting opinion has yet surfaced on the topic of Goenka's courses. Most of the commentor's opinions here seem over- enthusiastic, describing the Goenka retreat as 'the best thing in [their] lives'. To me, this is a serious warning sign.

I have attended a 10-day retreat, which ended prematurely (on the evening of the 8th day) because of my refusal to watch the evening lectures. While I must agree with the commentors' positive opinions on the meditative experience, I feel compelled to inform any potential attendees that despite claims of 'scientific rationality', the Goenka group is a regular religious sect.

When the course begins, the Vipassana technique is presented as an objective tool to achieve insight into the workings of one's own mind and emotions. Mr. Goenka presents himself as someone who merely happened to learn of the technique and is now passing it on.

As the days progress, the scarce sleep and food schedule begins to take its toll on the participants' bullshit detectors, and subtle religious notions are slipped into the agenda. These include:

* The primacy of Vipassana over other religions * The unreasonableness of other religions * The unquestioned truth of reincarnation * The fact that our bodies are made up of some kind of vibrations

I remember finally 'snapping out of it' when Goenka started talking about bubble chambers( <https://en.wikipedia.org/wiki/Bubble_chamber> ) and the depressed scientist who made them. My bullshit detector went on overdrive and I thought about all the things that had been said up to that point during Goenka's video lectures and his taped talks. I realized that I had been subtly manipulated into accepting certain ontological truths, such as the above. These were not things I would have readily accepted coming to the retreat.

I also realized I had been planning to continue with Vipassana practice and make it part of my life. Maybe help organize retreats in my home country. Become part of this 'wonderful' movement headed by Mr. Goenka. This was definitely not something I had in mind when I signed up for the course.

I know I will not convince anyone who is already engaged with the Goenka group to rethink how they got involved, but I would like to sincerely warn anyone who is considering attending:

You will not be alone with your thoughts and emotions during these ten days. You will be accompanied by _THE RECORDED VOICE OF GOENKA_ , for 2-3 hours of each and every day. Every meditation will start with his talks and chants, every meditation will end with his talks and chants. From what I understand, but have obviously not experienced firsthand, the grueling 9-day course ends with a Love-bomb (<https://en.wikipedia.org/wiki/Love_bombing>). I believe these 'free' courses are set up to lure susceptible people into the organization.

Please look into some critical views on any religious group before engaging with one. On the internet, smears abound, but you may also find some grounded critique that may shine light on the structure and purpose of the organization.

<comment><USER28> Having completed a full course, I agree with you regarding the video content. It went beyond practice guidance and veered into disappointing endorsements of woo-woo.

The nightly videos start out objective and practice-focused, but by the middle of the course Goenka suggests quantum-scariness explanations for the sensations meditators are feeling.

As one retreat participant noted, in one of the nightly video talks [0]:

> [Goenka] mixes quantum mechanics with "psychology of enlightenment"

What makes it a bit more of a breach of trust is that by the time he breaks out the full-on woo-woo, you're deep in the process—the days are flying by—you purposely don't have your critical thought facilities active.

Aside from that sour note, I'd still suggest this course to others. Hitting a stride with practice and tuning your ability to focus your attention on minute sensations is a fascinating experience, especially if you've taken any courses on proprioception, sensation and perception.

If you're a rationalist who already firmly rejects the ideas of mind-body separation and reincarnation then you have nothing to worry about.

[0]: <http://www.vipassanaforum.net/forum/index.php?topic=1871.0>

<comment><USER27> The arguments made by the poster in your link really resounded with me. It goes against my ethics to recommend these courses to people who are actively seeking to jump-start their meditative practice.

As far as being a rationalist goes, I'm not confident that I can always keep my mind free from unwanted social/religious influence, or allow myself to be influenced in a controlled manner. I'm glad that this one off-hand remark about bubble chambers actually awakened me from a cognitive stupor, but I can't be sure that, under slightly different circumstances, I wouldn't have ignored the doubts and the red lights going off somewhere in the back of my head.

Cognitive blind spots are a real thing, and my guess is that many Eastern religious groups are very sophisticated in how they use these blind spots to appeal to intelligent, but unsuspecting, individuals. Eastern techniques of religious indoctrination greatly differ from the guilt-based Christian approach that most of us are familiar with.

I stand by my position: these retreats are not simply meditation courses. They are a religious ritual held by a well-organized religious group. Moreover, they are deceptively advertised as non-religious, and herein lies my problem with them.

If someone wishes to learn meditation from a religious source, I think it is wise to refer to an organization that is overt in its practice and beliefs.

</comment></comment></comment><comment><USER29> I am building a hardcore vipassana app based on the noting technique. You can sign up for the beta at <http://mindblownapp.com/>

</comment><comment><USER30> As an adjunct to this article, I should mention an experimental project I launched just yesterday called VOW of NOW. It's a 30 day program that tries to instill meditation as a daily habit by leveraging powerful incentives: <http://vowofnow.com>

A bunch of people are taking the vow already, and I'm excited and hoping it works for them and is the beginning of a more zen life. We'll see. I welcome everyone to try.</comment>


Apple gave Uber the ability to copy your iPhone screen - <AUTHOR> http://mashable.com/2017/10/05/uber-copy-iphone-screen-apple-watch <comment><USER2> [https://hn.algolia.com/?query=apple%20uber&sort=byDate&prefi...](https://hn.algolia.com/?query=apple%20uber&sort=byDate&prefix&page=0&dateRange=all&type=story)</comment>


Scalable and accurate deep learning for electronic health records - <AUTHOR> https://arxiv.org/abs/1801.07860 <comment><USER2> As one of the apparently few remaining people on HN who like Google and are excited about the technology they're producing, I applaud this effort. I've had multiple doctors in my life who were worse for my health than seeing no doctor at all. Two of them outright looked up my symptoms (in a book in one case, and some shitty intranet in the other) to tell me what I already knew from googling it myself. We can do better than that. This is the way forward.

<comment><USER3> until you realize its going to be people like insurance companies using this data to deny coverage and other stuff aimed soley at profiting on the people its harvested from.

How much benefit does the average person get from being profiled like crazy across the web? Besides the (dubious) benefit of free services, there isnt much positive you can point to from such activity, and a shitload of really scummy exploitative crap.

I dont see how such medical analysis is really going to pan out much differently at scale.

<comment><USER4> Here in socialized Europe it will be very beneficial. Our health system focuses on solving problems, not selling us stuff. So this will be another great tool assisting with making us aware of what options we have in improving health outcomes.

<comment><USER3> Plenty of private for profit providers in europe too.

<comment><USER5> Yes, but since the mainstay of the _financing_ isn't really based on private insurance coverage, the main objection doesn't apply. Really, denying insurance is pretty much the only legal-but-bad use case for such information. All other legitimate uses are good for the patient, balanced by an increased risk of crime and loss of confidentiality that may be embarassing but not profitable for anyone (well, except if there's some blackmail case).

<comment><USER3> "all other legitimate uses are good" is a tautology

<comment><USER5> No, saying that they're good _for the patient_ is not a tautology as there certainly could exist legitimate uses that are good for someone else at the patients expense.

</comment></comment></comment></comment></comment></comment></comment><comment><USER6> I have very little hope for this succeeding, although I hope it does.

I say this because this problem is vastly more complicated than people believe for the following reasons:

* A substantial amount of data about patients is encapsulated in clinical notes which are free text. There is no standard format for notes; notes wildly vary in terms of quality, detail, and accuracy; and notes are full of non-standard abbreviations and clinical shorthand.

* A large amount of the data in an EHR is incomplete or incorrect. Information may be out of date or recorded incorrectly. Patient medication lists are a great example of this, and they frequently cannot be trusted by physicians even during the course of a single hospital stay (medications are started or discontinued without being properly recorded). Then there is the issue of non-adherence: patients frequently claim to be adherent to treatment regimens that they are not actually adherent to which is going to distort outcome data. In fact, the majority of people are non-adherent for many different treatment regimens. More generally, diagnoses may have been incorrectly determined and data may be incorrectly labeled or missing.

* Data exists all over the place even within an EHR, let alone across facilities. I myself have been to at least 5-10 hospitals and more outpatient clinics over my life, none of whom share data with each other. No one entity has all the relevant data about a patient, including insurers, specific care teams, or the hospital systems they got this data from.

* Data placed into a EHR may be intentionally falsified for simplicity or to get insurance coverage. It is not uncommon for physicians to intentionally misdiagnose a patient so their insurance will cover a diagnostic / treatment / procedure, or to input a less specific diagnostic code to save time when a more specific one is available.

Trying to train with data sets this fucked up is an enormous challenge.

<comment><USER7> I feel like Google have the power behind them to try to at the very least data mine the free text data for information. Possibly they even have enough sway to convince doctors and other individuals towards a form that is far less random, or at least to provide a platform to share information between patients. Note that NLP seems to have come a long way in five years, who knows where we'll be in a decade or more.

In terms of incorrect records, I'm not sure this would be a major problem over a large dataset. Making this up, but I imagine the range of errors is fairly random across all fields and with enough data you can always identify an approach that appears to save a life more than some other approach, even if the diagnosis is incorrect.

Although I totally agree with you that this is going to be an enormous challenge for Google, I feel like it would be underestimating them to assume this has very little hope of succeeding.

<comment><USER6> I agree that if Google throws enough money and talent at the problem then they can certainly make improvements, but I seriously doubt that they are going to create some sort of revolutionary clinical decision support system which is going to massively improve outcomes.

</comment></comment></comment><comment><USER8> This is a case where discussion will likely be better if HN points to the paper rather than an information-poor popular treatment.

We changed the URL from [https://qz.com/1189730/google-is-using-46-billion- data-point...](https://qz.com/1189730/google-is-using-46-billion-data-points- to-predict-the-medical-outcomes-of-hospital-patients/).

</comment><comment><USER9> If this is for personalized recommendations about the Diagnosis or Plan for a particular patient based on past medical record history on that patient, this might be something that a hospital would want integrated into their current environment.

If this, however, is something that takes multiple patients records into consideration, this would most likely never see the light of day. Most hospitals don't want anyone knowing that, for instance, their city has the highest number of leukemia related deaths per capita than any other city in the US.

<comment><USER10> Doesn't the CDC know that anyway?

</comment></comment><comment><USER11> Electronic bureaucracy has been developed to replace 1 or 2 brains plus paper records. However, put simply, it doesn't work.

(e.g. I had to give my address _three times_ to the same outfit last month just to get a flu jab.)

But we can't go back to paper records so we'll try building a giant brain (AI) to try to decipher and manage millions of electronic records. Trouble is that that brain still won't have _written_ the records or _talked_ with the people they're about. So I doubt it will work.

</comment><comment><USER12> The veracity of any analyses is directly correlated to the quality of the input. I have no doubt that Google's methods are state of the art. My concern is with the input those methods were fed. Sample size needs to be much, much larger on a patient population basis.

</comment><comment><USER13> > Google obtained de-identified data of 216,221 adults...

How much did this data cost?

<comment><USER14> Having been privy to some of the contractual details of deals that Google has made with other medical centers, I'm betting that they probably got it for "free", as in they didn't directly pay a set fee to the universities. Google most likely provided funding in the form of donations (tax write off), free cloud compute resources and/or cloud storage (write off), and the opportunity for university researchers to co-author high impact publications (everybody wins).

Also, 200k patients is actually kind of small. Granted this dataset is far more granular/robust than what you'd typically find in commercially available healthcare datasets, but to give you some frame of reference, the healthcare datasets I work with contain > 20 million individuals (again, with orders of magnitude fewer features).

</comment></comment><comment><USER15> How did they get this data?

<comment><USER16> From the third sentence in the article:

> To conduct the study, Google obtained de-identified data of 216,221 adults, > with more than 46 billion data points between them. The data span 11 > combined years at two hospitals, University of California San Francisco > Medical Center (from 2012-2016) and University of Chicago Medicine > (2009-2016).

I know I'm not supposed to say 'Did you read the article?', so I won't say that.

<comment><USER17> That de-identification is probably the absolute minimum they have to do to achieve compliance, but don't mistake it and think that the data is actually anonymous. We've seen many studies show that 90%+ of the people can be re- identified from such datasets.

Unless Google starts using fully homomorphic encryption or something similar, then you should consider that data non-anonymous. And until that happens, Google and national healthcare agencies should also require consent from the patients before giving the data to Google or any other company for such studies. Otherwise, I hope class action lawsuits will be started against both.

<comment><USER18> Homomorphic encryption currently doesn't exist in any useful sense. So you are asking for the impossible. If we followed your advice, medical record research would be impossible. All research comes with risks. Society is all about balancing those risks with there potential benefits.

Come back with your criticisms when you have actual evidence of Google mishandling data or misleading the IRB boards.

<comment><USER19> > All research comes with risks.

And the core foundation of medical ethics is that the patient gets to decide if/how much of that risk to take.</comment></comment></comment></comment></comment>


Python Headless Web Browser Scraping on Amazon Linux - <AUTHOR> http://fruchterco.com/post/53164489086/python-headless-web-browser-scraping-on-amazon-linux <comment><USER3> PhantomJS is brilliant, but Selenium is a questionable choice for this task. For some reason, the creators of Selenium have decided that passing HTTP status codes back through the API is and always will be outside the scope of their project. So if you request a page and it returns 404 you have no way to find out (other than using crude heuristics). This makes Selenium completely unusable for anything I would have used it for.

Fortunately you can do it by using phantomjs directly instead of going through the Selenium WebDriver API. Maybe one day the phantomjs WebDriver API implementation (ghostdriver) will extend the API to pass HTTP status information back to the caller. Until then, this API is unusable (at least for me).

<comment><USER4> Well, I think the matter is a bit more complicated than that. When dealing with a full browser, you fetch a lot of resources. The status code for the first page fetch may be easily obtained, but your API gets very wonky as soon as you want to get status codes for all linked resources. Even if you managed that, any Ajax requests would complicate things, especially if they have deferred loading. And then you have WebSockets.

There are tools, such as BrowserMob Proxy, far better suited for monitoring HTTP traffic. And they'll get you all the headers. You can even capture to HAR so you measure performance.

<comment><USER3> Difficult edge cases are never a good reason not to support the 99.9% case.

Also, phantomjs has access to all the information you want and the WebDriver API already has a capabilities negotiation facility.

[Edit] Don't forget that the original URL is the only one supplied by the client of the API. It may be incorrect for very different reasons than all the other resources included by the page itself. That's why it is justified to treat it as a special case.

<comment><USER4> These aren't edge cases. They're asked about constantly. Most people are using Selenium because they care about everything on the page. Otherwise, your stdlib HTTP client would be sufficient.

That aside, if PhantomJS already has the info, you can always fetch it with executeScript.

If you do feel that strongly about the status code part though, I'd urge you to comment on the public draft of the W3C spec: [http://www.w3.org/TR/webdriver/](http://www.w3.org/TR/webdriver/)

<comment><USER5> From the point of view of simulating actual users, the fact that some random third-party resource on the page failed to load is not particularly relevant. That happens all the time as I browse around the web, and I never have to care about it as long as the site continues to function. So it very much is an edge case compared to the page itself failing to load.

<comment><USER4> A JavaScript file failing to load will bork most pages. A CSS file failing to load or a key image will cause most people to quit. And an Ajax request failing in a single-page app will render it useless.

But, my point of view is from actual Selenium users. This is framed by providing support on the IRC channel, on the mailing lists, triaging the issue tracker, and by interacting with people at SeleniumConf and the local Boston meetup. It's not some fringe use case and I'm not arguing the point for the sake of arguing it. The original supposition that it's an edge case is not accurate. And sure, the web breaks. That's why people using Selenium would like a way to catch that. And that's a big part of why the BrowserMob Proxy project exists.

<comment><USER6> "A JavaScript file failing to load will bork most pages. A CSS file failing to load or a key image will cause most people to quit."

Wha?

Sure, if, say, "app.js" fails to load, you have a problem.

But an analytics script?

A 3rd party ad script (which is what the GP gave as an example)?

These things can and do fail all the time.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> For anyone using PhantomJS I'd recommend checking out CasperJS ([http://casperjs.org/](http://casperjs.org/)) . It adds some nice features to PhantomJS and takes out a lot of the pain points

</comment><comment><USER8> I find it preferable to determine the requests that jQuery is making and perform them myself to extract the necessary data, rather than load up a whole browser just to do the same thing.

Selenium is _terrible_ , performance wise, and requires a _significant_ investment in environment in order to work reliably. I try to avoid it except when I absolutely cannot.

<comment><USER9> I wound up doing this myself, after spending an undue amount of time struggling with a morass of insanely written Javascript. Fiddler proved indispensable for observing the actual interaction with the web server.

</comment></comment><comment><USER10> If you're writing Python and need to do something like this, you could try using Phantompy, a Python port of PhantomJS: [https://github.com/niwibe/phantompy](https://github.com/niwibe/phantompy)

It's still "in an early stage of development" but it's on my list of libraries to keep an eye on for when I have time to tackle the JS-heavy sites of the world.

</comment><comment><USER11> For scraping phantomjs or casperjs is the best way to go but you will have to use some JavaScript [1]. Both give you access to everything a WebKit browser user does with either a Node-style callback syntax (phantomjs) or a procedural/promises-style syntax (casperjs). Easy to setup, simple to use and fast enough for scraping but only WebKit (for now).

For testing on browsers other than WebKit (or vendor specific WebKit edge cases) use Selenium. Harder to setup, more complex, probably faster (still slow for testing) but not limited to WebKit.

[1] Sorry folks but some JavaScript is required to programmatically interacting with the web - also need some HTML and CSS.

</comment><comment><USER12> One more thing, has anyone used BeautifulSoup for forever? Is the project still active? I mean the website is cute and all, but I find pyquery ( Also based on lxml) so much easier with parsing the scraped data.

<comment><USER13> Something to consider is that the trend the past year has been to use headless browsers over BeautifulSoup, cURL, etc.. because headless browsers are harder to detect by anti-scraping systems and can interpret JavaScript.

<comment><USER14> That's what the OP is about ;-). But BeautifulSoup isn't a way to retrieve a web page, it's a way to parse HTML. You can get the page with a headless browser, and then transfer the DOM into a BeautifulSoup tree to do your scraping.

</comment></comment></comment><comment><USER15> I recently tried to get back into Selenium for a work-related project and, despite its frustrations, it is one my favorite open source gems I found in the last several years. When showing it uninitiated web devs their heads almost exploded from joy and amazement. Your setup with Selenium intrigued me since the pain point for me has become how difficult it is to maneuver some browsers with Selenium IDE to throw together ideas, if that is even encouraged anymore.

</comment><comment><USER16> You are installing some devel-packages, but i don't see anything compiling? Does the selenium installation build native extensions? Then the commands should probably the other way round. Or is phantomjs compiling something on the first run?

Minor nitpick: I don't think it is a good idea to copy a binary directly to /usr/bin, without a package manager. You could just put it into /opt and symlink to /usr/(local/)bin.

<comment><USER17> The file that he is fetching ( phantomjs-1.9.1-linux-x86_64.tar.bz2 ) is the executables for his platform, with some examples on usage and a readme.

<comment><USER18> That doesn't seem like a very safe thing to do... dont they have sc for PhantomJS one can checksum and run ./configure > make > sudo make install?

<comment><USER19> PhantomJS is pretty big. IIRC, building it takes quite some time. I think they bundle webkit and the necessary parts of Qt, and you'd have to be out of your mind to build _that_ from source if you can avoid it.

Using official distribution packages would be a better idea, but their freshness can vary, especially on RHEL.

<comment><USER18> Fair enough if you want to get straight to doing what you were planning to do.

</comment></comment></comment></comment></comment><comment> j-kidd Off topic: it is perfectly fine to install things like PyQt / PySide on a headless server. I suppose the problem is because the distro doesn't provide these packages?

Also, PhantomJS works fine in this case because the binary in the tarball is statically compiled. You can find a whole lot of qt stuffs inside PhantomJS source repository. There ain't no such thing as "truly headless".

</comment><comment><USER20> Wow was searching something similar. Actually was trying to build a app which scraps data from movie ticket booking sites and provides data via SMS to user that whether tickets are still available or not. Because everyone doesn't have access to internet in India yet.

@Steven5158 thanks for the share.

If anyone here wants help in building SMS apps do contact me.

</comment><comment><USER21> We do quite a bit of web scraping / parsing on headless servers with Selenium. What we did was just install some X packages and run VNC server on the headless clients with Firefox. Cool thing about that is you can then go watch the scripts executing if you connect to the VNC session and take a screenshot on failure, etc.

<comment><USER22> Brilliant! I've been using Xvfb for headless operation, didn't even consider using VNC.

</comment></comment><comment><USER23> I am under the assumption the python-requests would have the same issue - <USER2> does not render the page, it only retrieves the original page response.

Very, very good to know when diving into scraping.</comment>


Reasons Apple has peaked—for real, this time - <AUTHOR> http://qz.com/22490/five-reasons-apple-has-peaked-for-real-this-time

<comment><USER2> This article is full on opinions, light on new information.

Horace, from Asymco, provided a [much better analysis]([http://www.asymco.com/2012/10/29/the-iphone-and-apples- margi...](http://www.asymco.com/2012/10/29/the-iphone-and-apples-margins/)) on why the recently announced lower margins are in line with Apple's historical performance during every product launch.

Also, you can get a better perspective by reading [previous comments on HN](<http://news.ycombinator.com/item?id=4714473>) of why Forstall and Browett leaving Apple is actually _positive_ for the company.

</comment><comment><USER3> You known when Apple will have peaked? A bit after the "Apple has peaked" headline meme will have run its course.

</comment><comment><USER4> More rubbish designed to get page views.

The fact is that Apple has a 2013/2014 roadmap of incremental updates that could all continue to sell very well. Retina iPad Mini. Retina MacBook Airs. New Mac Pro. NFC/Passbook iPhone.

Not to mention that Johnny Ive's iOS 7 and OSX 11 could easily reinvigorate both platforms.

And then from the business side Apple is only just starting to tap into India/China both of which see the iPhone as a status brand ala Louis Vuitton. Expect major growth here.</comment>


Stephan Wolfram Stanford Seminar - alok-g http://www.youtube.com/watch?v=x8ZLNxnYslM

<comment><AUTHOR> I do NOT believe this! I read the title of this submission, which glaringly misspells Stephen Wolfram's name. But, before complaining once again about people's inability to spell, I clicked the link to be sure it's not just a case of repeating someone else's error, and the linked page has the name spelled wrong too. Then I played the video, and the title page in the video, originating with Stanford University, ALSO misspells Wolfram's name.

This is unbelievable -- an illiteracy trifecta! The Stanford Center for Professional Development is ultimately responsible, but any of the intervening authors could have caught the error -- and none did.

Is literacy dead? Is text-based search in jeopardy? The man's name is STEPHEN WOLFRAM:

[http://en.wikipedia.org/wiki/Stephen_Wolfram](http://en.wikipedia.org/wiki/Stephen_Wolfram)

<comment><USER2> "did you mean?"</comment></comment>


I now have a chip on my shoulder - <AUTHOR> http://www.andrewcross.ca/2012/11/22/for-the-first-time-in-my-life-i-have-a-chip-on-my-shoulder/

<comment><USER2> This can be pretty powerful motivation. I left law school a couple years ago, and the decision was pretty universally viewed as 'stupid'.

Proving that label wrong really motivated me for the first year. I accomplished more than I ever have.

I sometimes wish I still had that urgency, but on the other hand, my life is more balanced now.

My challenge now is finding a way to be as productive as I was then, but without letting work be everything in my life.

<comment><USER3> It's amazing how fired up you can get with a little adversity. What were you working on after you quit law school?

<comment><USER2> A book. I'm an LSAT instructor. Originally these were sold through an affiliate (still are) and this year I published it.

[http://www.amazon.com/Hacking-LSAT-Explanations-Official- Dia...](http://www.amazon.com/Hacking-LSAT-Explanations-Official- Diagrams/dp/0988127903/)

I also built up a local LSAT tutoring business.

<comment><USER3> Congrats!

<comment><USER4> I find it a little hilarious that the Amazon product page doesn't even explain the LSAT acronym. (Had to look it up)

Then again I'm from Germany and couldn't be less interested in Law Schools - but is that really such common knowledge in countries where it applies?

<comment><USER2> Fairly common knowledge amongst college-educated 19-30 year olds in the US and Canada. Not well known outside of those groups.

My book is only of interest as a companion to a specific set of LSAT tests. So if someone doesn't know what the LSAT is, my book is completely inapplicable to them.

Pretty niche, but not a bad niche.

</comment></comment></comment></comment></comment></comment><comment><USER5> I would be careful.

I don't recommend continuing a startup when the primary motivation is to show up others who didn't believe in you. You should be working on your idea because you believe it will be successful and ultimately help people.

A day may come... it might take a few months it might take a few years. When you realize that your startup has very little chance of success but you think "I can't quit... If I do then those who didn't believe in me won!"... I think generally that's not a good idea.

Yes, there have been countless success stories... "When I was 12 my teacher said I wouldn't amount to anything!" And while yes that motivation can (rarely) drive a company to success, it can more often than not lead to misery.

YMMV

<comment><USER3> Thanks for the warning. By no means is showing up others my primary motivation, but it will be very useful when the times get tough (which they will).

</comment></comment><comment><USER6> Am I the only one who was expecting an article on electronic implants?

<comment><USER7> I expected a similar pun. Maybe someone will now have a chip on their shoulder to have a chip on their shoulder.

</comment></comment><comment><USER8> Without commenting on your product I'd like to be frank about why this mentality can be unhealthy.

Motivation is great, but sometimes the fuel that motivates you isn't always healthy. The way that your article reads comes off as having a point to prove _because_ you were rejected.

Analyze the psychology of that for a second. You're indirectly implying that you will make decisions because of rejection. Your problem solving and decision making will now gravitate towards whatever bias that "chip on your shoulder" created.

This may seem obvious, and you may be thinking you're impervious to such a fallacy, but let me point out a few warning signs in your article.

1) "... Investors look for it since it's an indicator of persistance. When you've got something to prove, you're more likely to blast your way to success. But how do you actually get this fire?"

Do investors really look for this? These statements come off like a confirmation bias.

2) "And with a chip on my shoulder, I find myself more focused, thinking more clearly and getting stuff done."

Can you quantify why?

I've been in analogous situations a few times in my past and observed it in many colleagues. I've rarely seen healthy work come out of it.

In summary, while there is room for personal bias and creative preference when building a startup, you shouldn't take your motivational fuel lightly. It's really easy to let the ego get in the way of doing what's right for your company.

<comment><USER3> Interesting take, I'll provide a bit of extra clarification. All good points to raise though.

"The way that your article reads comes off as having a point to prove because you were rejected."

In order to prove a point, my understanding is that I would have to have been rejected or given negative feedback. If it was a positive response, everyone would be in alignment and I wouldn't have a point to prove.

"Your problem solving and decision making will now gravitate towards whatever bias that "chip on your shoulder" created."

The chip on my shoulder/bias is a pure desire to succeed. The rejection was an implicit statement that all the other companies appear to have a higher chance of success. I see this desire to succeed as a very good context to make decisions from.

"1) "... Investors look for it since it's an indicator of persistance. When you've got something to prove, you're more likely to blast your way to success. But how do you actually get this fire?"

Do investors really look for this? These statements come off like a confirmation bias."

Mark Suster wrote a whole article on it. [http://www.bothsidesofthetable.com/2012/05/17/what-to-do- abo...](http://www.bothsidesofthetable.com/2012/05/17/what-to-do-about-that- chip-on-your-shoulder/)

"2) "And with a chip on my shoulder, I find myself more focused, thinking more clearly and getting stuff done." Can you quantify why?"

I don't track my time so I don't have any quantitative numbers. With that said, I find I'm able to fight distractions more effectively with an "I've got something to prove" mindset.

</comment></comment><comment><USER9> How is this different from FlightFox (YC S12)?

</comment><comment><USER10> maybe you should think about why you're doing your startup again if rejection is what you need to get more passionate or fired up about it.

<comment><USER11> We don't always know what we need to get fired up until it happens; it's often different for each person. There are plenty of famous people who do credit rejection as giving them the kick they needed; Michael Jordan does very clearly, for example. Or, "Can't sing. Can't act. Can dance a little.". That one worked out okay too.

</comment></comment><comment><USER12> Finally.

I'm back in town in Jan, lets meet up...just say if I can help.</comment>


Silicon Valley's rank and file prepare to fight Trump - <AUTHOR> http://www.recode.net/2017/1/10/14201450/tech-trump-secret-meeting-labor-strike-pressure <comment><USER2> I'm going to have to chalk this all up to virtue signaling. Where were all of these heroic freedom fighters during the last eights years of unprecedented mass surveillance and growing executive power?

But now that Trump and his insane cabinet are in power I'm to believe these people are serious about personally "fighting" this beast? If the future of freedom hangs on the whims of out of touch SV tech workers, I'm afraid we're all in for a rocky ride. These people will give up as soon as they can't get their morning cup of Philz.</comment>


A method to use Google for DDoS. Bug or Not?  - <AUTHOR> http://chr13.com <comment><USER2> Nice catch. I'm not so sure about:

      A simple fix will be just crawling the links without the request parameters so that we don't have to suffer.

Many links would fail/have different content if the request parameters were removed from the URL. Perhaps the crawler could use some kind of reverse bloom filter [1] to be more careful/back off if it receives the same content from multiple URLs. However nothing is simple at Google scale so there are probably issues with this approach too.

[1]: [http://www.somethingsimilar.com/2012/05/21/the-opposite- of-a...](http://www.somethingsimilar.com/2012/05/21/the-opposite-of-a-bloom- filter/)

<comment><USER3> You can always change that to

        =image("http://targetname/1.jpg")
        =image("http://targetname/2.jpg")
        =image("http://targetname/3.jpg")

<comment><USER4> But what if 2.jpg doesn't exist? Or is a trivially small file?

The advantage of the querystring-method is that you can just find one suitable (i.e. huge) file and force Google to pull it down many times.

</comment></comment></comment><comment><USER5> I'm not surprised at Google's response, since this looks to me along the same lines as putting lots of images in your signature in a popular forum; although in that case it is really is a _D_ DoS.

Maybe Google should consider putting a bandwidth limiter of some sort on that (or even better: use hashes to avoid duplicates), but I think screaming "security! vulnerability!" is not a good action to take here...

<comment><USER6> How could Google use hashes to avoid duplication? They'd have to download each link before they could hash the contents thereof, so the damage would still be done.

<comment><USER7> _How could Google use hashes to avoid duplication?_

Rate limit per website (e.g. don't download more than 10 images per domain per second)

Limit the total number of images it downloads per document, so a single user can not cause too much traffic.

<comment><USER8> In that case, users may notice a performance decrease in spreadsheets for images from certain websites.

</comment></comment></comment></comment><comment><USER9> I've seen this bug floated around a few times, with the request parameters and all. Interestingly enough, you do not have to use an image either, and can link to any document on the server. In addition, it will work with nondeterministic values. So you can do (for example):

        =image(CONCATINATE("http://example.com/?", RAND()))

If you add this to a spreadsheet and fill a few thousand rows with it. Each time the spreadsheet is loaded, google will hit the server a few thousand times.

</comment><comment><USER10> The other huge problem here is that Google's FeedFetcher doesn't respect robots.txt. (Their reasoning is that it is acting at the direct request of a human to retrieve a specific resource, so it doesn't count as a bot.) Because of this, there is no easy way to _stop_ it from hitting your site.

<comment><USER11> You can block the user agent, I believe "Feedfetcher-google" should work.

<comment><USER10> True, but (while possible) it's not straightforward to block access to specific files only. The same user agent is also used for Google Custom Search if you're using that. And it's still going to be hammering your firewall (although admittedly that's less catastrophic than trying to download a 10MB file repeatedly).

</comment></comment></comment><comment><USER12> I've decided to stress test this idea. This is what google errors on:

"It's taking a while to calculate formulas. More results may appear shortly."

I set the spreadsheet document to load images like so: =image("[http://example.com/image?id=146&r={increment](http://example.com/image?id=146&r={increment) here}")

After 30 or so images, google starts to slow down its fetch rate.

<comment><AUTHOR> Multiple spreadsheets ?

<comment><USER12> 30 per spreadsheet it would seem. It seems to be that if the document takes longer then 2 seconds to load, it starts limiting itself.

</comment></comment></comment><comment><USER13> I wonder how much visibility on HN it requires for the switch from "not a bug" to "definitely, for sure a bug" to happen?

</comment><comment><USER14> > Since Google uses multiple IP addresses to crawl it's also difficult to > block these type of GET flood

It wouldn't be too hard to block by User-Agent: Mozilla/5.0 (compatible) Feedfetcher-Google; (+[http://www.google.com/feedfetcher.html);](http://www.google.com/feedfetcher.html\);) if you notice the traffic.

Feedfetcher does not fetch robots.txt though; so you'd have to do something in your server config.

[edit: fixed a typo, and agree with the update]

<comment><AUTHOR> Thanks, I've updated the post.

</comment></comment><comment><USER15> This is about two years old: [http://www.behind-the-enemy- lines.com/2012/04/google-attack-...](http://www.behind-the-enemy- lines.com/2012/04/google-attack-how-i-self-attacked.html)

I would hope that Google is able to detect abuse of their infrastructure for (D)DOS.

<comment><AUTHOR> Indeed, I've quoted that article. But it doesn't talk about random parameters which makes it so easy to attack any website not just your own where you know what the urls are.

<comment><USER16> Nothing mind blowing, same vulnerability really, there are many ways to extend the core issue.

</comment></comment></comment><comment><USER17> Nice catch!

I don't think removing the parameters would be ideal, though, since some sites might legitimately serve up different images based on different parameters.

Just limiting the amount of traffic to a single server, or outbound from a single spreadsheet, seems like a good solution, though.

<comment><AUTHOR> Yes of course. But then do those dynamic images serve any purpose on a spreadsheet ? If a user needs a dynamic image he can download it to his own machine and upload it. Of course if he need many dynamic images, then that's another question.

<comment><USER5> > If a user needs a dynamic image he can download it to his own machine and > upload it.

Doesn't that somewhat defeat the purpose of a dynamic image?

</comment></comment></comment><comment><USER18> I'd say if Google can send this much traffic automated it is most certainly a bug. They should engineer some sort of upper limit to the amount of traffic sent to a single ip so as not to perform denial of service attacks on it.

</comment><comment><USER19> Doesn't Facebook do something similar for preview links in chat and/or wall posts? You're probably limited by the number of messages/posts, but I wonder if that could be exploited with n number of fb accounts.

<comment><USER20> There are reports skype does it to.

<comment><AUTHOR> Don't want to disclose anything at this point but at least one other huge bandwidth owner suffers from this type of attack. Combined, it is clearly a disaster for any small-medium business.

</comment></comment></comment><comment><USER21> A better solution might be to throttle multiple requests to the same domain. Also, they could prevent it from fetching the same content multiple times and use a cache instead based on ETAGs and file information.

</comment><comment><USER22> Fetching the content without request parameters is not a fix, as the author claims. This depends entirely on the server and the way the content is stored, but request parameters can be used to determine what sort of content is fetched, and removing those parameters can form an invalid request.

</comment><comment><USER23> I guess a way to protect from this would be to set some forwarding at the web server level, so that any url for static files would forward to its canonic representation. After the forward I think the file shouldn't be downloaded again.

</comment><comment><USER24> Many of the questions here are already answered officially by Google: [https://support.google.com/webmasters/answer/178852](https://support.google.com/webmasters/answer/178852)

<comment><AUTHOR> May be they should visibly put that FeedFetcher can crawl your website can incur 1TB bandwidth in couple of hours ? May be apache and other httpd should block these crawlers by default ? How else would anyone know about a certain Google feature that could be disaster for them ?

</comment></comment><comment><USER25> DaaS - DDOS as a service :)

<comment><USER26> DDOS as a Service has been around for years, the difference is that Google is trying to muscle in and undercut the competition with a free service. Unlike the extant players their customer service is nonexistent, so I'm sure the big DaaS companies won't lose too much business...

<comment><USER27> Imagine the DDOS providers filing an antitrust case against google :P

</comment></comment></comment><comment><USER28> Even kiddies like me can DDoS now... Where shall I begin... ;)

</comment><comment><USER29> That's pretty clever, good find! I wonder if you can do the same stuff in Office 365, etc.?

</comment><comment><USER30> Any googlers wanna comment on this, especially if you're from the docs division?

</comment><comment><USER31> My hat tip to you. Nice find!

</comment><comment><USER32> Reminds me of the kind of article I used to read in _2600_. Nice post!

</comment><comment><USER33> please post a how to tutorial. like where do i post the image list?

</comment><comment><USER34> Fascinating!</comment>


Obligatory Designer Clock - <AUTHOR> http://www.saikatbiswas.com/web/Projects/Obligatory_Designer_Clock.htm

<comment><AUTHOR> I've seen that page before but then had a hard time today to find it again (lots of google and HN searches!). The idea is interesting and simple.</comment>


Show HN: Apply to YC directly from a Google Doc - <AUTHOR> https://docs.google.com/a/fireid.com/document/d/1tRb6lFr07u6KACmT71dsGc2pin8-6RWnuyXsX7c2byI/

<comment><AUTHOR> A quick hack that lets you update (not submit) your Y Combinator application from a Google Document. We found using a Google Document for collaborative editing of our YC application worked great, but pasting in the data to the form manually is a pain. So I wrote a script to automate it.</comment>


It's Not Just HP And Autonomy - <AUTHOR> http://techcrunch.com/2012/11/20/its-not-just-hp-and-autonomy-the-enterprise-software-space-is-a-giant-stinking-mess/

<comment><USER2> Kragen Sitaker wrote a wonderful message about what "Enterprise" software really is: [http://lists.canonical.org/pipermail/kragen- tol/2005-April/0...](http://lists.canonical.org/pipermail/kragen- tol/2005-April/000772.html)</comment>


Boston Airbnb regulations cause owner of apartment building to make it a hotel - <AUTHOR> https://www.universalhub.com/2018/boston-airbnb-regulations-cause-owners-small <comment><USER2> It was a hotel before and it is now properly labeled as such. Airbnb is for hotel rooms w/o regulation, so good that Boston got the building properly relabeled.</comment>


The Poor Mans Spotify is awesome - <AUTHOR> https://pypi.python.org/pypi/Poor-Mans-Spotify

<comment><USER2> This just seems to be a cli app for pleer.com, which seems to have a massive cache of mp3s. Has nothing to do with spotify, and is downloading music from that site is likely breaking copyright laws in the us and other places.</comment>


Ask HN: Machine learning for scientists? - <AUTHOR> What's your recommendation to get up to speed on current practices in machine learning for someone ~Phd-level in the hard sciences? In other words, someone experienced in linear algebra, statistics, programming, and experiment design+evaluation, but not in the specifics of machine learning.<p>I'm thinking courses, tutorials, blog posts, papers, books, software, instructive problem sets, etc. <comment><USER2> ML is a huge field. What kind of problems are you trying to solve?</comment>