The End of Reflection - <AUTHOR> http://www.nytimes.com/2016/06/12/fashion/internet-technology-phones-introspection.html?pagewanted=all <comment><USER2> Donald Knuth on email:

"Email is a wonderful thing for people whose role in life is to be on top of things. But not for me; my role is to be on the bottom of things. What I do takes long hours of studying and uninterruptible concentration."

[http://www-cs-faculty.stanford.edu/~uno/email.html](http://www-cs- faculty.stanford.edu/~uno/email.html)

<comment><USER3> I've never heard of or been part of any organization (academic, non-profit, corporate, open source, etc.) that respects the basic need of developers for uninterruptible concentration.

Note also that it is a particular word choice by Knuth: uninterruptible ... _not_ uninterrupted. It harkens back to what Paul Graham wrote in both _Maker 's Schedule, Manager's Schedule_ and in _Great Hackers_ \-- even just having a mid-morning meeting on the calendar at all can prevent you from even attempting something hard at the start of your day, because you know the time is already interrupted.

You need more than just happenstance of not being interrupted in a given time window. You need to know that you _can 't_ be interrupted, at least not unless it's a sky-is-falling kind of emergency (and you need to be able to trust that your colleagues know the difference).

This is far and away the number one thing I need as a developer. I can write passable software that I'm not proud of if it's loud, violently anti-privacy, over-saturated with "collaboration" generally in some ultra-Agile open-plan nightmare setting. But the only times I've ever written software that I was proud of -- software that made huge, huge differences in the organizations I was part of -- was when I could finally find a loophole or something to create a situation when it was _not possible_ to interrupt me with the usual bullshit.

It amazes me that even still, status effects and incredibly poor understanding of where the value of software engineering resides leads to so many organizations that not only have interruption-is-the-norm open-plan disasters for offices and a million Agile meetings, but actually praise that garbage and hold it up as if it is an example of positive collaboration or effective engineering, despite it being the very antithesis of positive collaboration or effective engineering.

<comment><USER4> Oh, it isn't just me! My wife doesn't understand why i have a horrified look on my face whenever she opens my office door to look for something or pass through.

<comment><USER5> I have a small office in a large closet upstairs that I use at night for work, my SO has a converted room downstairs as her office for her home business, so there's not a lot of cross traffic. Unfortunately, that means she IM's me quite a bit, which since it's even lower cost than walking 20-30 feet if she's already at a computer, I'm not entirely sure isn't worse. :/

<comment><USER3> The number of people here who have both a private home office and a significant other depresses me.

<comment><USER5> Why else would you want a _private_ home office? If you don't have a significant other, there's no need for a separate private space.

In my case, I work from home two days a week, and I have children. I separate space is _essential_ to get work done. It's a matter of having a private space that allows me to be home if needed, and save time for with my family, or not being productive enough to make it economical for the business to allow.

<comment><USER3> I was trying to express that I am envious of people who are able to have a significant other and a comfortable home with space. Having even just one of the two sounds like a dream world to me.

<comment><USER5> Fair enough, I interpreted it differently. Good luck on the search, in both respects. :)

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> Anecdotally, switching back from an iPhone to a flip phone has been very revealing for me. I don't know about all the generalizations made in this piece, or decrying overall social change. And I expect I am an outlier as being far more distractible than most.

But I do think it's worth spending a week living your normal life but without a smartphone to see how you personally are different. Don't just go on vacation and "disconnect". Try living your life but disconnected. It's interesting. You have to get places earlier. You have to make plans that have pre-arranged fallback plans. You have to have something to think about to not get bored. (Or, at least, I do.)

<comment><USER7> I want to do this - except I have 2-3 apps I feel I could not live without.

Google Auth for 2-factor been the main one.

<comment><USER8> If you use 1Password, they have 2-factor auth built-in.

Read more here: [https://support.1password.com/one-time- passwords/](https://support.1password.com/one-time-passwords/)

<comment><USER9> I didn't realize they had that feature. Doesn't that kind of defeat the purpose of 2FA to have your password and your second factor in one single app?

</comment></comment></comment></comment><comment><USER10> I am actually worried about myself for spending too much time with the smart phone. Even at the bathroom, while I stand peeing I browse stuff that I actually don't care that much about. It is a compulsion. Probably should get treatment. I find that I've started to lose the ability to read and concentrate on longer pieces of text. I hardly read books anymore, which goes into things at any depth.

I used to take long walks where I thought about difficult problems and found solutions. I would ponder these problems on the bus, or subway. I never mined being alone with my thoughts before. I never understood how people could be bored when there are so many things one could be thinking about.

Yet here I am in 2016 incapable of being alone in my own thoughts for more than 5 minutes before pulling up the iPhone. I starts to feel more like a bad drug, than a usable tool.

If anybody has good suggestions for phones or ways of configuring the iPhone so that I can only read email. I am not addicted to email. It is the browsing functionality which is so addictive.

I also wish I was not wasting my time on HN writing this comment ;-)

<comment><USER11> Just get a dumb phone. I don't have a smartphone because I'm sure I'd fall into the same trap.

It's occasionally difficult, but usually just freeing.

</comment></comment><comment><USER12> I read the first two sentences, and then I immediately closed the tab. I then decided to give it a second thought seeing the response on HN, and I see my first gut reaction was correct.

I'm 27, and I know people who are younger who surprise, surprise are reflective. I stay about 45 minutes away from my office by bus. By bike it takes 35. I like the bike ride, but initially, I missed the bus rides because I had time to contemplate on the bus...yes, with my phone in hand. May be I am the minority, but even with my phone I think a lot because I decide to and to be honest, my phone isn't as interesting as my thoughts...frequently. Technology has always been an augment to my life, it isn't my life. When I didn't spend 5 hours on my phone I probably spent 5 hours playing video games or 5 hours doing something else. This feels like another thinly veiled $(echo Rock and roll is corrupting the youth | sed 's/Rock and roll/Marijuana/' | sed 's/Marijuana/Grunge Music/' | sed 's/Grunge Music/Video Games/' | sed 's/Video Games/the Internet/') argument--it also is another article that indirectly targets Millenials ( attacking the selfie, then the tweet ) and that their culture is "unreflective"...yes, the same generation that drinks, smokes, is more educated, and has less teen pregnancies than prior ones. Possibly, some of those 5 hours were spent self-inflicting one's self with cancer in generations past, which I would argue is quite unreflective.

The phone and the internet is not going away, many introspective introverts like me still do think and still like it. Just because we have written language, it doesn't mean the corrupt youth know less since they no longer have to memorize. In fact, as Aristotle was dead wrong on Newton's 1st Law, I offer that written language, like all technology, just makes us better thinkers...it all comes down to how we use it.

<comment><USER13> Agreed. The article speaks more to the awareness of the author than the collective awareness of the public.

Every tool will be used by people however they choose. Tech can enhance or lower intellect. It's how you decide to use it that matters.

Collectively, I'm not convinced this age is becoming less aware of itself. There are some awesome things out there. Creative, artistic things in all walks of life. I have my own list of things that won't apply to everyone. It just depends what you like. If the world isn't producing something that you think should exist, then make it. I suppose, for this author, that is this article.

</comment></comment><comment><USER14> Active reflection on past experiences has many positive effects on psychological wellbeing and I'm quite surprised that they weren't mentioned in this article.

Artie Konrad (disclaimer: a labmate of mine until he graduated recently) focused on the effects of Technology Mediated Reflection (TMR) which deals with user's keeping a sort of personal diary and being prompted to reflect on past experiences routinely. Overall this increases mental wellbeing significantly with some interesting effects on how memories are evaluated over time and how the reflective process affects current moods.

<comment><USER15> I would expect the opposite - ruminating over past problems will make them worse. It clearly does for some people, but you're saying it's different for other, do you know where the distinction lies? If you ever find out please send me an email. Thanks!

<comment><USER14> Great question. Their result in this area did surprise me. Conscious reflection on past negative memories slowly turns them into less negative and less painful memories. There is a lot of work here done by Pennebaker about how writing about past negative events transforms our memory of them [1]. And the work done by my lab started off with a system that facilitated these reflections through technology [2]. There is more recent work on this in press.

I think the difference lies in the structure. Rumination is symptomatic of depressive thinking and often characterized by intrusive negative thoughts. Reflection on the other hand is the conscious process of sitting down and reevaluating a past situation (often in writing).

1\. Pennebaker, J. W., & Chung, C. K. (2011). Expressive writing: Connections to physical and mental health. In H. S. Friedman (Eds.), Oxford handbook of health psychology (pp. 417-437). Oxford, England: Oxford University Press.

2\. Isaacs, E., Konrad, A., Walendowski, A., Lennig, T., Hollis, V., & Whittaker, S. (2013). Echoes from the past: how technology mediated reflection improves well-being (p. 1071). ACM Press. [http://doi.org/10.1145/2470654.2466137](http://doi.org/10.1145/2470654.2466137)

<comment><USER13> Meditation/Mindfulness might lead you to some more insight on this subject. It teaches to observe pain when it comes up rather than ignore it. The book Mindfulness In Plain English is pretty good. Or, Tuesdays with Morrie. Observe emotion, then let it go.

To the scientific mind, mindfulness might seem hokey. I certainly thought so for years until I saw it promoted on HN with some scientific research backing its efficacy. Then I tried it and was impressed with the results.

<comment><USER16> To add another book recommendation: "Mind Illuminated", which reads like a college textbook, straight and to the point yet very in depth.

<comment><USER13> Sweet thanks, I'll check it out.

</comment></comment></comment></comment></comment></comment><comment><USER17> I get constantly distracted at work by people asking me questions, needing something etc. My solution has been to leave the office and go for a walk. No one knows where I am and I get to think quietly to myself while walking.

I have my phone with me and I just leave it in my pocket.

</comment><comment><USER18> This whole question of what to do with one's mind. Introspection sounds nice but it's not always useful.

What I'm kind of trying to do with technology is study or learn for many hours every day, it's just that I'm not doing a great job, because the stressful approach makes it hard to find and approach real valuable material.

I'm also trying to keep up with some kind of professional community as well as some cultural forms of community. I appreciate the professional or hobbyist communities because basically they don't revolve around Donald Trump and I can pretend the world is basically sane.

I've reflected a lot but not really gotten anywhere. I really value two different three month periods in the past few years when I haven't had internet access at home at all, though. Having hours every day when my mind wasn't hooked up to high frequency information systems was pretty wonderful.

There's something in Bob Dylan's memoirs, the first book which is the only one I read (I don't finish books anymore), about how his "news" for a while was the history of the Byzantine empire (or something) because that's what he was studying at the time.

I'm more and more interested in the visceral feeling of "offline" and I think it's a source of refreshment that is becoming scarce (this ought to be a considerable driver of interest in meditation retreats).

<comment><USER13> > Having hours every day when my mind wasn't hooked up to high frequency > information systems was pretty wonderful.

I feel like this is what many people would describe as a period of introspection. You are temporarily closing out far away communications and narrowing focus to immediate surroundings.

You can get more of this by focusing on one object, or closing your eyes. If you do it for 20 minutes a day, you might find yourself more capable of handling other stresses. The skill is currently called being mindful or aware, and you can strengthen it, as you observed, by closing out externalities. After closing your eyes, the idea is to close out thought. It is a challenging and useful exercise. There are some books on mindfulness that could say more.

</comment></comment><comment><USER19> _That_ kind of reflection. I was starting to think inspecting the types from within a language wasn't considered good anymore ...

</comment><comment><USER20> Try this 4x4 digital detox. Repeat every 4 months. Customize to your habits.

Prereq = Delete feed apps, turn off notifications

Day 1, 2 = No feeds, forums, news

Day 3 = Introduce News, forums limited to 2 slots of 15 mins

Day 4 = Introduce feeds limited to 1 slot of 20 min

<comment><USER21> Not sure why you're being downvoted. This might not be everyone's cup of tea, but this is certainly a valid idea.

Tangentially, I've thought about resetting my phone once a month to make sure I only have installed what I want/need. But as a first step, I disabled _23_ notifications I didn't need -- kept just SMS and WhatsApp.

</comment></comment><comment><USER22> The thing about the way the human mind works is that you often can't find out what would be most rewarding, or why you are acting as your are, good or bad, unless you spend some quiet time slowly paying attention to your unclear feelings and various experiences.

Meditation can help a great deal with this, ditto keeping a journal. So can engaging in artistic expression such painting or writing poetry. Ditto reading great thinkers or novelists. I particularly like the focusing method developed by the philosopher and psychologist Eugene Gendlin.

You have to understand, social media corporations don't want you to know what you really want, because it often would turn out to be something other than what they are selling. So they keep you distracted with mental candy and an unending worry that you are missing something important.

</comment><comment><USER23> _The last remaining place I'm guaranteed to be alone with my thoughts is in the shower._

Teddy Wayne clearly hasn't discovered LifeProof cases.

When I first got a LifeProof case many a friend received a photo of water coming out of the shower head and the words "just having a shower".

But anyway, yeah we spend too much time on our phones. But so what? There are much worse things. Out of the 20 or so <25 year olds at my work place only three of them smoke cigarettes. Who cares if we use our phones too much. If we go to work, pay the bills, and generally don't get too involved in fucking people over, do whatever you like on your phone.

At least smartphones are interactive, at least to some extent.

<comment><USER24> > Teddy Wayne clearly hasn't discovered LifeProof cases.

Tangentially related but I hadn't discovered LifeProof cases prior to you mentioning it. These things look _amazing_. I'm ordering one for my phone right now. Less for using in the shower (I take really fast showers) and more for using in the rain and not worrying about an accidental drop in a puddle.

(And maybe I'm the outlier here but I don't want to go back to having a not- smartphone. I remember using flip phones like my StarTac as well my complete tank of a Nokia 3390--complete with Snake--but I also enjoy being able to e-mail and text with my friends and read books and play music all from one device. And I'm in my mid-30s so it's not like I've never lived when we didn't have smartphones.)

</comment></comment><comment><USER25> I wanted to check my phone less, and one thing that helped was setting the screen to grayscale (iOS: Settings > General > Accessibility ). When things are less shiny, you're less inclined to compulsively have a look. Tip from The Atlantic: [http://www.theatlantic.com/video/index/480240/adventures- in-...](http://www.theatlantic.com/video/index/480240/adventures-in- grayscale/)

<comment><USER26> Thanks for the tip. Is grayscale better for your eyes also? I'd imagine it would be.

</comment></comment><comment><USER27> All this thread makes me wanna walk on a mountain and do yoga for a few months.

<comment><USER15> Go to Vipassana instead. 10 days make dramatic difference.

</comment></comment><comment><USER28> When I want to reflect, I go for a drive. It takes me through places I don't see every day, and ensures I keep my hands off my phone. When I get home, I will have cleared my head, and I often feel I just had an adventure.

<comment><USER21> It also works just as well to leave your phone at home and go for a walk. We existed for a long time with just landlines; you weren't at home when Bob called? That's fine. Just because we _have_ cell phones, we're not required to be accessible 24/7.

</comment></comment><comment><USER29> The single best thing you can do for your productivity and peace of mind is leave your phone on airplane mode unless you are expecting an important call, or need to make a call. Anyone trying to reach you that you actually want to hear from will leave a message, and the hassle of having to switch out of airplane mode keeps you from compulsively surfing, using facebook/twitter/etc.

<comment><USER21> I've found that the act of logging out of Facebook after using it generally has the same effect. If I find myself casually loading Facebook, but it takes me to a login screen, I remember that I wanted to be purposeful about using Facebook and put it away.

<comment><USER30> I removed all social media apps from my phone. I will still occasionally access them from my laptop now but usually when I'm using my laptop I'm working so the temptation is a lot weaker.

</comment></comment></comment><comment><USER31> Two observations: first, I would probably underestimate my smartphone use by the criteria they provide. It wouldn't occur to me to describe checking the time as 'using' my phone, but it would count, because it turns the screen on. I'd guess that half the occasions I turn my screen on, it's to check the time.

Second, I easily use my phone for five hours in a typical day, because I like to listen to music. But listening to music is no barrier to reflection or introspection at all, rather the contrary, at least for me.

</comment><comment><USER32> Maybe I reflect less now than I did before I had a supercomputer in my pocket. I am not certain. However, today I have more and better information to reflect upon than back then. Of that I am certain.

When I turn off, tune out and drop into a book it's more out of the habit of an older generation than some quest for a quality inaccessible to the well wired. I don't think that being wired has made people less contemplative so much as it facilitates encountering those who aren't now and would not have been in the past.

</comment><comment><USER33> Agree that reflection and contemplation is important.

Likewise, writing your thoughts out is also powerful and sometimes, being forced to edit your point down to one or two 140 char snippets is not such bad thing either.

Similarly, being exposed to the views of people you would likely never meet in pre-internet/social media times is no bad thing either IMHO.

"Everything in moderation" and "Variety is the spice of life" are aphorisms that persist for a reason.

<comment><USER34> We were writing out our own thoughts before the internet and also cutting the length down(see graffiti, sms).

Being exposed is not the same as being constantly bombarded with _new_ content. By the way, you're not really being exposed to that differing views as Google, Facebook, etc are putting you inside a bubble filled with things you like.

Moderation is hard when companies are working hard to make you use their tech as much as possible (more looking, more clicking -- more money).

</comment></comment><comment><USER35> So increased smartphone usage may be impairing our reflective abilities, but to play devil's advocate, what is it imparting instead which is beneficial? As the article notes, a thought may be tweeted out to others instantly and developed in a public forum instead of in isolation in the thinker's head. That may very well result in better thoughts, particularly for people who never had very complex ideas in the first place.

</comment><comment><USER36> For the last few weeks I've been not listening to podcasts or music on my morning walk to work (half hour). I deliberately did this to give myself time to be more introspective or creative. Realizing after X minutes that you've just been lost in thought is kind of satisfying.

</comment><comment><USER37> The only problem I have with my feature phone is the fact that it has opera mini which is capable enough to write this comment. On the first week I broke the email when trying to update it. Facebook on the other hand is kind of interesting. it takes 2min to send a message.

</comment><comment><USER38> Initially thought this was an argument against meta-programming till I saw the source. I got very defensive for no reason. |D</comment>


Free Minimal Resume PSD Template - <AUTHOR> http://despreneur.com/free-minimal-resume-psd-template/

<comment><USER2> The template looks nice but having it in PSD means it can practically used only as printed copy. Translating it to a doc (any format that can be exported to PDF) or html/css would be much more valuable.

still, good job.

<comment><USER3> PSD > PDF is easier than DOC > PDF.</comment></comment>


Damn Cool Algorithms: Fountain Codes - <AUTHOR> http://blog.notdot.net/2012/01/Damn-Cool-Algorithms-Fountain-Codes <comment><USER2> This sounds really similar to secret sharing[1]. While not doing _exactly_ the same thing, a simple secret sharing algorithm is also interesting to read about now--it's a different algorithm chunking up and reassembling data in a similar way with different properties.

One difference is that with secret sharing unless you have enough chunks you do not have _any_ information about the result. (That's why it's called secret sharing--you can share a secret with a bunch of people without any of them knowing it unless a minimum number get together.) The neat trick is that _any_ subset of the chunks big enough will get you all of the information.

[1]: <http://en.wikipedia.org/wiki/Shamir%27s_Secret_Sharing>

In fact, the algorithm (at a high level) is exceptionally simple: encode your data as the coefficients of a polynomial of degree n and transmit it as a bunch of points in the polynomial. You then need n+1 points to recover the polynomial, but which particular points you have does not matter.

What I love about this algorithm is that it is useful, nontrivial and can be explained in a single sentence (by a better writer than I :)).

<comment><USER3> Another way of phrasing the same concept:

An infinite number of Euclidean planes can be drawn containing a single point, but you only need three of these—and it can be any three—to find the original point.

<comment><USER4> (if the original point is contained within a line that is also known. For example, if the original/desired point is on the x-axis)

</comment></comment></comment><comment><USER5> Alas, digital fountains are tied up in patents, and people have been leery:

[http://www.eecs.harvard.edu/~michaelm/postscripts/itw2004.pd...](http://www.eecs.harvard.edu/~michaelm/postscripts/itw2004.pdf)

<comment><USER6> I know there are lots of anti-patent folks on HN, but if anything deserves patent protection, digital fountains certainly do. This is useful, novel, and certainly non-obvious stuff. And really quite beautiful, IMHO.

Its always bothered me to read about inventors who discovered beautiful and amazing things, only to die penniless while powerful interests reaped all of the financial benefits. Hopefully the people who discovered these codes have been richly rewarded for their work.

<comment><USER7> > I know there are lots of anti-patent folks on HN, but if anything deserves > patent protection, digital fountains certainly do.

I'd suggest that the usefulness of algorithms like fountain codes (and other innovative algorithms like RSA and many compression techniques) leads to the opposite conclusion: such algorithms prove so broadly useful that patents on them have even more of a negative effect than usual.

<comment><USER8> How does it have negative effect? Consider three cases: 1) an algorithm is designed and patented, and the inventor has the right to use or license it. 2) an algorithm is designed and never published, i.e., held secret for proprietary use of the inventor. 3) an algorithm is designed and put into public domain.

With 3), the inventor never gets a chance to commercialize his invention: as soon as he publishes the algorithm, he will be destroyed (or assimialted) by large players. With 2), we never know how many cool, useful algorithms are rotting away in commercial products. As far as the society is concerned, the algorithm in question does not exist. With 1), the inventor benefits, AND also the society (in the long-term).

IMO, the patent system should be reformed so that having a large portfolio of _unused_ patents becomes a liability. For example, the patent holder would have to periodically (e.g., every 2 years) apply [with fee, of course] for patent renewal. For this, the holder must be able to document that he is actively exploiting the patent (1). There should also probably be a minimum limit for the required income from patent exploitation. Failing to do so, the patent would automatically expire. The patent office should publish an on-line database of expired patents.

There should also be restrictions on what can be patented: for example, work funded by government funds must not be patentable: the inventors' (in this case, mostly researchers at universities) day job is _by definition_ designing things that will benefit society, are paid for that by the society, and therefore have no further rights to patent protection. (This is, IMO, fair since research is [should be] free, so the society has no firm guarantees that the researchers will produce something valuable, or anything at all.)

(1) By exploitation I mean either making a device covered by the patent, or licensing it to somebody who making such device.

<comment><USER7> You seem to suggest that algorithms won't get published if their authors can't patent them and commercialize them. Given that the vast majority of algorithms don't get patented, and that the vast majority of patents never get licensed except in bulk cross-licensing deals, this seems almost entirely untrue.

Aside from that, I'd point out that in the rare case when an algorithm _does_ get published as a patent and licensed, that makes it entirely unusable in Free and Open Source Software as well as in open standards, for a period of ~20 years. At least when the algorithm remains unpublished, someone can re- discover it (or something more useful that would build on it); patents force willful ignorance of an entire class of algorithms, as well as anything in the general vicinity of those algorithms.

</comment></comment></comment></comment></comment><comment><USER9> Why does the described algorithm only decode transmitted blocks when the receiver has decoded all but one of the constituent blocks? For example, if the receiver has received the following 3 blocks:

      1. A ^ B ^ C
      2. A ^ B
      3. B ^ C

it can combine #1 and #2 to decode C, then combine C and #3 to decode B, then combine B and #2 to decode A.

The algorithm as described would in this situation cause the receiver to wait until it's received a single block, either A or B or C, before decoding anything. This strikes me as inefficient.

Edited to add: I think this is analogous to forward vs. backwards chaining: you can either start by combining single blocks with composite blocks to simplify them, or start by combining blocks whose component block lists differ by only a single block. Or you could apply both, which should get the greatest simplification with the smallest amount of input.

<comment><USER10> Normally one would use some distribution of the number of encoded source blocks in a block. So one would send 30% single blocks, 30% double blocks, 15% triple, and so on until you get some relatively small number of high order encoded blocks. This means that while you can find a solution the way you posted it is a rare case that you have no single block for decoding and since in any case you need to receive a bit over 100% of the blocks before you can decode anything there is no real reason to go to exquisite lengths to find the perfect solution when the trivial case will work 100% of the times (well 99.9999% if you cared to be accurate :-)

The algorithm is also not so much about computational efficiency, to check at any point if you can or cannot decode requires quite a bit of work anyhow. The main advantage of the algorithm is in its relative low complexity and high efficiency on the transmission line.

<comment><USER9> My point isn't about computational efficiency, but bandwidth efficiency (or alternatively, decoding the full message as early as possible). As I understand it, the Moore's law doubling time for bandwidth is longer than for CPU speed, suggesting that processing is getting cheaper relative to bandwidth, so using more CPU in order to have to send fewer blocks is probably a long-term win.

<comment><USER11> The decoding amounts to solving a system of linear equation where each equation corresponds to one received packet. Assuming the file to be sent was broken into n parts, minimum n independent equations are required for reconstruction. To do that optimally, receiver needs to keep trying to solve the linear equation system once it has received at least n packets. This means O(n^3) in general case for each received packet. If the equations are generated randomly, the expected overhead seems to be a mere 1.61 extra packets! [http://virtual.vtt.fi/virtual/abi/abi- seminar07/abiseminar07...](http://virtual.vtt.fi/virtual/abi/abi- seminar07/abiseminar07-tuomas.pdf)

In practice however, depending on the receiver, you may not want it to solve a general system of linear equations for every packet it receives when n is big. It might be better to stick to a simple algorithm (which has a better time complexity if certain conditions are satisfied by the linear system) and just wait to accumulate enough equations for it to become applicable.

<comment><USER12> Raptor codes require O(1) time to generate an encoding symbol. Decoding a message of length k with a belief propagation decoding algorithm require O(k) time for the appropriate choice of inner/outer codes. [http://en.wikipedia.org/wiki/Raptor_code#Computational_compl...](http://en.wikipedia.org/wiki/Raptor_code#Computational_complexity) See also: [http://www.inference.phy.cam.ac.uk/mackay/itprnn/ps/555.573....](http://www.inference.phy.cam.ac.uk/mackay/itprnn/ps/555.573.pdf)

</comment></comment></comment></comment></comment><comment><USER13> Where do people keep these algorithms around? I've asked in the past about reference guides for stuff like this they throw Knuth's "Art of Computer Programming" which is a neat book but doesn't have esoteric and useful but hard to find stuff like this.

Where do I find stuff like this?!

<comment><AUTHOR> <http://en.wikipedia.org/wiki/List_of_algorithms> (seriously! Scan it, choose something that looks interesting and you'll have hours of fun ahead of you.)

<comment><USER13> Thanks, but neither fountain nor rateless codes appear on that page. :(

<comment><AUTHOR> No, but it does link to Forward error correction, which links to fountain codes as well as rateless codes.

</comment></comment></comment></comment><comment><USER14> Whats the advantage of Fountain Codes over more well known and deterministic forward error correction techniques like Reed-Soloman Codes?

The D.J.C.MacKay paper linked from the article lists applications in broadcast and data storage, but Reed-Soloman would work equally well in both those applications.

Hypothesis: With Reed-Soloman and friends, if you're unlucky and loose the /right/ n blocks, for small n, you can't recover your data. Is that n larger for LT? If I could maliciously corrupt your LT blocks, would I have to corrupt more with LT?

Hypothesis: If I randomly corrupt blocks with Reed-Soloman, does my likelihood of not being able to recover all my data drop faster than if I did so with LT?

Can someone say authoritatively?

<comment><USER15> The advantage is you don't need to know the channel characteristics ahead of time. In particular, if you used a Reed-Solomon code, you'd use one designed for one particular erasure rate. But that code would be too weak for a channel with a higher erasure rate (it would fail to decode) and it would be too strong for a channel with a lower erasure rate (it would be wasteful and use too many parity bits). The advantage of fountain codes is that they work efficiently whatever the channel's erasure rate.

By the way, I've been talking about channels with an erasure rate (the Binary Erasure Channel) because fountain codes were originally invented in that context. But you can find in the literature papers on "Rateless codes for noisy channels" that show that the fountain codes also work fine for bit- flipping channels like the binary symmetric channel and the additive white Gaussian noise channel.

<comment><USER16> > But that code would be too weak for a channel with a higher erasure rate (it > would fail to decode) and it would be too strong for a channel with a lower > erasure rate (it would be wasteful and use too many parity bits). The > advantage of fountain codes is that they work efficiently whatever the > channel's erasure rate.

So could we think of fountain codes as a kind of adaptive on-the-fly RS code in which one sends a block protected against one's best guess of the erasure rate and if the recipient fails to decode it one resends with a higher erasure rate, or if the recipient succeeds one sends the next block with a lower erasure rate?

<comment><USER15> More or less, but it's really best not to think of them in terms of RS codes at all. The reason they're called "fountain codes" is that they're used like a water fountain. The transmitter just starts sending out bits for a file like a water fountain spews out water. The receiver(s) collect bits until they've got enough to reconstruct the file. It's like they're collecting water in a bucket, and when they've got enough water (good bits) they can reconstruct the file. The receivers with a good channel are like people standing close to the fountain who can catch lots of bits. Those with a bad channel are like people standing further away. But everybody can eventually fill up their bucket. And the transmitter is indifferent to what channel they've got.

</comment></comment></comment></comment><comment><USER17> Didn't know about that algorithm, pretty cool. Also, the way the author explained it made it a joy to read.. clear, simple examples, meaningful graphics. Can't wait to start reading the previous blog posts (About other cool algorithms).

</comment><comment><USER18> Seems the website has gone over quota. Shame, because the content is absolutely fantastic. Had it opened and had begun reading it on my computer at home.

<comment><USER19> Here's a mirror: [http://blog.notdot.net.nyud.net/2012/01/Damn-Cool- Algorithms...](http://blog.notdot.net.nyud.net/2012/01/Damn-Cool-Algorithms- Fountain-Codes)

<comment><USER20> NYUD gives me a 324 error (no bytes returned). This one worked for me: [http://webcache.googleusercontent.com/search?q=cache:blog.no...](http://webcache.googleusercontent.com/search?q=cache:blog.notdot.net%2F2012%2F01%2FDamn- Cool-Algorithms-Fountain-Codes)

</comment></comment></comment><comment><USER21> It wasn't immediately clear to me the ideal soliton distribution always summed to one. There is a simple proof by induction. I wrote it up here: <http://kevinlawler.com/soliton>

</comment><comment><USER10> It is a damn cool algorithm, I was introduced to it in a startup I worked in and we actually tried to make use of it. The algorithm as described has a problem to converge fast enough and we had to make some extra structure on top that made it converge very fast and in a very consistent way. We used it for multicast communications where feedback to the server is impractical and such codes are making it a very effective way to distribute the data.

</comment><comment><USER22> Luby had a startup named Digital Fountain in SF in the late 90's where they attempted to productize this, primarily for video streaming.

Looks like they were acquired by Qualcomm a few years back: [http://www.qualcomm.com/solutions/broadcast- streaming/media-...](http://www.qualcomm.com/solutions/broadcast- streaming/media-delivery)

</comment><comment><USER23> Fountain codes look very interesting. There was a project at OLPC, I think written in Forth for OpenFirmware, that used something like fountain codes to reflash massive numbers of computers. It worked like this. Turn one XO-1 on in OFW mode, insert a USB with the disk image you wanted to be on all machines, and turn it on as the source server. Boot into OFW, the other XO-1s you want to reflash, and tell them to receive a new image. I think there was a key shortcut of some kind that told a machine to receive. The server XO would continue to send packets until all other machines had received enough packets to make up a complete disk image. If I remember correctly, they used it to reflash 3000 machines in the field that I heard about. There may have been more elsewhere or since.

</comment><comment><USER24> Gray on black text. :~(

To make the page readable, pull up the dev console and run `document.body.style.color='black'`

<comment><USER25> Or you could zap colors:

[http://www.russellbeattie.com/blog/the-zap-colors- bookmarkle...](http://www.russellbeattie.com/blog/the-zap-colors-bookmarklet)

<comment><USER26> That bookmarklet comes from squarefree (and looking at his about page, Russell Beatie should learn to credit other people): <https://www.squarefree.com/bookmarklets/zap.html>

<comment><USER25> I guess it's an encoded version of the same JavaScript, all right.

I expect I was first introduced to it through squarefree, but the Beatie page is a simpler place for a surfer to land for that one specific thing, which is why I chose it. I didn't realize it was also a straight-up, uncredited copy.

</comment></comment></comment></comment><comment><USER27> It's somewhat intuitive, but I feel like there is some cool math or attempt at explanation behind this that was left out.

Why is this better than just broadcasting messages that always contain a single block? What do you gain by forming combinations? You still have an expectation of eventually getting all the information. I assume it just turns out to take longer on average because your expected gain in information is lower from single chunks once you already have many of them?

<comment><USER7> If you only broadcast messages that contain a single block, anyone who already has that block gets no value out of the message. On the other hand, if you broadcast messages generated via fountain codes, messages give you almost a full block worth of information on average.

If you assume the same network environment that the article does, namely one where no feedback occurs from the recipient to the sender, then if you miss a single block in the message, you'll have to wait for the sender to send that one block again. Since the sender has no information about what block you missed, you'll have to wait on average for half the blocks to get retransmitted, resulting in 50% overhead or more. It gets worse if you miss enough blocks that you have a non-trivial chance of missing those blocks _again_. (On top of that, some transmission mechanisms have properties that make particular data patterns particularly prone to loss, which makes it significantly more likely that you'll lose particular blocks repeatedly.)

With fountain codes and similar mechanisms, on the other hand, every message provides several chances to obtain the missing blocks, resulting in much less overhead.

</comment></comment><comment><USER28> One wonders how it does rate limiting to avoid pointless resource consumption of faster, near links when the packets are going to be dumped by a slower, farther link.

In particular, if you intend to coexist fairly with TCPIP on an internet you might find your rate limiting code carries all the most of the work and timing issues of TCPIP anyway.

<comment><USER10> The fountain code itself is not concerned with it, it's merely the coding layer. There needs to be a communications layer that takes care of that and the way it does that will need to consider co-habitating with TCP.

In the application I worked on we used multicast and implemented a form of multicast congestion control that was orthogonal to the coding but exploited the fact that we can send however much data as we wanted and the only thing that changed was the time it will take to complete the transfer. It worked beautifully with the coding and co-habitated nicely with TCP.

</comment></comment><comment><USER29> Isn't this network coding?

<http://en.wikipedia.org/wiki/Network_coding> <http://research.microsoft.com/pubs/67246/tr-2004-80.pdf>

</comment><comment><USER30> This seems most useful for broadcast. You could just broadcast the fountain code of a file out as long as you have listening receivers; the only feedback needed would be "send to me" and "done"

</comment><comment><USER31> Thanks for this post. Patented or not, this is damn cool.

<comment><USER32> Arg. I thought algorithms weren't supposed to be patentable. I know there are workarounds (like referring to the "algorithm" as a "system"). But it's frustrating nonetheless.

For the curious, Knuth wrote an interesting letter on the topic back in 1994: [http://www.pluto.it/files/meeting1999/atti/no- patents/brevet...](http://www.pluto.it/files/meeting1999/atti/no- patents/brevetti/docs/knuth_letter_en.html)

<comment><USER33> We had a patent officer visit our senior design class in engineering school and this question came up. We were all of the opinion that algorithms were purely mathematical expressions, as Dr. Knuth insists in the letter you cited. I don't recall the patent officer ever giving a precise answer on the subject, but I do recall considerable dancing as he attempted to explain why math isn't always math to a room full of engineers.

<comment><USER32> Actually, I ran into a relevant discussion on the topic with a few choice quotes (from [1]):

 _When filing an algorithm patent, patent lawyer will tell you to remove every instance of the word "algorithm" and replace it with some other word. This seems to be necessary for getting the patent through._

 _[...] at some point you may find yourself pulling diagrams from thin air and thinking "why am I doing this?" (but then remembering that it's because of the bonus)_

The complacency in the latter quote is (probably) why so many well-meaning engineers inadvertently exacerbate the problem.

[1] [http://yaroslavvb.blogspot.com/2011/02/how-to-patent- algorit...](http://yaroslavvb.blogspot.com/2011/02/how-to-patent-algorithm-in- us.html)

<comment><USER7> Having worked with a patent attorney in the past, I can give you specific wording examples. The software patents I've seen stated the claims in three different ways, to take advantage of different patentable areas: "A method for $foo", "A system for $foo" (as in a computer system), and "A computer program product for $foo".

Those same software patents also included, in addition to all the illustrations of the algorithm, a diagram of a stack of computer discs capable of containing the program, in order to demonstrate that (at least in some embodiments) the patent actually covered a physical _thing_ rather than an algorithm.

</comment></comment></comment></comment></comment><comment><USER34> heavily patented</comment>


Anti-Piracy Outfits Launch Attack on BitTorrent Protocol - <AUTHOR> http://torrentfreak.com/anti-piracy-outfits-launch-attack-on-bittorrent-protocol-120519/

<comment><USER2> Leave it to TorrentFreak to take a couple sentences of hypothesis from a thoroughly researched analysis and blow it into a story. Reading the original piece, it is clear that the source and purpose of these packets is unknown, and behavior has been observed which wouldn't necessarily match up with a poisoning attack. It is unfortunate that TorrentFreak couldn't just report the anomaly accurately, and instead had to spin it into something completely unfounded.

<comment><USER3> Well it seems very similar to the Pirate Pay system, [https://torrentfreak.com/microsoft-funded-startup-aims-to- ki...](https://torrentfreak.com/microsoft-funded-startup-aims-to-kill- bittorrent-traffic-120513/)

</comment></comment><comment><USER4> TorrentFreak are just amazing at pandering to sites like HN.

"The security researchers don't make any conclusive claims about the origins of the attacks, but they do note that anti-piracy groups are a possible source."

</comment><comment><USER5> Pirate Pay? LOL, at least they have a sense of humor.

Are they only corrupting certain files -- i.e. the movies they target -- or do they cause a general damage to BT? It wasn't really clear in the artice if these "false positives" are only showing when downloading certain files, or generally on BT, regardless of what you exchange.

</comment><comment><USER3> This might well violate FCC rules. It's equivalent to jamming a communications channel.</comment>


Ask HN: Getting into low level audiovisual codecs and graphics - <AUTHOR> As an intermediate programmer, what are the goto resources to get into low level programming on audi video codecs and computer graphics? <comment> d--b Before getting to the code, you need to understand the principles, which are getting quite complex. I would start by reading how mpeg-2 worked, it's sufficiently simple. Then move on to h.264, you'll see a shitload of innovation there, not all of which is easy to grasp. And then look into state of the art stuff.

The recent video codecs are incredibly complicated, and as a solo developer, it would be pretty hard to develop a codec by yourself. You may be able to invent some feature that improves the coding slightly, but coding efficiency is getting pretty hard to beat.

What to read and what code to learn is difficult to say depending on your objective. If your objective is getting a job in video coding, it's not too hard cause these jobs are quite academic and you don't get paid much, so if you know some coding and show some interest in learning the field, I don't think it'd be too hard for you to find an intern position in a lab or something.

<comment><AUTHOR> Can you recommend a reading resource? I was looking for MPEG-1 and MPEG-2 specifications, and to the best of my knowledge, they are behind a paywall. I will probably buy them if there is no other choice other than the specs.

<comment><USER2> There are a lot of relevant-looking links in the references section of this page:

[https://en.wikipedia.org/wiki/MPEG-1](https://en.wikipedia.org/wiki/MPEG-1)

Isn't everything relevant linked to there? If not, you'll be able to get it free here [http://gen.lib.rus.ec/scimag/](http://gen.lib.rus.ec/scimag/)

</comment></comment></comment><comment><USER3> If you really want to do a practical hands on deep dive in Graphics: I would strong recommend [1].

Just work through this book and you are set. There is a a LOT of demand and short supply for good 3D graphics programmers. I would say it should be either Media or Graphics; both may be an overkill! Good luck.

[1] [http://www.amazon.com/gp/product/B00LAMQYF2/](http://www.amazon.com/gp/product/B00LAMQYF2/)

</comment><comment><USER4> video codec is an undocumented mine field. If you want to know how to decode and encode video, looking at libav's example code or gstreamer's example code is the best way.

If you want to understand the fundamentals. The best I know is this [https://people.xiph.org/~xiphmont/demo/daala/demo1.shtml](https://people.xiph.org/~xiphmont/demo/daala/demo1.shtml)

</comment><comment><USER5> if you want a fun introduction, I suggest becoming part of the demoscene !</comment>


Tech's gender gap is getting worse, not better - <AUTHOR> http://www.techrepublic.com/article/techs-gender-gap-is-getting-worse-not-better-report-says/ <comment><USER2> If girls don't want to pursue Computer Science, why force them to? The majority of girls simply don't enjoy programming. Some (like my sister), do. The rest are free to pursue whichever other career they prefer. Although the media tries to paint it as a matter of nurture, it's mostly a difference in nature between men's and women's preferences.

The only reason this push to get girls into STEM exists in the first place is because of all the money and opportunities that are pouring into tech careers. You don't hear the media complain about the gender gap for boiler repair technicians or the reverse gender gap in nursing.

Fred On Everything shared a similar opinion: [http://www.fredoneverything.net/Coders.shtml](http://www.fredoneverything.net/Coders.shtml)

</comment><comment><USER3> Considering most male-dominated occupations are due to them being physically demanding, I do agree that software development stands out as a bit of an odd- one out if you look at the list below.

[https://www.dol.gov/wb/stats/occ_gender_share_em_1020_txt.ht...](https://www.dol.gov/wb/stats/occ_gender_share_em_1020_txt.htm)

But as you can see, the gender imbalance problem is no worse than those of healthcare sectors.

Where is the same political outcry about under representation of male nurses, even though RN is highly paid, physically demanding work. Why aren't we pushing men to go into caregiver fields where the imbalance is much worse?

Is it so taboo to say that perhaps it's because women excel at some of these fields than men? And vice versa?

Look, it's good to acknowledge that the imbalance exists, and do an honest root cause analysis and try to remedy it.

But this has be become such a politicized topic that they're making documentaries about this, and even private sector CEOs are getting involved. Why?

<comment><USER4> Also where's the outrage that there aren't as many female car mechanics? And how come we're not pressured into referring to them as "she"? And why don't we call nurses "he" like we're supposed to call programmers "she"?

</comment></comment><comment><USER5> How about "larger, not smaller" rather than implying a moral judgement?

<comment><USER6> These days, inequal outcomes are inherently immoral. God forbid individual members of society have different preferences and capabilities...

</comment></comment><comment><USER7> That is what happens when women choose not to study computer science.</comment>


The oldest problem in computing - Recipes - How do you solve it? - <AUTHOR>

When trying to explain to the general public why a computer in your house would be useful, it always seemed to boil down to recipes. From the $10,600 Honeywell H316 Pedestal (with cutting board) in 1965 to 1980s Apple II ads in the 80s to articles about the internet in the early 90s. I even observed an argument for a tablet here on HN where convenient  access to one's recipes was the primary demand.
I still can't do any better online then I would do with a modest kitchen bookshelf. Obviously it's no problem pulling up a recipe for anything. ABout.com seems to have a top search result for just about an 'recipe x' query. But there's no way if figuring out which is better. You usually even end up with less pictures then you get in a cookbook. If I want to go a step further (eg 'an introduction to making pâtés), I get nowhere.
I have two questions for HN:
- How is it possible that this isn't solved yet?
- Where do a learn about pâté? <comment><USER2> I think the answer lies in "choice and themes". When you select a book for the kitchen shelf you have selected an author whose recipes you trust or you have selected a particular niche. Hunting for a recipe in your bookshelf is relatively "safe" and the limited choice an advantage when you come to actually making a selection (albeit on available ingredients or the style or mood of a meal).

Compared to the relative intimacy of a recipe book a computer based search (or worse, probably, an Internet search) seems sterile in comparison.

The answer might be somewhere in web 2.0 land though. Would you trust the recommendations of friends more than those on your bookshelf? Could the software learn what you like and come up with better selections in future? Solving this problem might require an understanding of just what it is in any given dish that characterises it and thus appeals or not to any given consumer. Analysing those characteristics could provide an excellent basis for a recommendations engine.

<comment><AUTHOR> Well one very web 1.0 solution would be a big online cookbook with good recipes & articles & such. I'd settle for that.

<comment><USER2> Such exist ( as a search for just "Recipes" in Google shows) but as I thought your question, that is the foundation of this thread, pointed out - they do not seem to represent a "solution" to the computer in the kitchen.

<comment><AUTHOR> Well, 'good' is important. I also need to know that it's good.

<comment><USER2> We are getting there - you want some sort of peer review.

However, will you just go with a majority or would you prefer the positive reviews of a selected group? Tastes vary! Much as I love my visits to the USA I think US food is far too sweet so I would probably avoid the guidance of west Coast hackers. I also know that food styles vary - I live in France and here food tends to be simply prepared (well in the countryside anyway) and dishes concentrate on a simple combination of flavours - perhaps with more dishes served in succession during a meal than would be normal elsewhere. Thus a recipe suitable for my neighbours might fail if presented to some visitors from the UK.

So - back to my original response I think...

<comment><AUTHOR> Actually, I wouldn't worry too much about differing tastes. I think that there is some sort of semi-objectivity to food. A good recipe is a good recipe.

Accounting for tastes could be taken care of reasonably well in well named recipes: 'country style ratatouille' Thai style ratatouille, a new twist on an old favorite.'

But anyway, I think there's more to it then just recipes. I want to make pate. Could be trout, could be pork, could be goose. Where do I start. What kind of pate do i want. A really good solution would let you find new foods.

<comment><USER2> Ah - I see. You are looking for a recommendation for an on line (presumably) recipe resource. Not quite the sort of thread for HN.

<comment><AUTHOR> Is it not? I'm saying that the problem identified as the home computer's raison d'etre is not solved. Is it is, I want the solution (like I said, I want to make pate). If it isn't, I think working out why is an interesting sort of a thread.

maybe not.

Anyway, if you can come up with a way to recommend a recipe specifically for my tastes, available ingredients & all that Jazz, I'm in. But first I'd like to know why the simple solution isn't working.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER3> Hardware. It's a hardware and interface problem. Books are pretty battle- hardened for the kitchen. They do not require batteries or a power source. They do not give up the ghost if you drop them into cake batter, and most importantly they carry on their pages a permanent record of previous adventures written in pencil and flecks of flour dough.

<comment><AUTHOR> Assume you're willing to use a computer. Still no real good solution.</comment></comment>


Russian Programmer 3D-Prints His Very Own T-800 Terminator Robot - <AUTHOR> http://www.odditycentral.com/news/russian-programmer-3d-prints-his-very-own-t-800-terminator-robot-complete-with-artificial-intelligence.html <comment><USER2> British Programmer 3D-Prints His Very Own Ultron Robot [https://www.youtube.com/playlist?list=PLpwJoq86vov-v97fBMRfm...](https://www.youtube.com/playlist?list=PLpwJoq86vov-v97fBMRfm-A9xv8CzX8Hn) [https://www.youtube.com/playlist?list=PLpwJoq86vov-C5SldDA-A...](https://www.youtube.com/playlist?list=PLpwJoq86vov-C5SldDA- AhxesVHPRk74x)

But it's not as flashy a title as with "Russian". :-)</comment>


Instagram Platform Update - <AUTHOR> http://developers.instagram.com/post/133424514006/instagram-platform-update <comment><USER2> Disappointed! I made an app for the Pebble watch this summer to let you view photos from your feeds, and now two of the three services I use are "deprecated".</comment>


Fan page with 26 million fans hacked and distributing affiliate stuff - <AUTHOR>

I have no idea if this will get any response, but I'm hoping someone in the HN community who sees this will have the contacts in or out of Facebook to get this sorted:
Rihannas Facebook fan page which has 26 million fans has been "hacked" for days now, it's repeatedly posting links to affiliate stuff (sign up to 10 rewards get a free ipad etc) and it's getting annoying. My voice won't get heard, so wondering if anyone can get it looked into? I went as far as emailing Mashable (who will report on anything) to hope they'd "investigate" it by using their contacts, but alas nothing happened.
I say hacked, I suspect it's either someone at her PRs firm doing it for personal gain, or they've approved an app for wall access that is being used by the developer for nefarious purposes.
http://www.facebook.com/rihanna/
http://www.facebook.com/rihanna/posts/10150100761776676
If they remove it again (normally the links stay alive for 15 mins and then are deleted: http://samuel.imgur.com/42A6V#cR0wd)
current link is: http://apps.facebook.com/rihannaspecial/
If anyone knows where I can send this and get it noticed, please say :-) <comment><USER2> Contact anyone at her manager, Rocnation: <http://rocnation.com/about/>

<http://www.linkedin.com/pub/megan-miller/20/8aa/941> <http://www.linkedin.com/pub/fab-leys/23/669/1a6>

</comment><comment><USER3> this hacker helps me alot. i would recommend him. his email is email@example.com</comment>