European Spacecraft Prepares to Land on Mars Next Week - <AUTHOR> http://www.space.com/34341-european-spacecraft-mars-landing-next-week.html <comment><USER2> Kinda sad to see that it does not have any cameras apart from the landing camera.

Rosetta just captured the imagination of loads of people, yet this one will be a yawn-a-thon for the average member of the public without pictures.

I am sure the science will be great, just us Luddites like pretty pictures :-)

</comment><comment><USER3> The landing sequence appears less complex and risky than the skycrane solution for Curiosity. Does anyone know what the tradeoffs between the two options are?

<comment> T-A Curiosity weighs 900 kg. NASA decided that it was too heavy for retrorockets (which would kick up too much dust and create holes in the ground around the rover) and so large (almost 3 meters across) that airbags would be too heavy [1].

Schiaparelli weighs 600 kg, closer to the size of the old Viking landers (576 kg), which did fine with retrorockets.

[1] [https://science.nasa.gov/science-news/science-at- nasa/2012/3...](https://science.nasa.gov/science-news/science-at- nasa/2012/30jul_skycrane)

</comment></comment><comment><USER4> Why is this touted as a European (only) mission?

It's a joint mission with Roscosmos they built the lander (ESA is building the rover and probe) and the rocket is Russian.

</comment><comment><USER5> From another article: [http://www.space.com/32254-exomars-mars-mission- launches-orb...](http://www.space.com/32254-exomars-mars-mission-launches- orbiter-lander.html)

> _But these instruments will likely operate for just a few days, until > Schiaparelli 's batteries run out. The probe's primary purpose is to prove > out the entry, descent and landing technology needed to get the life-hunting > ExoMars rover on the ground several years from now._

It seems _crazy_ that they're shipping this thing all the way to Mars to get just a few days of data, and then to let it rust forever on the surface.

<comment><USER6> I agree, it seems like the marginal cost of making it a longer-lived and more powerful rover are low once you've gone to the trouble of putting 600 kg on the surface of Mars. Worth noting, though, that this mission is also putting a satellite into orbit.</comment></comment>


Ask HN: What is social media? - <AUTHOR> I am curious, how do you understand the phrase "social media", of which we talk constantly. What, for you, are defining characteristics of a social medium? How recent of a phenomenon are they? Obviously, Facebook or Twitter will likely jump to the front of your mind, but would you call a blogging platform (such as wordpress.org or medium.com) social media? Would you call an old-school forum (a bulletin board powered by phpBB) social media? Would you call reddit social media? What about GitHub? What about Slack or Discord (and eventually, irc channels)? Finally, what about Hacker News itself?<p>(Asking because I usually hear the phrase "social media" used as a synonym for "Facebook or Twitter".) <comment><USER2> How easy it is to get your name out there, based on market factors. They tend to include: Coverage, Reach, Cost, Consumer confidence and Frequency. Which is to say anything and everything in form of media that gets your name/business/trade/news/and form out there where others can communicate based on your thoughts/ideas/forms for a fully open feedback-loop. In terms of economics, it's an offering and those that would purchase/consume your offerings having an open space to communicate their wants/needs/complaints.</comment>


What do you think of our new team page? (type "dance" for party mode) - <AUTHOR> http://wistia.com/about/team <comment><USER2> Weird how many negative comments you're getting. I for one absolutely love it. I think it's energizing and the awkwardness+energy is fun.

It's definitely not the "typical SF thing" etc, but to anybody outside California, this feels refreshing.

Great job

<comment><USER3> Agreed definitely something that should and possibly be shared virally and or bookmarked or stumbledupon.

A start-up is about creating/doing your own thing, making a living off of it(hopefully profit from it) and most of all having fun!

</comment></comment><comment><USER4> Too many chiefs and just one craftsman. This definitely rings a bell. It's such a shame for startups to "enterprisy" their job roles.

<comment><USER5> I think that can come in handy in a business-to-business situation. For example, if you have two engineers you can either call them each a "Software Developer" or you can have a "VP of Engineering" and a "CTO".

Now when your customer, Huge-Corp(TM), who is going to pay the bills for the next 3 months calls, would you rather tell them "let me put one of our developers on the phone" or "let me put our vice president of engineering on the phone"?

<comment><USER6> In a similar vein, I introduce myself as "I'm in charge of the product" when returning calls.

</comment></comment></comment><comment><USER7> Personally, I really liked it. The only thing I didn't was how the pictures are cut at the bottom when you "dance", looks a bit weird to me.

</comment><comment><USER8> Grrr IE....

Webpage error details

User Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; InfoPath.3; MS-RTC LM 8) Timestamp: Thu, 10 Mar 2011 21:39:09 UTC

Message: 'beats' is null or not an object Line: 303 Char: 9 Code: 0 URI: <http://wistia.com/about/team>

<comment><AUTHOR> Thanks for the details. I didn't test dance mode in IE. I thought I'd cut myself some slack since it's just an easter egg. I'll see what I can do though!

</comment></comment><comment><USER9> Dance mode made my day. Well played.

</comment><comment><USER10> Nice page, but you won't be able to read the text on smaller screens. You may want to consider making the height of that huge graphic dynamic (within limits).

Also, the dance thing is just awesome.

</comment><comment><USER11> Really digging your Company Overview video too: <http://wistia.com/about/company>

<comment><USER12> Thanks. It was made by Chris Lavigne at <http://www.crlvideo.com> He's UBER talented.

<comment><USER13> Ditch those S107s and get yourself some Blade MCX2s!!

I'm diggin' the company profile page as well as the team page, nice work!

</comment></comment></comment><comment><USER14> On my 14" screen I can't see the staff and each profile at the same time.

</comment><comment><USER15> Awesome and fun!

Who else looked at the source to see what other words you could type in? Only to discover that it's just dance =(

</comment><comment><USER16> FYI Dance does not work in Chrome 10.0.648.127 on Win7-x64

<comment><USER17> Same prob on osx.

</comment></comment><comment><USER18> Where did the Jay-Z/Modern English track come from?

<comment><AUTHOR> It's the last track on Girl Talk's All Day: <http://www.illegal- art.net/allday/>

<comment><USER19> Great page, and thanks for the link to Girl Talk. It's massaging my eardrums right now. Creative Commons license FTW!

<comment><USER20> You might appreciate this as well <http://mashupbreakdown.com/>

<comment><USER21> This site and Girl Talk are the coolest things I've seen/heard in a long time. If you like music (who doesn't?) you have to check this out.

</comment></comment></comment></comment></comment><comment><USER22> I like yours!

Here's something polar opposite: <http://carbonmade.com/about>

<comment><USER12> Thanks, we like yours too!

</comment></comment><comment><USER23> I think it's cool.

Good for you guys for posting something creative and fun on your site. (No idea why anyone would be critical of this.)

Coincidentally, I opened a Wistia account just last week. Very cool service, keep up the good work.

<comment><USER24> Thanks for your kind words.

</comment></comment><comment><USER25> Jeff Vincent: _You can always finding him listening to good beats._

I'm thinking this is a typo. I mention it in hopes of helping you present a good public image. God knows, I can't proof read my own crap.

<comment><AUTHOR> Mz, thanks again for spotting that and taking the time to let us know. Just fixed it on the website. You rock!

</comment></comment><comment><USER26> Haters are gonna hate! Keep pushing forward!

<http://www.youtube.com/watch?v=QePjIIBI-sI>

</comment><comment><USER27> It's awesome. You might want to consider preloading the hover states though, it seemed to take a while for them to load on my screen.

</comment><comment><USER28> It's pretty nice, but you might want to look at the edge refine too in Photoshop CS5 to get the cutouts a bit neater.

</comment><comment><USER29> I've added Brendan Schwartz to the 'where-people-are' file on the Brown CS filesystem...

</comment><comment><USER30> Absolutely love it. Great work.

</comment><comment><USER31> The transitions can be a bit smoother and less instant

</comment><comment><USER32> Did you guys remove the 'party' mode easter egg?

<comment><USER33> Note: if you have flashblock enabled, it won't run; you have to enable flash for this site first.

I realized why it was probably breaking after seeing comments here about the music.

</comment></comment><comment><USER34> Great stuff. Well done. Had a good laugh there.

</comment><comment> brown9-2 What happens when you hire someone new?

<comment><AUTHOR> I made hi-res cutouts of each person. The plan is just to shrink everyone slightly and add the new people to the left and right. I'm not sure how it will look, but that's the idea at least.

Also, at first I was worried about matching the lighting in the new photos, but these cutouts are all from different photos and the lighting was quite a bit different between each shot already. I'm no expert, but I just played with the levels in PhotoShop until things looked right. I think it came out looking pretty good.

</comment></comment><comment><USER35> Keep up the good work!

</comment><comment><USER36> Love it.

</comment><comment><USER37> Doesn't work vimperator, the first D deletes the tab ;)

<comment><USER38> Why don't you learn how to use it? Ctrl+Z to enable pass-through mode.

</comment></comment><comment><USER39> I think you need to prepare for your DMCA take-down notice.

<comment><USER40> I very much doubt that Gregg Gillis will send them a takedown notice. (They are technically violating his copyright since the track is under a non- commercial/attribution Creative Commons license and they seem to violate both those terms. Gregg Gillis, however, doesn't strike me as the kind of guy who would send out petty takedown notices.)

<comment><USER39> At least put the attribution in the HTML as a comment, no?

This company arbitrarily decides what copyrights to acknowledge and violate?

So I am supposed to host my video with them, but how will I know they will respect my intellectual property?

If you are a good coder, do creative commons licenses not apply? Is this the tip of the iceberg?

Help me out here.

</comment></comment></comment><comment><USER41> 6 up-votes, so I take even your VP of sales now got the HN account :)

(edit) The page itself - I am trying to like it, but something about it rubs me the wrong way. It might be that it is _too_ personal and overly playful, and people look somewhat unnatural, almost as if they were forced to look having fun. This coupled with sudden image swaps creates the impression of a twitchy teenager trying to look cool rather than a smug bunch of sociable geeks. Also your CEO needs a shave, looks kind of unkempt, and Jeff's photo is of a different scale, so he looks like a person with a small head.

Oh, and the dance mode doesn't appear to be working in FF3 on Windows. Typing "dance" does not do anything.

<comment><USER42> Nitpicking about the shave? Seriously? He doesn't look unkempt, he looks relaxed and like he's enjoying his life.

<comment><USER41> Not nitpicking, no. He is a head of the company, and his appearance contributes to the overall company's image. Consider what your first impression of a company would be if its CEO were a pierced dude with dreadlocks. Same goes here, "relaxed" and "approachable" is nice for socializing, but in business context it might be less desirable than "reliable" and "organized".

I will take back the _unkempt_ , but I would still insist on Chris needing a shave. Now, back to the downvoting frenzy.

<comment><USER43> Not exactly a counterpoint, but I immediately thought of this: <http://earbits.com/play/#/about_us></comment></comment></comment></comment>


React Concurrent Mode - <AUTHOR> https://reactjs.org/docs/concurrent-mode-intro.html <comment><USER2> While this page went to the top, I see a lot of comments that show people haven't seen the other pages.

I know it's a lot to ask. But this really is a different programming model. It takes some time to see what it allows.

I spent the last 48 hours documenting it. Here are the most important two pages:

1\. [https://reactjs.org/docs/concurrent-mode- suspense.html](https://reactjs.org/docs/concurrent-mode-suspense.html)

2\. [https://reactjs.org/docs/concurrent-mode- patterns.html](https://reactjs.org/docs/concurrent-mode-patterns.html) ( _especially_ this one!)

It would mean a lot to me if you could go through both of them (including the examples) before forming a final opinion. Thank you.

<comment><USER3> Thanks to Dan and the React team - great work as always.

The most useful thing to provide with this would be a small self contained example of the absolute core of all web programming - a form that gets data, submits it to the back end, and then displays a list of records. Nothing more. Super simple. All web development knowledge derives from this start point - for me anyway.

This is what underpins all web development and when I come into a new approach like this I always aim to implement that basic display/submit/list form workflow.

It would be SO helpful if that existed as a small standlone application.

The profile stuff is in the right direction but not it.

When there is not a basic example of that exact workflow then the cognitive load instantly skyrockets because I now need to work out how to do that and even then I don't know if I got it right because it's my guess using a technology I do not understand. So much better for the technology designers to show how to do the core of all development "the new way". Ideally with a standard form and not graphql.

<comment><USER4> Rails is probably a better fit for your needs. If you're primarily concerned with sending forms to and showing data from a server, React isn't going to give you much.

React is great for Figma, Dropbox Paper, Coda, and Pagedraw: apps that are basically entirely local, and may or may not happen to be syncing with a server in the background.

CRUD is a special case of app, and the modern HTML/CSS/JS browser is pretty well tailored to handle all the needs for the client side of one. Maybe sprinkle in some Turbolinks [1] if you want some fancier page transitions.

It'll take time before someone makes a good CRUD framework on top of React. Until then, Rails is a great CRUD framework, and isn't missing much I'd expect to see React solve.

[1] [https://github.com/turbolinks/turbolinks](https://github.com/turbolinks/turbolinks)

<comment><USER5> If you're not living in a first world country (or living in Germany lol), you usually don't have a stable internet connection. I've developed a lot of CRUD apps that are core to many businesses on field and without ServiceWorkers (or the manifest based appcache before) and client side databases that are better suited to use from SPAs, I'd rarely have success in any of those projects.

You don't really need a framework for this. Forms are incredibly easy to handle from React. A framework for declaratively caching relational data with server side components (like poachdb but for SQL server) would have been amazing though.

I don't know Turbolinks enough but it doesn't help with the connection problems I assume?

<comment><USER6> No, it does not. Turbolinks keeps a local cache of previously visited pages so transitions to those seem instantaneous, but this is session-only and can't be populated in advance.

</comment></comment></comment></comment></comment><comment><USER7> I'm a developer who primarily uses ReactJS for the front-end. I chose the library years ago because it just made sense to me. I really liked the syntax (JSX), the lifecycle methods and the quick hot reloads. The only thing at the time that I really didn't like was the build system (browserify, gulp, webpack, babel, etc) as I found it overly complicated. Now all of that is basically invisible with "create-react-app" and I can just get right to building applications in seconds - life is pretty good.

With each React update however, I worry that ReactJS's scope/concerns is growing far beyond the view layer that is was initially designed to be. I worry that it will become just another bloated "framework" with unnecessary complexity and drive me into the lighter weight and more focused alternatives such as VueJS or Svelte. Perhaps the React team is getting bored?

IMO I'd like to see a base ReactJS with these advanced features being add-ons in some fashion. Keep the version I like to use lean and the complexity just where I like it.

<comment><USER8> > I worry that ReactJS's scope/concerns is growing far beyond the view layer > that is was initially designed to be.

It is. Because through experience developers realized these concerns need to be addressed, and handled... by something. It is unavoidable to have a useful application. Having the framework integrated to help you do so is much less painful than piecing together separate tools, or having 20 different libraries construct their own way to do so ( _if_ it is solveable in a uniform way that simply needs to be advocated and agreed upon).

But in any case, why can't you just not use them?

What is the pain point caused by ignoring the changes... and choosing separate tools to solve these problems if that's what you prefer?

<comment><USER9> If the Jquery authors said 'if u don't like sizzle selector, feel free to use standard dom functions to grab your elements'

We'd all be pissed if the sizzle selector wasn't awesome and well thought out api.

What I keep seeing with React is really bad api design with things like hooks (because classes are so complicated, someone link that documentation post) and redux is horrific boilerplate to manage global state.

Ok no problem, don't use it. Discourse ends there right?

No I'm sorry, reducers, hooks, 'suspense' stuff is not intuitive. This needs to be properly discussed.

Can I possibly even start the discussion at why the hell these things are being named in this way? Is that not a start? Functions and Classes are stupid? A debounce function that's possibly a promise, stuff we know about, is suddenly like the 'suspense' api? Does any thought go into this anymore for how average developers have to approach this horseshit?

<comment><USER10> Those are advanced resources. For what they do, they are pretty straightforward, but if you are just starting, you don't need to care about reducers, hooks, suspense. You don't even need to do a SPA to take advantage of react, thus, you don't even need a router to start.

<comment><USER9> You sound like such a shill. You don't need anything in this life dude. But let's say you do, or let's say a subset of people do, and the feedback they comeback with is 'this thing could be a lot better', and shills like you come back with 'well you probably didn't need this'. Give it a rest.

<comment><USER10> Well. You're free to improve on it them. Maybe doing something useful would help you to control your anger issues, dude.

</comment></comment></comment></comment></comment></comment><comment><USER11> Throwing promises to emulate algebraic effects seems really hacky.

I wonder if React would've needed to go down this path if they were built by a different company.

For example Google/Apple/Microsoft all have huge influence over browser implementations (chrome/safari/edge respectively). They might've been able to push a ECMAScript proposal for proper language support instead.

It's a very clever hack regardless but it only seems necessary because Facebook doesn't have control over the whole stack.

(it does seems like Facebook trying to fix this by building their own React- optimized javascript engine but only for react native on android right now: [https://facebook.github.io/react- native/blog/2019/07/17/herm...](https://facebook.github.io/react- native/blog/2019/07/17/hermes))

<comment><USER12> > push a ECMAScript proposal for proper language support instead

I don't think this is the way it does (or should) work, though.

No company should be able to just come up with essentially an application layer pattern and immediately push that pattern down the stack, just because it's their idea and they think it's good.

A pattern needs to get a lot of practical usage, be iterated on a lot, and have strong consensus behind it before it's pushed down the stack to become a more fundamental building block in ECMAScript / browser APIs.

It's a form of make it work -> make it good -> make it fast. The process should take _years_. It's what happened with jQuery which eventually inspired the native DOM selector APIs, and I think the same _is_ happening with React.

I honestly wouldn't be surprised at all to see JSX standardised as part of ECMAscript within the next 5 years. The rest of the stuff they're doing with the fibre architecture etc may follow too, eventually, if it turns out to be a really good idea after years of people using it in practice and agreeing it really ought to be part of the stack.

<comment><USER13> I agree. JavaScript obviously has problems but it's also the most used development platform of all time.

Part of that is because the runtime API really doesn't change much. And for many years the only changes allowed were very tightly scoped: a single new function call, that takes two strings. Stuff like that. The attitude was "what is the smallest thing we can do to make it technically possible for to get the job done" and leave developer ergonomics to the web frameworks.

It forces people to build real platform layers on top of the browser. That causes fragmentation but it is also a hot crucible for competition. It has led to arguably the most competitive developer tool landscape of all time.

It's easy to say "browsers should just standardize on X" and be right.

However it's exactly the very slow acceptance of standards that had led to such a strong ecosystem of ideas. So in the aggregate, as you indicate, it's not good to land on standards too fast.

</comment></comment></comment><comment><USER14> Any framework that allows me to write _simple code_ which then gets _automatically_ highly optimized to run efficiently and with high/low priority based on the user-interaction and effectively removes race-conditions and makes it more intuitive to deal with loading states is a good thing for me.

In fact this feels so advanced stuff (React's runtime I mean) that it will probably one day land in the browser. So React's team is just one step ahead of the others, they are thinking: "what's the least amount of code we can let developers write so that we can optimize away all the rest of the stuff?".

One thing I'm worried is whether with all these amazing optimizations and very different code-execution model, we have to rethink some of the valuable simple lessons of software architecture we learned over the years. For instance, we have to rethink a bit about how to organize code and how to test it. Some things that before we could confidently say "this piece of code will run at this time" will now be a bit more like "not sure when this piece of code will run".

Cool stuff, but some tradeoffs to consider.

</comment><comment><USER15> This is a good development. Honestly one thing I very much think is a downside about React is the fact that your template is also JavaScript. Don't get me wrong this is incredibly powerful and can be very expressive, but it comes at a cost.

That cost has the same cost as other JavaScript - it's blocking, cannot be paused and has all the possibilities of de-optimizations that might occur in the JIT. The Ember team faced a similar problem in the past and I find their solution to this problem quite intriguing, even in it's obscurity. Instead of compiling their templates to JavaScript they instead compile them to opcodes in a binary format which is executed by a virtual machine.

This has several advantages. Since the template file is represented as an optimized binary the size of the file itself is incredibly small, excellent since templates can be a large part of your eventual code. Not only that but since the instructions are opcodes, this mean the VM can pause at a moment when the browser is too busy with other stuff.

Since a large part of your templates is probably static and not prone to any changes the template file also indicates which parts of the instructions are dynamic and which are static. This allows the VM to generate a separate set of instructions just for subsequent updates, only updating those parts which are dynamic.

Now I am not saying this is a silver bullet, it is however an interesting and fresh approach. In case you're interested in this check out the Glimmer site or Tom Dale's excellent talk (can't recommend this enough) on the subject:

\- Tom Dale's talk: [https://www.youtube.com/watch?v=nXCSloXZ- wc](https://www.youtube.com/watch?v=nXCSloXZ-wc) \- The Glimmer JS site: [https://glimmerjs.com/](https://glimmerjs.com/)

<comment><USER16> > a downside about React is the fact that your template is also JavaScript

Everytime I read this (and it still seems to happens in 2019) I just don't get it--I don't get why people want to stick to the past. It's the best thing which could happen, everything is JS, FE development finally got bearable, no, enjoyable! It freed us from all the subpar patterns before.

<comment><USER17> What is it you don't get? The parent post clearly states why this can have performance costs. Other modern frameworks like e.g. Svelte solves rendering without making templates JS.

<comment><USER16> > performance costs

They are overstated. You are not building Photoshop in the browser 99% of the time. Dev productivity should have a higher prio => everything in JS and nobody needs some FE VM with bytecode (wtf, I mean when we already talk about Ember, Ember interfaces are not really the fastest one despite having their own small VM).

This is what I don't get, sticking to the past and requiring stuff which doesn't make sense.

<comment><USER17> Sticking to the past? You talk as if JSX is the final solution in dev productivity, after which nothing else is even worthy of consideration. JSX has pros and cons, as does e.g. Vue's and Svelte's templating approaches. Something even better will surely appear, or already has.

</comment></comment></comment></comment></comment><comment><USER18> This is a wonderful example of a well engineered solution to the wrong problem. React is a wonderful view library, and if limited to that use it is remarkably fast and easy to use.

But the usual way of building things is to treat React as a total web app framework. The terminology is very confused: they still call it "rendering", even though it now usually includes data fetching, complex manipulations, event processing, etc. You just `React.render` once when your page first loads and then child components take care of the rest. It should really be called `React.start`.

> The reason for the stutter is simple: once rendering begins, it can't be > interrupted.

Rendering should never _need_ to be interrupted. If your app is architected to have any rendering operation take long enough that it's perceptible to the user, STOP DOING THAT THING DURING RENDERING.

Want a performant web app? Apply separation of concerns properly, and limit your use of React to the view layer.

<comment><USER19> > If your app is architected to have any rendering operation take long enough > that it's perceptible to the user, STOP DOING THAT THING DURING RENDERING.

At my workplace we frequently encounter cases where the rendering itself - just the evaluation of the pure React functions - takes hundreds of milliseconds, if not several seconds. This is after memoizing every part of it that can be memoized, after profiling it for places where work can be shaved off, etc. Sometimes there really is just that much rendering, and unlike other types of computation, today's React gives you _no_ options for putting that in a worker or breaking it up. Tabs' single threads mean you can't even _scroll_ in the interim. Concurrent Mode is the only hope we have of solving that case, other than avoiding it with paging, which is what we currently do.

<comment><USER20> > React gives you no options for putting that in a worker or breaking it up.

Isn't that React Fiber? [1]

"Fiber breaks down animation into segments that can be spread out over multiple frames."

How does that differ from React Concurrent? Are they compatible?

[1] [https://en.wikipedia.org/wiki/React_Fiber](https://en.wikipedia.org/wiki/React_Fiber)

<comment><USER19> So, I'm not sure exactly how Fiber works, but given the context and what I've read/watched, here's my guess:

Confusingly, there are multiple tiers of "updates" happening in a React app. One is when your render functions are called to recompute their _React_ element trees. Work can be avoided here via shouldComponentUpdate(), etc.

Once React has this internal data structure, it uses it to do _another_ level of updates to the actual DOM. This part is totally black-boxed from the application code, because React has all the information it needs internally to avoid unnecessary updates at this level.

I think, maybe, that Fiber is specifically focused on this latter stage. Concurrent Mode would then be the stage 1 equivalent of what Fiber does in stage 2. This would be why it has an effect on how application code can actually be written, where Fiber didn't really.

<comment><USER21> "Fiber" was the rewrite of React's internal render logic (primarily how it processes the component tree), and was released in 16.0. It's named after the data structure that is used to track individual pieces of work (roughly on a per-component basis).

Fiber does specifically divide the overall update process into two general phases. "Render phase" involves walking the component tree, asking components what the desired UI should look like, and calculating the diff and necessary changes based on the previous requested output. Once that's been calculated, it is applied all at once in the "commit phase".

Concurrent Mode is, loosely, additional modifications to that rendering logic to enable pausing partway through the "render phase" process, tracking one or more in-progress versions of render passes, prioritizing them based on the source of the requested update (user input vs network request, etc), and recalculating in-progress renders if another higher-priority render was pushed to the front and committed to the DOM first.

</comment></comment></comment></comment></comment><comment><USER22> This seems quite complicated. Rarely the big perf hit is in js. The big perf hit is usually layout, composite and paint cycles. They are eons slower than manipulating a virtual dom tree in js.

So I don't get it. You can't interrupt the browsers native paint cycle. Once you say node.appendChild or change some attr/style. The browsers gonna do what the browsers gonna do.

React was simple. With hooks , concurrent mode, virtual events and mumbo jumbo. I don't get it anymore. It's not a lean mean library that takes minutes to run and one can't read the source in a couple of hours

<comment><USER23> >React was simple. With hooks , concurrent mode, virtual events and mumbo jumbo. I don't get it anymore. It's not a lean mean library that takes minutes to run and one can't read the source in a couple of hours

I feel that.

One of the top voted questions was "how do I do a simple form submission?" and the response was "it all depends on your state management caching behavior". I think that highlights just how beginner unfriendly some of this stuff has become.

<comment><USER24> > the response was "it all depends on your state management caching behavior". > I think that highlights just how beginner unfriendly some of this stuff has > become.

What's the alternative? In Angular, the whole problem would've been solved with their template engine + built-in forms module. But Angular has a huge API surface.

With React, you have your template engine. Form state can be managed with a third-party library, like final form. It's no broader of an API Surface than Angular would be.

So the question is, when were these things ever "beginner friendly"? Were forms easier 5, 10 years ago than they are to implement today?

<comment><USER23> >Were forms easier 5, 10 years ago than they are to implement today?

Yes. Less was expected of us. Things have grown and evolved in such a way that user interfaces can provide much better experiences than they use to be able to and as the capabilities have grown, so have the expectations. Along with it the barrier to entry has gone up.

</comment></comment></comment></comment><comment><USER25> React.js is literally the only reason I'm still in software engineering. I know I should learn Scala and be a full stack developer again, but at almost 34, I really feel like I've found my niche in frontend React development. I trusted the React team when they released hooks, and I've never loved coding more than now. Everything just makes sense, and it seems to always work when you follow the best practices.

I'm excited to start learning more about Concurrent Mode and Suspense.

</comment><comment><USER26> As I understand it, the principal behind the VDOM is that renders are cheap and the cost is is negligible compared to real DOM updates. If this is the case, why do you need to keep concurrent VDOMs in memory at once? Why can't you just render the VDOM you need when you need it? I think this is where I'm getting lost on this new feature.

<comment><USER2> The thing that's slow is _arbitrary JS code people write in components_. VDOM is not the point there. We want to make arbitrary JS code people write non- blocking, by using components as the "units of work" where we can yield.

Really though, you're over focusing on the CPU part. The IO part is much more interesting.

<comment><USER26> When you say "yield", you're talking about render functions throwing a Promise, right? Is there another kind of yield that _arbitrary JS code people write in components_ can do that I'm missing?

<comment><USER13> Render functions, to my knowledge, can't return a promise.

(And you don't throw a promise, you return and reject it)

The issue at hand is fully synchronous render functions that, e.g., go through a giant list of objects and call a bunch of other render functions on each one synchronously.

This change allows that one function to finish, and interaction to continue before all the children render.

<comment><USER26> The new suspense API works by throwing Promises when the data isn't ready yet.

<comment><USER13> I'm not sure what you mean by "throwing promises"...

You mean returning unresolved promises? Or returning rejected promises? Or just throwing an error?

<comment><USER21> Literally, _throwing_ a Promise.

React will catch promises thrown during rendering, and treat that as a request to "suspend" the component and come back to it later once the promise has resolved.

</comment></comment></comment></comment></comment></comment></comment><comment><USER27> The version control analogy in the description was spot on, but a good demo is worth a thousand words, so I will be waiting for a good demo to make conclusions.

<comment><USER2> It might be asking a lot but to form the right impression, I would really suggest that you get through these docs:

1\. [https://reactjs.org/docs/concurrent-mode- suspense.html](https://reactjs.org/docs/concurrent-mode-suspense.html)

2\. [https://reactjs.org/docs/concurrent-mode- patterns.html](https://reactjs.org/docs/concurrent-mode-patterns.html)

I spent the last 48 hours writing them, and I think they answer your questions and contain many, many demos.

<comment><USER28> I'm still working my way through the suspense doc (very nice so far...) so please bear with this "in flight" thought.

Does having an external state tracking system like Redux combined with a tracking a "loading state" solve the in-flight rendering item? My initial take is that this allows for a simplified model where data is loaded directly in components without the wrapping dance for tracking client load status.

<comment><USER2> The meat of this thing is in the 'Patterns' doc. That's the kind of stuff Redux isn't equipped to do.

</comment></comment></comment></comment><comment><USER29> "For IO-bound updates (such as fetching code or data from the network), concurrency means that React can start rendering in memory even before all the data arrives, and skip showing jarring empty loading states."

What is there to render but loading states when there's no data?

<comment><USER30> Buttons, headings, labels... Will all be ready when your data arrives.

<comment><USER24> You can already do that without concurrent mode, though. Let's say you have a navbar (your website's header), a list of things asynchronously retrieved from an API, and a footer.

const myApp = () => (

      <Header />

      <Body />

      <Footer />

);

Body can make the network request, and store the result in state. Since the network request took 200ms, Header and Footer have already rendered.

</comment></comment></comment><comment><USER31> On a purely technical level, this is really cool. But I can't help but wonder: if this level of complexity is the answer, maybe we're asking the wrong question.

<comment><USER32> I am huge React fan, but when I see how complicated are some of the modern React codebases in some "leading" companies I have a strong feeling that we will soon need to go back to drawing board and hit some kind of reset button. As I said, I love React ecosystem, but there is something fundamentally wrong with most front-end developers today. They build bigger and bigger SPAs because they are confident that React ecosystem makes it maintainable, well good luck maintaing React/Redux/Graphql/hooks/bazillion-libs after 5 years. Abstraction is good, but it has its limits.

<comment><USER33> Sometimes frontend developers overengineer their solutions, yes.

Sometimes you need a more complicated solution for a complicated set of requirements.

Do you need GraphQL? Nope!

Do you need GraphQL for an application that needs to fetch a lot of highly relational data with nested objects? No, but it sure would be more optimal to just ask for what you need in one request in the shape you expect, instead of either:

a) Multiple REST API calls that you need to combine and coordinate in the frontend

b) A specialized REST API call that gives you all the data you need in the shape you want, but it isn't very RESTful

c) A REST API call with a bunch of `with[]` params (a la Laravel) that indicate what nested objects you'd like, but not their exact shape

So, which is more maintainable then?

—

The other wild misconception I see time and time again is this:

Of the five things you mentioned "React/Redux/Graphql/hooks/bazillion-libs", only the first one is React, and the second-to-last (Hooks) of which it is optional.

The rest are dependencies you opt for, hopefully after weighing pros and cons. You don't need to use Redux, GraphQL, react-table, whatever other dependency in your React web application.

Please separate out React from React-based codebases from code architecture. They're not one in the same.

<comment><USER32> I just don't buy this idea that there is some kind of separation between React library and React as a platform. I'm building SPAs since 2008, been to too many front-end meetups, have friends in so many different companies its hard to count. NOBODY is using React as a library. Banks, shops, news sites, you name it, everyone is using React+Redux+Typescript+bazilion-libs. Graphql is not a standard yet but it is already very popular, basically every new hired Dev adds new libs or switches to something else. It's a mess.

<comment><USER33> The thing is, I don't disagree that a large (if not majority) number of developers reach for dependencies by default, instead of carefully considering patterns first.

I don't think bundling React the view library, Redux the state management library, Typescript the Javascript superset, GraphQL the API query language, Webpack the bundler as one "React the platform" is helpful.

I'd like to see the developers that automatically merge all of these technologies together by default because everyone else does it (or worse, because X Big Tech Co does it) to start questioning and exploring other options. I'm not sure how to get there, and maybe my comment above is my contribution towards convincing readers to reconsider their understanding of the React ecosystem.

You're right though, it is a mess. I think the mess comes from people and how they code, not so much this specific library nor the language.

<comment><USER34> These new frameworks and things like "create-react-app" have made it easier and easier for junior developers to make immensely complicated apps with immensely complicated tool chains. I think that has a lot to do with how things have gotten to the point described in your comment and many of the others. As you start reading tutorials about Vue or React they tend to quickly stray into other dependencies, and webpack or whatever. The web has gotten more complex. The only way to navigate it wisely is to become wise through experience. There are no shortcuts but we've definitely made it easier for new developers to stumble into a mess.

</comment></comment></comment></comment></comment></comment><comment><USER35> Pretty interesting. Hope someone also writes a summary of how it works. Are they computing the tree in a worker, or have they just modified the rendering api itself to somehow be interrupt-able?

<comment><USER35> Seems someone has written some of it up here: [https://github.com/acdlite/react-fiber- architecture](https://github.com/acdlite/react-fiber-architecture)

<comment><USER36> acdlite has written most of the implementation work that lead to this feature

</comment></comment></comment><comment><USER37> Huh. Throwing promises to manage concurrency is kind of neat and scary (because it's new and different). I remember there was huge performance hit from functions containing `throw` statements in V8, has that been addressed (in V8 or in this project)?

<comment><USER2> If you're blocked on network, throw is not gonna put a dent in that. We only use throwing when work is blocked on IO.

</comment></comment><comment><USER31> Rich Harris' excellent talk Rethinking Reactivity has an interesting section where he compares a React Concurrent Mode demo to Svelte: [https://youtu.be/AdNJ3fydeao?t=1128](https://youtu.be/AdNJ3fydeao?t=1128)

<comment><USER2> This comparison focuses on just one aspect of Concurrent Mode and misses the big picture. I'll believe it when I see these demos ported to Svelte:

[https://reactjs.org/docs/concurrent-mode- patterns.html](https://reactjs.org/docs/concurrent-mode-patterns.html)

<comment><USER31> So you're saying Svelte might have the best performance, but Concurrent Mode makes it much easier to achieve medium performance (there certainly would be value in that)? Sorry, haven't had time to read all the material yet.

<comment><USER2> I'm saying that I don't see how you're going to implement these "pending loading state" patterns in Svelte but I'd sure like to be proven wrong.

It doesn't make sense to try to summarize that page. I've been writing it for two days and it's the shortest that I could compress the essence. I'd be happy to respond to questions you'll have after reading :-)

<comment><USER38> You deserve a vacation!

</comment></comment></comment></comment></comment><comment><USER39> I'm relatively new to ReactJS and professional web development, and I've been using (and learning) react-redux. The layer schema makes sense to me; React handles user input, redux manages state, and react-redux ties them together.

If react does a better job at speedily handling user input to the point of seamless interaction, then I suppose that would be good. What are we losing from choosing to make a layer itself responsible for the speed of returning output (rendering) from input (user interactions)?

</comment><comment><USER40> As someone who mostly uses ember.js in the day-to-day, I'm wondering how this compares to the amazingly great ember-concurrency library that uses generators?

<comment><USER41> Here's their official position on generators: [https://github.com/facebook/react/issues/7942#issuecomment-2...](https://github.com/facebook/react/issues/7942#issuecomment-254987818)

<comment><USER40> very insightful, thank you

</comment></comment></comment><comment><USER42> It's certainly a problem I sort of dance around in React planning how and when something might render based on what is going on, the amount of rendering and such, and the dance is somewhat never ending.

<comment><USER43> One of the main weaknesses of vdom is that React now has to solve these issues that browsers already do natively.

<comment><USER44> Can you elaborate? At a minimum vdom is just doing work you would have had to do manually with DOM apis.

<comment><USER43> With regular dom/javscript, you can be typing into an input field and updating unrelated DOM and the typing should not be impacted much (unless you are changing a lot of DOM and spiking the cpu).

But with React it has to reconciliate the input field.

<comment><USER21> "controlled inputs" are certainly the recommended React idiomatic pattern, but you can use standard uncontrolled inputs if you want.

</comment></comment></comment></comment></comment><comment><USER45> This was the biggest pain point I had when developing a vanilla react app circa two years ago. It's great to see that they've come up with a neat solution that works out of the box!

</comment><comment><USER46> As an experienced front-end developer I must say that I don't really understand why the complexity of building websites is growing instead of going down. We have build thousands of front-end frameworks and all of them try to solve the same issues that should have been solved by the browsers themselves a long time ago. We implement complex "routers" and "server-side-rendering" to show specific content when the URL in the address matches a specific value, while the entire web architecture was made exactly for this purpose, to show some content when you go to an URL. Tens of years of "evolution" and we just keep making everything slower and more complex for no reason, apart from being bored or wanting to be trendy.

I know all those frameworks exist to solve a specific problem, but while doing so we are creating hundreds of problems that never existed before.

<comment><USER47> You likely don't have the problems that Facebook is trying to resolve with React.

Namely, developing something once and having it work ideally everywhere on the planet (slow network vs. fast network).

They're worried about shaving kilobytes here and there - concurrent mode helps to dynamically render/load items in an order that makes sense for people on slow networks where it wouldn't even matter for people on fast networks.

<comment><USER48> And this is fine but why the facebook-scale problems solver becomes a mainstream library then? How many companies have these problems?

<comment><USER49> This is how things have worked in tech for at least the last 10 years (which is when I first noticed it). People that don't have the same problems as FAANGs cargo culting FAANG tech because it works for FAANG.

</comment></comment></comment></comment><comment><USER50> I like what I see from Suspense. I've had to implement that pattern myself a couple of times before in Angular; it's common enough to justify a standard approach.

EDIT: This is better than what I've implemented. My solutions have involved data-load waterfalls. I'll have to crack the code open and see how Suspense is pulling this approach off; it's pretty cool!

</comment><comment><USER51> Anybody knows whether generator functions were considered and if they were, reasons why they were not used for this?

<comment><USER41> Yes! I was also curious about this, because I'm sure you could make generators/yield work for asynchronous interruptible rendering, but here's what the React team has to say: [https://github.com/facebook/react/issues/7942#issuecomment-2...](https://github.com/facebook/react/issues/7942#issuecomment-254987818)

</comment></comment><comment><USER52> > It would be nice if we could "skip" it and wait for some content to load > before transitioning to the new screen.

Yes please! I do hope this pattern catches on. Not only are full page loading screens on every interaction obnoxious, I'm sure they also require an epilepsy warning.

</comment><comment><USER53> This isn't new. You've always been able to build up a DOM element in memory before adding it to the tree. You can even use something like requestAnimationFrame to incrementally build and not block (useful for things like long lists etc).

<comment><USER2> For sure. But I'd love to see a demo of features described here that's as expressive.

The challenge isn't the low level mechanism. It's how to make it composable so people can build their own abstractions.

[https://reactjs.org/docs/concurrent-mode- patterns.html](https://reactjs.org/docs/concurrent-mode-patterns.html)

</comment></comment><comment> c-smile Or just implement React natively so speed will not be an issue: [https://sciter.com/sciter-4-4-0-0-with-reactor-on- board/](https://sciter.com/sciter-4-4-0-0-with-reactor-on-board/)

</comment><comment><USER54> Is this just using generators to allow things to be interrupted and scheduled? Or is it more fancy than that?

<comment><USER51> Last time I seen an explanation for this, they were talking about throwing and catching promises. So you could probably implement suspense with an error boundary just checking if the error you got is actually promise and if so wait for it to resolve until rerendering.

But reading this I think there is a bit more to it and I would assume for some things to work you have to actually try rerendering on some scheduler not just wait for promises to resolve (like they mention in how multiple Suspense components are aligned and rendered together if they resolve in close succession)

<comment><USER41> Here's a comment from Sebastian M (React core team member) about why they chose not to use generator functions: [https://github.com/facebook/react/issues/7942#issuecomment-2...](https://github.com/facebook/react/issues/7942#issuecomment-254987818)

</comment></comment></comment><comment><USER55> I'm left scratching my head a bit from this blog post. In [https://reactjs.org/docs/concurrent-mode- suspense.html](https://reactjs.org/docs/concurrent-mode-suspense.html), current pitfalls of fetching data are explored including waterfalls (sequential loading spinners for nested components) and data races.

The way we currently solve these problems, which for some reason didn't make it into the examples, is with Redux:

      fetchProfileData()
      const ProfilePage = connect((state) => ({ user: state.user, posts: state.posts})(ProfilePage)

In truth, we'd also track a loading and error state as well. This solution is not subject to data races or waterfalls.

One could argue that their new implementation provides a better interface... but I'm not so sure. The final ProfileDetails won't work outside a Suspend because the Suspense API is Throwing Promises [1]. In fact, it crashes the app :-/

      Note how we eliminated the if (...) "is loading" checks from our components. This doesn't only remove boilerplate code, but it also simplifies making quick design changes.

Yeah, I did note that, but is it a good thing?! Rendering in React is currently localized to the render function of the Component itself! Its rather easy to miss the function call that is throwing a promise during rendering to cause itself to not be rendered! Suspense is a hidden interface to delegate rendering to a parent - perhaps an even more error prone Context API.

All in all, this feels like Hooks to me - the trivial examples in the Docs seem perfectly cromulent, but in practice, the ergonomics are so bad that any time I try to use them, I am forced to spend several hours debugging stuff because they lose predictability when composed. Perhaps this won't be so bad in practice, but I'm having a hard time imagining how to compose components that interact with Suspense. Of course, I'd love to be wrong (about the API) or proven wrong and everything turns out to be magical. Maybe TS makes it perfectly obvious in practice, but maybe it generates a ream of gibberish instead.

The true sleight of hand in most of the examples is to kick off `fetch`ing before a component gets mounted that needs the data:

      // Kick off fetching as early as possible
      const promise = fetchProfileData();

      function ProfilePage() {}...

As far as I can reason, there is no good way to do this. The data dependency graph is naturally defined by the component graph: after all, components know what data they need to render, but that graph can't be resolved until after they are actually rendered. So, you end up defining it in two places with an implicit coupling or via HOC or get a waterfall. I'd be open to some magic here.

1\. [https://codesandbox.io/s/frosty-hermann- bztrp](https://codesandbox.io/s/frosty-hermann-bztrp) \- see fakeAPI.js (couldn't figure out how to link to it.

<comment><USER2> >The way we currently solve these problems, which for some reason didn't make it into the examples, is with Redux:

As the _author_ of Redux I can assure that it's not designed for — or capable of — providing the features described on this page:

[https://reactjs.org/docs/concurrent-mode- patterns.html](https://reactjs.org/docs/concurrent-mode-patterns.html)

Hope that makes sense.

Regarding fetching, as the page says, in practice we use Relay for that. If you want to look at how it works in production, the page does link to Relay docs.

<comment><USER55> > As the author of Redux I can assure that it's not designed for — or capable > of — the providing thee features described on this page... Hope that makes > sense.

Sure! Seems like we are talking past each other (these problems being waterfalls and data races and the examples coming from [https://reactjs.org/docs/concurrent-mode- suspense.html](https://reactjs.org/docs/concurrent-mode-suspense.html)).

<comment><USER21> Right, and as a _current_ Redux maintainer, I can also assure you that Redux does nothing to solve those problems :)

Redux is simply a synchronous external data store. It does nothing in regards to how you fetch your data. While React-Redux does trigger UI re-renders when the store is updated, we rely on React's own scheduling capabilities to handle actually updating the UI. So, no, Redux definitely does not solve the patterns described in those docs.

I'm hopeful that we may be able to eventually build some integrations between Redux and Suspense at some point in the future, but frankly we've been waiting for actual docs and examples of how all this stuff is _meant_ to work. Conveniently, we now actually have some (thanks Dan!), but it's still going to take a long time to absorb what the APIs are and what techniques may be possible.

</comment></comment></comment></comment><comment><USER56> These frameworks should be architected to support async/await scenarios natively instead of all these convoluted workarounds. Promises/async have ben around long enough.

EDIT: JS is already dynamic. If a function returns a promise then it should be awaited and render blank or nothing in the vdom. This is obvious, except in JS where architecture prefers synchronous-only design with async being an afterthought.

<comment><USER57> React's prioritization of features has led to it being one of the most successful frameworks of all time, I'm not sure how useful it is to suggest they should have started with this from the beginning. It's been overwhelmingly successful without this feature.

<comment><USER58> The illogical adoption of React must not be questioned, even in the face of its poor performance and ugly syntax. One must never note that perhaps it is simply a cultish adoration, or faddish adherence to popular mass opinions, that leads to "popularity" mattering more than practicality.

So what if React is massively bloated, stupid syntax, can't use regular HTML, can't use regular CSS, can't use async/await, have to do everything a special way...literally, SO WHAT. It IS REACT! OMG behold it's glory. It came from FB, maybe if we use it we can scale like FB!

Omg the idiocy. But keep on slavishly following the next front-end dev trend. I can't tell whether front-end dev culture is a massive circle-jerk or cargo cult. And don't mention the irrational hate attempted to be heaped onto anyone who _dares_ question the programmatic thinking of the official line of the React Appreciation Society.

<comment><USER59> Is there anything more in-bad-faith than assuming everyone who uses X is just an idiot, trend-follower, or beginner? Unlike you, the Enlightened One?

Why not just ask people why they use X? Plenty of veteran developers on HN could tell you why they like React despite a sea of alternatives, but it's easier to just cast aspersions instead of understanding.

It's a shame to see this on HN which is supposedly a community of craftspeople who build things. If you can only come up with insults for why another group of people do something, it's time for you to get up off your ass and ask someone in that group. And this extends beyond just engineering, it's how you understand other people in general.

</comment></comment></comment></comment><comment><USER9> I just want to take a moment and say , while Dan is smart and has probably the right patterns we all need to implement, he is a horrific API writer.

So none of this could be abstracted right? Like, jesus, not functions but reducers, not straight for 'cancelRender' but 'Suspense' Api.

There was a guy once upon a time that said '$('.comments')', that makes sense , it's intuitive. These APIs are just getting dumb from a common sense perspective.

<comment><USER60> Dan didn't write the entire API himself. I'm sure the whole React core team contributed in some fashion and it went through various code reviews. No need to bash.

<comment><USER9> Well you know not to be too snarky, but some of these framework writers now days would argue 'bashing has its place, not everyone needs to be bashing'

I'm just showing you that a shitty version of discourse is available if you need it for your use case.</comment></comment></comment>


Ask HN: Networking tutorial for DevOps - <AUTHOR> I have trouble understanding the networking parts in Docker and Kubernetes. Do you have book/tutorial recommendation for developers? <comment><USER2> It was designed to be as magical as possible, so that programmers that work with network don't need to learn how to configure networking (which is a dumb idea on its own). I don't think there's a good description anywhere.</comment>


Kinect + Javascript: Control webpages with gestures (MIT Media Lab) - <AUTHOR> http://vimeo.com/17180651 We have the Microsoft Kinect talking to Javascript with some Computer Vision recognizing basic gestures.
You can directly read out the Kinect data, or use our high-level gesture events. On any web page!
See the demo video at http://vimeo.com/17180651
The source is available at http://github.com/doug/depthjs <comment><USER2> Nice, although I'm not sure that this method of navigation will catch on. You can also do this with stereo vision, although at a higher computational cost. For example, here's an overhead view of a point cloud from a stereo camera (Minoru webcam):

<http://www.youtube.com/watch?v=kQPIm2Kw5kc>

One possible useful gesture based form of navigation might be to use a hand wave in order to change viewports.

<comment><USER3> Thanks! Our current gesture set is more focused on showing what's possible so far and inspiring some new ideas. Navigating the web is only one application of the framework we built - we envision all sorts of applications that run in the browser, from games to specific utilities for specific sites. The great part is that now web developers who specialize in Javascript can work with the Kinect without having to learn any special languages or code. We're hoping this allows a new set of interactions beyond what we first developed.

</comment></comment><comment><USER4> So can this be done with just a regular camera without depth perception?

<comment><AUTHOR> Right now its Kinect only--but it's called DepthJS versus KinectJS because we'll eventually infer depth using web cams.

</comment></comment><comment><USER5> a) this is wicked cool b) in person, these guys are rockstars, esp aaron</comment>


Feds Threaten To Arrest Lavabit Founder For Shutting Down His Service - <AUTHOR> http://www.techdirt.com/articles/20130816/14533924213/feds-threaten-to-arrest-lavabit-founder-shutting-down-his-service.shtml

<comment><USER2> Forget the 4th, how is this not a violation of the 13th Amendment?

</comment><comment><USER3> Why is this blatant blogspam allowed? The original article is perfectly readable and to the point and isn't filled with self-links to more blogspam.

Original article here:

[http://investigations.nbcnews.com/_news/2013/08/13/20008036-...](http://investigations.nbcnews.com/_news/2013/08/13/20008036-lavabitcom- owner-i-could-be-arrested-for-resisting-surveillance-order?lite)

<comment><USER4> Techdirt has been a consistently better aggregator of "anti-establishment" tech news than NBC, and does do original reporting as well. I'd guess the submitter has an RSS feed or bookmark there.

</comment></comment><comment><USER5> Who is John Galt?

<comment><USER6> I get that a terse Rand quote doesn't really garner much respect, but there's a point here undeserving of the downvotes - the government is effectively saying that Mr. Levison should be forced to remain in business for the sole purpose of serving the government's interests (by threatening to arrest and prosecute him for the action of going out of business) - a circumstance which is depicted as one of rings in the downward spiral of the crumbling dystopia in _Atlas Shrugged_.

<comment><USER2> I don't think anyone downvoting the GP failed to get the point of the quote.

I think they were downvoting based on the perceived value of reference (whether GPs terse one or your verbose one) to Rand's novel in the discussion.</comment></comment></comment>


California adopts nation's broadest gun seizure laws - <AUTHOR> https://www.foxnews.com/politics/california-expands-gun-seizure-law <comment><USER2> Society clearly needs some kind of control over who can own weapons/chemicals/etc capable to kill (a lot of) people. That's simple and almost everyone would agree with that. What people disagree about, is what public institution should be responsible for that. Some people say it's government's job, others are afraid that government being too powerful is too big of a risk. So who is right? Is there an alternative to government for such a job? Is it any better?

That is how discussion about gun control should look like, instead people just scream some nonsense at each other.

<comment><USER3>
      weapons... capable to kill (a lot of) people

A _rental truck_ was used to kill 86 and injure 458 in _one incident_ (Nice, France, 2016). That's a greater toll than most USA mass shootings since then _combined_.

The Murrah Building bombing in OKC, killing 168, likewise needed only a rental truck and conventionally available components.

<comment><USER2> Completely agree.

I think society is too focused on instruments invented for violence and defense instead of violence(and defense) itself.

And yet my original statement is still accurate, as i said there is a consensus that dangerous things should be somewhat limited in use by 'dangerous' people. Like cars are.

Regarding guns. Instead of arguing should everyone have unrestricted access or not(like media is portraying it) people should talk about solutions that wouldn't involve government, because it's the only compromise those two camps can find(pro and anti gun).

P.S. most 'mass shootings' in us(in terms of media, when a few people got shot by single shooter or something like that) happened with illegal guns, which means instead of trying to create more regulation, people should figure out why current regulation doesn't work. And if there is no clear reason. Try other methods.

<comment><USER4>
      Instead of arguing should everyone have unrestricted access

Who is arguing for _unrestricted_ access?

<comment><USER2> I think there are people who argue for that, but regardless, media and supporters of gun confiscation(meaning radical position on that side) certainly portrays everyone who think that current regulation is enough as unrestricted access supporters. Which makes it look like restricted/unrestricted conversation, when in fact it's not or shouldn't be.

</comment></comment></comment></comment></comment><comment><USER5> This is a good example of what frustrates me about gun control: I think that America needs more gun control, but the people who propose gun control laws don't seem to take the time to find ways to protect legitimate gun owners.

It reminds me of a problem that I feel happens with software security: People think about the security of a policy/design by trying to identify whether the privacy/security protections are correct. Instead they should be thinking about how the system could be incorrect (i.e. how it could be exploited to hurt people).

</comment><comment><USER6> Wow. The ACLU generally stays away from fights over gun rights. But even they don't like this:

 _The American Civil Liberties Union (ACLU) opposed the bill, saying it "poses a significant threat to civil liberties" because a restraining order can be sought before a gun owner has an opportunity to dispute the request._

<comment><USER7> ACLU does not believe the Second Amendment provides an individual right to gun ownership, but they have become more willing to insert themselves on issues where gun owners _other_ civil liberties are being trampled on.

[https://www.aclu.org/blog/civil- liberties/mobilization/aclus...](https://www.aclu.org/blog/civil- liberties/mobilization/aclus-position-gun-control)

</comment></comment><comment><USER8> CA owes its progress to the laws that ban non-compete agreements. IIRC, the anti gun laws were enacted to prevent black people from getting any real power in the era when segregation existed.

<comment><USER3>
      anti gun laws were enacted to prevent black people from getting any real power in the era when segregation existed

CA was never a "Jim Crow" state with legally-enabled segregation, but there was plenty of _de facto_ segregation accomplished using zoning[0], deed covenants, etc.

Concealed carry bans with discretionary permit exceptions have racist roots (so sheriffs/chiefs could limit permits to "good white folks").

A clear example of CA legislation motivated by race-war fear was when the Black Panther Party demonstrated with long guns with live rounds in them. Until that point, long guns were not considered "loaded" unless a round was chambered. That law was quickly changed, with an Urgency Clause allowing it to take immediate effect.

[0] The recent series "Blue Sky Metropolis" (recommended) discussed the postwar influx of black workers into southern California aircraft plants. They often had to commute long distances because new suburbs were limited to whites, forcing non-whites into areas like Chinatown.</comment></comment>


First job in a startup considered harmful - <AUTHOR> http://wrongsideofmemphis.com/2014/02/17/first-job-in-a-startup-considered-harmful/

<comment><USER2> It's also good experience to immerse yourself in sulphuric acid, to better appreciate clear water.</comment>


Google AI invents its own cryptographic algorithm - <AUTHOR> http://arstechnica.co.uk/information-technology/2016/10/google-ai-neural-network-cryptography/ <comment><USER2> "no one knows how it works" \- That caught my attention, having seen computer evolved ASTs. Probably no one knows because no one has put effort to reverse engineer the internals - most of which would be unnecessary complexity that is very likely with neural nets.

Many years back, I had once experimented with PyEvolve to develop a custom trading strategy which was kind-of code with a custom instruction set. The output of a strategy was a string (program AST) that was uglier than most obfuscation outputs with many unnecessary statements. For example "a=1;b=2;c=a+b;d=c/3;if(d==1).." \- expand that to many more variables. The program evolved strategies that made small profit and therefore the output was worth analysing. But decompiling that output used to take me hours - and few of those were tweaks of well documented strategies. Others I never understood because it was too much effort for a relatively small profit (and other trading parameters).

<comment><USER3> Could you not automate that too? Have another stage of evolution where you just randomly delete statements from the 'code' and the fitness function is that the performance doesn't get worse.

This should eventually reduce the 'code' to it's minimal state, removing any unnecessary parts.

<comment><USER4> There already is automation for exactly this, it's the optimization step of any compiler. Sounds like OP's lacked an optimizer

<comment><USER5> That's not too helpful (especially with python due to interpretation) because what you want to see is the output of the optimizer _in the source language_ with only optimizations that make things clearer (as opposed to more opaque like many optimizations) applied.

</comment></comment></comment></comment><comment><USER6> I get the sense that many commenters misinterpreted the point of this study. This isn't about show-boating some newfound, great cryptography method, this is about teaching computers to re-write their own code.

If you can write software that improved itself better over time, our species as a whole would advance at incredible speeds. Think about all the possibilities that could evolve from self-maintained programs. Obviously the thought is a bit scary, but it's also incredibly exciting!

<comment><USER7> Please don't editorialize. Finding the best parameters that optimize an objective function can hardly be called "re-writing its own code." Granted, the article is just as guilty. (Edit: The article also doesn't mention self- modifying code so I'm not sure where you're getting that idea from.)

For the curious, here is the preprint: [https://arxiv.org/pdf/1610.06918v1.pdf](https://arxiv.org/pdf/1610.06918v1.pdf)

The technique is an off-the-shelf GAN that attempts to learn a transformation (not arbitrary code!) from input to encrypted output. The learned model is not Turing complete, and most importantly, is _not_ self-modifying. The optimization procedure is what modifies the network, not the network itself.

GANs have been used before to create realistic pictures. They're not new here -- the application is. It's a cool application, sure, but doesn't involve self-modifying code.

<comment><USER8> I don't think it's entirely inaccurate to call neural networks "rewriting their own code". For a long time people experimented with algorithms that could actually rewrite human programming languages, like genetic programming. And it's mostly a failure, because most mutations to most programs just break them.

Neural networks fix this problem by being sort of 'continuous programming language' where any small change to the net results in a small change to the output. And because of that we have algorithms that can find good changes to make much easier than with code. They are theoretically turing complete under some circumstances. But even when they aren't completely recurrent as in this case, they can still learn very sophisticated functions. Theoretically a neural net can learn anything a fixed number of logic gates can learn, which is still pretty powerful.

I don't think parent comment meant by computers that can "re-write their own code" was that the programs are self modifying. But _the computer itself_ is self modifying, as one program can change another on that computer, so the computer has modified itself.

<comment><USER7> In both the examples you cite (genetic algorithms and neural networks), the _optimization process_ is what modifies the parameters. The network doesn't modify its own parameters.

It's like calling pencil and paper self-modifying because somebody could use a pencil to write down plans to build a better pencil.

<comment><USER8> As I said, the parent comment didn't say the network was self modifying. The exact quote was "this is about teaching computers to re-write their own code" which is _roughly_ correct. If you see the weights of neural network as a sort of code, then the computer is rewriting it's own code.

<comment><USER9> And if you see the universe as "a sort of code", the network is rewiring the universe!

How's that for a bombastic editorial title?

(To be clear, I'm 100% agreeing with gcr here. Parameter optimization is a tremendously powerful technique, and quite possibly the "true currency" of our universe, but there's a few abstraction layers missing between that and a claim of "self-modifying AI")

<comment><USER8> The universe isn't code, and only gcr used the term "self modifying". There is absolutely nothing bombastic implied by the statement that a program can rewrite code.

<comment><USER10> But it's fundamentally one module [re]writing a completely different module. "rewrites its own code" is a very misleading way to describe that.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> This seems pretty click-baity to me. Especially "no one knows how it works."

Alice generates ciphertext based on plain text and key. Bob generates plaintext based on ciphertext and key. Eve generates plaintext based on only ciphertext. Train the three networks across a variety of plaintetxts,

No doubt it's cool, but I would be very surprised if it offered any insight into, or represented an advance for strong cryptography.

<comment><USER12> The paper [1] is by Abadi [2], which is a pretty big name in crypto. So I can assume it's something with value, wether the press can or can't convey that is another story.

[1] [https://arxiv.org/pdf/1610.06918v1.pdf](https://arxiv.org/pdf/1610.06918v1.pdf)

[2] [https://en.wikipedia.org/wiki/Mart%C3%ADn_Abadi](https://en.wikipedia.org/wiki/Mart%C3%ADn_Abadi)

<comment><USER13> Pretty big name is a bit of an overstatement. A pretty big name in cryptography would be someone like Goldwasser/Goldreich/Micali/Bellare/Rivest/Bernstein/... and probably about 200 more before we get to Abadi.

</comment></comment></comment><comment><USER14> This is most probably a rather lousy hack put together by Alice and Bob, but it maps out a very interesting future were we one day maybe can stop worrying about how our algorithms work and make that the job of the computer.

It also is totally terrifying to live in a world were computers can hide their messages from their own creators.

<comment><USER15> It doesn't. As an AI developer you have access to the AI code and memory.. How could you not replay the encryption process used by the AI since you know the state of what it "knows" constantly? Both have a encryption and decryption process that you can reproduce.

<comment><USER16> > As an AI developer you have access to the AI code and memory.

Until you don't. Sufficiently advanced AI can devise methods to mislead the developer.

<comment><USER17> Or until you only have access to the AI code and memory that designed the higher-level AI that designed the algorithm.

</comment></comment></comment></comment><comment><USER18> This is the link to the research paper ("Learning to protect communications with adversarial neural cryptography"): [https://arxiv.org/abs/1610.06918](https://arxiv.org/abs/1610.06918)

</comment><comment><USER19> TFA's title is a little clickbaity by adding "noone knows how it works" which apparently means that:

The researchers didn't perform an exhaustive analysis of the encryption methods devised by Alice and Bob, but for one specific training run they observed that it was both key- and plaintext-dependent. "However, it is not simply XOR. In particular, the output values are often floating-point values other than 0 and 1," they said.

Just saying...

<comment><USER20> Outputting floats instead of discrete values actually sounds like a very exploitable hole to me.

It reminds me of the classic "check if your model corresponds to reality" example where a one-time pad is broken: the standard said "<0.5V volts is OFF" but your machine was outputting 0.1V for OFFs that came from computing 0 xor 0 and 0.2V for OFFs that came from computing 1 xor 1.

</comment></comment><comment><USER21> I'm not sure what conclusions we're supposed to draw from this. Just because they were able to hide the messages from a 3rd AI doesn't mean the encryption was good. Shouldn't a human examine it and see if they can codebreak it? Isn't the goal better encryption?

<comment><USER22> I don't want to tell people what to take away from it, but my own view is that the contribution of the paper is the framing of the problem and a preliminary demonstration that you can achieve the goal of defeating a NN via an adversarial training formulation. In a lot of ways, I view our work as creating a challenge problem for the AI & crypto communities. It shows an intriguing possibility that is far from being practical, and itself represents a stepping stone in work on, e.g., training DNNs to learn to communicate.

Can this be extended to generate cryptanalysis-resistant schemes? Can we create a formulation that can learn to use discrete operators such as modular arithmetic? (The latter reflects one of my personal complaints about the state of DNNs -- it's very hard to have discrete "blocks" of functionality if that functionality isn't differentiable). Can we identify mechanisms that can train networks such as these more robustly? (It fails to train roughly half of the time; this is a fairly common problem in adversarial training of DNNs today.)

<comment><USER23> NNs/ANNs/DNNs/xNNs have already[0] been framed as easily fooled.

[0] [https://arxiv.org/pdf/1412.1897.pdf](https://arxiv.org/pdf/1412.1897.pdf)

</comment></comment></comment><comment><USER24> Isn't that a bit like "security through obscurity"? Unless I'm misunderstanding something, the cryptographic algorithm that the AI comes up with isn't guaranteed to be based on a provably NP-hard problem, so there aren't any formal guarantees. It would also be very hard to reason about, inspect, and prove correct.

Please note that I'm in no way dissing the researchers' work. What they did is pretty cool, but I can't see an obvious way to use these AI-generated algorithms in production systems where you may need to certify their correctness.

<comment><USER25> You may be right. It was mentioned in the paper:

        Our chosen network structure is not sufficient to learn general implementations of many of the mathematical
        concepts underlying modern asymmetric cryptography, such as integer modular arithmetic.
        We therefore believe that the most likely explanation for this successful training run was that Alice
        and Bob accidentally obtained some "security by obscurity" (cf. the derivation of asymmetric
        schemes from symmetric schemes by obfuscation (Barak et al., 2012)). This belief is somewhat reinforced
        by the fact that the training result was fragile: upon further training of Alice and Bob, Eve
        was able to decrypt the messages. However, we cannot rule out that the networks trained into some
        set of hard-to-invert matrix operations resulting in "public-key-like" behavior. Our results suggest
        that this issue deserves more exploration.

<comment><USER22> Note that this was for the public-key formulation. For the secret key formulation, it _is_ mixing in the key, so it's not pure security through obscurity. It may be -- probably is -- relatively poor cryptography, but it's not just doing a transformation of the plaintext without using the key.

The public key part is in there mostly because of the formulation, not because we actually had any noteworthy success in training a network to _learn_ pubkey. The result we mentioned was probably an anomaly, but it's an interesting anomaly to dig into more in the future.

But, in general, the parent is correct. There are few guarantees about a scheme derived in this way. It's one of the things I think most interesting for future work: Can one formulate "can't be cryptanalyzed by a mathematically sophisticated adversary?" as an adversarial training goal? Certainly, it's solvable for the one-time-pad-like scenario we used, but what about for more general cases that permit key reuse?

<comment><USER26> > Certainly, it's solvable for the one-time-pad-like scenario we used, but > what about for more general cases that permit key reuse?

Can't you force this simply by making the message a lot longer than the key? If the network can encrypt variable length messages, surely the method must be more advanced than a one time pad.

</comment></comment></comment></comment><comment><USER27> It's not clear from the article whether they train the networks with the same shared-key in every iteration, or if they randomize it. Any info on that?

<comment><USER22> It's a random key paired with a random plaintext for each input. In the experiments, the key is the same length as the plaintext.

Happy to answer more questions about it. (I'm one of the authors.)

<comment><USER28> So basically the NN is learning a one time pad?

<comment><USER22> Kind of. Except that there's no restriction that there has to be a 1:1 correspondence between the key and plaintext bits (or characters) that get mixed, as there would be in a conventional OTP. And, indeed, the DNN doesn't learn that - it mixes multiple key and plaintext bits together. Probably in a way that's worse than a true OTP -- the adversary is more successful than it should be were the encryption scheme a "correct" OTP with XOR.

<comment><USER28> Cool! Also, have you tried giving some bits of the key to the adversary?

<comment><USER22> I haven't. Interesting - that'd be a nice way to try to probe how strong the encryption is (i.e., "bits recovered vs. key bits supplied to adversary"). I'll have to think about that more - thanks for the idea!

</comment></comment></comment></comment></comment></comment><comment><USER29> > While it seems improbable that neural networks would become great at > cryptanalysis, they may be quite effective in making sense of metadata and > in traffic analysis.

How does that square with the fact that the best cryptanalists appear to have nothing but their own neural networks to work with?

This reminds me of the genetic algorithm that came up with a tone discriminator that at first glance looked like it could not work (parts connected wrong or not at all, and yet, crucial to functioning).

[https://www.damninteresting.com/on-the-origin-of- circuits/](https://www.damninteresting.com/on-the-origin-of-circuits/)

<comment><USER22> Because humans have invented tools - math, notably - that they can use to be more formal and precise when they deal with problems of this sort. To date, neural networks can't -- they're doing very well at the kind of "intuitive" pattern matching that our brains do, but less well on an algorithmic front. It's one of the really interesting frontiers of AI right now. The folks at DeepMind recently showed, for example, that a variant of their "Neural Turing Machine" architecture could learn to do things involving more like what we think of as discrete algorithms ([https://deepmind.com/blog/differentiable- neural-computers/](https://deepmind.com/blog/differentiable-neural-computers/) ), but all of this is very early work.

</comment></comment><comment><USER30> Schneier's Law: "Anyone, from the most clueless amateur to the best cryptographer, can create an algorithm that he himself can't break."

</comment><comment><USER31> Poor taste with the Terminator graphic. A new level of lame fear mongering by Ars Technica?

<comment><USER32> I was hoping psychedelic dog heads would become the default stock photo for AI news, rather than red eyed robots

</comment></comment><comment><USER33> "The researchers didn't perform an exhaustive analysis of the encryption methods devised by Alice and Bob, but for one specific training run they observed that it was both key- and plaintext-dependent. "However, it is not simply XOR."

I think this says it all.

</comment><comment><USER34> Really interesting that for the first ~6500 steps Eve was better at decrpyting the messages than Bob, which had access to the key.

Would be cool to see a setup like this in which Eve is a cohort of decryption tools, otherwise your Alice network will just overfit against its mechanics. Ie, if Eve uses lockpicks to open a door, an Alice solution would eventually replace the key-lock with a number pad - not necessarily any more secure, but will foil Eve every time.

<comment><USER35> >Really interesting that for the first ~6500 steps Eve was better at decrpyting the messages than Bob, which had access to the key.

This is probably because Alice wasn't good at encrypting yet.

<comment><USER34> On a second look, seems like a combination of Alice not being good at encrypting (in a sense that it had to encrypt in a way that makes decrpytion possible), and Bob needed to get up to speed on decrypting (even with the key).

As soon as that relationship clicks, Bob get better very quickly, while Eve improves at a slower rate for a while, before Alice adjusts (causing bob to get worse for a bit, until it catches up)

Would be interesting to see a set up where Alice / Bob get trained first, then Eve is activated and see how they adjust.

</comment></comment></comment><comment><USER36> Next: Google AI encrypts google's servers and demand Ransom

</comment><comment><USER37> Isn't this just the automated version of "anybody can invent crypto they can't break themselves"? It doesn't sound like the neural nets were provided with any high level cryptanalytic primitives. Just basic math and left to bodge something together. I can't imagine it would be easy to hill-climb towards an understanding of the _why_ behind crypto. Algorithms that work look a lot like ones that have glaring flaws - but those flaws only glare if you know the right way to come at them.

</comment><comment><USER38> It must be the time for Cryptography + Machine Learning. I had an O'Reilly Security 2016 talk "Machine Learning to Improve Random Number Generators" accepted this June 2016:

[http://conferences.oreilly.com/security/network-data- securit...](http://conferences.oreilly.com/security/network-data-security- eu/public/schedule/detail/53732)

EDIT: I thought the reference was relevant but please do critique/comment if you feel I've broken some convention.

</comment><comment><USER39> Hill climber climbs hill, news at 11?

<comment><USER40> Interesting choice of words!

</comment></comment><comment><USER41> Interestingly, Eve maintained a better than random chance at guessing the message. Consistently stayed a little better than 50%.

</comment><comment><USER42> This reminds me of Peter Watts's novel <Blindsight>. Two captured "Scramblers" (the highly intelligent alien without self-consciousness) managed to communicate with each other in a way defying all human analysis, even with human knowing what they are saying (the plain text).

</comment><comment><USER43> For those that can handle old pre CG movies there's a great old movie where computers make their own language. Highly recommended

Colossus: The Forbin Project (1970)

[http://www.imdb.com/title/tt0064177/combined](http://www.imdb.com/title/tt0064177/combined)

</comment><comment> martin-adams At an open day for an animation course they showed an AI derived walk cycle of a character. The results were amusing. The condition was that if it feel over that would be no good. The end walk cycle was a character doing forward flips in the air and landing on its feet. Very unexpected results.

</comment><comment><USER44> "No one knows how it works"

Yea, someone might also have said that about the GOST sboxes:

[https://eprint.iacr.org/2015/812](https://eprint.iacr.org/2015/812)

</comment><comment><USER45> Given I saw this as top on /r/futurology I figured it was clickbait BS. But since it was posted to HN I thought I'd give it a read.

It's clickbait BS.

<comment><USER46> Its interesting, but needs someone to re-submit with a proper write up about it without the fearmongerng titles.

</comment></comment><comment><USER47> Teach the AI to communicate with each other in a way humans can't eavesdrop on... What could go wrong with that?!?

</comment><comment><USER48> Google should teach an AI to do prime factorization instead.

<comment><USER49> Why?

</comment></comment><comment><USER50> maybe another AI system do crypt-analysis and break it. and nobody will know how it was broken.

</comment><comment><USER51> By the time we finally realize we're not as smart as we think we are, it's going to be too late.

</comment><comment><USER52> Am I the only one wondering about this and Ethereum...

<comment><USER53> Do you mind elaborating?

If you are talking about a smart-contract being able to do stuff with encrypted data without other people being able to access the data, I don't think this would help with that, because whatever system the contract would use would be available to whoever is trying to get the data from the contract.

Though, things may be enough to make it hard for other contracts to get stuff from a particular contract? I'm somewhat skeptical of that, but maybe. Also, in any case, the "attacker" contract could just make an incentive for people to give it the information, which I think would work. (and that might be necessary for it to get the info from the defending contract anyway? I don't remember. I don't think it is, but I'm not sure.)

However, I'm not sure that is what you meant, and also I'm not sure of what I'm saying.

edit: well, there are other systems that have been designed to let contracts keep information secret but I think those are still too expensive to be used, and relied on a secret sharing scheme iirc, so would require that not all the info be on chain.

</comment></comment><comment><USER54> > this is about teaching computers to re-write their own code.

Maybe I've seen too many movies, but I hope we can make sure they don't rewrite the "protect the humans" part.

<comment><USER55> but don't we then just unleash the huge EMP bomb?

<comment><USER56> Sounds good. Sadly, six months ago the bot brain predicted the day we were planning with .98 confidence.

<comment><USER57> But then it got sucked in to TVTropes, and the world is saved.

</comment></comment></comment></comment><comment><USER58> Duplicate of [https://news.ycombinator.com/item?id=12800228](https://news.ycombinator.com/item?id=12800228)

</comment><comment><USER59> What could ever go wrong with this....</comment>


Researchers at MIT Develop The Fastest Possible Data Transmission Method - <AUTHOR> http://bostinno.com/2012/02/12/researchers-at-mit-develop-the-fastest-possible-data-transmission-method/

<comment><USER2> Based on a paper in 2007: [http://allegro.mit.edu/pubs/posted/journal/2007-erez- trott-w...](http://allegro.mit.edu/pubs/posted/journal/2007-erez-trott- wornell-it.pdf) Patent: <http://www.google.com/patents/US8023570>

<comment><USER3> Thank you for this, it's quite difficult to follow the paper trail to the exact algorithm they're talking about. I had seen a talk about an extension to Reed-Solomon that promised more efficiency and was hoping this would talk about that, instead it looks like a wireless-optimized fountain code.

</comment></comment><comment><USER4> Really interesting, but definitely this has little to do with 'fastest possible data transmission'. It just is a nice way to find the optimal code rate for the current channel. Current systems just adapt the code rate and send all data again. I wonder if something similar exists or could be developed for source encoding, i.e. progressively enhanced video streaming, so that if you want higher quality you would just combine the original with additional data.

<comment><USER3> > It just is a nice way to find the optimal code rate for the current channel. > Current systems just adapt the code rate and send all data again.

Actually, this is likely an improvement on rateless/fountain codes. A key feature of these codes is the ability to reconstruct an entire message block if you collect "enough" subcodes, regardless of order. These actually obviate the need for retransmission and are crucial for things like reliable multicast wireless: it would be a nightmare to keep track of each of your receivers and retransmit lost packets to each one, with rateless codes you theoretically don't even need to know your receivers are there. Just keep spewing out data and eventually they will receive enough to make up for missing blocks.

A casual glance at this paper implies that they've found a model of these codes that takes into account standard models of noisy wireless channels; using this model allows you to optimize your code to have less error correcting overhead (meaning you need to collect fewer packets to successfully reconstruct the original message).

More info, and perhaps a more coherent explanation: <http://en.wikipedia.org/wiki/Fountain_code>

</comment></comment><comment><USER5> The bottom part of this Quora snwer is a analogy of the technique in the paper for sending physical money which might help you better understand things:

[http://www.quora.com/What-is-the-safest-way-to-send- someone-...](http://www.quora.com/What-is-the-safest-way-to-send- someone-1-000-cash-physical-bills/answer/Ben-Maurer)

</comment><comment><USER6> I half expected to be reading about quantum entanglement. Rats!

</comment><comment><USER7> Does this have any bearing on AI or machine learning. According to Marcus Hutter, the best AI is one that can compress the most (the hutter prize for compression). If compression is the same problem as data transmission, than this might an optimal AI algorithm as well.

<comment><USER8> This isn't about compressing data, it's rather the opposite: expand the data so that even if a certain portion of it is affected by interference, the original data can still be reconstructed without errors.

To take a really simple code, let's say you have this data:

10110

Then you append a checksum (actually a parity bit here) to the data 1+0+1+1+0 = 1 (in binary)

101101

Now let's say there's interference and a bit flips:

100101

The receiver calculates the checksum and sees that the sum is 1+0+0+1+0 = 0, which does not fit since the sum is supposed to be 1. Thus the receiver knows that an error happened and can request a retransmission. More advanced codes like those discussed here can allow arbitrary levels of error correction, instead of only detection of single errors like the parity bit; but higher levels of error correction come with an overhead, and apparently the innovation here is a technique to reduce this cost in wireless networks.

<comment><USER9> > This isn't about compressing data, it's rather the opposite: expand the data > so that even if a certain portion of it is affected by interference, the > original data can still be reconstructed without errors.

The two problems (data compression, and noisy channel coding) are tied together quite neatly by information theory though: <http://www.inference.phy.cam.ac.uk/mackay/itila/book.html></comment></comment></comment>