Zero-cost futures in Rust - <AUTHOR> http://aturon.github.io/blog/2016/08/11/futures/ <comment><USER3> This is huge.

This allows one to express concurrency in a natural way not prone to the typical errors of problems of this nature, with no runtime overhead, while competing with C in terms of the constraints of the runtime.

Big kudos to Aaron Turon and Alex Crichton. You guys knocked it out of the park.

<comment><USER4> Agreed, and very cool... I do think that async/await syntax is very nice for reducing a certain amount of nesting bloat. Even with chaining, it doesn't look as nice imho, though better than deeply nesting callbacks.

</comment></comment><comment><USER5> As mentioned in the post, given Rust wants to operate in the same space as C, this approach makes sense. However from a higher level, building more complex concurrent systems, dealing with futures/deferred-s/promises and/or a central select/epoll/kqueue reactor loop gets daunting and doesn't mix with complex business rules.

Deferred based approach has been a round for many years. I experienced it by using Twisted (Python framework) for 5 or so years. And early on it was great However when we switched to using green threads, the logic and amount of code was greatly simplified.

So wondering if Rust provides any ability to add that kind of an N:M threading approach. Perhaps via an extension, macro or some other mechanism.

Note that in C, it being C such things can be done with some low level trickery. Here is a library that attempt that:

[http://libmill.org/](http://libmill.org/)

And there were a few others before, but none have taken off enough to become mainstream.

<comment><USER6> > So wondering if Rust provides any ability to add that kind of an N:M > threading approach. Perhaps via an extension, macro or some other mechanism.

I don't want M:N threading as Go implements it. It's a big loss of performance for marginal benefit over futures. In particular the libmill approach was tried in Rust and the results were far worse than 1:1.

However, assuming this takes off I would like to see async/await style syntactic sugar over futures down the road to make it easier to write code that looks blocking but actually isn't. Crucially, this sugar would maintain the zero-cost nature of futures. With that approach, we'd have the ergonomics of languages like Erlang and Go without the significant performance tax associated with M:N.

<comment><USER7> > I don't want M:N threading as Go implements it.

As someone who writes code for enterpricey businesses doing a lot of I/O bound stuff, golang style M:N threading is a godsend over Java's standard library, and other common platforms in that space. Being able to express your code in a sequential manor and still gain the performance offered by implicit fiber,goroutine,w/e scheduling is pretty awesome. With futures, there's still some language semantic overhead from the point of view of the developer.

In my problem space, M:N threading is simply worth it. And if I need anything better performing, I can still switch to C for specific use cases.

<comment><USER6> > Being able to express your code in a sequential manor and still gain the > performance offered by implicit fiber,goroutine,w/e scheduling is pretty > awesome.

You don't gain as much performance. On Linux, you don't actually gain that much if anything over 1:1 threading. Most of the benefits of goroutines actually comes from the small stacks, which don't have anything to do with M:N and 1:1 to begin with—they're a feature of GC.

As the blog post states, our end goal is to achieve the ergonomics of Go-style M:N without sacrificing nginx levels of performance. With this futures library, we've established the foundations. It would make no sense to give up before even trying.

> And if I need anything better performing, I can still switch to C for > specific use cases.

Why would you write networking code in C in 2016, when there are better alternatives available (like this one)?

Rust's philosophy is to have _both_ performance _and_ ergonomics. It rejects the idea that optimal performance requires a cumbersome programming model. I'm not about to give up on that.

<comment><USER8> > Why would you write networking code in C in 2016, when there are better > alternatives available (like this one)?

Support for kernel bypass networking libraries like ibverbs, DPDK (has an old unmaintained Rust wrapper) and other IO kernel bypass libraries such as SPDK and IOAT.

If Rust supported these libraries, I'd much prefer the future based Rust code to a massive event loop in C.

<comment><USER9> I'm one of the authors of SPDK (which includes an NVMe driver and an IOAT driver). If the community wants to add rust bindings to those two components I'd be very supportive.

</comment></comment></comment></comment></comment></comment><comment><USER10> TLDR;

 _I've claimed a few times that our futures library provides a zero-cost abstraction, in that it compiles to something very close to the state machine code you'd write by hand. To make that a bit more concrete:

\- None of the future combinators impose any allocation. When we do things like chain uses of and_then, not only are we not allocating, we are in fact building up a big enum that represents the state machine. (There is one allocation needed per "task", which usually works out to one per connection.)

\- When an event arrives, only one dynamic dispatch is required.

\- There are essentially no imposed synchronization costs; if you want to associate data that lives on your event loop and access it in a single- threaded way from futures, we give you the tools to do so._

This sounds quite badass and awesome. I'm not sure what other language implementations take this approach, but this is clearly an extremely beautiful, powerful, and novel (to me at least!) concept. Before reading this, I thought rust was great. This takes it to the next level, though.

<comment><USER11> that sounds great but I guess that will make the compilation time bigger.

<comment><USER12> In Rust it's frequently the case that slow compilations are dominated by generating and optimizing LLVM IR. This codegen step (generating LLVM IR) often takes awhile just because we're generating so much IR.

Rust takes an approach with generic functions called monomorphization which means that we generate a new version of each function for each set of generics it's instantiated with. This means that a future of a String will generate entirely different code from a future of an integer. This allows generics to be a zero cost abstraction because code is optimized as if you had substituted all the generics by hand.

Putting all that together, highly generic programs will generally trend towards higher compile times. With all the generics in play, there tends to be a lot of monomorphization which causes quite a lot of LLVM IR to get generated.

As with many aspects of Rust, however, you have a choice! Rust supports what we call "trait objects" which is a way to take a future and put it behind an allocation with a vtable (virtual dispatch). This forces the compiler to generate code immediately when a trait object is created, rather than down the line when something is monomorphized.

Put another way, you've got control over compile times if you're using futures. If you're taking a future generically and that takes too long to compile, you can instead take a trait object (or quickly convert it to a trait object). This will help cut down on the amount of code getting monomorphized.

So in general futures shouldn't make compilation worse. You'll have a choice between performance (no boxes) and compile times (boxing) occasionally, but that's basically already the case of what happens in Rust today.

<comment><USER13> Hm. I've heard arguments that C# or Java is slow for multiple reasons, but never because of the minuscule overhead of a virtual method dispatch when using objects behind interfaces (kinds similar to trait objects).

It's interesting that this is seen as significant here. Are we dealing with much shorter timescales, or just being eager to optimise everything?

<comment><USER14> Virtual dispatch per se is not terribly slow, as long as the branch is predictable by the CPU. The problem is that virtual dispatch prevents the sort of aggressive inlining and interprocedural opimizatios that C++ compilers are known to do. C# and Java JITers get around that via runtime analysis and speculative inlining, but that is done at runtime and eats away some of the precious little time available for optimisations.

Edit: spelling

<comment><USER13> Put it this way:

Cost of a branch misprediction is 10s of cpu cycles. (1) Measured in gigahertz (10^9 cycles per second).

Time to turn around a web request is, if you're very lucky and have done the work, mainly about getting a value from an in-memory cache at multiple milliseconds (2). That's 1 / (10^3) seconds.

If you're not lucky, 10s or 100s of milliseconds to generate the response.

It seems that the second duration is best case around 10^6 times longer. I would not sweat the first one.

1) [http://stackoverflow.com/a/289860/5599](http://stackoverflow.com/a/289860/5599) 2) [http://synsem.com/MCD_Redis_EMS/](http://synsem.com/MCD_Redis_EMS/)

<comment><USER14> Contrary to popular belief, not all C++ programs (or rust FWIW) are web servers serving HTTP requests over the Internet.

<comment><USER13> Yep, that's why I'm asking about the use-cases in the grandparent comment.

<comment><USER14> As an example, many real-time systems are often a giant ball of messy asynchronous code and state machines. Futures can help with that, although lately I have found that somtimes the best, cleanest, way to implement a state machine is to make it explicit.

<comment><USER15> How much do you attribute that to the benefit of creating a high barrier to entry for modifying that code? Could this be summarized as: code that inexperienced devs can't understand, stays performant because they can't figure out how to change it?

<comment><USER14> None of the teams I've worked with had such a policy and certainly I wouldn't work in a team like that.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER16> I dabbled with rust in the past and was really fascinated with it, but haven't played around lately. One thing caught my eye in the post:

        fn get_row(id: i32) -> impl Future<Item = Row>;

That return type looks odd to me. What does it mean to return an "impl", and is that a new feature in rust, or just something advanced that I missed in my exploration before?

<comment><USER17> This is an exciting upcoming feature in Rust, which you can read more about in a couple places:

\- [http://aturon.github.io/blog/2015/09/28/impl- trait/](http://aturon.github.io/blog/2015/09/28/impl-trait/) \- [https://github.com/rust-lang/rfcs/pull/1522](https://github.com/rust- lang/rfcs/pull/1522)

This feature allows you to return any struct that implements the trait, without having to type the name of the struct (which can sometimes be quite big). It also means that clients only know what traits are implemented; the concrete type is invisible to them. The above links have a bunch more detail.

(And this feature is set to land in nightly Rust very soon! Shoutout to eddyb :)

<comment><USER18> So, existentially quantified types?

<comment><USER19> Yes. However, there isn't full support of existential types in the horizon – only returning them from a function (and using them inside the caller, as the type inference allows this). This resolves some specific pain points of the current Rust experience.

There may or may not be some extensions (like storing them in fields of structs) in the future.

Note that Rust supported existentials before too, in the form of trait objects. But this was only possible behind a pointer and using virtual dispatch, so the performance story wasn't perfect. The impl Trait syntax is supported through monomorphization.

<comment><USER18> Yeah, since rust is eager existential types seem like a necessary evil.

</comment></comment></comment></comment></comment><comment><USER20> In my opinion - as someone with some background in CS - the name "future" is a little too overloaded here. It is not only used for the deferred computation of a value so much as it also means the composition of computations. This is not wrong per se, but calling the result a "future" alone oversimplifies what's happening below and hides some properties about the combinations.

The first observation one can make - which is not mentioned anywhere in the article - is that the composition of futures here can be understood as a monadic composition. This by itself gives a big hint why this interface is so powerful. Second is that this library could be understood as an implementation of process and process combination from pi-calculus [1] - <USER2> combination, joining, selection, etc - so it could be formalized using its process algebra.

From the practical side, one example of a mature library that implements similar concepts is the LWT [2] library for OCaml, which has the same idea of deferred computation, joining and sequencing, but calls the computations "lightweight threads". One could also argue about naming in this case, but it seem to reflect a better the idea of independent "processes" that are combined on the same address space.

Finally, as much as these concepts of futures and processes look similar on the surface, they each have their own properties - so it's always good to consider what better fits the model. By looking at the research and at other similar solutions, one can make more informed choices and have a better idea of what to expect from the implementation.

[1] [http://www.cs.cmu.edu/~wing/publications/Wing02a.pdf](http://www.cs.cmu.edu/~wing/publications/Wing02a.pdf)

[2] [http://ocsigen.org/lwt/manual/](http://ocsigen.org/lwt/manual/)

</comment><comment> nv-vn Anyone else find the f.select(g)/f.join(g) syntax unintuitive/awkward? I'm confused as to why they wouldn't go with the (IMO) more logical select(f, g) and join(f, g) in this case (since neither Future is really the "subject" in these cases). Not that this is a major concern (it would take only a few lines of code to change within your own program using an alias for the functions), just interested in knowing the rationale behind the choice.

<comment><USER21> You can write it both ways. t.method() is the same as T::method(t).

</comment></comment><comment><USER22> Finally a nice async/io interface for rust, always felt that it was a big missing piece, couple of questions for peps familiar with async in other languages :

1 - Isn't the state machine approach the same as C#/.net async/await is using ? But the with the added convenience of the syntactic sugar ?

2 - no allocation : , does'nt the lambda closure need to be allocated somewhere ?

3 - I would have love some comparison (booth performance wise and on theory) with C++ up comming coroutine work, from my understand the C++ approach is even more efficient in term of context switching and have the advantage of even less allocation.

<comment><AUTHOR> 1\. I am not sure exactly how async/await is implemented, but I believe it is very similar. Some people are also working on implementing similar sugar in Rust, but it's not done yet.

2\. Closures are on the stack, not the heap, by default, in Rust. If you don't see a Box, they're not heap allocated.

3\. I agree!

<comment><USER23> Stack-allocated closures are one of my favorite features of Rust. They make it easy to write functional-style code that performs well.

</comment></comment></comment><comment><USER24> Little benchmark rs-futures vs lwan ([https://lwan.ws](https://lwan.ws)) on my machine Core i5

futures-minihttp(singlethread):

      $ wrk -c 100 -t 2 -d 20 http://127.0.0.1:8080/plaintext
      Running 20s test @ http://127.0.0.1:8080/plaintext
        2 threads and 100 connections
        Thread Stats   Avg      Stdev     Max   +/- Stdev
          Latency   823.09us  449.37us  20.98ms   98.69%
          Req/Sec    62.15k    10.51k  105.24k    48.63%
        2479035 requests in 20.10s, 340.44MB read
      Requests/sec: 123335.77
      Transfer/sec:     16.94MB

lwan(singlethread):

      $ wrk -c 100 -t 2 -d 20 http://127.0.0.1:8080/
      Running 20s test @ http://127.0.0.1:8080/
        2 threads and 100 connections
        Thread Stats   Avg      Stdev     Max   +/- Stdev
          Latency   596.45us  573.31us  24.46ms   99.33%
          Req/Sec    86.17k    13.15k  119.71k    76.00%
        3429720 requests in 20.01s, 624.73MB read
      Requests/sec: 171404.15
      Transfer/sec:     31.22MB

For lwan i use http server example from lwan.ws main page.

As you can see in this example C http server much faster than simple http Rust server.

* futures-minihttp release build

* lwan -O3

<comment><USER25> On Reddit [1], Alex mentioned that the single-thread case was not optimized (yet). What do the numbers for multi-threading look like?

[1]: [http://reddit.com/r/rust/comments/4x8jqt/zerocost_futures_in...](http://reddit.com/r/rust/comments/4x8jqt/zerocost_futures_in_rust/d6dm0f5)

<comment><USER24> futures-minihttp(2 threads):

      $ wrk -c 100 -t 2 -d 20 http://127.0.0.1:8080/plaintext
      Running 20s test @ http://127.0.0.1:8080/plaintext
        2 threads and 100 connections
        Thread Stats   Avg      Stdev     Max   +/- Stdev
          Latency   633.87us  808.22us  24.66ms   98.41%
          Req/Sec    86.72k     4.84k   94.90k    85.75%
        3452383 requests in 20.01s, 424.73MB read
      Requests/sec: 172546.21
      Transfer/sec:     21.23MB

lwan(2 threads):

      $ wrk -c 100 -t 2 -d 20 http://127.0.0.1:8080/
      Running 20s test @ http://127.0.0.1:8080/
        2 threads and 100 connections
        Thread Stats   Avg      Stdev     Max   +/- Stdev
          Latency   375.26us  618.54us  20.60ms   98.21%
          Req/Sec   116.82k     5.56k  124.07k    92.25%
        4647906 requests in 20.00s, 846.62MB read
      Requests/sec: 232339.90
      Transfer/sec:     42.32MB

</comment></comment></comment><comment><USER26> This is cute. This is clever. Whether or not it's too clever time will tell. A year ago, I noted that Rust was starting out at roughly the cruft level C++ took 20 years to reach. Rust is now well beyond that.

All this "futures" stuff strongly favors the main path over any other paths. You can't loop, retry, or easily branch on an error, other than bailing out. It's really a weird syntax for describing a limited type of state machine.

I'm not saying it's good or bad, but it seems a bit tortured.

<comment><USER27> How does futures count as cruft? It's a pure library. This is like saying GObject is cruft in C. It is a library that some folks don't like (like most libraries), but it is not part of the language and nobody is forced to use it in their code.

Why do you think Rust has cruft anyway? It has a lot of typesystem features, yes, but this is no different from languages like Haskell. These features work together nicely and are useful. C++ has lots of features which for better or for worse have been hacked in to the language (can't be made an organic part of the language because backcompat). This is not the case with Rust (or D, which is to me a cruft-less organically-designed C++)

<comment><USER26> Yes, it's a pure library. But this sort of thing is becoming standard for Rust. If your code isn't full of "foo.and_then(|x| ...)" it's uncool. This isn't "functional"; these functions have major side effects.

The bothersome thing is that the control structures of the language are hidden under object-specific functions. "Things really start getting interesting with futures when you combine them. There are endless ways of doing so." I'd rather have "there's only one way to do it", as in Python, rather than "endless ways of doing so". That usually leads to code that's hard to read and debug. As in "how did control get there?". At least in traditional code, you can see control flow easily. Adding an user-definable level of abstraction hides that.

<comment><USER27> > This isn't "functional"; these functions have major side effects

uh, no, I've rarely seen adaptors like and_then being used with side effects (folks use regular loops if they want that). Rust doesn't have a strict notion of purity, but that doesn't mean that most rust code isn't pure.

There's nothing wrong with having lots of adaptors scattered around the code, either. It's not less readable, it's just _different_.

> There are endless ways of doing so." I'd rather have "there's only one way > to do it", as in Python

Uh, "there are endless ways of combining future adaptors", not "there are endless ways of solving a problem". Each combination of adaptors solves a different problem (mostly).

> That usually leads to code that's hard to read and debug

This is async code. This has always been hard to read and debug. Futures make the control flow more explicit, if anything (especially if you have async and await), because the flow is now in one place, at least. Grokking manual event loop code is much more annoying.

Sure, the async argument doesn't apply to regular iterators and Option. However, these "object specific functions" are not object specific. All futures have the same adaptors. All iterators have the same adaptors. The only special objects with their own set of such methods are Result and Option. These share the names of the methods, and these are used often enough to justify it. There aren't that many of them either, so this really isn't that big a deal. You just need to know what each of this small number of methods does. It only hides control flow if you're not aware of these methods, which is a state of mind that goes away quickly after the first few rust programs.

Besides, because of the closures it's pretty obvious that _some_ tweaking of control flow is happening, so it isn't hidden. You can check the docs for that function to know exactly what the tweaking is.

I also don't know what you mean by "traditional code", this pattern is exceedingly common in languages which aren't C or C++.

</comment></comment></comment></comment><comment><USER28> I'm confused by

        .map(|row| { json::encode(row) })
        .map(|val| some_new_value(val))

Over

        .map(json::encode)
        .map(some_new_value)

Is the explicit extra layer of lambda generally prefered in Rust over just passing the functions?

<comment><AUTHOR> It's just a style thing, some people prefer one way, some another. I personally prefer the latter. They compile to the exact same thing.

<comment><USER28> The makes sense. Do you know if Rust already has an idiomatic correct style?

<comment><AUTHOR> I don't think that there's ever been an explicit discussion about it. I'm not even sure how many people know the latter is possible, to be honest, it's a bit harder to learn about.

</comment></comment></comment></comment><comment><USER29> I love the direction this is going, and the performance it achieves.

Debugging promises/deferreds in other languages has given me nightmares, compare with erlang/golang debugging where you get a simple stacktrace.

Does this provide some nice way of debugging complex future chains? Are there plans towards making it super easy to debug?

Cheers!

<comment><USER30> Native promises in JavaScript suffered from debugging issues - but there are hooks that can help you with those - like unhandledRejecton and the like.

Debugging promises in JavaScript is very easy at the moment.

</comment></comment><comment><USER27> I'm rather surprised by the benchmark; I would expect the Go benchmark to be faster than Java (and the fact that it isn't may indicate some improvements that can be done to fasthttp by learning from rapidoid or minihttp). Then again, the difference isn't that much, so it just could be implementation details that would require a total refactor to fix.

<comment><USER31> You may find this makes somewhere more sense to think of it as ~5.3 microseconds per request for fasthttp vs. ~4.8 microseconds for Java vs. ~4.3 for Rust. It's 40 microseconds or so for the standard lib Go. I'm just eyeballing the graph but this should be close enough (dominated by local CPU variances and such). Just as some people point out that "gallons per mile" is a more intuitively useful way of thinking, I think that at this scale "overhead per request" is a better way of thinking about it.

I'm not generally a big fan of measuring the "ping time" response for web servers, but in this case I believe it is justified since we really are trying to establish that this future library is very fast. I fear, based on experience, that some developers well be looking at this and will sit there trying to choose fasthttp vs. rapidoid vs. minihttp based on this one graph, without considering what they really mean. Overhead-per-request I think makes it more clear that for the vast, vast majority of purposes, all of these, including Go and Node, are "way way faster than your code", and unless you _know_ you're building a server where you seriously need to answer an API call at several hundred thousand responses per second, all of these are "fast enough" and the real criteria for choosing should be "everything else".

(One last edit... remember, it's "milli/micro/nano". Micro seems to be forgotten since it seems like many things that we care about fit into nano- or milli-. It's quite a challenge in the web world to get your request out in under a millisecond usually, so .040 milliseconds added as HTTP overhead is rarely the problem.)

<comment><USER17> Thanks for this comment! I actually totally agree with this perspective, and wish I'd used your suggested scale in the post.

</comment></comment></comment><comment><USER32> Is there any special handling for Futures that complete with an error?

Also, how do you debug code that's hung or taking too long? It might be useful to get a list of all the jobs (incomplete Futures) that are currently running, much like running 'ps'.

<comment><USER17> Yes -- the blog post didn't go into details about this, but Futures in general have an error type as well, and all the combinators know how to propagate errors correctly. (There's also a notion of "cancellation" for a future -- we'll get into this with later posts).

In terms of debugging, there's not infrastructure currently, but the kind of thing you're talking about should be easy to add!

<comment><USER33> I've used async tooling extensively on several platforms now, including Go, C#, Java, Clojure's core async, the new async methods in JS.

The challenge of these state machine codegen abstractions is not performance. It's making source-facing debuggers do the right thing and handle errors in a way that doesn't break the illusion that the code running is the code in source.

If you can match the degree of support C# has for error handling, you'll be in an amazing place.

</comment></comment></comment><comment><USER34> While the benchmarks are looking a lot better than many other similar Rust libraries in this space, I'm not sure the code is in a state where they're actually meaningful yet: [https://github.com/alexcrichton/futures- rs/blob/master/futur...](https://github.com/alexcrichton/futures- rs/blob/master/futures-minihttp/src/response.rs#L30-L35)

<comment><AUTHOR> I agree that good benchmarks are important. This is the Techempower benchmark, which only requires this, and is how it's implemented in most languages for Techempower. See [https://github.com/TechEmpower/FrameworkBenchmarks/blob/180d...](https://github.com/TechEmpower/FrameworkBenchmarks/blob/180d44e1064abc6fe8c703b05e065c0564e6ee05/frameworks/Java/rapidoid/src/main/java/hello/SimpleHttpProtocol.java#L37-L77) for example.

Extra tests for more complex stuff would be great.

</comment></comment><comment><USER35> This looks really impressive. I'm curious what the story is around propagating errors through chains of futures. Traditionally future libraries don't pay much attention to that which can make debugging excruciating, which it doesn't have to be. But then rust does errors differently so maybe it's less of an issue there?

About the naming though, I was a little disappointed. Out of future, deferred, and promise, "promise" is the better term. The two others imply that something will happen later which is misleading because it's fine to have promises stick around long after they're fulfilled.

</comment><comment><USER36> This is cool and validates Rust, but I just want to add that even 2kb stacks as mentioned in sibling comments is bigger than Erlang's process stacks. In Erlang 19.0.3, even with dirty-schedulers enabled, a process's default size is 338 words.

</comment><comment><USER37> Why you didn't compare it to C++ or C ? If you want to compete with C/C++ it would be natural to compare those in benchmarks. Java and Go have GC. It's like comparing super car with street cars when you should compare it to other super cars.

<comment><USER27> Because they're benchmarking the implementation of async IO, and showing the results against libraries that folks use. This is not a common pattern in C/C++, so the libraries there are fewer and less people have used them.

But, if you want C++, check the data from the original benchmark which was linked: [https://www.techempower.com/benchmarks/#section=data-r12&hw=...](https://www.techempower.com/benchmarks/#section=data-r12&hw=peak&test=plaintext) . The Java one is slightly faster than the C++ one (more like "just as fast", since the difference is tiny). And there are 7 more java libs (half of them with recognizable names) before the next one (which I've never heard of), which underlines my point about this being more common in Java than C/++.

The benchmark wasn't shown to prove that Rust is the fastest language in the universe. It was shown to prove that the Rust futures implementation is competitive with the others in use today, ones which people would recognize.

</comment></comment><comment><USER38> The recent flurry of activity around async IO in Rust has been really exciting; to me, it indicates that the core team's decision to stabilize the language was a smart bet that is paying off in rapid ecosystem growth.

One quibble I have with this post is that it talks about futures as a zero- cost abstraction. That might be true (or close to true) from a performance perspective, but in my (admittedly inexperienced) opinion, it seems to have a significant ergonomic cost that is not accounted for.

While futures help us deal with multi-threaded coordination of data from multiple sources, that overhead isn't necessary for situations where you're running in a single thread dedicated to doing IO operations.

Dealing with futures in your code is not non-trivial. Browsing through the futures version of the HTTP server, I had a hard time following along:

[https://github.com/alexcrichton/futures- rs/blob/master/futur...](https://github.com/alexcrichton/futures- rs/blob/master/futures-minihttp/src/lib.rs#L92-L133)

And it requires a bunch of helper code to go with it:

[https://github.com/alexcrichton/futures- rs/blob/master/futur...](https://github.com/alexcrichton/futures- rs/blob/master/futures-minihttp/src/io2.rs)

The blog post mentions Tokio, another high-level abstraction on top of mio (by the same author). Because it doesn't require the futures abstraction from top- to-bottom, it offers similar (maybe even a little better) performance with what, to my eyes, is far simpler code:

[https://github.com/tokio-rs/tokio- minihttp/blob/master/src/l...](https://github.com/tokio-rs/tokio- minihttp/blob/master/src/lib.rs#L21-L44)

I'm still learning Rust and spend most of my time in JavaScript. The analogy I'd use is: imagine if in the Node programming model, every API required you to use JS Promises, even at the very lowest level. Even if you could reduce the cost of creating new Promise objects, interacting with them over simple values could make the code you write more verbose. In Rust, that problem is exacerbated by the much stricter type system and the fact that you have to do cross-thread coordination.

I'm a total beginner to systems programming, and a lot of this stuff is above my pay grade. However this shakes out in the community, I'm very happy to see Rust on the way to becoming the fastest, most productive way to write high- performance web services.

<comment><USER17> Thanks for the thoughtful reply!

I'm a little confused about the snippet you're pointing out. It's not actually _using_ futures at all! In fact, that code is just part of setting up threads for the server. The reason it's more complicated than the version in Tokio is that minihttp supports _multiple_ event loop threads (which gives some performance benefits), and this code is handling that setup.

At a broader level, Tokio's services -- the main thing users write -- are based on futures, in exactly the same way as minihttp. So for people writing actual servers, the ergonomics should be the same.

Now, stepping back, it's definitely true that ergonomics are a cost to be aware of, and it's one we've thought carefully about in the design of futures. We've had a lot of experience and success in Rust with iterators, which share a lot of the same API design philosophy.

That said, I do anticipate that over time we'll want to layer sugar on top of futures. As I mention a couple times in the blog post, async/await (or something like it) is the obvious way to do it, and there's plenty of prior art. But I think we should walk before we run -- let's make sure we've got the core abstraction right, and then we can sprinkle some sugar where it's needed.

</comment></comment><comment><USER39> One thing I have not seen in discussion is - Work vs. Worker abstraction.

Your application work - computation logic/business rules, should be decoupled from the type of worker.

The worker can be - blocking or non-blocking - Futures/Continuations/Co- routines.

</comment><comment><USER40> Not sure if I missed this in the post, does this depend on any unstabilized features or can we use this today on 1.10.0 stable?

Awesome stuff btw, love the iterator inspiration.

<comment><USER12> You can indeed use this on stable Rust today! Right now 1.9.0 is the minimum supported version due to the usage of `catch_panic` in a few places.

I'd recommend a beta compiler for now though to compile some of the examples. There's a bug in the stable compiler which causes them to take up to 8x longer to compile, but beta/nightly are both speedy!

<comment><USER40> That's so awesome. I was hoping that was the case.

Serious Kudos, it feels like a lot of the promises of Rust are really paying off here.

<comment><USER41> "promises" _applause_

</comment></comment></comment></comment><comment><USER42> Big downside is now you will have a dichotomy of functions that block using futures and functions that block at the OS level and no sane way to intermix them. Rust essentially becomes two languages. Async/await sugar doesn't fix this.

Would be great if functions could be written in a general way for both IO models and users could select the implementation at their convenience.

<comment><AUTHOR>
      > no sane way to intermix them.

My understanding is, the idea is to put the blocking stuff in a threadpool with [https://github.com/alexcrichton/futures- rs/tree/master/futur...](https://github.com/alexcrichton/futures- rs/tree/master/futures-cpupool)

<comment><USER42> Right, there may be a lot of back and forth marshaling between using thread pools and not depending on whether the library you're using is futures based or not.

Maybe you use one library that is futures-based and one that isn't. Maybe the library you use is mostly non-blocking except for one use of sleep() or another esotorically blocking call. It's just annoying and prone to error. Most people may not even be aware of the subtly blocking nature of the code they use in their futures-based project.

This is what I mean by having two different languages. Libraries written for one aren't always/simply compatible with the other. You'll have a growing community of nominally "futures-based" rust libraries too.

This is why people use Go or Erlang. It just removes the need to have to think about this. Not saying they are generally better than Rust, and some Rust people may even like having a futures-based sublanguage, but I suspect most programmers will be loathe to have to deal with the extra mental tax.

<comment><USER6> > Right, there may be a lot of back and forth marshaling between using thread > pools and not depending on whether the library you're using is futures based > or not.

So just like if you use cgo. You can't get away from having to deal with the issue entirely; the most you can do is to punt it to the FFI layer. There is the question of how much of the community is using blocking vs. nonblocking I/O, to be sure, but Go has a version of that too: how much of the community is using cgo vs. how much of the community is writing in pure Go.

> This is what I mean by having two different languages.

Calling them "two different languages" is a huge exaggeration. You simply block or switch to a thread pool: it's very easy.

> I suspect most programmers will be loathe to have to deal with the extra > mental tax.

I like having the low-level control over blocking vs. not, especially in situations where I can't use async I/O everywhere (for example, my work on Servo). In fact, it's essential.

Ultimately this is going to come down to "you should be willing to pay a performance and control tax for a more ergonomic model" vs. "you shouldn't give up performance and control for a small amount of ergonomics". Yes, there is a tradeoff here. That's fine. Taking Go's side of the tradeoff would make Rust unusable for my domain, and for many others (which is why M:N was the most controversial issue ever in the Rust community, with most of the community demanding it to be removed, while in Go nobody questions it). Some people may not want Rust's side of the tradeoff, and that's fine too.

<comment><USER42> I see your cgo analogy but at the same time it's much less pronounced there since the programming interface is the same, the programmer is supposed to assume everything will work as it should (even if it doesn't always). In this case it's a different programming interface and I think that stresses the issues.

Regarding your comment on preferentially having control over blocking/async code. I think that's right. At the same time, some C++ programmers would say that they prefer having to think carefully about how memory is managed in there program (say, for the benefit of fast no bounds checking). C++ draws a line, Rust draws a line, Go draws a line, Java draws a line, and Python draws a line. These lines are somewhat about technical superiority and somewhat about programmer identity/preference but they are mostly about domain-specific constraints and necessary tradeoffs. This futures-based approach will be sufficient (if somewhat inconvenient) where Go/Erlang can't be used, e.g. where GC pauses are absolutely intolerable.

<comment><USER6> It's not just about GC pauses. Rust isn't "little Go" that you reach for only when you can't afford a GC. Many people choose Rust for the cargo package manager, generics, pattern matching, mature optimizer that prioritizes runtime speed over compilation speed, ability to write libraries callable by any language, fast FFI, compiler-enforced data race prevention, memory safety in multithreaded mode, etc. etc. These benefits apply to servers too. And many of these benefits are what lead to the futures model being more appropriate than the M:N model for the language.

Go has its benefits too, of course! One of those benefits is that blocking I/O is a simpler mental model. Both languages can happily coexist without one being in the shadow of the other.

</comment></comment></comment></comment></comment></comment><comment><USER43> Wohoo. I was waiting for this. I hope that at a later point this will also mean that we get some sort of syntax support for it once it's stable and entered std.

<comment><USER44> What sort of syntax support do you have in mind? (Personally I think any language-level support is unlikely, given that Rust doesn't make a point of privileging any particular approach to concurrency.)

<comment><USER43> See Python 3.5, ES6 and C# and their await stuff. They generally use a form of future internally to abstract this.

<comment><USER44> Ah yes, I wasn't thinking of async/await. I suppose there's enough precedence from other languages to warrant that, though this is probably far-future work (I don't even expect futures to land in the stdlib for quite a while, though I would like them to eventually).

</comment></comment></comment></comment><comment><USER45> Back when futures and promises were a new concept to most people, if someone asked me to explain why you would want to do such a thing, my favorite example was loading images in a web browser. You wouldn't want to load the same image four times just because it appears in four places on a page, would you? Yada yada promises etc etc.

Seeing articles like this makes me feel like a circle has finally been closed.

</comment><comment><USER22> For those who are curious about how does that fair against a coroutine based approach : [https://www.youtube.com/watch?v=_fu0gx- xseY](https://www.youtube.com/watch?v=_fu0gx-xseY)

</comment><comment><USER46> > a simple TCP echo server;

How convenient. I've been exploring/learning Rust, and writing a simple echo server and comparing it to a reference version I've written in Perl is my first semi-trivial program I wanted to do to compare.

</comment><comment><USER47> Curious if someone has tried this and Eventual[0] with any thoughts on how they compare.

[https://github.com/carllerche/eventual](https://github.com/carllerche/eventual)

<comment><USER12> The futures crate is intended to be the successor to eventual, the author of which, Carl, helped us with some key insights in the futures crate as well.

<comment><USER47> Cool! Thanks for the reply :) It's nice to see some collaboration as opposed to trying to decide between competing crates.

</comment></comment></comment><comment><USER48> Aawesome. So to cancel a future, I just drop it! Awesome.

</comment><comment><USER49> Unfortunately, it seems that you still need to use callbacks and lots of and_thens to write this async code.

Wouldn't it be possible to add coroutines to Rust instead?

</comment><comment><USER50> How are futures handled for open() and disk i/o?

<comment><AUTHOR> Nothing special yet; those are still implemented in a blocking way, and so should be put into a threadpool.

<comment><USER50> A threadpool with futures support, I gather. Very nice.

<comment><AUTHOR> [https://github.com/alexcrichton/futures- rs/tree/master/futur...](https://github.com/alexcrichton/futures- rs/tree/master/futures-cpupool)

</comment></comment></comment></comment><comment><USER51> I'm genuinely interested in knowing what the problem is with an event loop using epoll and a threadpool for IO that blocks but epoll can't poll. I've used proprietary event loops at 2 giant companies, libuv with C, asio with cpp and nodes async, and the async IO was never the problem in terms of performance or complexity. What is the problem that's trying to be solved?

<comment><USER52> The problem being solved is better UX (in this case, developer experience). It _is_ an event loop using epoll and a threadpool, just in a more palatable (and composable) interface, without much overhead.

</comment></comment><comment><USER53> How does the zero-cost abstraction work?

Say we make a Future<Int> and then chain `.map(|x| x+1)` on a dynamic number of times (N). Presumably this requires storing at least N function pointers.

How can we store these N function pointers with zero cost? If it only takes one allocation, where does the N-1 future store its function pointers?

<comment><USER54> A "zero-cost abstraction" really means that abstraction doesn't impose a cost over the optimal implementation of the task it is abstracting. Some things—like chaining a dynamic number of (arbitrary) closures—fundamentally require some sort of dynamic allocation/construction, and so a zero-cost abstraction would be one that it only does that dynamic behaviour when necessary.

If you don't need the dynamic behaviour, the library is a zero-cost abstraction, by statically encoding all the pieces at the type level: like Rust's iterators, each combinator function returns a new future type that contains all information about its construction and operations. To add to this, a closure in Rust is not a function pointer, each one a specialized struct containing exactly the captures, and thus this all gets put into the type information too, and everything can be be inlined together into a single pipeline.

However, if you are dynamically constructing the future you'll have to opt-in to a uniform representation for the parts (i.e. erase the type information about the different constructions). This does indeed require allocating and storing pointers, but AFAICT this is required in any implementation, i.e. this library imposes no/little extra overhead over the optimal hand-written implementation.

Furthermore, the static and dynamic parts can work together: if you have parts that are statically known, these can be constructed as a single static type (with no function pointers or allocations), and then boxed up into a dynamic future as a whole unit, which can then also form part of other static chains, meaning allocations and dynamic calls only need to happen when absolutely necessary.

<comment><USER53> Thanks for your reply. I'm still trying to understand what the restrictions look like.

Say I sometimes have an outstanding asynchronous operation, i.e. validating some text in a document. I want to represent this by storing a Future representing this operation:

        struct Document {
            text:String,
            validating: Option<Future<Bool>>
        };

It seems like there's a problem: in order to have a struct of this type, we need to have the type of the Future, but the only way to have the type of the Future is to create the Future (and here we have None).

It this struct possible? How would I initialize it with {"foo", None}?

<comment><AUTHOR> One solution is to put the Future into a Box.

<comment><AUTHOR> And I can't believe I missed this, but the other way is to make the struct generic over a type implementing Future.

<comment><USER53> Does this work though? How do you create an instance of the struct without an instance of the Future?

<comment><AUTHOR>
        trait Future {}

        struct DynamicFoo {
            text:String,
            validating: Option<Box<Future>>
        }

        struct StaticFoo<T: Future> {
            text:String,
            validating: Option<T>
        }

These are the two options. In the first, Box<Future> stores a "trait object", that is, a tuple of (data ptr, vtable ptr). In the second, there will be a struct for each type used to generate a StaticFoo.

        impl Future for i32 {}
        impl Future for f64 {}

        fn main() {
            let d = DynamicFoo {
                text: String::from("foo"),
                validating: Some(Box::new(5) as Box<Future>),
            };

            let s1 = StaticFoo {
                text: String::from("foo"),
                validating: Some(5),
            };

            let s2 = StaticFoo {
                text: String::from("foo"),
                validating: Some(5.0),
            };
        }

Works just fine.

</comment></comment></comment></comment></comment></comment></comment><comment><USER55> What would a rough sketch of async/await syntax sugar look like implemented with Rust macros?

<comment><AUTHOR> One vision of that [https://github.com/erickt/stateful/blob/master/examples/gene...](https://github.com/erickt/stateful/blob/master/examples/generator.rs)

</comment></comment><comment><USER56> So will this become the official part of the language / standard library?

<comment><AUTHOR> That, if it happens, is still fairly far in the future. This isn't the only work on this space in Rust (see [https://medium.com/@carllerche/announcing- tokio-df6bb4ddb34](https://medium.com/@carllerche/announcing-tokio- df6bb4ddb34) for example, which also uses this library), so it's still in an early phase. Once everyone has actually used things and found it satisfactory, then such things can be discussed.

</comment></comment><comment><USER57> What's the meaning of "zero cost future" in this context? I googled the phrase and got a bunch of irrelevant material.

<comment><AUTHOR>
      > C++ implementations obey the zero-overhead principle: What
      > you don't use, you don't pay for [Stroustrup, 1994]. And
      > further: What you do use, you couldn't hand code any better.
      >
      > – Stroustrup

So, in this context, the idea is that if you hand-rolled your own state machine, you should see no difference than using this library. And, we measured: the overhead in a benchmark comparing the two was 0.3%, that's three tenths of one percent.

<comment><USER58> The other concept that GP needs to understand the title is [https://en.wikipedia.org/wiki/Futures_and_promises](https://en.wikipedia.org/wiki/Futures_and_promises)

So the title/TFA is about "futures" (in the sense of my link) that have "zero cost" (in the sense of steveklabnik's comment).

</comment></comment></comment><comment><USER59> select is an odd term choice for what is essentially a race condition. What's the thought behind the naming of that method?

<comment><AUTHOR> [http://man7.org/linux/man- pages/man2/select.2.html](http://man7.org/linux/man-pages/man2/select.2.html)

</comment></comment><comment><USER60> This is awesome, but I have an off-topic rust question:

Why can't we have some syntactic sugar to get rid of .unwrap()?

<comment><USER61> Instead of looking for "syntactic sugar to get rid of unwrap()" I think the better option would be to prove that the Option isn't None. In my experience in other functional languages with Options, if you can't prove it, you're usually making a mistake somewhere upstream in your code.

</comment></comment><comment><USER62> I guess it's cool that Rust is getting zero-cost futures, but they have a long way to go to catch up to C++'s negative-overhead coroutines!

</comment><comment><USER63> I suppose you could say, this way of programming is _the future._

<comment><USER64> _get off my lawn_ this isn't reddit _grumble grumble_ doesn't add to the conversation _blah blah_</comment></comment>


Is Sitting a Lethal Activity? - <AUTHOR> http://www.nytimes.com/2011/04/17/magazine/mag-17sitting-t.html <comment><USER2> I really wish the NYT would stop attaching words like "lethal" and "toxic" to activities which slightly increase one's risk of illness or injury.

(This is the second such article I've seen in the past few days: [http://www.nytimes.com/2011/04/17/magazine/mag-17Sugar-t.htm...](http://www.nytimes.com/2011/04/17/magazine/mag-17Sugar-t.html?pagewanted=all) )

<comment><AUTHOR> It has been reading a bit sensationalist. The brain cancer stuff too.

They are doing a nice job of though hyperlinking evidence to back up the claims. Where else can I find similar work? Can I trust them more or less?

</comment></comment><comment><USER3> I have a desk that Ikea used to make called the "Jerker" (I have no idea what that translates to). The desk surface can be configured to be at any height allowed by the array of holes for the bolts along the two vertical sidebars. I had it configured as a standing station for several months while I was in a less active part of my life. When my life changed and I became more active, I configured it as a sitting desk.

There used to be a website where people would share pictures of their Jerker setups. I wonder why, with such a loyal fan-base, they stopped making it.

 _Edit: the Wikipedia entry for Ikea offers this explanation: "Because IKEA is a worldwide company working in several countries with several different languages, sometimes the Nordic naming leads to problems where the word means something completely different to the product. While exotic-sounding names draw attention, e.g., in anglophone countries, a number of them call for a snicker. Notable examples include "Jerker" desk, "Fukta" plant spray and "Fartfull" workbench. Also, the most recent new product, Lyckhem (meaning bliss)."_

Coincidentally, I got a relevant web-advertisiment this morning. I like the idea of being able to easily reconfigure the desk more often:

<http://www.geekdesk.com/>

</comment><comment><USER4> It may be, but people are going to continue sitting.

Maybe instead of telling people how dangerous things are (knowing full well that they will still do them) medical science should start working on a cure.

A pill with the same effects as an hour of exercise will do more to save lives than penicillin.

</comment><comment><USER5> Between this _sitting-is-death_ and the _sugar-is-death_ piece, there should be tens of millions of dead office workers who died before 60 in the past few decades.

Since social security is in trouble, I'd venture a guess there are plenty of people making it to 65?

Instead of torturing lab rats they should give us quantitative factual numbers about how many office workers are now dead from sugar and sitting.

I'm not saying these studies are wrong, I am saying someone is either exaggerating or not providing what should be easy real-world examples.

<comment><AUTHOR> Causation doesn't reveal her bosom willingly.

Answer why Japan lives two years long on average, than the US and the UK, and you could help a lot of people:

[http://www.google.com/publicdata?ds=wb- wdi&met=sp_dyn_le...](http://www.google.com/publicdata?ds=wb- wdi&met=sp_dyn_le00_in&idim=country:USA&dl=en&hl=en&q=life+expectancy+chart#met=sp_dyn_le00_in&idim=country:USA:GBR:ARE:JPN:CHN:IND:BRA)

</comment></comment><comment><USER6> Living is a lethal activity.. always ends in death</comment>


Becoming an exploit developer/security researcher help? - <AUTHOR> Hello everyone. I want to become one of those guys like Spender (grsecurity.net) that can easily find bugs in code, and show people like Linus Torvalds how insecure their things are. To be able to do this, I realize that i have to learn C and assembly, (not really sure about shell scripting or python/perl) but could someone recommend a _way_ to do this, books i need, just steps required in order for me to get to where i am going??
I know a lot of people will probably mock me but thats ok, i can take being made fun of if i will get a few good answers
btw i am on osx 10.5 (leopard) and most of the work i want to learn will be on that but i can install a vm if need be.
osx is a unix so it might be nicer to do since linux has apparently (via google) been explored so much osx is a nice new area that might be good enough for me i think
thanks everyone <comment><USER2> The first step is to learn how to do things for yourself. It's an essential requirement for discovery, which is what bug finding is. I hate to be trite, but it's true and that's why no one's voting you up.

</comment><comment><USER3> <http://ompldr.org/vNWloOQ></comment>


Gravity suit mimics Earth's pull for astronauts - <AUTHOR> http://www.bbc.co.uk/news/science-environment-11686304

<comment><USER2> Why not make the suit require even more energy to move around rather than just trying to replicate the work required to combat Earth's gravity? That would save them some exercise time.</comment>


Skype Outage – I run a supernode and here's what I think happened - <AUTHOR> http://blogs.voxeo.com/voxeotalks/2010/12/23/skype-outage-i-run-a-supernode-and-heres-what-i-think-happened/ <comment><USER2> Maybe this will cause Skype to take Linux and Mac more seriously. If you've used the Windows version, you know almost all development has occurred there, whereas the Mac and Linux clients have looked the same for the last four or five years. The Linux client did have a major version bump about a year ago iirc, and that brought some needed features, but it was mostly the same.

Also, hopefully this will teach Skype to do more with open-source. I really hope they open the client up. This bug may have been caught, and things would definitely have turned out differently if Skype ran freely on other platforms. Maybe someone could even factor out a "Skype server" instead of an exclusive policy of client supernodes. Even serious torrenters rent a server somewhere to host their torrents -- P2P doesn't have to be strictly consumer-level connection, and really shouldn't be.

<comment><USER3> Or, maybe this will cause users to take Jabber/XMPP more seriously and stop using proprietary technology for corporate IM.

I've worked at places where management is completely gaga over Skype and would push me to support it despite the fact that I had no ability to block spam, troubleshoot messaging problems or integrate our IM system into existing Asterisk, SSO, monitoring and collaboration solutions.

IMHO, Openfire is far more flexible, extensible, secure, reliable and most importantly - manageable as a service.

<http://www.igniterealtime.org/projects/openfire/>

disclaimer: I do not work for Ignite and have no vested interest in their business.

<comment><USER2> When an XMPP solution that's as easy to use as Skype comes out I'll be all for this. As it stands, even Google's video chat is much more difficult for end- users to install and use.

I used to use Ekiga for video chat and it was much shoddier than Skype, constantly dropping calls, refusing to release the audio or video device so that we couldn't call back, and other serious bugs. Skype "just works".

I really hope that someone comes up with a decent free software competitor, but it doesn't really exist at this point. The fastest way to solve this problem would be for Skype to become free software.

</comment></comment></comment><comment><USER4> Just checked my Windows event logs... as I also run a supernode and yup, one Application Error at 2010-12-22 15:51:10 from Skype.

Tech data shows version 5:

        Faulting application name: Skype.exe, version: 22.214.171.124, time stamp: 0x4cb31516
        Faulting module name: KERNELBASE.dll, version: 6.1.7600.16385, time stamp: 0x4a5bdbdf
        Exception code: 0xe0fafafa
        Fault offset: 0x0000b727
        Faulting process id: 0xd20
        Faulting application start time: 0x01cba1ea3b70b5ef
        Faulting application path: C:\Program Files (x86)\Skype\Phone\Skype.exe
        Faulting module path: C:\Windows\syswow64\KERNELBASE.dll
        Report Id: 4b2781d7-0de3-11e0-bbdc-005056c00008

I didn't upgrade voluntarily.

</comment><comment><USER5> Nice to see the CEO of a large company that knows how to walk the talk. I've also seen him post on HN. Does Steve Ballmer know the difference between a micro and monolithic kernel?

<comment><USER6> I was thinking the same thing. What other big company CEO's would even think of investigating why their Skype was more jittery in some hotels, let alone actually know how to do it.

<comment><USER7> probably busy running big companies

<comment><USER8> CEOs have no free time? They don't eat, sleep, or rest. They will not stop until the stock prices surge?

<comment><USER9> Wow, Ballmer must be pretty tired by now

</comment></comment></comment></comment></comment><comment><USER10> tl,dr; a forced software upgrade kept crashing all skype windows clients, taking down supernodes as well.

</comment><comment><USER11> Well, the big question is not if Skype went down becase lots of supernodes crashed. The question is _why_ they crashed, and if the trigger was external. There are messages on p2p-hackers list arguing just that:

      I have evidence that suggests this is being done
      by a ddos attack on the supernodes' object list
      cmd parameter.

</comment><comment><USER12> I had the same issue: Shortly before Skype went completely offline, my Windows Skype client suddenly started crashing every few minutes. The Mac client didn't have that problem.

Some bug in the new version is probably the likeliest explanation, but if someone deliberately attacked the Skype network, it would probably look similar, right?

<comment><USER13> You're not going to get full client crashes without a major bug. Some amount of attack is possible but I've been getting crashes for a couple weeks now and skype should have fixed things.

</comment></comment><comment><USER14> Nice write up, pretty interesting that Skype has not come out and said anything about WTF happened. Also ironic that their security blog's last posting it titled "The importance of updating" :) ( <http://blogs.skype.com/garage/> )

-John

</comment><comment><USER15> A wonderful reason to have pre-release testers for _every_ piece of software that communicates with other software, any time you plan to _push_ updates. With pulls, your early updaters are your test beds; with push, you have to _create_ your testing groups.

<comment><USER16> The problem with a big adaptive p2p network is that it is close to impossible to test under real production loads.

<comment><USER17> My guess is that the average supernode is on something like 6M/1.5M DSL. This is easy to simulate.

Now, if most of your nodes are on machines directly connected to 10GbE, then that's a problem. But most Skype nodes aren't. (I do imagine there are a few connected to 100M+ connections. But only a few.)

<comment><USER18> I imagine quite a lot in countries like Sweden. I had 100Mbit in my apartment for cheaper than my shitty comcast which barely breaks 5.

</comment></comment></comment></comment><comment><USER19> [http://arstechnica.com/software/news/2010/12/skype- brought-d...](http://arstechnica.com/software/news/2010/12/skype-brought-down- by-double-whammy-of-overloaded-servers-client-bugs.ars)

</comment><comment><USER20> Are Skype relying on third parties with no contractual relationship for an essential component of the network?

<comment><USER21> Yup. But they do mention it when installing. You can also disable it.

If you are not firewalled or NATd you are a supernode automatically - on the plus side you get better sound quality for voice calls.

Normally the bandwidth used is pretty low since you are mostly forwarding text messages.

When they said mega-supernodes they meant machines controlled by them that do nothing else, and are on high bandwidth connections. (My bet is lots of amazon instances.)

<comment><USER22> I've looked around for the ability to turn off supernode on Linux and Mac clients for a while but have come up empty. Do you know which section of the preferences/settings it's supposed to be in?

<comment><USER23> I don't think there's a UI element for it, but you have to add a registry key, see here:

[https://www.cs.uwaterloo.ca/twiki/view/CF/SkypeConfiguration...](https://www.cs.uwaterloo.ca/twiki/view/CF/SkypeConfiguration#Disable_Supernode)

<comment><USER8> How pray-tell, does a Windows registry entry disable super node functionality on Linux and/or MacOSX?

<comment><USER9> Well barring effective virtualization, booting into Windows disables supernode functionality on Windows and MacOSX in the same sense that a brownout does

</comment></comment></comment></comment></comment></comment><comment><USER24> I, too, run a Skype supernode - completely unvoluntarily, barring the possibility of something hidden behind an asterisk in the EULA. Does anyone know if there is a way to "opt out" of this? Everytime I start skype up it makes a million connections to everywhere and without even having a call on the line my measly 2.5 mbps upstream is permanently choked.

<comment><USER25> [https://www.cs.uwaterloo.ca/twiki/view/CF/SkypeConfiguration...](https://www.cs.uwaterloo.ca/twiki/view/CF/SkypeConfiguration#Disable_Supernode)

Found this link lower down :-)

<comment><USER24> Sorry, I should've mentioned that I'm an OS X user :)

With that said, I've had a look-see in Skype's .plist and its various Application Support files, but oddly there's nothing "grep'able" to be found anywhere...

</comment></comment></comment><comment><USER26> I seriously doubt the Skype supernodes actually run in regular clients on people's desktops. I'm pretty sure these are special servers placed strategically on the net.

If you see thousands of connections on your Skype at home then there probably is some weird P2P problem going on.

<comment><USER27> I'm sorry to be the one who has to point this out, but you have no idea what you are talking about.

<http://www.skype.com/intl/en-us/security/universities/></comment></comment>


What Is Passive Building: Passive House Principles - <AUTHOR> http://www.phius.org/what-is-passive-building/passive-house-principles <comment><USER2> tl;dr: You'll live in an air tight plastic burger box and like it.</comment>


Computational complexity may make accurate derivative pricing impossible - <AUTHOR> http://www.cs.princeton.edu/~rongge/derivative.pdf

<comment><AUTHOR> A less technical explanation of why certain kinds of widely used derivative pricing methods are NP-complete: [http://www.freedom-to- tinker.com/blog/appel/intractability-f...](http://www.freedom-to- tinker.com/blog/appel/intractability-financial-derivatives)</comment>


The Case of the Bumbling Spy: A Watchdog Group Gets Him on Camera - <AUTHOR> https://www.nytimes.com/2019/01/28/world/black-cube-nso-citizen-lab-intelligence.html <comment><USER2> > They also seemed to be trying to provoke the Citizen Lab researchers into > making anti-Semitic or otherwise offensive comments [...] He held the cards > in one hand, while in the other he held and awkwardly pointed a pen that > appeared to contain a video recorder, Mr. Scott-Railton said.

Ah interesting. The tried and true tactics, bait them into making offensive comments, then blackmail them to withdraw reports with the threat of exposure.

> In a phone conversation, he had told Mr. Scott-Railton that he had a son > about his age. When they met, he said the child was a daughter.

Someone needs to take more "basic tradecraft" courses I think.

All in all I think Citizen Lab did pretty good job identifying and thwarting this attack. It would a bit harder, as they found out, for whoever sent the fake "Michel Lambert" to keep sending him, he was already recognized at least once. It's back to a desk job for them I suppose.

</comment><comment><USER3> I wonder if the Israeli government will end up creating their own enemies? Surely at some point some of these spies will defect (money, blackmail, etc) and this creates a blind spot for them?

</comment><comment><USER4> Since NSO so emphatically denies any connection to this activity, who else would be willing to pay money to investigate Citizen Lab's work into exposing NSO?

If there are no plausible alternative candidates...

</comment><comment><USER5> Previous discussion: [https://news.ycombinator.com/item?id=19006477](https://news.ycombinator.com/item?id=19006477)

<comment><USER6> This is not a dupe, it is an additional, new article.

<comment><USER5> Sure, about the same incident previously discussed on the HN front page. I'm not saying it's a dupe, I'm saying we've previously discussed the Michel Lambert situation and providing a link to that commentary.</comment></comment></comment>


The Moment of Silence Link Just Shown on HN - <AUTHOR> http://www.causes.com/causes/807161-stand-with-sandy-hook/actions/1716727

<comment><USER2> It wasn't "gun violence", it was "disturbed individual violence perpetrated with a gun". A gun didn't suddenly become sentient and start mowing people down.

<comment><USER3> <http://www.youtube.com/watch?v=baPgr_tw79Q>

I think I just realized how old this argument has gotten. "It wasn't the gun's fault, it's just a tool, it was the person's fault".

OK, fine.

How did that person get a gun?

If gun advocates like yourself are actually concerned about protecting their gun-owning rights, they should be organizing and policing themselves to make sure that things like that video above _can't happen_.

Putting an advanced weapon in the hands of a crazy or mentally unbalanced person does far more to harm your right to own firearms than any liberal or politician. Every time some gun dealer at a public event displays a weapon and sells that weapon to an individual without any kind of background check or waiting period or any other due process, the odds of _yet another_ school shooting in the U.S. goes up. Eventually, the odds get to be > 1, there's another huge example of senseless stupid violence in the news, and once again it becomes politically beneficial to limit your right to own firearms.

So, you want to have guns? Fine, then keep them out of the hands of crazy people.

If you can't do that, then I can't defend your right to them.

<comment><USER2> I'm not a "gun advocate", I'm just against the politicizing of a national tragedy. I'm against this one in particular because this tired rhetoric of blaming the gun distracts from the core issue, and that is that there was a person mentally disturbed enough to want to kill a bunch of children, and guns happened to be how he carried it out.

How did that person get a gun? From his mother, whom I understand acquired her weapons legally and after numerous background checks. Nobody sold this guy his guns, Congress _never had the opportunity_ to come between him and his weapon of choice.

If we're going to talk about further legislating guns to prevent this specific type of crime, we've only got one single option available: you go out to every gun owner in America and tell them to give up their guns, so that there is no chance at all that some crazy person can steal it and go on a rampage.

And that is the reason this argument is so old, and why it feels so tired. Because you can blame guns and gun owners all you like, but that will never solve the problem.

</comment></comment></comment><comment><USER4> For me this was really disturbing. I was digging in an old thread, because I often use HN for research on specific topics, and suddenly it popped in my browser and I had no idea what was going on. It did not feel like _me_ controlling my laptop at that moment ...</comment>


NoSQL: Comparing the Different Solutions - <AUTHOR> http://www.readwriteweb.com/cloud/2010/11/nosql-comparison.php

<comment><USER2> Why link to readwriteweb instead of the real comparison?

[http://perfcap.blogspot.com/2010/10/comparing-nosql- availabi...](http://perfcap.blogspot.com/2010/10/comparing-nosql-availability- models.html)

</comment><comment><USER3> I applaud this effort to compare NoSQL databases, but I'd like to inject a word of caution. The various NoSQL databases are _amazingly_ different, and it's hard to compare them properly using a single scenario.

Adrian Cockcroft's scenario, for example, favors a simple key/value store with good partition tolerance. (It also contains some design decisions that fit together poorly: He asks about partition tolerance, but his application is designed to write to one availability zone and immediately read consistent data from another availability zone. He should either combine these two operations into a single operation, give up partition tolerance, or look for better load-balancing options.)

I could easily describe other scenarios which favored S3, MongoDB, CouchDB, or offline Hadoop/Hive queries of S3 buckets. And that's just the systems I've used. We need much better NoSQL comparisons, but we won't get there without investigating a wide mix of use cases. Otherwise, we're going to wind up with the equivalent of a comparison of Clojure, Haskell, Mozart and F# written by a "Blub" programmer: All the genuinely interesting questions will fall outside the basis of comparison.

<comment><USER4> > I could easily describe other scenarios which favored S3, MongoDB, CouchDB, > or offline Hadoop/Hive queries of S3 buckets. And that's just the systems > I've used. We need much better NoSQL comparisons, ...

Please do - that would be VERY useful.

kb

</comment></comment><comment><USER5> The title is a bit misleading. No comparison has been made yet, in actuality the author is requesting developers from each NoSQL solution to describe how their solution handles some problems he lists. I would love to see the resulting article though once more have replied.

<comment><USER6> The Cassandra and MongoDB articles are already available.</comment></comment>


Blog Hosting: The Ultimate Guide - <AUTHOR> http://simpleprogrammer.com/2015/04/27/blog-hosting-the-ultimate-guide/

<comment><USER2> You can actually juice out free Wordpress hosting from cloud platforms like Heroku, Cloud Control or perhaps even Bluemix. Definitely wouldn't suit beginners, but worth a look.</comment>


Radars versus Dashboards: A Life/Death Choice - <AUTHOR> http://blog.trailmeme.com/2011/01/radars-versus-dashboards-a-lifedeath-choice/ <comment><USER2> This was painful for me to read, because the author clearly doesn't understand the roles played by different types of avionics in building situational awareness (SA), and the whole analogy he tried to build on top of this misunderstanding was therefore deeply flawed. I think he may be on to something with his dichotomy between "conflation" and "discrimination," but it doesn't really apply to the example case he used to introduce it (avionics), which makes the whole thing give me a headache every time I try to think about it. I wish he had just left out the whole cockpit analogy and this post probably would have been much better.

The fact is that U.S., former-soviet (FSU), and other cockpit designs all include elements that are "dashboards" (instruments, really) and "radars," (sensors, more broadly). The difference betweent he "wall of dials" and the glass cockpit is simply a difference in how this combination of elements is displayed. The glass cockpit, if executed properly, can be far better than even the best steam-gauge cockpit, and either style can be executed so poorly as to become a net drain on SA. The Indian pilot in the story preferred the steam gauges for one reason more than any other: _it is what he was used to_. It may also be the case that the only glass cockpit he was ever exposed to happened to be an especially bad design.

Glass cockpits are actually _better_ for "management by exception:" you don't have to ignore irrelevant or unimportant information because it _simply is not displayed_. All of the same gauges exist virtually, with a computer monitoring them constantly in the background. The things you need to see all of the time are shown all of the time, except with more real estate dedicated to them. The things you don't need to see most of the time only pop up when they leave the range of numbers which would have been green on an old-fashioned dial. Not only does this free up more space for the stuff you need to look at frequently, it also is more likely to catch your attention when you _do_ need to look at it.</comment>


6 Reasons Why JavaScript's Async/Await Blows Promises Away - <AUTHOR> https://hackernoon.com/6-reasons-why-javascripts-async-await-blows-promises-away-tutorial-c7ec10518dd9 <comment><USER2> I am a bit saddened to see so many languages adding sugar for promises (or the equivalent abstraction in that language). Promises are just a monad and async/await is a poor mans version of Haskell's do notation. Exactly the same amount of syntactic overhead (actually a tad bit more, since async/await are longer than do/<-), but significantly less expressive. Sigh.

<comment><USER3> I agree, the sugar is pouring in from every crack and hole, how many ways do we need of doing the same thing. Sometimes the sugar is a genuine productivity booster and you don't want to be a Luddite but recently, in JS it seems it's coming faster and faster and that's why it is so important, whether you are working individually or as a team, to choose a subset of language features you will use to keep sane. No one likely agrees with me but I always found callbacks are more readable for my brain than code rewritten to use promises, so may be this new way is something I might actually like in use! Select your flavor of sugar.</comment></comment>


Analyzing the XKCD Passphrase Comic - <AUTHOR> http://xato.net/passwords/analyzing-the-xkcd-comic/

<comment><USER2> 100 mutations seems high, and mutations make things harder to type. I'd rather add a 5th word (in fact, I do!) to get the complexity up where I want it.

I love the idea of including non-dictionary words, though. Numbers 0-999 gives you an easy thousand, any one of which should be easy to remember in a password. Proper nouns are also likely to be memorable, although remembering which spelling variation might be more trouble.

Also, if you speak a language other than English, that's another dictionary you can pull in (gaining a number of words dependant in part on your fluency...).

<comment><AUTHOR> "100 mutations seems high"

JTR has more than 180 rules, so I don't think 100 is an unfair number.

And yes, mutations do slow down typing which is why I most often use a random non-word that sounds like it would be a word (i.e., lickering, fauxd, glasster, etc.). Another is capitalizing in the middle of the word. And another is sticking a hyphen in the middle of the word. And another is adding a suffix like .com to your word.

Of course my favorite mutation is swapping the first letter of two of the words (i.e., cog drap).

But yeah you could also just add a fifth word and not even worry about mutations.</comment></comment>


Pg's "roots of lisp" Lisp eval function - adding defmacro - <AUTHOR> http://stackoverflow.com/questions/3683388/given-the-the-following-lisp-eval-function-what-is-required-to-add-defmacro

<comment><USER2> Wow - this is really about the fundamentals of LISP. I don't think pg had touched on this level before.</comment>


Bitfinex's banking problems solved: Snitches get stitches - <AUTHOR> https://davidgerard.co.uk/blockchain/2018/10/17/bitfinexs-banking-problems-solved-snitches-get-stitches/ <comment><USER2> If deposits go direct to withdrawers...

How does bitfinex know the transaction is completed?

How do they deal with one side claiming they never received the money, with the other side claiming they sent it?

This is exactly the issue local bitcoins has, and they need an entire reputation system to combat it, and still fraud rates are pretty high.

<comment><AUTHOR> That's why there's a bit of question as to this, 'cos it's a frankly nuts system when we're dealing with a minimum of $10k a transaction.

(OTOH, this is Bitcoin, so "but that's stupid!" doesn't mean it isn't done that way.)

But something like a deposit->withdrawal hookup might explain the multiple days' delay.</comment></comment>


Military oxygen device set to help children in developing countries - <AUTHOR> http://www.ploughshareinnovations.com/blog/military-oxygen-device-set-to-help-children-in-developing-countries

<comment><USER2> Another point to prove why military-funded research is quite necessary for technological advancement.</comment>


Python List Comprehensions Explained Visually - <AUTHOR> http://treyhunner.com/2015/12/python-list-comprehensions-now-in-color/ <comment><USER2> Maybe its me, but "Explained Visually" sounds more like it would have some 'visual' aspect to it besides a GIF of code (don't get me wrong, I think most code should be made an image to limit copy/pasting).

A website like VisuAlgo ([http://visualgo.net/](http://visualgo.net/)) is a great example of exampling something visually should mean

<comment><USER3> I agree. I was using "List Comprehensions: Now In Color" as a working title but I thought that was too nonsensical. It was actually the colorized code and not the gifs in particular that made me think "visual".

Sorry for the misleading title!

<comment><USER4> What's the font you're using in that gif? It looks good.

<comment><USER3> Inconsolata: [https://en.wikipedia.org/wiki/Inconsolata](https://en.wikipedia.org/wiki/Inconsolata)

<comment><USER4> Ah, cheers :)

</comment></comment></comment></comment></comment><comment><USER5> Coming from Ruby to Python...I could not believe how damn hard it was to comprehend the beautifully explained and simple code (just 21 lines!) in Peter Norvig's writeup of how to create a spelling corrector [1], purely because of the list comprehensions (and a few other visual differences, such as partial list notation)

For example:

         inserts = [a + c + b for a, b in splits for c in alphabet]

It's no different (conceptually, at least) than doing:

         inserts = []
         for a, b in splits:
            for c in alphabet:
                inserts.append(a + c + b)

Or in Ruby (OK, maybe this isn't perfect Ruby):

         inserts = splits.reduce([]) do |x, (a, b)|
             alphabet.each{|c| x << a + c + b }
             x
         end

But the list comprehension was so difficult for me to read that it might as well have been a brand new concept. But I found I got quickly over it after rewriting all of his comprehensions as for-loops, and then trying to convert them back into comprehensions. It was a good reminder that sometimes the barrier is just plain muscle/memory reflex (and probably much more so for beginning programmers...)

[1] [http://norvig.com/spell-correct.html](http://norvig.com/spell- correct.html)

<comment><USER6> I most frequently use List Comprehensions in a nested manor, whenever I do that I /ignore/ the normal python syntax and do this for readability.

        inserts = [ a + c + b
                    for a, b in splits
                    for c in alphabet ]

It has the effect of showing the nested loops stacked like they would be in code, but without intending.

<comment><USER7> For the projects I manage, I would have the dev rewrite that as a proper for loop.

That's way too confusing and hard to parse. List comprehensions are great for simple operations, but once you start nesting, you're just being a jerk to the next dev that has to read your code.

Code should be clear first, elegant second.

<comment><USER8> > Code should be clear first, elegant second.

I disagree: code should be correct before it is clear. And because it's so easy to mess up a for loop (for me at least), I choose list comprehensions where reasonable.

> you're just being a jerk to the next dev that has to read your code

That's not very charitable to either party. You're assuming that the motive of the author is to be a jerk, and you're assuming that the reader won't understand. If I had a dev on my team who couldn't read a list comprehension, I'd (a) wonder how they were hired, and then (b) teach them.

<comment><USER9> I've written a fair share of Haskell, and I still get tripped up by list comprehensions in python - even my own. It's not that it's impossible to understand, and when there's only one predicate, I think it's fine:

      for row in [[i*j for i in range(1, 8)] for j in some_list if j % 2 == 0]:
        some_op(row)

vs

      for j in some_list:
        if j % 2 == 0:
          row = [ i * j for i in range(1,8) ]
          some_op(row)

I would compare it to sentences and paragraphs. The former feels like a run-on sentence, while the latter is more obvious, cause it has one predicate per line. Also, list comprehensions are a bit like yoda-speak - it introduces the verb before the subject. You have to untangle the order of operations, rather than having the order read top-down and left-right.

Of course, I'm a rubyist, so I'd prefer:

      some_list.select { |j|   j % 2 == 0 }
               .map    { |j|   (1..8).map { |i| i * j } }
               .map    { |row| some_op(row) }

Though definitely need to do something about that second line.

Haskell list comprehensions are a bit easier to parse because they have symbolic delimiters, the fact that Haskell is naturally more terse, and because you can always check the type of the list

      [ [i*j | i <-[1..8]] | j <- [1..4], j % 2 == 0 ]

</comment></comment></comment></comment></comment><comment><USER10> Cool. As a math guy, I enjoy the similarity between comprehensions and set- builder notation:

[https://en.wikipedia.org/wiki/Set- builder_notation](https://en.wikipedia.org/wiki/Set-builder_notation)

<comment><USER11> In the opposite direction, having knowledge of Python's list comprehensions (and to some degree, an understanding of lazy evaluation) helped me a ton when set-builder notation was introduced in my first discrete math class.

</comment></comment><comment><USER12> Are Python List Comprehensions really hard for people? I feel like my memory of learning them was I saw one in production code I was modifying, intuited what it did, pulled open a REPL and played with it for a bit, and then moved on.

<comment><USER13> I agree. It really surprised me to see this getting so much traffic. I sometimes get hung up with nested comprehensions when I haven't been using them in a while, but other than that, they are quite mundane.

</comment></comment><comment><USER14> Is it just me or were there zero visualisations in this post?

<comment><USER15> It might not be what you were expecting, but there's an animation if you ctrl-f "Here's the transformation animated:"

<comment><USER14> :) I did see that one and indeed it wasn't what I was expecting.

</comment></comment></comment><comment><USER9> That translation is kind of why i don't like python list comprehensions though. It's often clearer as a for-loop or map/filter, cause the different predicates are broken out into separate lines, rather than lumped together in one long line.

<comment><USER16> It is not much easier to understand the explicit loop:

      doubled_odds = []
      for n in numbers:
           if n % 2 == 1:
               doubled_odds.append(n * 2)

compared to the list comprehension:

      doubled_odds = [n * 2 for n in numbers if n % 2 == 1]

Here's the explicit loop without newlines for comparison:

      for n in numbers: if n % 2 == 1: doubled_odds.append(n * 2)

Notice that the for-loop and the if-statement are in the same order in the loop and in the list comprehension. Only the value expression (n * 2) is out- of-order. OPs jumping around is often unnecessary in the gif.

The one line list comprehension is as much readable for people familiar with Python as the variant with the explicit loop.

It is not uncommon to use several list comprehensions in a function. If you rewrite them as explicit loops then it may increase the line count 4 times e.g., it can convert a simple 3-lines function that could be understood at a glance into 12 lines function that you have to read line by line to understand -- it is easy to keep track of 3 values on 3 lines but it is not clear whether it is true for 12 lines.

<comment><USER12> I think the problem here is that nested comprehensions and filter comprehensions are almost always too complex. If you are doing something that requires that, you should break it up and give the pieces names:

        odd_numbers = range(1, limit, 2)
        doubled_odd_numbers = [n * 2 for n in odd_numbers]

</comment></comment></comment><comment><USER17> Frankly, comprehensions are one of Python's worst features. Some properly implemented functional syntax would be both more readable and concise. Plus double-nested (I'm not sure what you'd call it) comrpehensions seem to be backwards. Rather than [ingredient for ingredient in recipe for recipe in cookbook] it's [ingredient for recipe in cookbook for ingredient in recipe] which makes it sound as if there's only going len(recipe) number of ingredients in your list. Plus it's so verbose and horizontal, half the time if you want descriptive names you're gonna write something so verbose it doesn't even make it that much more readable.

<comment><USER18> When reading code I find _comprehensions_ easier to comprehend than combinations of _map_ , _filter_ and _reduce_. That is considering that I played around with some Common-Lisp before getting into Python.

The parts in the comprehension

        [*wanted* *iteration(s?)* *condition?*]

compares well to

        *iterations(s?)*
            *condition?*
                *append wanted*

for normal looping behaviour without changing order of the iterations.

One of my favourite comprehensions is of the sort

        [item for item in items in for i in (0, 1)]

of course this can be the same as

        list(items) * 2

but the comprehension is more versatile

        [item for item in items in for i in (0, 1) if items[0] != 'a']

For most once the mental fog clears comprehensions become natural very quickly as they are not that far removed from normal loops.

</comment></comment><comment><USER19> > Sometimes a programming design pattern becomes common enough to warrant its > own special syntax. Python's list comprehensions are a prime example of such > a syntactic sugar.

I disagree with this thesis, although the post is otherwise a very nice guide to list comprehensions. Special syntax for specific types indicates a failure of the language to be generic enough.

List comprehensions anoint lists as a special _thing_. You don't get to play with list comprehensions, unless you're using the type that the language has decided to let you play with. If you decide to use a different type for some reason, you... can't.

map and filter can be part of a typeclass/interface/protocol (in Python these would just be informal), so you can use them on arbitrary types. If you want to switch your list type, it should just work.

I write a lot of Swift, and I'm constantly frustrated that optional chaining (?.) and exceptions are special _things_. I can't implement ?. for my type (say, a Result). Only the language creators can use it. Somewhat confusingly, ?? _is_ a thing I can implement.

In Python's case, I think that list comprehensions are necessary because the language's support for first-class closures is poor.

<comment><USER20> > List comprehensions anoint lists as a special thing. You don't get to play > with list comprehensions, unless you're using the type that the language has > decided to let you play with.

That's one reason why I like Scala's comprehensions; they have the conciseness of list comprehensions, but are more generic.

<comment><USER21> aka monads ;)

</comment></comment></comment><comment><USER22> One basic misstep here for me is a comparison of why list comprehensions are better than loops -- speed, ease of reading, etc. There are a lot of comparisons, but no discussion of WHY list comprehensions.

I work on a project that's written mostly in Python; on occasion, other coworkers have to edit/work with my code and they're not used to Python at all, so I find myself writing for loops instead of list comprehensions for clarity.

<comment><USER23> I find the utility of list comprehensions is maximized when I'm in a Python shell and dealing with indentation when writing a loop can become obnoxious to edit. I tend to have a Python shell tab open in my terminal throughout the day, and it's nice to bang out a quick loop in a one-liner.

When writing actual code, I tend to stick with regular for loops.

</comment></comment><comment><USER24> For those confused - the visual part is done in a GIF (i.e. [http://treyhunner.com/images/list-comprehension- condition.gi...](http://treyhunner.com/images/list-comprehension- condition.gif)).

I think animating the transformation from a normal loop to a list comprehension is a great way to show how the syntax translates between the two forms. Very awesome and comprehensive post.

<comment><USER25> Maybe I'm an FP snob but it would have been nice to mention the analogy to set builder notation (though lists aren't sets - maybe multisets or bags) to indicate the existence of higher abstraction; not just syntactic sugar on the for-loop.

However, from a practical point of view I think it is great introduction for procedural programmers to begin using list comprehensions.

<comment><USER20> > though lists aren't sets - maybe multisets or bags

arrays/lists (like maps/hashes, though with a more narrow restriction on the indices) are (or at least are isomorphic to) sets of pairs of (index, element).

They carry information about order which does not exist in multisets/bags.

</comment></comment></comment><comment><USER26> I've been reviewing Python this week and this was a great brush up. Thanks!

</comment><comment><USER27> nested comprehensions are kind of a nightmare to read, but basic list comprehensions seem incredibly straightforward to me. So much so that the explanation is more complex.

</comment><comment><USER28> Not sure how this is visual, neat post either way!

</comment><comment><USER29> When I started to learn Python, I was quite lost with this feature, but now I love using it.

</comment><comment><USER30> Is it really that hard to grasp?</comment>


The Edge Computing opportunity is not what you think - <AUTHOR> https://blog.cloudflare.com/cloudflare-workers-serverless-week/ <comment><USER2> WebAssembly is a great platform to build upon, but it's still very rough for most languages to build the necessary features for seamless interop with WASM. WASI is still not standardized.

</comment><comment><USER3> Working with Cloudflare workers has been such a treat, and now they're going to enable Python support! Such an awesome tool.</comment>


Apture acquired by Google - <AUTHOR> http://techcrunch.com/2011/11/10/google-buys-contextual-rich-news-browsing-startup-apture-to-beef-up-chrome/

<comment><USER2> "Never heard of it" ... "Apture Highlights" ... "oh".

I hate those things, I often highlight as I scroll to keep my focus on where I'm reading and every time I accidentally trigger these.

<comment><USER3> I am really saddened to read of this purchase.

Apture breaks the web in many ways. As you note it screws up and steals/hijacks mouse behavior. It also round trips to their servers any selection you make, making it a privacy leak. They make it hard to opt-out of their javascript menace, and it clutters up a page.

"a browser extension" if it were solely a browser extension I wouldn't mind it so much. The behavior that is most borrowsome is the stuff they license to folks like the NYTimes, the Apture Hotspots.

<comment><USER4> One of the reasons I finally gave in to Adblock+ was as a convenient means of implementing a filter against the JS that perpetrates that NYT annoyance.

My concern is as to if/when this gets rolled into core Chrome. As long as the functionality is loading over the wire, I'll remain hopeful that I can block it and that sites will remain usable in that state. If it's an extention or plug in that I can disable, I can live with that as well, I guess. If it's baked in... Well, I guess it won't be baked into Firefox et al.

I don't mind useful, opt-in lookup features. For example, the Babelfish Firefox extension that does select and click translation. But such features continue to be killed off. I suspect initiatives like this purchase are instead intended more to "game" me into further staying/interacting with "blessed" sites and their subsidiary or partner content.

It's their content, I guess. But I'll probably "opt out".

</comment></comment></comment><comment><USER5> Congrats to the Apture team. But to be honest, I never found sites having Apture useful. It kind of distracted me in many ways.

I hate anything that puts underlines / icons into the texts I'm reading. Not to mention, it is borderline spamvertisement and even virus-like.

</comment><comment><USER6> I have very different reactions to talent, product, user and technology acquisitions.

I wish someone would track which kind of acquisition is made and let me track it by acquiring company. From my limited vantage point, Google seems to make mostly talent acquisitions and that generally makes me sad.

Anyone know which this was?

</comment><comment><USER7> Apture helped enhance web sites with supporting media functionality. For smaller sites that did not have time or talent to get widgets and such installed, Apture offered a simple way to do media feature upgrades with a very small investment of time. I will miss those features.

</comment><comment><USER8> How interesting. I worked on something very similar back in 2005 (but never launched): RocketMenu

[http://www.yarone.com/2011/02/rocketmenu-unobtrusive- search-...](http://www.yarone.com/2011/02/rocketmenu-unobtrusive-search- and.html)

</comment><comment><USER9> What's a typical acquisition compensation package for a non-founding engineer being acquired by Facebook, Google, eBay,etc? Anyone know?

</comment><comment><USER10> They removed Apture from the Chrome Store just now

</comment><comment><USER11> I have never heard of Apture up until now, but I love the idea. Congrats to both Tristan and Can, keep up the great work.

</comment><comment><USER12> I love contextual search on tablets, and really hate it on the browser.

</comment><comment><USER13> Google takes one step closer to becoming skynet and GLaDOS.

Apture Science, we do what we can because we must.

<comment><USER14> Larry Johnson and Cave Brin

</comment></comment><comment><USER15> Love it. Can't wait to see how they will incooperate it into Chrome

</comment><comment><USER14> Someone else also read APERTURE?

<comment><USER16> Google acquired Aperture Science?!?!

<comment><USER17> I'm guessing they wanted to, but this was the closest they could get</comment></comment></comment>