Intrusive lists in Doom 3 - <AUTHOR> http://gpfault.net/posts/intrusive-lists-doom3.txt.html <comment><USER3> They moved away from intrusive lists to vectors in the BFG edition for performance reasons. More about that here: [http://fabiensanglard.net/doom3_documentation/DOOM-3-BFG- Tec...](http://fabiensanglard.net/doom3_documentation/DOOM-3-BFG-Technical- Note.pdf)

<comment><USER4> Very interesting post - it seems that, like many things in computing, intrusive lists make a lot of sense _until_ you start thinking about cache locality. Perhaps on machines of the time, branch prediction was slow enough that conditional code would be a limiting factor... but on modern architectures, it's all about that cache, 'bout that cache, no thrashing.

<comment><USER5> You can strike the intrusive. Your comment is valid for all kinds of linked lists.

<comment><USER6> Hell it's valid for everything period. Even vectors! Splitting struct a into hot/cold chunks is pretty common. Minimize the sise of data being iterated on to maximize cache usage. Can result in big wins.

<comment><USER4> Huh, hadn't heard of hot/cold struct splitting before. This paper seems to indicate it has a nontrivial effect even in Java code: [http://research.microsoft.com/en- us/um/people/trishulc/paper...](http://research.microsoft.com/en- us/um/people/trishulc/papers/definition_distr.pdf)

</comment></comment></comment></comment></comment><comment><USER7> Instead of allocating each element of the list individually you can do this one better.

Allocate an array of elements. The prev and next don't point to memory but to indexes in the array.

How do you handle empty elements? Have a special flag that means "this element is blank" and use prev and next to point to the empty elements in the list! (So you are essentially storing two interleaved lists as the same time.)

Also store the first empty (and first full) element separately of course.

You will need to initialize the memory before you use it, filling all the prev and next pointers for the empty elements.

When you "allocate" an element you just update the prev and next pointers of all 5 elements, as needed.

If you think about it - malloc also needs to store a list of unused memory areas, so despite being a bit more complex you can speed things up quite a bit by doing this.

An optimization: Since filling all the empties can allocate memory, even if you might never use it, have an optimization that if the next pointer is 0 (or -1) that implicitly means the next array index, but an uninitialized index.

By doing this you can allocate a 1GB array without worry, and allow the OS to only fault in pages as needed (i.e. there is no need to ever resize the array).

<comment><USER8> That's great until you start caring about address space use or total process commit charge. All you've done is build a dubious special-purpose heap allocator.

</comment></comment><comment><USER9> This approach is more common in lower level software where each allocation is carefully managed. The Linux kernel uses them, for example:

[http://www.makelinux.net/ldd3/chp-11-sect-5](http://www.makelinux.net/ldd3/chp-11-sect-5)

as does the Windows kernel:

[http://msdn.microsoft.com/en- us/library/windows/hardware/ff5...](http://msdn.microsoft.com/en- us/library/windows/hardware/ff563802%28v=vs.85%29.aspx)

<comment><USER10> And BSD / OS X. Check out queue(3).

<comment><USER11> And on AmigaOS, pretty much everything in Exec (the OS kernel) was held together by doubly-linked-lists (of the intrusive kind). Coming from C I'd say that std::list is the 'strange kind' of lists, and intrusive lists are the 'common kind'. On the other hand I haven't been using lists for years, since growable arrays (like std::vector) are almost always faster.

</comment></comment></comment><comment><USER12> I am somewhat confused by this article since to me it sounds like the writer has wanted the reader to think intrusive lists are "trickier and weird", when they have been my bread and butter during all my C work... And I guess I am not alone.

<comment><USER13> They _are_ trickier though because you either have to create a new type for each list and all all accompanying functions to work on that type (the PointListNode in the article) or you can make it more generic by discarding type safety and using tricks like "container_of" (that's actually my preferred approach when writing C, at least it lets me use the linux kernel's list.h and I don't have to reinvent the wheel).

In C you don't have much of a choice to make a generic container but in C++ the templated std::list is much safer.

I'm a bit wary of that article to be honest, it's interesting but it oversells the intrusive list a bit IMO and I can definitely imagine inexperienced C++ coders deciding to use intrusive lists instead of the STL because of it. But at any rate that would be very premature optimization in my book. Even _if_ intrusive lists happen to be faster in a certain use case (and I'd like to see benchmarks before I believe it) I'd feel much more comfortable using std::list while developing/debugging.

<comment><USER14> The problem with std::list is that it's almost always worse than the alternatives. I would recommend using std::vector as the "default" container, and then use another container if the performance of std::vector is not good enough. Or, put another way, I think std::list is a premature optimization, because you are saying that insertions and deletions are frequent enough that you need them to be O(1), even at the expense of extra memory usage and tons of cache misses for simple traversal.

The std::vector container will outperform std::list surprisingly often. Just think of std::vector as your default container.

</comment></comment></comment><comment><USER15> I really prefer intrusive data structures, for the exact reasons outlined in the article.

I've myself written many implementations of intrusive structures. But now I'd like to present the greatest of them, a super-generic implementation of an intrusive AVL tree in C. See [1], and example [2].

\- It relies on a user-defined concept of a "link". In particular, this means the structure can be in an array and linked with array indices, and can be copied correctly with a single memcpy. Of course the user may still use pointers as links.

\- It supports implementation of user-defined associative operations (e.g. sum). This consists of operations: CAvl_AssocSum: Calculate the sum of all the nodes. CAvl_ExclusiveAssocPrefixSum: Calculate the sum of elements from the first one up to but not including node X. CAvl_FindLastExclusiveAssocPrefixSumLesserEqual: Find the first node where the sum of all preceding nodes is >=X. Of course these all have O(log n) time complexity.

\- The implementation is made generic by include files which rely on macros a lot. Other than the macro madness, I think this is better then C++ templates, due to easy #if, where in templates one would need to jump through hoops due to non-existence of static_if.

[1] [https://github.com/ambrop72/badvpn/tree/master/structure](https://github.com/ambrop72/badvpn/tree/master/structure) (see CAvl_* [2] [https://github.com/ambrop72/badvpn/blob/master/examples/cavl...](https://github.com/ambrop72/badvpn/blob/master/examples/cavl_test.c) (also see cavl_test_tree.h in the same dir defining some parameters for the structure)

<comment><USER13> > The implementation is made generic by include files which rely on macros a > lot. Other than the macro madness, I think this is better then C++ > templates, due to easy #if, where in templates one would need to jump > through hoops due to non-existence of static_if.

And either throw away type safety completely or end up with a huge macro soup (which is even worse to debug than template soup). I also don't understand your static_if remark.

I'm not saying intrusive lists are never useful but saying that intrusive lists are generally better than the STL is just terrible advice.

I'm not a huge fan of C++ but if there's something I miss dearly when writing C it's generic containers.

<comment><USER15> Well I don't want to get into a flame-war regarding intrusive/containing or C/C++, but I will say that type safety and intrusive/container are orthogonal things. One can have a type-unsafe container as well as a type-safe (generic!) intrusive structure. See my code for proof :) I think we'll agree that it's not too complicated and easy to use.

Implementation: [https://github.com/ambrop72/aprinter/blob/master/aprinter/st...](https://github.com/ambrop72/aprinter/blob/master/aprinter/structure/DoubleEndedList.h)

Usage: [https://github.com/ambrop72/aprinter/blob/master/aprinter/sy...](https://github.com/ambrop72/aprinter/blob/master/aprinter/system/BusyEventLoop.h) (the BusyEventLoop keep track of QueuedEvent objects).

If you wonder what's the purpose of the Accessor, it's for when you can't use the simple member-pointer based interface due to C++'s eager evaluation, and can hack around it with forward declaration of a custom Accessor class, followed by the complete definition later.

I think Boost intrusive structures are type safe too, but I'm not 100% sure. But Boost is another matter altogether.

</comment></comment></comment><comment><USER6> I like intrusive lists. Another consideration is that a std::vector<FooPtr> is as fast to iterate over as intrusive list of Foos. The memory access pattern is the same. (Note: typed FooPtr instead of Foo Asterisk because the single asterisk italicized everything)

Another perk is that an element can remove itself from the list without actually having to know anything about the list. This makes a lot of shutdown code super easy and clean.

I do like intrusive lists to be inherited I think. That makes the interface for adding or removing objects from lists super clean. It also eliminates the extra obj pointer since the obj root address is known at compile time. Storing one object in multiple intrusive lists gets a little hairy and you need a tag parameter but that's a relatively rare need in my experience so it's fine.

One thing I _don 't_ like is lack of good debugging info in visual studio. For c++ at least their customization tools are seemingly nOT powerful enough to handle intrusive lists well. Not being able to clearly see all objects in the watch window is wildly frustrating. Things that make code harder to debug make me very, very cranky.

<comment><USER16> > Another consideration is that a std::vector<FooPtr> is as fast to iterate > over as intrusive list of Foos.

A "normal" linked-list of strings or vectors always makes me sad -- the reasoning goes,

\- The list elements need to live on the heap because they might live forever, \- The string object has to be fixed-size because objects are always (at least officially) fixed-size. \- The string storage has to live on the heap, because it can be arbitrarily large.

So printing out a linked-list of strings gets two pointer dereferences per string instead of one. I guess that's the cost of abstraction, though.

(This actually fits into the C++ zero-cost-abstraction narrative pretty well, though:

\- Linked-lists are not zero cost, \- C++ programmers don't like non-zero-cost abstractions, \- C++ programmers don't like linked lists.

:)

<comment><USER17> > _C++ programmers don 't like non-zero-cost abstractions_

I think you're confusing zero cost with zero overhead.

What C++ programmers like is that the compiler does not add overhead, for example calling `a[10]` will read the word at address `a + 10` and return that. No bounds checks are inserted, no function call is done to lookup the element in some implementation defined sparse data structure, etc.

But C++ programmers are generally not opposed to using things that has a cost associated with it (basically everything has a cost) — for example `std::map` is quite popular and certainly not "zero cost", though it has its running time defined, and the implementation is actually using a self-balancing search tree rather than a hash table because we can give guarantees about the running time of the former, not the latter.

<comment><USER18> std::map is implemented using a tree and not a hash table because it's specified as an ordered data structure.

<comment><USER17> I am quite sure (based on interviews I've read) that Alexander Stepanov's primary concern was having a data structure with known (fixed) complexity rather than a sorted container. The latter is just a nice side-effect.

<comment><USER19> In the worst case scenario the time complexity of a hash table equals that of its bucket, which doesn't have to be a list. So the only advantage of the map over the unordered_map seems to be the ease of use, it only needs a comparer.

<comment><USER18> unordered_map pretty much has to have linked buckets due to its interface and complexity requirements.

</comment></comment></comment></comment></comment></comment></comment><comment><USER20> "There can be a lot of projectiles, they appear and disappear quickly, and every time that happens we incur the cost of allocating/deallocating memory multiple times."

I write a lot of shoot-em-ups (both 2d and 3d). I never do this. I preallocate all of the bullet entities on level initialization and never destruct them during runtime. If the number is large enough, I do sort them by status (alive or dead) and exit processing when I encounter a dead projectile, but usually that's unnecessary.

<comment> enjoy-your-stay Do you pre-allocate all bullets per shooter? Or have one big bucket that all shooters use?

I take it that having a finite number of bullets in the game at any time is obviously an acceptable trade-off.

</comment></comment><comment><USER21> Just like that old joke -

      There are two ways to wash dishes -
      after a meal or before it.

Same with data containers, there are two mindsets. Either data is primary and can be optionally organized into containers or containers are primary and they essentially own the data that's put in them. STL model is of latter and "intrusive" lists are of former. I leave it up to you to decide which one is an ass-backward approach in the name of type safety :)

<comment><USER22> It depends on whether you have generic code (e.g. map, filter, sort etc.) that you want to apply to the list, vs efficiency concerns, or whether your code needs to iterate through the list from an item.

There's no one true way here. I personally find having a library of operations like map and filter more productive for most code that I write, rather than needing to specialize them per structure.

</comment></comment><comment><USER23> I didn't think these "non-intrusive lists" were actually used much in real world programs. Even if allocations weren't a big issue performance-wise, the sheer amount of work required manage the allocations themselves would put me off using them.

<comment><USER24> Just use Boost.Intrusive: [http://www.boost.org/doc/libs/1_57_0/doc/html/intrusive.html](http://www.boost.org/doc/libs/1_57_0/doc/html/intrusive.html)

It makes the job much easier.

<comment><USER25> Except: boost.

</comment></comment></comment><comment><USER26> You can also make these using this structure:

        template <class T>
        struct node { node<T> *next, *prev; };

        template <class T>
        struct intrusive_list : private node<T> { };

Your type foo then subclasses the node<foo> type, and in some places intrusive_list<foo> has to statically cast them back down.

This means you don't need that messy "T *owner;" pointer anywhere.

The downside is that if you want your type to be a member of multiple intrusive lists, you have to make spurious parent classes and inherit from those.

<comment><USER16> One possible alternative is having your objects be members of "lists of slightly different type". For example,

        template <ptrdiff_t offset, class T>
        struct node {
            node<offset> *next, *prev;
            T& operator*() { return *(this-offset); }
        };

Maybe? It has been a while since I've written C++. I guess you have a big problem setting up your base class in a portable way, though, because the types of its member nodes depend on their offsets, and I don't think you can do that.

        class Foo {
            int data;
            node<offsetof(Foo, list1_elem), Foo> list1_elem;
            node<offsetof(Foo, list2_elem), Foo> list2_elem;
        };

:/

<comment><USER26> I'm not sure if that would work, because computing the offset of list1_elem could depend on the template instantiation of list1_elem because that could (ostensibly) affect the alignment. But you can drop the template altogether on the list element type and put the offsetof expression in the list type declaration. An example of that is linked by some other comment in this thread: [http://www.codeofhonor.com/blog/avoiding-game-crashes- relate...](http://www.codeofhonor.com/blog/avoiding-game-crashes-related-to- linked-lists)

The base class option is, I think, best only for the case where you have only one kind of list you want the object to be part of.

Also your pointer should be cast to char before you do offsetof arithmetic on it.

</comment></comment></comment><comment><USER27> I don't understand. Is it really well explained ?

Aren't ListNode just a way to index a list ? Isn't this Point2D just "marked" to be on a certain list ?

Why does he say "less dereferencing" ?

> Dereference the appropriate "next" pointer, get the next object from memory.

the prev/next pointer is in the ListNode, not in the object itself, so it's obviously a dereference...

I did not understand it very well... I can understand that a linked list will be faster thanks to branch prediction, but what does it have to do with an intrusive list ?

<comment><USER28> > the prev/next pointer is in the ListNode, not in the object itself, so it's > obviously a dereference...

The ListNode is embedded in the Point, so their storage is actually combined. The memory layout would be something like

      0x0  x
      0x4  y
      0x8  next
      0xc  prev
      0x10 owner

with the total structure being 20 bytes if pointers are 32-bit.

> Aren't ListNode just a way to index a list ? Isn't this Point2D just > "marked" to be on a certain list ?

The list itself is formed entirely by the ListNodes themselves. There is no backing array to index into; the storage for points (and nodes, which are contained in points) is created with an individual malloc per point. The way you reference a list is by holding a reference to the head, which is just a regular ListNode.

<comment><USER29> If you use a std::List<T>, doesn't it uses nodes like

        T data
        p next
        p previous
        p owner

anyway? It shouldn't create a pointer to the data, just like a std::vector<T> isn't an array of pointers to T.

<comment><USER30> Let's say I have a bunch of _Bullet_ objects, and each _Bullet_ belongs to multiple lists. In that case, std::list<Bullet> doesn't work and I have to use std::list<Bullet*> instead, thus the extra pointer dereference.

</comment></comment></comment></comment><comment><USER31> The thing about linked lists is their Big O time looks nice, in theory, but in reality they do not work well with how most CPU's are designed. (Because it makes the CPU cache mostly useless, and clock speeds aren't really improving, so the CPU cache is pretty much the driver of better performance at this point). I'd rather just use a vector/array and take the hit when I need to resize it, most of the time.

</comment><comment><USER32> Oh, look, it's my post :)

</comment><comment><USER33> Here's another example [1] of the same. Blizzard games from WC2 to SC2 use this code nearly verbatim (testing 1 instead of INT_MAX for 64-bit compat in newer titles).

1\. [http://www.codeofhonor.com/blog/avoiding-game-crashes- relate...](http://www.codeofhonor.com/blog/avoiding-game-crashes-related-to- linked-lists)

</comment><comment><USER34> Wayland also uses intrusive circular linked lists - but it doesn't need owner pointer, since that pointer can be computed from pointer to link, and offset of that link in owner structure. I wonder if doom 3 really needed it.

</comment><comment><USER35> It's funny how this technique looks like something coming from a newbie programmer. This reminds me parallel areas. Looks like done by a profane, but hey, it's actually cache-efficient these days.

</comment><comment><USER36> Does anybody know what would the performance gain be in this case?

<comment><USER18> There isn't one. std::list<T> has the same space requirements, memory locality and capabilities of an intrusive list. Period.

The naivety of an 'intrusive' list, where all your objects are always 'list ready', is that it allows the insertion and removal of elements without _copying_ them, which is beneficial if your lists experience a lot of churn. The thing is, you can achieve this without intrusion anyway by keeping all your 'not in a list' items... well, in a list. You then get memory management for free and can use splice()[0] to efficiently move nodes around.

My philosophy on lists is: if you've never used or needed to splice then you probably don't really need a linked list. Splicing is what linked lists are all about.

[0] [http://en.cppreference.com/w/cpp/container/list/splice](http://en.cppreference.com/w/cpp/container/list/splice)

<comment><USER37> What about modifying the elements while keeping them on multiple lists?

With intrusive list you need only modify any element on a single place (A bullet location as an example).

How could you achieve this with std::list?

<comment><USER38> If you're storing pointers to elements in the list, you only have to change the pointed object.

<comment><USER39> OK, then.

I'll give you a pointer to an element and you remove it from all containers that it is currently on, in O(1).

The only way to do that is to keep a bunch of iterators in the item itself, which is virtually identical to the intrusive container model _except_ it involves more pointers, more heap allocations and introduces an extra invariant into the data model (the iterator stored in the item must point at a list that actually contains said item). And the only benefit you get for all this pain and suffering is that you don't have to do any "funny" casting via container_of() contraptions.

Point being is that once data items sit in more than one container, intrusive container model generally results in a simpler code that is also more efficient.

</comment></comment></comment></comment></comment><comment><USER40> I think I remember Turbo Pascal's container library using a similar approach.

</comment><comment><USER41> > " Every time a new projectile is created it has to be added to a bunch of > lists. There can be a lot of projectiles, they appear and disappear quickly, > and every time that happens we incur the cost of allocating/deallocating > memory multiple times." Lists can be pre-allocated though

</comment><comment><USER42> The article contains many false claims and its main statement that intrusive lists are better than non-intrusive ones is simply wrong.

Outright wrong:

1) There is no additional dereferencing needed when using his 'ListNode' implementation, since the object is stored directly in the ListNode.

2) If you want objects to be non-pointer members of ListNodes of several lists, you can achieve that with both intrusive and non-intrusive lists in exactly the same way (contrary to the authors claim).

The article is a disgusting example of someone trying to proof a point he heard somewhere else. So it is not his original idea and he is failing hard.

Why are there no comments on HN (except for jokoon) addressing that? Instead everybody uses the comment system as a platform to show off his own bad ass C++ skills... stupid. You deserve each other

<comment><USER43> Your contribution would be very welcome here and upvoted, if you kept it factual (your points #1 and #2 are interesting and fine; your speculation that the author heard this elsewhere is almost okay, if you had elaborated.)

But in your last two paragraphs you switch to using the language of trolls ('disgusting example', 'stupid', 'you deserve each other'). no need - you went from being an obvious upvote to an obvious downvote and probably risk being hellbanned. (Where you see your contributions but nobody else does - <USER2> nobody can see you no matter how good your points are.)

see commenting guidelines: [https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

I hope you will read that and be a positively contributing part of this community.</comment></comment>


Ask HN: Best tech Twitter accounts to follow - <AUTHOR> Hi!
I've recently replaced Reddit with Twitter to get my Tech news, and I'm wondering what the community thinks are the best accounts to follow to stay up to date in Tech.
Appreciate it :) Claro <comment><USER2> @pinboard for excellent snark + tech political activism

</comment><comment><USER3> @SwiftOnSecurity. Excellent tweets on security, general programming stuff, nerd humor, and women in tech.

</comment><comment><USER4> News wise? Tech journalists.

Though I think the better use of twitter is to follow people like @naval and similar accounts. More of general life/interesting thoughts.

<comment><USER5> Stay away from tech journalists unless you want to spend time reading tech mixed with politics these days. Follow engineering blogs of Facebook, Netflix, Spotify, LinkedIn, etc.

</comment></comment><comment><USER6> Not @dhh --- the guy tweets to prove he is right about everything and that the rest of SV is wrong. I bought into it for a little while but then he got all buddy buddy with that Gary V dude. the last straw was him celebrating the outing of Uber's CEO for a whole week.

</comment><comment><USER7> There are loads of good ones out there but here's a few IMP:

-MadewithARkit (brand new, lots of cool stuff)

-SwiftonSecurity

-VentureBeat

-Kara Swisher

-Christopher Mims

-Google Developers

-Google Research

-Steve Jurvetson

-Siraj Raval

Honorable Mentions, the Parody Tech Accounts:

-Not Jonny Ive

-Bored Elon Musk

</comment><comment><AUTHOR> Thanks! I'll check out naval@

</comment><comment><USER8> @BenedictEvans

</comment><comment><USER9> I like to use Otherside to create different lists on different topics[1]. You just pick the person most likely to be following a certain niche of users, and Otherside creates a private list for you. You can make the list public and share it with others if you want.

[1]: [https://otherside.site](https://otherside.site)</comment>


Status.GitHub.com – Internal Server Error - hilbert-

https://status.github.com/ <comment> hilbert- Status Feed:

14:15 UTCService has recovered, we are currently processing hooks from the backlog

14:05 UTCWe've identified an issue with external connectivity, affecting hooks, avatars and other services.

13:38 UTCWe're investigating an issue with hook delivery. 8:50 UTCEverything operating normally.

8:06 UTCWe have identified the root cause and are now restoring access to customer repositories.

7:32 UTCWe are investigating an issue preventing repository access for some customers.</comment>


Experiments with WebSocket Performance (for HTML5 games) - <AUTHOR> http://blog.artillerygames.com/2012/06/websocket-performance.html

<comment><USER2> The description of the retransmission timeout is kinda wrong. All modern TCP stacks (at least Linux and Windows, not sure about OS X) implement the selective acknowledgement ("SACK", see RFC 2018) option by default[1]. The receiver will notice the out of order sequence numbers and immediately push a SACK package back to let the sender know about the drop.

And if you look at the charts, that seems to be exactly what happens. At low- but-nonzero packet loss rates, the maximum delay is never more than one or two RTTs. With high packet loss rates, you start to see a long tail of longer times due to double-loss events, I suspect.

[1] "By default" becuase it can be turned off. SACK can be used as a DoS vector by forcing the sender of a large transmission to buffer and reprocess essentially all of it repeatedly by pretending it "lost" a packet.

<comment><USER3> Thanks, I spent some time trying to figure out how I could configure this retransmission timeout. The SACK solution you describe sounds like a great idea.

Is that the optimum solution though? Could one, say, have two TCP connections and send the same data on both? This way, maybe when one connection experiences packet loss, the other can carry on. This probably doesn't actually work due to high packet loss correlation or something (I have no idea about networking), so it would be interesting to hear what experts think.

Are there any other relevant parameters to tweak in order to make TCP's performance slightly more like that of UDP? Working on an MMO game myself (for fun), so would be great to know.

<comment><USER2> It's a complex subject, but don't think about it as TCP being "low performance". In fact TCP will _vastly_ outperform basically any hand-written UDP protocol at large sequential transfers over fat networks (or I guess it would be better stated that any hand-cooked protocol that performed as well as TCP would be isomorphic to TCP).

The characteristic you want from UDP is out of order sequencing: you want to see the most recent packet as soon as it arrives, even if some got missed before. No version of TCP does that, because TCP is a reliable protocol which presents a stream metaphor. Packet loss will always show up as latency burps, though as I point out generally that burp is one extra RTT.

<comment><USER3> Thanks for the reply. Yeah, I understand TCP enforces packet order and works really well for large transfers etc.

But when writing a game where people interact in "real time" through their browser via websockets, anything to get TCP latency down is a plus in my book.

While packet loss is bound to cause some latency issues, I just want to minimize the latency hit for packet loss as much as possible. For instance, in a high packet loss situation, to overcome double loss (where a retried packet gets lost, causing 2xRTT of extra latency), perhaps TCP can send multiple retries at once after a single lost packet. For all I know, maybe this is a standard configuration parameter that I just have to set appropriately.

</comment></comment></comment></comment><comment><USER4> Will WebRTC alleviate this at all? Looking over the spec[1] it seems that there is a "reliable" flag on the DataChannel, although the implementation of the underlying transport is not specified.

[1] <http://dev.w3.org/2011/webrtc/editor/webrtc.html>

<comment><USER5> Yes, it may work but this is yet to be implemented or standardized.

<comment><USER6> WebRTC is available in developer versions of Google Chrome. There are some demos listed here: <http://www.webrtc.org/running-the-demos>

<comment><USER5> I have been playing with WebRTC but the API for the data channel is yet to be implemented or available in any of the browser. Currently only MediaStream and PeerConnection has been implemented in Chrome.

</comment></comment></comment></comment><comment><USER7> No later than yesterday I spent couple hours researching Websocket servers to no avail. I want to run it on Apache/Windows with PHP (there is some python implementation, but I wish it would be as simple as mod_websocked and opening concurrent port). Unfortunately solutions like Heroku are tremendously expensive in my opinion, if you want to build something that will have 1,000 websocket connections at the same time. Could not find anything reliable. This and fact IE/Opera do not have websocket implemented, makes me come back to good old Flash technology - building a simple socket module that will trigger Javascript functions inside a webpage.

<comment><USER8> Opera does support Websocket - <http://www.opera.com/docs/specs/presto27/websocket/>

As will IE10 - [http://html5labs.interoperabilitybridges.com/prototypes/webs...](http://html5labs.interoperabilitybridges.com/prototypes/websockets/websockets/info)

Full matrix at <http://caniuse.com/#feat=websockets>

<comment><USER7> I stand corrected in regards to Opera. I have 9.8 (which I downloaded not so long ago) which does not support Websocket.</comment></comment></comment>


The Ripley Scroll (2009) - <AUTHOR> http://bibliodyssey.blogspot.com/2009/01/ripley-scroll.html <comment><USER2> FWIW: If you're in the LA area, the Getty currently has a Ripley scroll on display.</comment>


Ask HN: Where are all the front-end devs at? - <AUTHOR>

We have had a posted job for a front-end developer position for some time. We posted it on all the popular job boards but we got very limited applications. Are all the good ones already in jobs they like? Just a hot market with not enough job seekers? So where are they?? <comment><USER2> I looked up your jobs page on your site. I liked the page and thought you did better than average at presenting the job and your company. However, you presented (in my opinion) two negative points which were requiring both Backbone and .NET, and requiring participation in 24/7 on-call rotation.

You are looking for a bit of a rare bird. I'm sure others are asking for the same thing and getting more applicants. Be sure you are comparing yourself only to them.

</comment><comment><USER3> I can't speak to your specific job posting, but I can speak to my own experience applying for jobs.

Job postings seem to either request mystical abilities, attainable only by the rockstar ninjas that don't need a job anyway, or have some sort of requirement that makes the job look like a miserable opportunity.

I suspect that most front end devs make more money working from home freelance than they do via salaried positions. In that case, why bind yourself to a salaried job at an office?

As a hiring manager, I feel your pain. I've had a hard time finding and keeping the good ones because of our old-school office environment. As a job seeker, I decided to stick to freelance opportunities and networking. I've landed more gigs by making friends than by sending resumes.

</comment><comment><USER4> It's hard to say from so little information. Generally speaking though, if you know that others are getting applicants then you're probably doing something wrong in comparison. Look through the /jobs board here for inspiration in that regard.

Requiring a formal Resume process to apply? Remote working? Relocation expenses? Too little equity+pay? Not a very interesting job post?

</comment><comment><USER5> Try the following:

* Monthly 'For Hire' threads on HN * HN Jobs * Authentic Jobs * GitHub Jobs

Best of luck!

</comment><comment><USER6> Good frontend developers are like unicorns, cuz they need to be comfortable with both technical and design worlds. I think that's why they are rare.

That said, I'd love to hear more about your position. Hit me up:

<http://dylanhassinger.net></comment>


Node and ARM - <AUTHOR> https://medium.com/@nodesource/node-and-arm-b5e3eebf449

<comment><USER3> The arguments against jvm vs node I'm not buying. You don't automatically get scalability for free with node any more than any other platform. If anything the JVM has been further optimized, particularly for servers, for years over v8. You also don't see ram helping node not because node is so much more efficient -- it's because the design of the v8 limits how much ram it can use.

I'm not sure even why it serves to try and make this comparison if you're talking about node running well on modern arm processors.

There are a number of arm hosts out there now, but none seem to have caught on big. Perhaps in a generation of processor or two someone like aws will start to offer it and there might be a plausible shift.

<comment><USER4> The only real advantage to node is that it doesn't have all the "enterprise" baggage: the apps people build with it are a lot smaller and a lot more decoupled.

<comment><USER5> That baggage is concentrated in one particular JVM language and is largely self-imposed. I write NLP software in Java without resorting to the AbstractFactoryFactory nonsense prevalent in the enterprise world.

<comment><USER6> It happens in both Java and the .Net world.. every time I see "Enterprise _" with a product description it makes me cringe... A lot of those abstractions only serve to scenarios; 1) Unit Testing and 2) service/storage portability. In most cases, neither are actually done. In the first case, I find actually writing unit tests in JS to be _SO MUCH* better than with .Net or Java. In the latter, YAGNI prevails in my mind.

IT's entirely possible to work with "enterprise" platforms without using the excessive design patterns and abstractions, most simply don't. Not to mention that async being an option with C# for _YEARS_ but most code I've seen even running on newer runtimes doesn't use it at all.

Node works very well for I/O constrained loads, unless you're risking running out of memory. I actually prefer the node way, and when you favor functional flows over OO design patterns, it gets to be _really_ nice.

<comment><USER5> Java certainly has its warts, but I still think nearly all of what you describe is cultural rather than technical. Whether it's slavish adherence to GoF patterns, or obsession with code coverage metrics, or religious devotion to OO purity it's about the culture of the developer and the team. Java and C#, unfortunately, are the predominant choices of the developers and teams who share this culture. It gives the languages an unfairly bad rap, one which I agreed with until I started my current job.

<comment><USER7> You can take Java out of the enterprise, but you can't take the Enterprise out of Java.

What's sad is a bunch of us saw this coming the moment a J2EE spec first came out, and complained loudly that it was a trap, to no avail. Version 1 was written by grad students who had no idea what the 7 Fallacies were.

Version 2 was so complicated only a few big companies could implement it. It was a sad trainwreck that was hard to watch. And then Spring was supposed to save us all, and now it's more complicated than any of the things it replaced.

Which is why I do a lot of front-end code these days.

<comment><USER8> _Version 1 was written by grad students who had no idea what the 7 Fallacies were._

At the risk of appearing a grad student, what were those?

<comment><USER7> [http://en.wikipedia.org/wiki/Fallacies_of_distributed_comput...](http://en.wikipedia.org/wiki/Fallacies_of_distributed_computing)

The interesting thing about the 7 fallacies is that we constantly find new ways to make the same mistakes. If we've been doing decentralized computing, someone thinks they can fix all of our problems by centralizing it. If we're in a centralized phase, someone thinks we can solve all our problems by decentralizing it.

Exactly like the node and microservices people are doing right now. If you want a front row seat to the next pendulum swing, start learning everything you can about centralized software models now. You'll be sitting pretty in three years when the backswing starts. Maybe we'll try software transactional memory again.

<comment><USER6> I think that breaking things apart can be very beneficial though... You can run microservices on the same system physically, and give preference to a local service. This does increase complexity of orchestration, but it's still beneficial. Most people wouldn't implement their own local database for a service that has to handle many thousands of simultaneous requests... yeilding to a RDBMS. By the same vein micro-services allow a similar distribution based on a type of work.

There's also a difference between writing a handful of microservices, which are a bit more broad vs. breaking every action into it's own service. Beyond this, if you can separate out of band actions via queues this can alleviate a lot.

One thing that does get me is how many services are using a SOAP or even REST interface over HTTP vs something lighter. In my current project the backend services operate over 0mq req/res interfaces, while the front-end will communicate over a REST composite interface. In this way there's less disconnect and overhead.

It's always a trade-off, but that doesn't mean an approach doesn't have merrit, it's just a matter of striking a balance. Which I would guess is the whole point of all of this.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> "Node is nimble; for example, we advise our clients to kill & quickly restart when their applications enter an unexpected-error state. You can't do this with a runtime that takes minutes to properly start and warm-up."

Hmmm. I'm running tiddlywiki (node.js app) on an ARM, and it probably takes a good ~30 seconds to start up.. Not good for an application that weighs in at 20k lines of code, 3/4 of which isn't even javascript. Worse yet, it has noticeable delays during operation, including about a 15 second delay if it hasn't been accessed in a few hours. I might blame it on a big dataset, but frankly it has less than a 100 tiddlers, and everthing is hosted on a SSD. This on a machine that generally is quite responsive, and capable (although its about 1/5 as fast as my desktop in some benchmarks I ran). It sure doesn't take that long to start apache/php on the same machine and start serving pages.

<comment><USER6> I am surprised that your application takes 30 seconds to start up in node. I have several utility scripts that run via babel-node instead of pre-build that don't take that long to start. When running pre-built JS, even using next generation syntax in source the start up is usually well under a second... Most of the lag time is in initializing database connection pools.

That said, how large is your application? Can it be broken up in to more right-sized services? Are you maintaining a lot of instantiated objects in scope? Are you using OO or FP orchestration? What kind of application is it?

(edited to be less abbrasive)

</comment></comment><comment><USER10> "we advise our clients to kill & quickly restart when their applications enter an unexpected-error state"

Wait, what?

Shouldn't your thing actually work. How unprofessional is it to advise clients to kill and restart things instead of fixing the issue that caused this to be needed?

Wow!

<comment><USER11> Or you can use a system designed to accomodate the occasional failure like Akka (running on the JVM they seem to hate so much) or Erlang.

<comment><USER12> This. So much this. We restart faster than X and therefore we're better? (not to mention all the other weird points of comparison the author chose to call out)

There's plenty of languages with runtimes that have fast start-up times. Does Node therefore suck because my Go application could potentially start-up faster? What about something Erlang on Xen where they're spinning up a _brand new_ Xen domU to service a web request in the hundreds of milliseconds range? Brand new VM, can't really "crash" and need to be restarted because it's only ever there for a single request.

I'm not really sure what the end goal of the whole article is, but it seems woefully written in terms of being technically rigorous.

</comment></comment></comment><comment><USER13> I've been using Node.js/io.js on ARM for a number of projects and have been pretty happy with it. For example using a cheap development board to create a build status light: [https://github.com/tantalic/build- light](https://github.com/tantalic/build-light). I'm just starting a new project where I am going to give Go a try for ARM development which I suspect may give even better results.

</comment><comment><USER14> > Node's development model encourages developers to think multi-process from > the beginning because we know that without the crutch of threads, the only > way we can scale our applications is to multiply the number of processes

>This means that Node applications scale as easily across clusters of servers as they do within a single server.

Yeah, the loss of efficiency that occurs by starting up an entirely separate V8 VM on the same machine is equal to starting one on two separate machines... But how is that a plus? Especially if you want to start more than one thread on a machine.

Also, that part had nothing to do with Node. Perhaps Node may be optimized for ARM, but that really isn't informative in how it performs compared against (say) Java on the same platform. Java's optimized for ARM, too...

</comment><comment> vezzy-fnord What exactly do they mean by "multi-process" in the context of an event loop like Node? I'm also not sure about their claims concerning clustering - <USER2> has no SMP capabilities, nor is there any real mechanism for enabling location transparency out of the box.

It's also interesting to note the conflict between the microservices and monolithic application camps. Both are just as adamant that theirs is the future and the other is awful. I'm sympathetic to KISS personally, but is there any real consensus forming?

<comment><USER15> Wow. Are people defending the monolithic app? Maybe it is where I hang out, but I'm not seeing many.

See for example, Martin Fowler: [http://martinfowler.com/articles/microservices.html](http://martinfowler.com/articles/microservices.html)

<comment><USER16> I think it depends on what kind of application you're building, as well as other factors like team size, experience, and time. Microservices aren't magic pixie dust you sprinkle over your applications to suddenly make them simpler and "web-scale". They have a cost in management and maintenance. You can end up taking microservices too far, just as you can end up making the monolith too monolithic.

Having your core API all be part of a single application is great. It's my impression that for a lot of applications you can't really achieve good decoupling of resources, so trying to break it up into multiple services ends up bringing you a lot of pain. Maybe this is my lack of experience showing, I'd love to be proven wrong.

I'm currently a fan of having a big/medium app that does everything that cannot be decoupled easily, and breaking everything else into services. This is a happy medium, because you don't have to worry about managing tons of services, but you also don't have to worry about having an absurdly complicated monolith.

</comment></comment></comment><comment><USER17> > While the JVM is right at the heart of the monolithic software stack and the > tooling that surrounds it, Node, or server-side JavaScript, is arguably at > the heart of the new SOA stack.

I love developing with node.js, but this doesn't sound right at all.

</comment><comment><USER18> Smaller ARM servers are an interesting trend, see [https://www.scaleway.com/](https://www.scaleway.com/)

<comment><USER6> I've looked at them... but the performance still doesn't match even the relatively small VMs that are offered from cloud providers in terms of raw throughput. Part of the issue is the system builders for ARM based server boards simply aren't at near the scale.

</comment></comment><comment><USER19> Interesting to see a high level comparison of languages without a comparison of the important numbers: performance.

I know that Java performance is pretty crap on RPis due to the poorer Java ARM support, and I'd imagine that it's largely identical for node. All those millions that went into optimising JS performance can't help if the code generated is hobbled.

I've found little evidence to either prove or disprove my view, apart from this fairly devastating graph from a year ago, where a Pi is perhaps 100 times slower than an Intel dual core from nearly a decade ago.

[http://smartmobilestudio.com/wp- content/uploads/2014/01/node...](http://smartmobilestudio.com/wp- content/uploads/2014/01/node.js-benchmark-2.png)

I'm not convinced that ARM will be the architecture of the server, except in specialised markets (statistical multiplexing will give a big advantage to the >64 core Xeon servers that are popping up all around the place), but I'm totally convinced that this press piece is not particularly informative.

</comment><comment><USER20> "Node applications are generally scalable by default." no bias there..

<comment><USER6> No kidding... look at the first pass of node applications written by people coming from .Net or Java... (I include myself)... it takes a while to get into the "node way" breaking your code down into discrete modules, and favoring functional flows over OO ones.

That said, once you get into it, some beautiful orchestration patterns emerge[1][2].

[1] [https://twitter.com/tracker1/status/558390877563801601](https://twitter.com/tracker1/status/558390877563801601) [2] [https://twitter.com/tracker1/status/593174888187240448](https://twitter.com/tracker1/status/593174888187240448)

</comment></comment><comment><USER21> Very nice summary on ARM but I don't really understand the pitch for node in this context, unless "small" is the overriding design criterion.

</comment><comment><USER22> So does this mean we'll get node.js on the arm64 port of Ubuntu soon?

[https://bugs.launchpad.net/ubuntu/+source/nodejs/+bug/143546...](https://bugs.launchpad.net/ubuntu/+source/nodejs/+bug/1435466)

</comment><comment><USER23> Where are the numbers? In my experience node is very slow on ARM.

Spin up a http server on arm and check out the latency for it to return "".

</comment><comment><USER24> I hope they can do something about the RAM usage for Node/io.js programs.

</comment><comment><USER25> Well Node.js is the best so.</comment>


Nvidia Is Now Entering the Table Market? - <AUTHOR> http://allthingsd.com/20130918/nvidia-looks-to-gain-ground-in-mobile-with-199-tegra-note-its-own-entry-in-crowded-tablet-market/

<comment><USER2> *Tablet</comment>


Show HN: Nuggets – Remember everything you read or learn - <AUTHOR> http://www.nuggetsapp.com <comment><USER2> From the source:

Blog

Download

Privacy

Help

You forgot to remove code from the app you copied this from.

Seriously?

</comment><comment><USER3> Any relation to [https://www.rooms.me/](https://www.rooms.me/) or are you just lifting their design without credit?

You forget to take out links to their site. (bottom right)

Pretty dumb move, considering rooms.me is by FB, and there's bound to a be FB people looking on HN.

<comment><USER4> In fairness, "nobody gives a shit how the sausage is made."

If this product adds value to people's lives, why does it matter if they lifted the website design?

<comment><USER3> Because it's illegal.

Copying licensed code, like this: (left rooms.me, right nuggetsapp) [https://www.diffchecker.com/gmudwop7](https://www.diffchecker.com/gmudwop7)

is I L L E G A L, and I M M O R A L. Letting it slide is not okay!

<comment><USER5> This sparked a debate in our office - clearly designers look to other sites for inspiration, but at what point does it actually become illegal? If the user didn't copy / paste and instead reverse-engineered the code, would it be legal? (but still immoral, of course).

<comment><USER3> IANAL (but work around a lot of them)

Even if it was just an interpretation, chances are FB would have a strong case in court. Pretty much every artistic and expressive UI/UX element from the original site has been copied and placed on the nuggets site. There's not really anything being creatively added or remixed by the nuggets creator.

If insane shit like patents on rounded app icons and the iTunes "music note" logo gets regularly brought to court and settled[1], then copying the entire L&F of a site is definitely infringing.

[1]: [https://en.wikipedia.org/wiki/Apple_Inc._v._Samsung_Electron...](https://en.wikipedia.org/wiki/Apple_Inc._v._Samsung_Electronics_Co).

</comment></comment></comment></comment></comment><comment><USER6> Hmmm, I saw your pitch to YC fellowship through topcharts yesterday and now you've made a Show HN with a lifted design of rooms.me? pretty bad move IMO.

The fact it's lifted really clouds anyone to look at the idea or the concept and provide feedback here, change it for the better.

</comment><comment><USER7> I've wanted an app like this for a long time. But the wholesale literal copy- pasting of the design just sucks and makes me sad.

</comment><comment><USER8> I like the idea. but, it turns out that the author is kind of "epic fail" because he exactly copying the landing page of rooms like other HNers described in the comment.

I don't know the landing page of rooms.me before. after reading the comments on this thread, i feel disappointed to the author of this app.

</comment><comment><USER9> The logo looks kinda like a hairy butt.

<comment><USER10> Balls, which is what Facebook will have him by for ripping off Rooms.

</comment></comment><comment><USER11> Setting aside this embarrassing directly-copied-layout issue, I have to say the concept itself is very nice. Is it original idea or is it also cloned from other app?

If former is true, that's a pity the author shoot himself in the knee. If the latter is true, could you point me to other, original app(s)?

</comment><comment><USER12> I don't understand the fuss and hate. It seems like a great idea and I downloaded the app. Sure, the landing page is copied. Sure, it is probably immoral. But it's just a landing page not the product. There are only so many kinds of landing pages and people copy them all the time.

</comment><comment><USER13> Great app! Just what I needed! Would be nice to have an option to edit my nuggets and save to Evernote.

</comment><comment><USER14> Great idea, and making this available in Kindle would be awesome!

</comment><comment><USER15> Wow. Bald face stealing of rooms.me.</comment>


How to change my apple id for icloud use? - <AUTHOR>

I recently updated my iPad2 to iOS, I have one problem not able to use icloud, as it expects my apple id to be like email, but mine is not. seems my image imports have vanished thank god had backup of most on facebook.<p>is it me alone or others who face the problem? how do i change my apple id for using icloud? <comment><USER2> If you came over from MobileMe your name could be email@example.com or you could also make a new ID (which I did by accident)</comment>


Show HN: Average steps humans take each day - <AUTHOR> http://i.imgur.com/1C0Klb8.png <comment><AUTHOR> Using elasticsearch to query all users daily step count. Then avg them by day of week, age range, and gender.

<comment><USER2> Source?

<comment><AUTHOR> [https://activity.club](https://activity.club) we have thousands of users submitting their steps each day.

<comment><USER2> It would be interesting to compare the the actual average number of steps vs that of people competing against each other. This sample is biased towards people competing with their friends to see who takes the most steps.

<comment><AUTHOR> yeah the data is quite fascinating. Many people install the app with the intention of competing but then end up just taking their normal amount of steps. We're going to release an SDK soon with some sample data and let everyone create cool stuff with [https://www.elastic.co/products/kibana](https://www.elastic.co/products/kibana)</comment></comment></comment></comment></comment>


Uber Is Ordered by Spain and Thailand to Halt Operations - k-mcgrady http://www.nytimes.com/2014/12/10/technology/uber-is-ordered-by-spain-and-thailand-to-halt-operations.html

<comment><AUTHOR> Cannot say I don't welcome Uber getting curtailed in all these places. Like many people have said, I doubt the drivers have the insurance to cover their passengers should something really bad happen whilst a passenger is in the car. Something to consider. Taxis are required by law to be insured in certain ways, as are bus companies and limo services.

Also, I like the traditional taxis and bus systems.</comment>


How Peer-To-Peer Payment Pioneer Venmo Grew Up and Got Serious - <AUTHOR> https://wp.fastcompany.com/blog/40400786/how-peer-to-peer-payment-pioneer-venmo-grew-up-and-got-serious <comment><USER2> This appears to be the wrong link - it asks me for a WordPress login.

<comment><USER3> try this one [https://www.fastcompany.com/40400786/how-peer-to-peer- paymen...](https://www.fastcompany.com/40400786/how-peer-to-peer-payment- pioneer-venmo-grew-up-and-got-serious)</comment></comment>


Qimo: A desktop OS designed for children - <AUTHOR> http://www.qimo4kids.com/page/What-is-Qimo.aspx

<comment><USER2> The first thing I tried to do was click on the center image. Several screenshots of Qimo in use would improve the site tremendously.

<comment><USER3> Here:

<http://www.quinncoincorporated.org/screenshots.php>

I had to do a little snooping first, only because their "donate" link figures so prominently on their site that it triggered off my scam detector. They're a legit non-profit founded by a couple.

</comment></comment><comment><USER4> I hope the keyboard repeat is set to off by default, because this is how my kid tttttttttyyyyyyyppppppppppeessssssssss

</comment><comment><USER5> Nope. Prove me wrong, Qimo: nobody is going to know what Qimo is 2 years from now.

* It uses the exact same desktop metaphor as OS X.

* My kids (starting ~age 5) have had no trouble picking up enough Finder and Firefox to use their Mac.

* Its one killer app (TuxPaint) is far easier to install on OS X than it is to install Qimo anywhere else.

* The remainders of its apps are pretty much just that, especially compared to what kids can get from the web now. Sorry; calling it like I see it.

A real, supported, up-to-date web browser for kids with a rock-solid "kiosk" mode, yes. Give me a Paypal link. An Ubuntu distro, no.

<comment><USER6> Are you reviewing this as a startup idea? Your response seems a bit harsh for an open source educational charity.

One benefit of open source is that Qimo could disappear tomorrow and others could carry the torch and build on anything of value they produced.

I use OS X myself and am aware that they have a simplified Finder for kids and old folks, but I see value in having something to put on old junker machines for kids, particularly younger ones that are more likely to spill paint on them or whatever.

You're right that the web is important for accessing the widest range of material. The only real knock that Linux has from the point of view of web kiosk for kids is the lack of shockwave plugin that some older content on e.g. the BBC kids sites uses.

Also, did anyone else think that Qimo sounds too close to Chemo (as in therapy)?

</comment></comment><comment><USER7> Their site runs in ASP on Windows? That's a bit messed up for a linux distribution.

<comment><USER8> What makes you say that? Looks like PHP/Apache running on Linux: [http://uptime.netcraft.com/up/graph?site=http%3A%2F%2Fwww.qu...](http://uptime.netcraft.com/up/graph?site=http%3A%2F%2Fwww.quinncoincorporated.org)

<comment><USER9> What-is-Qimo.aspx is the URL to the home page.

I mean, you can of course play with MIME types and serve arbitrary file extensions, but why would you want your Linux distro to seem as though it was served by IIS?

<comment><USER10> Maybe they have better things to do (like work on the distro) then care about some nitpicks. The audience here isn't geeks. It shouldn't matter that much.

</comment></comment></comment></comment><comment><USER11> Just a quick note that you may want to consider a new name, when I read the brand name I mentally pronounced it as 'Chemo' as in chemo-therapy.

</comment><comment><USER12> great idea! reminds me a bit about the olpc-approach to design ui's especially for children and exploring new concepts.

i couldn't find out too much about qimo on the homepage though... i think it would be a good idea to write how qimo differentiates from other distros. and link in the screenshots! :D

Keep up the good work -- We need more projects dedicated to children!</comment>


How to Blog about Code - <AUTHOR> http://garann.com/dev/2013/how-to-blog-about-code-and-give-zero-fucks/ <comment><USER2> By not following the patterns or advice herein</comment>


Marshall McLuhan: The Playboy Interview (1969) - <AUTHOR> http://www.nextnature.net/2009/12/the-playboy-interview-marshall-mcluhan/

<comment><USER2> McLuhan is one of the most relevant thinkers to our time--if you can figure him out. I've read most of his books, and written several papers on him and still don't quite get it, feel like there are layers of what he was talking about that I may never grasp.

Entry McLuhan for anyone interested -- The Medium is the Massage, a picture book that originally was a misprint (The Medium is the Message, being the phrase he coined), but he liked it and kept it. Kind feels like reading it is a drug trip, but you start to figure him out.

Understanding Media -- Seminal work, he lays out the medium is the message, media hot and cold, media as sensory extension, and other concepts as well as catalogues how he sees various media affect us.

The Mailer/McLuhan debate from CBC is really interesting as well, Mailer as existential/nihilist and McLuhan as nearly an analytic -- neither really talking philosophy. [https://www.youtube.com/watch?v=PtzxWR-j1xY](https://www.youtube.com/watch?v=PtzxWR-j1xY)

<comment><USER3> It's all a bit stream-of-consciousness. Some of that is I'd guess due to the fascination then with psychedelics. I still run into people who wax poetic about psychedelics as a social change mechanism and technology for personal growth.

MacLuhan talks about how Nixon adapted to TV; turns out that was most likely Roger Ailes' doing.

The present day "conservative" (tea party) movement is largely founded on ideas from MacLuhan. One of the curious things about Fox is that it's a media channel based on being against the media.

The indispensable idea from MacLuhan ( at least that's where I got it ) is that of identity and that people have a ... shortage of it.</comment></comment>


My first HTML5 game. "Stellar Alien", physics-based space game - <AUTHOR> http://play.stellaralien.com

<comment><AUTHOR> I'm a young developer looking for some feedback on my first game. It is also listed up on the Chrome Web Store if Chrome users would like to install: <http://r.cx/stellar-alien> It is also in review for the Firefox Marketplace.

</comment><comment><USER2> You should add some instructions in the beginning. I'm on a laptop and had a hard time understanding the objective

<comment><AUTHOR> Sorry about that. Will definitely try to introduce the basics of the game at the start. Did you finally understood the objective? You need to get the alien, Starbli, to the star before it turns into a red dwarf and dies.</comment></comment>


Confirmed: Qik will become part of Microsoft, too - <AUTHOR> http://www.geekwire.com/2011/confirmed-qik-part-microsoft

<comment><USER2> God dammit. A bunch of important tools I use every day basically got a death sentence today. IS NOTHING SACRED??</comment>


How Cape Town found water savings California never dreamed of - <AUTHOR> http://www.latimes.com/world/la-fg-south-africa-drought-20180401-story.html <comment><USER2> > Recycled water, including purified effluent, offers a cheaper source than > desalination, but many governments are reluctant to embrace this. Purified > effluent is used for drinking in Singapore and Namibia, but that would be a > hard sell here.

I don't get this. If it's been cleaned thoroughly, why not drink it? And if it hasn't, how can we discharge it into streams in good conscience?</comment>


Introducing Apex – Serverless Architecture with AWS Lambda - <AUTHOR> https://medium.com/@tjholowaychuk/introducing-apex-800824ffaa70 <comment><USER2> I'm curious if anyone has deployed a real app at any scale on Lambda/API Gateway. So far in my experience, the default limits and throughput are far lower than would be remotely useful for a moderately busy webapp. And to hit those rates would be outrageously expensive.

That may improve over time. But even more than that "serverless" is misleading. True, you don't have to personally manage the servers that are running your code, but that means that your operations team is actually inside Amazon. They have economies of scale, but that means they also don't always notice when one of their servers isn't performing up to par. More than once we've had to ask our support reps to get Lambda instances behind the scene restarted. At that point I start to wonder what the point is at all.

<comment><USER3> The guys at [http://acloud.guru](http://acloud.guru) are running their platform on Lambda/Firebase, and have given a few talks about how they've managed to achieve it. Disclaimer: I know a founder of theirs.

<comment><USER4> They have written a blog post that goes into more detail on how they piece together various components with Lambda: [https://read.acloud.guru/serverless- the-future-of-software-a...](https://read.acloud.guru/serverless-the-future- of-software-architecture-d4473ffed864#.thhfdvbc9)

I'm pretty sure they have another post coming in the near future too.

I am impressed by how fast they built their platform given the size of their dev team. We (I'm a course author there, not developer) also use Lambda for custom slack bots too.

</comment></comment></comment><comment><USER5> I tried to use and love lambda, spending a lot of time making my use-case work for their poor development and deployment environment (a zip file, really??). In the end, I had to abandon the effort.

They only allow a certain fixed execution time and 500 mb of temp space, max. And they won't raise these limits.

During testing I noticed that frequently I would have the same contents of empirical storage (/tmp) -- i.e. i'd be running on the same host.

I have a sneaking suspicion that the entire purpose of lambda is to resell capacity on hosts that aren't busy -- but could be -- thus the low CPU time limits and storage spaces. I'm sure someone would not notice 500 MB or 5 minutes of CPU time vampired on their less busy, not monitored hosts from time to time. And if the _real_ user does need the capacity, they can just kill the process, which had happened many times in my testing (process would be killed or die for no reason).

Sounds great and amazing in theory, but execution is horrible and unusable except as a toy.

<comment><USER6> I had a similar feeling when I first tried it. The UX is really bad, I'll give you that! For my current use-cases though I think it'll be just fine.

That's interesting about the random kills, I'll keep an eye out for that, good to know. Interesting theory though haha, wouldn't surprise me too much.

</comment></comment><comment><USER7> At Cronitor we've just finished building out lambda infrastructure for a fan- out where I would otherwise use a worker application and Supervisord.

Lambda has a lot of strengths, and it gives you solid primitives like immutable versions, version aliases/pointers and stateless functions.

The problem I had was that it didn't give me any more than that. Here are just a few of the things we had figure out and then build ourselves:

1\. The development workflow. How can I get a REPL workflow going that doesn't make me go crazy.

2\. Deploys and Rollbacks. How can we safely deploy and rollback, especially in cases where you have the same lambda function in multiple regions and each region is at a different published version (because the version number is monotonically incrementing)

3\. Permissions and Policies. The broad strokes are clear. But you want your S3 bucket or SNS topic to trigger lambda? Get ready to spend an hour trying to figure out what you've done wrong and what's missing from the vague directions. Hope you get that REPL flow solid first.

In the end, we built several scripts (create_build, publish_version, promote_to_prod, etc) and we use these directly during development and from a Fabric-based deploy script. When I have time I plan to release this tooling open-source.

If I had to do it all over again, I wouldn't. And I wouldn't use Apex (at least not yet). I would just use a t2.small instances with a simple worker.

</comment><comment><USER8> I love the idea of developers working on code, and not infrastructure, and so I love open source projects that solve this problem (by automating infrastructure).

But I will never support anything that locks me into a particular vendor.

There are many reasons, from geopolitical to economic, that any open source infrastructure effort should be vendor neutral.

<comment><USER9> Would be nice to see something like Apex that can work on Lambda or within some kind of deployable service, so that the environment it actually ran on was abstracted, avoiding vendor lock-in. I haven't seen any projects yet that try to make the Lambda interface portable though.

</comment></comment><comment><USER10> There is a stream and batch processing framework called Apache Apex. Might be a good idea to change the name. See [https://www.datatorrent.com/apex/](https://www.datatorrent.com/apex/)

<comment><USER11> Salesforce also has a language called Apex [https://developer.salesforce.com/page/Apex](https://developer.salesforce.com/page/Apex)

<comment><USER12> And Oracle, god make their sandals rot, have "Application Express", often abbreviated to ApEx.

<comment><USER13> The product itself ([http://apex.oracle.com](http://apex.oracle.com) and [http://apex.oracle.com/ut/](http://apex.oracle.com/ut/)) isn't bad: it's tailored toward citizen developers versed in the Oracle DB.

Off topic: if you're reading this, Vlad my boy, I will think about sending you an email at some point. Because we all know how you love emails.

<comment><USER12> It's ... a thing. I don't miss it.

</comment></comment></comment></comment></comment><comment><USER14> There's some limitations with both lambda and the API gateway that you need the be aware of -

API gateway has a fixed timeout of 10 seconds that you can't currently extend. Should be fine for most use cases, but something to be aware of if you have any functions that will take longer.

Lambda functions cannot return binary data.

<comment><USER15> Another big limitation is that you currently can't run lambdas in a VPC, so using Lambda with RDS is pretty much off the table if you don't want your database to be open to the world.

</comment></comment><comment><USER16> Looks great, and the discussion on HN is always helpful to find other related projects.

I've just finished prototyping a A/B testing backend running over Lambda / API Gateway. I think I bumped into some of the rough edges others are mentioning.

What troubled me the most however is how difficult it is to share / open- source the code for such a project. There are no environment variable support, so to configure things you have to embed them in code. No way to easily package something that users can just run a few simple commands and get up and running (I think having an AWS CLI + keys already set up is a pretty hefty upfront requirement). So you end up with a huge readme or a blog post just to explain how to set up all the moving parts. A small mistake anywhere along the process and this thing isn't working any more...

I hope tools like Apex can also solve this use-case. Something akin to the heroku / digitalocean button would be truly awesome.

</comment><comment><USER17> Looks like "serverless" is the new buzz word for App Engine or Heroku?

<comment><USER18> Which is too bad because there is a lot more interesting things you could do if you really didn't need the cloud, and connected apps could run client only.

<comment><USER17> Especially after VoLTE, all 4G data are IP based and even voice data are actuall controllable in Application Processor, it's basically SIP+RTP on a port like 5060. Someone need to develop a VoLTE p2p.

</comment></comment></comment><comment><USER19> I think of Lambda like I think of database triggers. It's a great solution for the specific thing that it's built for but people have a tendency to go very overboard with it.

Built for: Triggering events when something happens within AWS Not built for: Replacing all of your servers

I once knew a guy who was very into Oracle and wanted to server up HTML from stored procedures. This strikes me as that line of thinking.

<comment><USER20> Serving up HTML from Stored Procedures was a thing back in the late 90's/early 2000's as enterprise vendors made the shift to web apps.

In fact quite a popular thing for client-server app's where the server was built in PL/SQL, and some clients just needed reports.

I agree, AWS Lambda is a hammer. Just like everyone in enterprise software jumped at J2EE and Enterprise Java Beans back in early 2000's, and "did it wrong", I expect the same to happen here.

I just wish I know of a decent Open Source project doing similar things in a super-light-weight fashion - ideally in Python.

<comment><USER19> Well, if you look at Lambda as a simple event notification system that integrates with AWS your closest equivalent is PHP.

PHP isn't integrated with the AWS stack, but you could drop a terabyte of code on a 256mb VM and it would all run when called. Scales down really well, which is great for housing little one off scripts to receive web hooks at minimum cost, utility servers, etc. It's not sexy, but very effective.

I'd imagine you could do the same type of thing with Python in a CGI mode or something along those lines, but most other languages purposed for web stuff tend to need to load a lot of libraries at runtime to operate that way.

</comment></comment></comment><comment><USER21> [https://github.com/garnaat/kappa](https://github.com/garnaat/kappa) targets the same feature set, and has more functionality afaics, albeit perhaps slightly different. It has more sources to bind lambda to, and takes care of more of the drudgery (iam, permissions, etc). It doesn't do the nodejs shim for go.

</comment><comment><USER22> Am I the only that can't fucking scroll down on this site. I didn't know this was so annoying.</comment>


JSON file containing all active airports - <AUTHOR> https://github.com/jbrooksuk/JSON-Airports

<comment><AUTHOR> After two years of this list, I've just updated it to include more information.

\- The code index is now named iata.

\- Includes the size of the airport if known.

\- Includes the type of airport, heliport, seaboat, airport.

\- Now includes the continent of the airport.

\- And finally includes the ISO code where the airport is located.

Hope it helps!</comment>


Winds of Change at Dyson - <AUTHOR> http://www.fastcodesign.com/3050256/innovation-by-design/winds-of-change-at-dyson <comment> chestnut-tree Not mentioned in this article is the James Dyson Award. An annual international competition for students with a wide brief: _" Design something that solves a problem...Products that have a significant and practical purpose, are commercially viable, and are designed with sustainability in mind."_

The projects submitted are often very impressive and show great skill and ingenuity. The 2014 award went to a 23 year old student called James Roberts who designed a inflatable incubator for prematurely-born babies:

[http://www.bbc.co.uk/news/technology-29900968](http://www.bbc.co.uk/news/technology-29900968)

If you want to be inspired, the archive of entries is well worth browsing:

[http://www.jamesdysonaward.org/en- GB/projects/](http://www.jamesdysonaward.org/en-GB/projects/)

<comment><USER2> This years winner was a very well engineered 3D printed design for a prosphetic hand iirc.

[http://www.bbc.co.uk/news/technology-34044453](http://www.bbc.co.uk/news/technology-34044453)

</comment></comment><comment><USER3> I used to work for Dyson as an intern some 15 years ago now. Obviously, the company will have to be different than I was there, but one thing that did strike me is that Malmesbury is really out there. [1] No main train line, at least a 40 minute drive from the nearest interesting city (Bristol). I can't imagine them attracting the best talent, particularly if they want to duke it out with sexier companies in sexier places in the IoT segment.

They couldn't pay me enough to go back there, and I think anyone worth their time won't either. It's a mistake to build their technology building there and not near London or at least a major city like Bristol/Manchester/Edinburgh. I don't think they realize that really good programmers are worth 10 bad ones (and at least when I was there, they really didn't have the best ones to put it mildly).

[1] [https://goo.gl/maps/A7A1v](https://goo.gl/maps/A7A1v)

<comment><USER4> I invite people to have a look at the street view at the map location you've given. It seems like a perfectly wonderful place! If I were to move back to the UK I would pick a place like that over London any day.

I don't think programming ability is actually related to love of large city centres :-P Certainly it is much easier to hire in large cities, but I'm sure you can lure the ones who prefer small towns out of London if you offer them a competitive salary. Whether Dyson is interested in doing that or not, I have no idea.

<comment><USER5> Yeah, that actually looks nice.

Some of the best programmers I know work from places like that. There's a lot to be said for getting away from big cities.

</comment></comment></comment><comment><USER6> I love the tech of Dyson. But I really dislike the Design Languages of many of their products. It feels like some sort of futuristic design, Mechanical, and sometimes "Alien".

For me I would like something that is Simple, Elegant, and Coherent.

<comment><USER7> For me Dyson and sons are verging on the pretentious with all their personality marketing. We have a Dyson vacuum cleaner - my wife bought it but it seems overly complex and over engineered to me without any particularly special attributes for the price.

The cult of personality and 'we are so clever' sales schtick can get really annoying, including this article

<comment><USER8> I have a Dyson and I can only second your opinion.

I bought a Miele and its lightyears ahead in terms of quality and effectiveness. Old Nilfisks are also incredibly well designed.

<comment><USER5> Second the Miele. Compared to a Dyson vacuum the Miele is unquestionably superior, IME.</comment></comment></comment></comment>