For Better Computing, Liberate CPUs from Garbage Collection - <AUTHOR> https://spectrum.ieee.org/tech-talk/computing/hardware/this-little-device-relieves-a-cpu-from-its-garbage-collection-duties <comment><USER2> IMO garbage collection is the epitome of sunk cost fallacy. Thirty years of good research thrown at a bad idea. The reality is we as developers choose not to give languages enough context to accurately infer the lifetime of objects. Instead of doing so we develop borderline self-aware programs to guess when we're done with objects. It wastes time, it wastes space, it wastes energy. If we'd spent that time developing smarter languages and compilers (Rust is a start, but not an end) we'd be better off as developers and as people. Garbage collection is just plain bad. I for one am glad we're finally ready to consider moving on.

Think about it, instead of finding a way of expressing when we're done with instances, we have a giant for loop that iterates over _all of memory_ over and over and over to guess when we're done with things. What a mess! If your co-worker proposed this as a solution you'd probably slap them. This article proposes hardware accelerating that for loop. It's like a horse-drawn carriage accelerated by rockets. It's the _fastest_ horse.

<comment><USER3> > It wastes time, it wastes space, it wastes energy.

But all of these are much cheaper than developer labour and reputation damage caused by leaky/crashy software. The economics make sense.

Anecdotally, I spent the first ~6 years of my career working with C++, and when I started using languages that did have GC, it made my job simpler and easier. I'm more productive and less stressed due to garbage collection. It's one less (significant) cognitive category for my brain to process.

Long live garbage collection!

<comment><USER4> > But all of these are much cheaper than developer labour and reputation > damage caused by leaky/crashy software.

And that's why so much effort has gone into making it fast and low latency, but it was a false dichotomy: We can have memory safety without garbage collection.

\- Automatic reference counting: Most people know about Objective-C's efforts in this space, but it's admittedly less automatic than programmers would like so perhaps it doesn't get enough attention. And yet it should, since q/kdb+ uses reference counting exclusively, and it holds top performance numbers on a large number of data and messaging problems.

\- Linear lisp[1] asked functions to fully consume their arguments making (cons x y) basically a no-op. Again, no garbage collection, and no fragmentation (at least as long as all objects are cons), and yet no matter how promising this path looked, garbage collection got much more attention.

\- Rust's borrow checker/tracker makes ownership explicit; somewhat of a middle-ground between the two...

There's other scattered efforts in this space, and I don't know about all of them but for more on "everything we know about languages is wrong", also consider that Perl5 uses plain old reference counting, executes the AST directly, and still outperforms python for data and IO[2]!

I think the thing to take away is that memory management has to be automatically managed for developer sanity, _not_ that garbage collection is the way to do it.

[1]: [http://home.pipeline.com/~hbaker1/LinearLisp.html](http://home.pipeline.com/~hbaker1/LinearLisp.html)

[2]: The asyncio stuff in Python looks really promising though...

<comment><USER5> This comment is just bad and misinformed all over.

(1) _Automatic Reference Counting_ doesn't work; its equivalent in interpreted languages is, well, _reference counting_ , which can be optimized quite a lot (though has some issues with multithreading), but cannot collect cycles.

(2) therefore, if you want reference counting, you have to either also have GC (for cycles), or program carefully to avoid creating cycles (which is then only marginally better than C++)

(3) your comment on Python vs Perl5 is just nonsense, Python uses reference counting as well (along with the occasional GC to collect cycles)

(4) linear / uniqueness types (not exactly the same, but both can be used to ensure safe memory management) impose significant mental overhead on programmers as they prohibit many common patterns

(5) Rust features a lot of "cheating" \- pointers that allow mutation from multiple threads, reference-counted pointers, etc. - so obviously just ownership&borrowing isn't good enough

 _conclusion:_ you can't have your cake and eat it too (at least according to current cutting edge research and implementations) - you either have GC _or_ you have to be very careful/restricted when writing code

<comment><USER6> > (1) Automatic Reference Counting doesn't work; its equivalent in interpreted > languages is, well, reference counting, which can be optimized quite a lot > (though has some issues with multithreading), but cannot collect cycles.

This is what weakrefs (or better data structures) are for. The Linux kernel uses reference counting incredibly effectively for almost every structure. I think that pretty much discounts any argument that reference counting cannot work.

> (5) Rust features a lot of "cheating" \- pointers that allow mutation from > multiple threads, reference-counted pointers, etc. - so obviously just > ownership&borrowing isn't good enough

"unsafe" isn't cheating and isn't even related to whether garbage collection is a good idea or not. Yes, Rust's ownership and borrowing model doesn't cover all programs and hence "unsafe" is required -- but "unsafe" doesn't use a garbage collector so the existence of "unsafe" doesn't change that Rust works without one.

<comment><USER7> The point with weakrefs, though, is that they still require developer intervention.

Also, you can use most memory management facilities in Rust (including reference counting) without `unsafe`.

<comment><USER2> Correct, leaks are considered "safe" \-- it's premature deallocations that are considered unsafe.

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> Azul Systems has asked Intel to do this once... but instead created their own processors with interesting memory barrier properties for awhile that greatly sped up JVMs beyond what was capable (at the time) on x86-32/ppc/sparc. Eventually they gave up and became a purely software company, but their "Java Mainframe" product was many times faster than the Intels of the age executing the same code despite much slower CPUs. Died a quick life despite the cool factor.

<comment><USER9> I had a mentor who worked at Intel labs when this was happening. The reason this died was because someone invented a gc algorithm which consistently outperformed this, leading Intel to drop their hardware gc plans.

<comment><USER10> Could that not be hardware implemented / augmented?

<comment><USER9> It could be, but then after investing a billion dollars what happens when someone develops another algorithm?

What happens when a programming language with different gc requirements become popular?

<comment><USER11> Seems to be one of the main risks for any specialized circuits, if I understand you correctly. You always have to guess "will this _really_ be relevant long enough to invest the money to bake it into hardware?" .. and if you guess wrong you just wasted a part of your silicon budget for something no one will use.

<comment><USER10> Right. Got it.

There may still be room for specialized GC assist circuits.

<comment><USER12> When i thought about this (using my programmer brain which knows nothing about hardware), i came up with the idea of a 'store pointer' instruction, which took two addresses and an offset, and stored the first address into the field of the object pointed to by the second address. And also, if the two addresses referred to different memory regions, recorded the pair of addresses into some kind of buffer on the processor. When that buffer got full, the processor would trap to some preconfigured location.

That could be used as a basis for a write barrier.

The devil would be in the detail of how the regions were defined.

And maybe the trapping would mean this wasn't even all that fast.

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> Readable copy of the paper at Berkeley:

[https://people.eecs.berkeley.edu/~krste/papers/maas- isca18-h...](https://people.eecs.berkeley.edu/~krste/papers/maas- isca18-hwgc.pdf)

ETA: this doesn't seem to be quite the paper that the story refers to, but undoubtedly describes the same work in enough detail for people to get the gist of it. Darn paywalls.

<comment><USER14> The paper is available on Scihub: [https://sci- hub.se/10.1109/MM.2019.2910509](https://sci-hub.se/10.1109/MM.2019.2910509)

</comment></comment><comment><USER15> They're comparing to an in-order CPU. Given that most CPUs are out-of-order (at least of the non-embedded variety, and GC is less used in such applications anyway), it would be better and more intellectually honest to actually compare to a typical CPU that performs GC. They kind of address this in the paper but only in a short aside: "Note that previous research [1] showed that out-of-order CPUs, while moderately faster, are not the best trade-off point for GC (a result we confirmed in preliminary simulations)." So they don't quantify what any of this means.

I think it's an interesting idea, but it doesn't bode well when they seemingly choose the wrong target for comparison and hand-wave away the difference as insignificant.

<comment><USER16> The comparison at least in the abstract is energy efficiency. It's quite likely that a small in order CPU is very good at chasing dependent pointers around the heap for its power consumption.

Imagine a linked list. Each pointer access is likely to miss to main memory, and no concurrency is possible. Both the highest and lowest end cores will sit around making a single request every 80ns.

They claim that the comparison was to the best alternative and I'd probably take them at their word barring any specific evidence.

</comment></comment><comment><USER17> > globally this represents a large amount of computing resources.

Much of which would just sit idle otherwise, on client machines. Of course, the energy savings still apply.

> He also points out that many garbage collection mechanisms can result in > unpredictable pauses, where the computer system stops for a brief moment to > clean up its memory.

This is more of a hard barrier that's being solved.

All in all pretty cool idea, but I think the impact would be different from what's discussed here. Truly high-performance computing is already written in non-GC languages. This hardware would give medium-intensity GC programs (read: web servers on JVM, .NET, Node, Ruby) a boost, and could also allow some higher-but-not-peak intensity software to be written with GC where it might not be today (games come to mind), although that could actually encourage _more_ energy usage than what it would save.

<comment><USER18> We wrote latency-sensitive and high-performance code in GCed languages back in 1980s - avoiding pauses or having predictable latencies (in fact, more predictable than usual manual memory management!) are more of "we don't teach people how to program" rather than issue with GC.

As for energy savings, many garbage collectors have amortized energy use lower than malloc/free. Even pretty simple ones (some of the simplest I've seen beat it so hard it's not competition, but they are specific to their application)

<comment><USER19> Could you explain further or give links to more information? I'd love to read about old-timey techniques for programming in GCed languages.

<comment><USER20> His point is it's not rocket science. Preallocate and pool what you might need, don't call new in your tight loop. Change the GC algorithm to something that never runs unexpectedly. If you're still allocating such that you need GC eventually, manually GC at an appropriate time like a load screen.

<comment><USER21> > Preallocate and pool what you might need, don't call new in your tight loop.

This piece of advice is also valid in languages without GC. In C++ it's well known one should avoid allocating memory in tight loops, and it's strongly advised to call e.g. `reserve` on vectors to avoid allocating.

<comment><USER18> The point is that you lose not only in tight loops. (Tight loop allocating with good GC is actually well known _good_ [1] case, it's deallocation that is PITA).

free() as well as malloc() and related have significantly complex algorithms behind them, unless one can simplify base memory management in certain ways - and those ways tend to also allow region-enabled GC to outperform them anyway.

[1] Allocating memory in many Garbage Collected systems can be down to one instruction, and if using per-thread heap, it can be atomic add without CAS

</comment></comment></comment></comment></comment></comment><comment><USER22> It reminds me of the days I was reading Knuth's quote "97% of the time premature optimisation blabla" every time someone was trying to make something faster.

CPUs are not getting faster, yet it seems using tools that makes things run faster are somehow taboo.

Wirth's law:

Wirth's law is an adage on computer performance which states that software is getting slower more rapidly than hardware becomes faster.

Why is java taught in university, and why is this language considered like some kind of standard? Most OSes are written in C, yet most of silicon valley frowns upon writing C because of arrays. Even C++ is getting a bad reputation.

<comment><USER23> > frowns upon writing C because of arrays

Do you mean horrifying security flaws?

<comment><USER22> You're right, but you cannot accuse the entire language of being insecure, ultimately it's the responsibility of the developer.

Also you can't always say we don't use C only for security reasons, as other languages also have their security issues. There are many modern ways to avoid those flaws. Like someone answered, it boils down to a matter of development cost.

I'm also quite skeptical when people always rise the objection of security when writing software. Security is not so simple, and so far it's its own specialty, and pretending that it's worth it to make things slower and that the security gain is actually there, is not really completely accurate.

Security is almost a post 9/11 paranoia knee jerk reaction.

</comment></comment></comment><comment><USER24> Objective-C ARC (automatic reference counting) solved the problem neatly for my iOS apps.

Is there some overhead? Maybe, but it's neatly spread out through the entire application life time, so there is rarely[1] a UI-freezing stutter associated with GC. To reduce the overhead I turned off thread-safety and simply never access the same objects from more than one thread (object has to be "handed off" first if it comes to that).

One wart on the body of ARC is KVO, which I avoid like a plague for many other reasons anyway.

The other wart is strong reference loops. This can be solved by the app developer by designing architecture around the "ownership" concept (owners use strong references to their ownees, all other links are weak references). This is a good idea in itself as it increases clarity of the program. I do make an occasional slip, which is where I need to rely on Instruments, and I do wish I had better tools than that, something more automatic that would catch me in the act. Maybe a crawler that looks for loops in strong references during the development process but is quiet in release builds. Or at least give me a pattern to follow that makes it easy to catch my errors. For example, we could assign a sequential number to each allocated object, and only higher-ranked object could strongly refer to lower-ranked object. This won't work for everyone but I wouldn't mind fitting my app to this mold if that gave me immediate error when I slip.

[1] if you release a few million objects all at once it may stutter for a second. Could be handed off to a parallel thread maybe.

<comment><USER25> ARC is just garbage collection that doesn't always work (circular references).

<comment><USER26> True! And ARC in Rust suffers from the same problem (note that while ARC in Rust and ARC in Swift are the _same thing,_ the "A" happens to stand for different words in each case).

<comment><USER27> They're not the same thing; Swift is "automatic" and Rust is "atomic"; Swift's ARC is implemented the same way as Rust's Arc, but Rust's is manual.

<comment><USER26> Hi! I think you may have repeated what I said in my comment, that the letter "a" stands for different things. Although they do the same thing: they are both atomic (in the sense of concurrency) and automatic (in the sense that you do not have to explicitly retain or release objects).

<comment><USER27> That's the thing: they're not automatic in Rust. You have to explicitly retain. (Release is automatic though.)

Furthermore, in Swift it's pervasive, and in Rust, it only happens for types that explicitly opt-in.

<comment><USER26> It sounds automatic to me. Maybe we disagree about what "automatic" is.

It also sounds like you are hunting for ways in which Swift and Rust are different. This is completely unnecessary! It turns out that I already know that they are different languages, and that the features do not map exactly 1:1 to each other. I hope next time you give me a little credit.

</comment></comment></comment></comment></comment></comment></comment><comment><USER28> I'm probably wrong, but didn't the Symbolics LISP machines have some kind of hardware support for GC?

I think for platforms like Android this makes a lot of sense. Should help quite a bit with battery consumption and responsiveness. Also makes sense for server loads in Java or Go.

<comment><USER29> A [https://en.wikipedia.org/wiki/Tagged_architecture](https://en.wikipedia.org/wiki/Tagged_architecture) yes. Tags make life a bit easier for the CPU when it accesses objects and is deciding what to do with them, but the CPU is still doing all the work and walk the RAM to do the GC. (I vaguely recall stories about Lisp machine users who would turn off GC while working, and then let it run when they left work and returned the next day.) The idea here seems to be to have an entire separate chip, a specialized CPU, which walks RAM independently of the 'main' CPUs, whose only task is freeing up memory.

<comment><USER30> There is a renaissance of that idea in ZGC [1].

You have a lot useless bits in a 64 bits pointers and thanks to the virtual memory, you can manage to have an untagged address and its corresponding tagged address referencing the same physical memory. This give you free bits that you can use to track the liveness/evacuation of a graph of objects.

[1] [https://wiki.openjdk.java.net/display/zgc/Main](https://wiki.openjdk.java.net/display/zgc/Main)

<comment><USER18> Lisp Machines didn't use tags for tracking liveness/evacuation of objects, though. They used them for safety, which automatically gave them _precise_ , as opposed to _conservative_ , GC which always knew whether it was dealing with a pointer. They also had special, CPU-handled type of forwarding pointers, which when accessed "normally" would transparently redirect you to forwarded location.

<comment><USER30> The forwarding pointer you are describing is equivalent to a ZGC colored pointer with the evacuation bit set that a GC barrier (a load barrier) will rewrite to the evacuation address.

and yes ZGC doesn't use colored pointer to track if a value is an integer or a pointer because Java unlike Lisp is typed so the VM derives those information from the bytecode.

<comment><USER31> Most Lisp implementations use typed memory via tags. Lisp doesn't have pointers, but references and usually knows if something is an integer or a reference - because it's encoded in the tags.

</comment></comment></comment></comment></comment></comment><comment><USER32> My day job is writing C for on an embedded real-time system. No manual memory management necessary... because we're forced to declare all struct and array sizes at compile time! Not a malloc or free in sight. Obviously, it's extremely limiting - pretty limiting as far as algorithms beyond "read data off bus, store in fixed array, perform numeric calculation, write back to bus." But I've gotta say, it's pretty freeing to write C in such a limited environment.

</comment><comment><USER33> From an environmental perspective, I wonder how much energy is consumed (and emissions generated) for garbage collection and interpreters. These things exist to make programming easier but are then duplicated across thousands of servers.

If everyone used some compiled language that was just a little simpler, a little safer, had just a little better memory management/tooling, or like here, had better hardware support, how much would that reduce global emissions caused by data centers?

<comment><USER34> Here's a benchmark that includes an energy comparison: [https://thenewstack.io/which-programming-languages-use- the-l...](https://thenewstack.io/which-programming-languages-use-the-least- electricity/) . It's interesting that speed does not directly correlate with energy consumption and the the energy consumption of functional languages was much higher than imperative.

<comment><USER35> That's more a measure of indirection than of garbage collection.

<comment><USER34> I don't disagree, but that indirection is necessary for the garbage collector, even ref counters usually add levels of indirection.

</comment></comment></comment></comment><comment><USER36> I'm skeptical; GC is closely tied to programming language run-times. How is some accelerator going to know which pointers in an object are references to other GC objects and which are non-GC-domain pointers (like handles to foreign objects and whatnot)? How does the accelerator handle weak references and finalization?

People aren't going to massively rewrite their language run-times to target a boutique GC accelerator.

<comment><USER37> I remember ruby had some approach with reusing previously allocated yet out of scope objects. I can very well imagine taking this concept above and beyond, and having virtually separated stacks for each type...

</comment></comment><comment><USER38> One talking point I'd like to ask is:

For small short lived scripts and applications, do we even need to free any memory these days? For example you write a script which takes several seconds to execute, moves files, computes stuff with strings, etc. Should we really invest time and effort in the script interpreter to free the memory, where instead we can just exit normally and let the OS handle the clean up.

I would imagine this kind of paradigm could be much faster to run because of less runtime work being performed. The allocator used could also be a simple linear allocator which just returns the next free address and increments the pointer. If using multiple threads there could be one per thread.

What do people think of this?

<comment><USER39> > For small short lived scripts and applications, do we even need to free any > memory these days? ..we can just exit normally and let the OS handle the > clean up.

That makes me think: why couldn't larger programs be composed of many such small, short-lived scripts/processes that give up all allocated memory upon exit?

I suppose there could be accumulated overhead for starting many such processes, and also the issue of allowing shared memory spaces that are explicitly _not_ automatically freed. I'm way out of my depth in this line of thinking though, so, just speculating.

<comment><USER40> That's essentially what happens if you use region-based allocation (aka arena allocation): [https://en.wikipedia.org/wiki/Region- based_memory_management](https://en.wikipedia.org/wiki/Region- based_memory_management)

</comment></comment></comment><comment><USER41> It does seem like just doing it in hardware may be a linear gain but isn't a fundamentally better algorithm. There's a proof that you do need to pause your program eventually, if you want to be sure you get all the garbage.

<comment><USER42> Hardware is fundamentally parallel, CPUs are fundamentally serial; it is possible for a hardware solution to have a super-linear speedup in time.

As a simple example, what is the time complexity of zeroing out n bytes of memory. With a CPU, this is O(n). However, with proper hardware support, this can be done in O(1) time.

For a simple garbage collecting example (no idea how their chip does it), consider a simple mark and sweep algorithm. Assume the chip has an internal object graph. At the begining of GC, only root nodes are tagged. At each step, the neighboor of every tagged node is tagged. With a CPU, this step takes at least Ω(n) time, where n is the number of tagged nodes. However, if this is done entirely by hardware, then (depending on how the hardware is designed) each node can independently look at its neighboors and complete a single step in just O(1) time.

Moving stuff to hardware gives you a set of primitives that is asymptotically different than the primitives you have on a general purpose CPU.

<comment><USER43> But hardware that has to interface with the main memory can't be fundamentally parallel, because of memory bandwidth limitations. If you want to make this _part of the memory_ , then my objection does not apply. But if it's an external chip to the memory, you still are fundamentally serial.

<comment><USER42> How much does it need to interface with main memory? If the external chip has its own memory then it can maintain its own view of the object graph. The software can update it when references are made/deleted and query it when an allocation needs to be made.

There is still the overhead of communicating when references are made/deleted, but you need that anyway, as something that only looks at memory doesn't know what is a pointer, or the size of objects.

The communication overhead is then linear in with respect to the amount of updates to the object graph, not the size of the object graph.

You could even go so far as to not put the chip on the memory bus at all.

<comment><USER43> I'm presuming some kind of a heap. That heap has something like a list of free blocks. Who keeps that list - the main memory, or this other chip? If the main memory does, then when you garbage collect blocks you have to change the main memory. If the chip keeps the list, then yes, the chip can do it all internally. But...

What about multitasking? If there are N programs running, and each has their own heap, the chip has to be able to keep track of each of them. Or it has to be able to keep track of one monster heap that uses almost all of the available memory, even if it's occupied by a bunch of small allocations. All this without using any (main) memory itself. This chip would have to have a large amount of onboard memory itself to pull that off. The very worst scenario would be to be a long way into a run of a program that thrashed the heap, and then the GC chip ran out of slots. You can't (easily) go back and re-do the heap to have the heap control in the main memory, and you can't continue with the chip managing the heap.

</comment></comment></comment></comment></comment><comment><USER44> So my idea for GC is to offload it to a separate machine through a communications channel. The main CPU sends messages to the co-processor whenever it allocates memory, or whenever it mutates (whenever it writes a pointer to allocated memory or to the root set- there could be special versions of the move instructions which send these messages as a side-effect). There is a hardware queue for these messages and the main processor stalls if it's full (if it's getting ahead of the co-processor).

The co-processor then maintains a reference graph any way it likes in its own memory. It determines when memory can be freed using any of the classic algorithms, and sends messages back to the main processor to indicate which memory regions can be freed.

This has some nice characteristics: the co-processor does not necessarily disturb the cache of the main processor (it can have its own memory). Garbage collection is transparent as long as the co-processor can process mutations at a faster rate than the main processor can produce them. The queue handles cases where the mutation rate is temporarily faster than this.

<comment><USER45> That would seem to eliminate all moving GC algorithms though.

<comment><USER46> Not really though you may want to. The main CPU could compact memory if it felt fragmentation was high enough to warrant it. By its nature the traffic is asynchronous so the CPU won't do a perfect compaction (it won't have processed all the inbound messages to mark garbage as free) but it would probably be close enough.

The big win here is making it all async. No need to stop the world, issue write barriers to application activity, or otherwise synchronize for GC. You'd have a bigger highwater mark for used memory but otherwise the main CPU can pickup the "the memory from x to y is now free" messages on any thread and whenever it is convenient.

<comment><USER45> Compacting without tracing can't really be done, and not all moving algorithms are collect and compact (though collect and compact seems to have won over other moving algorithms in the SMP erra)

</comment></comment></comment></comment><comment><USER47> You can usually precisely control garbage collection by turning it off or forcing it to run. The cognitive load to handle memory manually is not insignificant. If you control memory manually, you eventually end up designing some kind of mechanism like ref counting or something else to handle memory cleanup automatically. And there's significant reasoning that ref counting might not be the most desired solution for all use cases. Best is a combination of the ability to handle memory manually, with some more automated garbage collection when there's a need to write stuff that doesn't necessarily have to be the absolute fastest. Kitchen sink languages like C++ tend to have both and don't force the developer in either direction. Best would be to #define out 'new' and make manual memory handling explicit.

</comment><comment><USER48> The irony of the thing is that in manual memory management languages you end up doing your own garbage collectors and in garbage collector languages you end up doing your own manual management. Unfortunately if you look in a language to solve such complex problems you are heading straight to severe disappointment land. Same shit different package. I still prefer dynamic languages by a long margin because of their ability to do decent metaprogramming and reflection which is essential for managing any form of data. Pick your poison and enjoy the hype while it lasts.

</comment><comment><USER49> >> consumes a lot of computational power—up to 10 percent or more of the total time a CPU spends on an application.

I stopped reading there. 10% is nothing. For such a useful feature as automatic garbage collection, for the vast majority of applications, I'd gladly give away 50% of the CPU.

In terms of ensuring code correctness and robustness, if I had to choose static typing or automatic garbage collection, I'd pick garbage collection every time. It adds a lot of value in terms of development efficiency and code simplicity.

<comment><USER13> > for the vast majority of applications, I'd gladly give away 50% of the CPU.

Don't you think you're over-generalizing a bit much from your own circumstances? If these issues don't matter to you then you basically don't belong in this conversation. They matter a lot to people who write software that runs on large numbers of servers, with both CPU and memory utilization pushed to the limits. That's a lot of us. If you have a million machines, even a 0.1% improvement on either axis is _huge_. That's racks upon racks worth of equipment that doesn't need to be installed, powered, or maintained. If I could save 10% of 10% ("nothing" according to you) across such a large fleet, I'd be a hero and I'd be rewarded accordingly.

<comment><USER50> Don't forget game development, especially users of the Unity game engine, where their "aggressive" GC has often been the bane of projects such as Kerbal Space Program. I have been struggling lately with my own Unity GC issues, where it's running the GC every frame, subsequently dropping my FPS from the 90 I need for VR down to 50 every few frames. Even the new experimental GC they have implemented seems to have no effect.

<comment><USER18> Would probably help if they had constant-time GC available, where you bind mutator ("application code") and collector (GC) to specific quanta of time to run, thus providing reliable timings.

The question is whether it would be fast enough.

</comment></comment></comment></comment><comment><USER51> Want to get away from garbage collection, retain safety, but think Rust is too invasive? Try compile time reference counting: [http://aardappel.github.io/lobster/memory_management.html](http://aardappel.github.io/lobster/memory_management.html)

</comment><comment><USER52> While we're at it, how about we liberate CPUs and caches from communication between threads and cores?

<comment><USER53> Many ARM chips have this.

<comment><USER52> Any useful links? What are the terms I should search for?

<comment><USER53> It's called a "weak ordering memory model." Synchronization requires explicit memory fence instructions.

RISC-V, btw, supports both weak and strong memory models as an implementation choice.

</comment></comment></comment></comment><comment><USER54> > but the automated process that CPUs are tasked with consumes a lot of > computational power—up to 10 percent or more of the total time a CPU spends > on an application.

Is that even a problem when most CPUs are idle 90% of the time even when doing typical daily tasks?

<comment><USER55> It is a problem if you have a server process running that you want to be extremely responsive (latency <100ms) and that then suddenly decides to do garbage collection for a minute or two before answering incoming requests.

</comment></comment><comment><USER56> The Kiwi scientific accelerator uses a similar approach with FPGAs I believe: [https://www.cl.cam.ac.uk/~djg11/kiwi/](https://www.cl.cam.ac.uk/~djg11/kiwi/)

</comment><comment><USER57> Hmm so there's a coprocessor that does the GC... doesn't it need to lock the memory away from the main CPU while it does that? And doesn't this lead back to unpredictable pauses and slowdowns?

</comment><comment><USER58> I never understood the need for Garbage Collectors. In my opinion, the difficulties of memory management are extremely overrated. I write code in C/C++ for almost 20 years and I never encountered a difficult bug that would have been avoided with a Garbage Collector.

If a coder really has a hard time with manual memory management it means he can't really code, this is a beginner problem...

<comment><USER59> I only work in GCed languages, so I don't know how manual memory management works, except segfaults in some courses at university. Guess I should quit my job, as you say I apparently can't really code :/ Thanks for letting me and others here know!

<comment><USER60> In my experience GC makes programmers sloppy in their resource usage. Just allocate a bunch of memory annnd... whatever, the GC will take care of that. But there are a lot of other resources besides memory that aren't automatically cleaned up like that. So what happens is people forget to close network sockets, forget to unsubscribe event handlers, forget to set certain references to null to actually allow the GC to do its work, etcetera... The existence and over-reliance on GCs has led to a mindset where many programmers are just not aware that what you create must also be destroyed at some point.

</comment></comment></comment><comment><USER61> Maybe I'm naive, but with multi-core CPUs, and parallel GCs, isn't it somehow the same? One core is mostly only used for GC, while the others do other things?

Edit: I guess they mention their chip itself can do it at a high level of parallelism, so that's probably one more advantage. But CPUs with additional slower cores and a lot more cores are in the works as well.

<comment><USER62> But the GC has to do it in a thread safe way which involves locking/synchronization. Otherwise you get nasty race conditions.

<comment><USER63> There's no need to lock or synchronize if you're working in a language with immutability and copy on write first class data structures.

</comment></comment></comment><comment><USER64> Hasn't Rust basically solved that problem?

But yeah, legacy stuff could profit from this.

<comment><USER26> No, because it is in general intractable to figure out object lifetime at compile time. Rust has just solved the problem for more cases than, say, C++ does, or perhaps just with more rigor.

</comment></comment><comment><USER65> Could this be applied to Chrome V8 for Javascript memory garbage collection?

<comment><USER66> That should be an ideal application. Usually the trouble with 3rd-party garbage collection is that it has to discern pointer vs. any other machine word. That's more of a problem with C; this is why the Boehm GC library calls itself "conservative". A runtime like V8 can follow a spec when it allocates memory so everything is properly marked.

</comment></comment><comment><USER67> I don't care about being absolutely fast when writing code. The convienience of not having to care about memory management is far more important to me. That's why I like GCs.

</comment><comment><USER68> Didn't the old Lisp machines also do this?

</comment><comment><USER69> Seems like we could just as easily stop using garbage collection. ... or even go back to reference counting / smart pointers and just live with the "limitation" that we can't have circular references.

<comment><USER70> doesn't seem cool to admire Apple technologies, but ARC seems to work amazingly well with zero CPU overhead

<comment><USER71> It's not at all zero CPU overhead, not even close. retain & release are thread-safe, meaning atomic ref count. Very comparable in cost to std::shared_ptr<T> or Rust's Arc<T>. Both of which also automatically insert the calls to inc & dec ref counts.

It's cool that you don't need to bother with specifying the type as being std::shared_ptr<T> or Arc<T>, but it's not particularly novel, either. It's "just" syntax sugar (or lack of syntax sugar I guess?)

<comment><USER72> Almost but not quite. Clang has very special rules around ARC that allow it to perform additional optimizations that would otherwise be illegal [1].

[1] [https://clang.llvm.org/docs/AutomaticReferenceCounting.html#...](https://clang.llvm.org/docs/AutomaticReferenceCounting.html#arc- optimization)

<comment><USER71> That just lets it release earlier, not retain/release less often.

ARC can't do anything magic here vs. something like really careful use of std::move & const references.

<comment><USER73> ARC optimizations do let it retain/release less often. For example, instead of a callee adding the return value to an autorelease pool then having the caller immediately retain the returned value, the autorelease+retain pair can be elided entirely.

<comment><USER71> > For example, instead of a callee adding the return value to an autorelease > pool then having the caller immediately retain the returned value, the > autorelease+retain pair can be elided entirely.

Are you just referring to standard RVO? If you return a std::shared_ptr in C++ today you won't get the equivalent of retain+release, either, RVO avoids that.

<comment><USER72> Did you even take a look at the doc I posted?

> For example, if the user puts logging statements in retain, they should not > be surprised if those statements are executed more or less often depending > on optimization settings

> In general, ARC does not perform retain or release operations when simply > using a retainable object pointer as an operand within an expression

> However, C and C++ already call this undefined behavior because the > evaluations are unsequenced, and ARC simply exploits that here to avoid > needing to retain arguments across a large number of calls.

> ARC performs no extra mandatory work on the caller side, although it may > elect to do something to shorten the lifetime of the returned value.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER74> In Java, I created thread local resource pools including strings which eliminate garbage collection in sensitive routines. Of course it's much faster in java to perform pooled string comparison with ==. Likewise I always use the indexed version of a for loop to avoid the iterator otherwise allocated.

GC in java is great for non-priority code which is most of the application.

</comment><comment><USER75> Ok, so saving 15 percent of 10 percent of power use by changing both how we build processors and how we write software. Doesn't seem worth it.

<comment><USER76> Seems worth it if the software developed is a fraction of all software written. Sure your VM/runtime will have to be refactored, but not the millions of programs that run on it.</comment></comment>


Is Shark Tank with Mark Cuban a realistic depiction of angels & VCs? - <AUTHOR> http://www.chubbybrain.com/blog/shark-tank-tv-reality-angel-investor-venture-capitalist/

<comment><USER2> There's only one Shark Tank and that's HP Pavilion where the San Jose Sharks play. [https://secure.wikimedia.org/wikipedia/en/wiki/HP_Pavilion_a...](https://secure.wikimedia.org/wikipedia/en/wiki/HP_Pavilion_at_San_Jose)</comment>


Issues in the Proof that P ≠ NP - <AUTHOR> http://rjlipton.wordpress.com/2010/08/09/issues-in-the-proof-that-p%e2%89%a0np <comment><USER2> _Head spinning._ At least I got that bit about LFP... mostly.

The coolest thing about this all is watching the Internet doing what it was built for. Real-time collaboration on a massive scale. The whole of the relevant scientific communities all in one virtual space clamoring at a hundred virtual whiteboards. I love living in the future.

<comment><USER3> I haven't looked at the proof in any kind of detail.

But comments by Jame S. Gates etc. raise a question in my mind. Any statical argument that a given phenomena is unlikely is based on the space of phenomena behaving very roughly "communatively" - AB and BA equally likely or at least correlated.

How do you apply that kind of reasoning to a totally arbitrary process of computation, a process which wouldn't necessarily have respect any kind of randomness? It seem like any statistical argument for P=/NP has to have a detailed and direct discussion of how you can escape this problem.

Anyway, that's my crude effort to translate the discussion into my mere math MA understanding...

<comment><USER4> Probabilistic arguments for deterministic systems are standard practice. Search for Erdős and probabilistic methods to see several examples in combinatorics.

In computation theory many proofs have the tagline "...with probability 1", meaning the author enumerated all desired possibilities and showed their probabilities summed to one. Most people accept this as proof, although it is less preferred, in the way that proof by contradiction is less favored than direct proof.

</comment></comment></comment><comment><USER5> Could it be that the million dollar millennium prizes are actually slowing down the resolution of important problems because they discourage crowd sourcing proofs?

<comment> j-g-faustus I may be out of tune with many others here, but I think certain constructive processes (as in creating a many-faceted proof, outlining overarching software architecture, making the storyline of a good novel/movie) are most efficient with small groups of people. Otherwise you run into issues of "design by committee" or "too many cooks".

It is easier to crowdsource once the main outline is in place, it structures a vague problem into a set of more specific subtasks that can (more or less) easily be distributed across many people.

Like in this case: Deconstructing an existing proof can work very well with crowdsourcing, since a proof contains a relatively small set of specific claims, and each "crowd" contributor can focus on one particular issue.

But this works because the crowd now has a common focus point and task list created by the paper being published. It's not clear to me how (or whether) the process could be reversed, that the same crowd and effort could be coordinated to cowrite an original paper with an alternative proof.

<comment><USER3> I'd agree that crowd sourcing might not always be the best way to approach things. But the general situation that the person who gets the final result _wins_ very likely does inhibit effective work on the solution of very difficult problems.

I mean, I think there's a consensus that if Wiles had been publishing his results as he went along, Fermat would have been solved earlier but probably not by Wiles himself. Whether the current N/NP proof is right or not, it also is clearly the product of long work in a vacuum. That's not the best way to do things if nothing else as a matter of sanity. If you're publishing as you go along, you've got a lot more of a sanity check. Further, if research is open, you can do single person, small-committee and crowd-sourced versions.

Still, I don't think this primarily a matter of the Millennium Prizes in particular but of the tremendous competition of academia in general - Wiles' effort happened before the Millennium Prizes - and he couldn't get Fields Medal either - too old.

Oddly enough, the Netflix prize actually seems to have produced a fusion of efforts in the end. Perhaps future prize creators could think about that. Both the Millennium Prize and the Fields Medal seem deeply problematic in their effect on mathematics in general.

<comment> j-g-faustus Good points, and I agree.

The Netflix prize is interesting. According to a summary post on the Netflix forum

        the early results were mainly by individuals...
        the team members began to coalesce and combine, and
        in the end, entire teams coalesced and recombined.

[http://www.netflixprize.com//community/viewtopic.php?pid=961...](http://www.netflixprize.com//community/viewtopic.php?pid=9616#p9616)

I can only speculate that the combination of a deadline, a problem that was too hard for any individual and a common forum for exchanging ideas helped encourage team formation. (I.e when you realize that you can't do this alone, teaming up is a rational thing to do.)

I assume that the problems we (as in society, humanity) want to tackle will grow ever more complex in the future, and may easily outgrow the capacity of individuals.

So yes, perhaps we could/should learn from how the Netflix prize played out...

<comment><USER6> I think some risk-averseness drove the combination as well: at the very end, two teams with very similar performance were looking at it being more or less random chance who'd finish a hair ahead, with one getting $0 and the other getting $1m. So they merged and split a much safer $500k each.

</comment></comment></comment></comment></comment><comment><USER7> It seems like a bit much to hope to understand and verify the proof without a huge investment of time and effort. The problem is exponentially compounded if you don't already do research in theoretical computer science and have the appropriate logic background.

But! All of this has put a spotlight on a theorem that I had somehow never appreciated. It is a beautiful theorem that is well within reach of a mathematically-literate computer scientist: P = FO(LFP) over finite ordered structures. If you start reading up on this, it's immediately clear that this is interesting, because FO(LFP) say nothing a priori about computational resources in the sense that P does.

Anyway, a sufficiently curious reader can look it up as theorem 4.10 in the following text:

[http://www.amazon.com/Descriptive-Complexity-Texts- Computer-...](http://www.amazon.com/Descriptive-Complexity-Texts-Computer- Science/dp/0387986006)

That book is pretty light on the basics of logic. I used this book:

[http://www.amazon.com/Mathematical-Logic-Undergraduate- Texts...](http://www.amazon.com/Mathematical-Logic-Undergraduate-Texts- Mathematics/dp/0387942580)

</comment><comment><USER8> Learn to use the DOM instead of HTML strings, and you'll almost never run into those XSS problems. You can replace all of your HTML sanitization functions with document.createTextNode(text).

<comment><USER9> I think you're looking for a different thread...

<comment><USER8> Oops, I meant to reply in the thread about that node.js chat application full of XSS holes.</comment></comment></comment>


Ask HN: When is a startup not a startup? - <AUTHOR>

How do you know that a startup has come out of the startup phase? More importantly, how does a startup know that its not longer a startup?
a) time since it was established b) making sales c) hiring more people
No single criteria seems to yield the correct answer. It may take a startup years to launch their product, there are companies that make sales, do hirings and even make money yet classified as startups. <comment><USER2> when its ajar?

</comment><comment><USER3> when you feel like your not.</comment>


Keeping PACER: Service offers a better way to search federal court records - <AUTHOR> http://www.abajournal.com/magazine/article/service_offers_a_better_way_to_search_federal_court_records_than_pacer/

<comment><AUTHOR> And of course they forgot about existing sites, like PlainSite, Justia, etc.</comment>


"If you are not paying for it..." - <AUTHOR> http://www.metafilter.com/95152/Userdriven-discontent#3256046 <comment><USER2> We've created a culture where we want everything for free, even going as far as to block ads with AdBlock... and then we complain when things don't go our way.

This can't be sustained forever. If we want legitimate content producers to take the Internet seriously as a platform, we need to start forking over money.

Edit: To anyone replying to me, we can debate this forever. The Chris Anderson v Malcolm Gladwell debate proved it's a controversial topic. However, I don't know where you're getting "close to zero." A good engineer or other employee is at least $100k a year. It may cost almost nothing for new headlines to show up on Digg every day, however Diggs 100 or so employees all need to put food on the table somehow.

<comment><USER3> "This can't be sustained forever. If we want legitimate content producers to take the Internet seriously as a platform, we need to start forking over money."

Right now there's only a tiny trickle of good content being created on the Internet. Of the hundreds of thousands of posts being written each day, there are maybe 5 at best that don't completely suck. What we really need is a system where the authors of the best 25 articles of the day each take home 100K or so. For the nominal cost of less than a billion dollars a year we could spark an intellectual renaissance to rival any other in human history.

<comment><USER4> "Of the hundreds of thousands of posts being written each day, there are maybe 5 at best that don't completely suck."

I'm sorry, but that's just silly. 1) You're not reading the whole internet, 2) you're not the intended audience for much of what's written, and 3) opinions vary.

Are you proposing we replace the free market of authors and readers with some central judging authority? How would these people filter through all this content, on what basis would they judge it, and how would they be funded? How do we protected against government meddling and political bias?

This is a pretty big problem you're creating to replace a system which, basically, seems to work fine. It would be nice to help creators make money, but you can't fix human taste.

<comment><USER3> "Are you proposing we replace the free market of authors and readers"

We don't have a free market of authors and readers. Readers don't currently pay authors, this is what needs to change.

"How would these people filter through all this content, on what basis would they judge it, and how would they be funded? How do we protected against government meddling and political bias?"

Not sure.

"This is a pretty big problem you're creating to replace a system which, basically, seems to work fine."

I don't think it works fine at all, judging by the fact that all of our biggest problems are marketing problems. In any given area of society the science is between 25 and 100 years ahead of the status quo.

<comment><USER4> "We don't have a free market of authors and readers. Readers don't currently pay authors, this is what needs to change."

OK. But 1) readers are free to reward authors IF they want to, and 2) authors can potentially make money from their readership by selling ads and t-shirts, getting speaking gigs and job offers and holding conferences, etc. So there's freedom, and there's a market. If people don't value something enough to pay for it, you can't force them to value it. They have a right to their own opinions.

Sure, 99% of authors make zip. But I bet a larger percentage of the population makes money creating stuff on the web than they did when you had to have a publishing deal to publish anything.

My point with "how would they be funded" was this: either it's voluntary or it's not. If you want to create a voluntary system, great - try to market some kind of micropayments thing or whatever. But if a committee is making the decisions about what's good, and I have no input, the only way I can see for them to get my money to put into the prize pot is to tax me. And I'm opposed to that, both because it's my money and because I distrust this hypothetical committee. What enlightened people do you imagine should be officially blessing The Best Of The Web? I bet my list is different from yours.

Oh! I know! Let's have lots of committees! In fact, let's all be our own committee! Freedom of thought and whatnot!

You see where I end back up?

"I don't think it works fine at all, judging by the fact that all of our biggest problems are marketing problems."

If you define 'people mistreat their neighbors for personal gain' as a marketing problem, maybe. I think it's a lot deeper than that. I'd call it a spiritual problem.

"In any given area of society the science is between 25 and 100 years ahead of the status quo."

Not sure what you mean by this, but science is a tool, not a solution. Psychology, for example, can help you understand what motivates people and how to influence them, but it can't tell you what SHOULD motivate people and how they SHOULD behave. It can as easily be used for evil as for good. Casinos use plenty of science to drain their victims of every last nickel.

</comment></comment></comment></comment></comment><comment><USER5> Sometimes even if you _are_ paying for it, if someone else is paying more. For example, newspaper subscribers are closer to being products than customers, since ad revenue is typically much larger than subscription revenue.

<comment><USER6> Very true. As I've been studying this topic lately I can even provide you some hard data.

A financially stable newspaper gets around 75-80% of its gross revenue from advertisements. That's why advertisers have so much power over content, even news content. I don't know the exact figure for magazines, and it could very well be lower, but this should give you an idea.

In fact if you check the literature you'll see that a common idea is that newspapers are in the business of selling customers to advertisers, and they will try to attract a certain type of reader in order to get certain type of advertisers.

<comment><USER7> > A financially stable newspaper gets around 75-80% of its gross revenue from > advertisements.

In the glory days, a solid 30-40% of revenue was from classified ads, which were _way_ overpriced, but the only game in town.

You may find this of interest: [http://newsosaur.blogspot.com/2010/09/newspaper-ad-sales- hea...](http://newsosaur.blogspot.com/2010/09/newspaper-ad-sales-head- to-25-year-low.html)

</comment></comment></comment><comment><USER8> This needs to be memorialized. Understanding this, of course, explains Google's approach to customer, er, consumer service.

Much to their nonpaying consumers' chagrin, naturally. Because consumers' support and service expectation does scale with the price they pay, but not all the way to zero!

On the business side, nonpaying consumers are cheaper to scale up than any product you can sell. Which is a blessing and a curse...

<comment><USER9> The guy who said it is Andrew Lewis. Might I suggest "Lewis's Law"?

<comment><USER8> I would've thought there would be a legitimate claim on the name already, but it appears not. Lewis's Law it is!

</comment></comment></comment><comment><USER10> Even if you _are_ paying for it you might be the product being sold. Consider hip night club or a star trek convention, in both cases people pay to get in, but the crowd is really the main attraction. People spend hours getting ready for these things. I'm told.

Any crowd is going to be part of the product being sold and the market you sell into. Any crowd, online or off, is going to draw performers, whether for free or professional.

People like to be seen, they like to be noticed, they like to contribute. I don't see user-generated sites as some kind of slave mine. Most people are online of their own free will.

<comment><USER11> Amended: "If you're not paying what it's worth, ..."

<comment><USER12> Apple announces on stage as part of its keynotes how many credit cards it has on file. In other words they're selling _you_ right there in the open. Can you really argue that you're getting your contract AT&T iPhone for less than it's worth?

<comment><USER6> But Apple cannot sell your credit card number to other people, they still need to get you to pay for it. Advertisement is different.

Advertisers pay the newspaper immediately in order to appear on the newspaper/magazine. And because they pay so much more than people buying the actual newspaper/magazine, it can be a great business for the later to attract more people... even going as far as giving the newspaper away for free.

Apple (pre iAds) was making money from you buying the phone: compared to that making money from apps sold is peanuts. So much so that it can be beneficial to let app developers in for free just to get more people buying the phone.

To summarise: Apple tries to attract customers with apps, newspapers try to attract advertisers with customers. Customers don't buy more newspapers because there are more ads, but advertisers buy more ads space if there are more customers.

</comment></comment></comment></comment><comment><USER13> That seems like an overly simplistic portrayal to me. You might not pay anything for using Google but that doesn't mean that Google can do whatever they want.

The relationship between Google and yourself wouldn't change much if they did away with the ads and charged you money. They want to keep you happy whether you are a customer (i.e. directly pay them money) or not. Paying customer or ad target, it doesn't really matter all that much.

Two sided markets with their network effects are certainly fascinating [+] but they are also nothing new, nothing extraordinary and certainly not abusive.

[+] It is possible to get negative prices (whenever you get anything for free you can immediately start to look for the two sided market and network effects) and prices that would otherwise be above monopoly prices.

</comment><comment><USER14> ...you're either the product being sold or you're enjoying a publically provided service whose costs are met by the taxpaying base.

<comment><USER15> i.e. You.

</comment></comment><comment><USER16> This is negative news and media is biased toward negative stuff. Moreover, it is a mention of a few well known companies and sites.

I hypothesize that the players are merely changing and that their strategy of getting large userbase fail because the quality of content is falling, rather than rising, thus decreasing the traffic quality.

This is also in conjuncture with the hypothesis that specialized interests tend to convert better than general social interests because specialized interests tend to have product related items they like to buy.

<comment><USER17> I think you're responding to the article, when the post links to a comment by blue_beetle, FYI.</comment></comment>


Predicting A Future Free Of Dollar Bills - <AUTHOR> http://techcrunch.com/2014/07/12/money-at-our-fingertips/ <comment><USER2> The whole time I was reading this all I could think was "man, this reads like a bad advertisement."

Then I noticed the disclaimer (horribly placed; I skipped over it the first time because it looked like a caption text for the banner image): _Editor's note: Simon Black is CEO at London-based Sage Pay._

edit: So, this is a long-form advertisement for a payment processing company. Why do people still read tech crunch?

Also, biometrics for payment? Christ, because payment processing hardware/support/service isn't expensive enough already! And has the past decade of massive privacy violations by both the private and public sectors taught us nothing?

<comment><USER3> Quite. Additionally:

"...will also make fraud virtually impossible..."

I'm pretty sure this line is trotted out every time a new payment technology is introduced. But one that's tied to some irrevocable, personally-assigned 'key'... yup: nothing could possibly go wrong there!

</comment></comment><comment><USER4> > Why go to the hassle of carrying loose change when you can swipe a card to > make a purchase within seconds?

\- Because cash is anonymous, and card transaction violate my freedom.

\- Because credit card companies charge 2-4%. And therefore raise the prices for everyone.

\- Because only cash is legal tender.

> A further benefit for us is that it will give us peace of mind as there will > be less concern over having money stolen.

I one steels my purse, I only lose the cash I have at hand. But if one steels my credit card, they can rob my complete bank account. Also credit cards are insecure by design, compared to EuroCheck cards. You do mot even to remember a 4 digit PIN to use them. The card number and expiration date is enough to rob me.

<comment><USER5> "Because credit card companies charge 2-4%. And therefore raise the prices for everyone."

And you think those armoured vehicles that transport the cash away from the shops do it for free?

<comment><USER6> Many small businesses do not use these services due to that cost. I know many people who run small stores that casually walk a couple of thousand dollars to the bank after a good day.

Hell, I used to work at a pub that had it's staff carry over $100k to the bank every Monday morning, though that amount is arguably grossly irresponsible both from a business perspective and with regards to staff safety.

</comment></comment></comment><comment><USER7> I hope cash will last for 50 years or so (at looking at how legacy systems are maintained it is realistic). One less point of identification - one less point of attack on a person. And I'm not even paranoid, massive spying on people using metadata and selling it to the highest bidder is already real today.

And situation of leaving stuff due to lack of cash is actually good for majority of people - you either a) monitor for finances before starting activity that involves payments, and then you do this regularly everywhere. Or b) you learn to discard and don't buy most useless products on the market. It's a win-win.

<comment><USER8> _And situation of leaving stuff due to lack of cash is actually good for majority of people_

I agree. My significant other is very poor at handling money but there are so many things companies and banks do to make it easy for them to spend money in a cash-free way. Overdrafts, clothes stores offering a 'card' which give you a discount but then lets you run up a balance, and similar bullshit. It's gotten so bad I'm going to get them a chargecard they put money on at the start of the money and then use like cash..

</comment></comment><comment><USER9> From the article:

"By removing cash, you reduce the chances of becoming a target of crime"

Reality:

"By removing cash, you increase the chances of becoming a tax criminal"

My suspicion is everyone will be a trivially detected tax criminal by design, but if you just vote the right way and keep your head down and avoid offending anyone in power and stay out of politics unless invited, that punishment will never be unleashed. It'll be a form of control. I can do without increased .gov control, we have enough, thanks.

Furthermore I predict we'll pretend the database is closed and private even though it'll be wide open to all .gov and by extension .com. Go to the wrong person's yard sale (perhaps an ex employee), HR will make sure you never get an interview, etc.

</comment><comment><USER10> This approach - giving your identity to prove you're going to pay for the item - seems overly careless in the age of privacy violations. Another system - where you can only give the proof, without the identity - would be better, but it's unclear at the moment how that could be reached.

Besides, putting an item on the shelf in case you don't have enough cash has another nice feature - an opportunity to think twice if you really need that item in the first place. If you pay too easily, it's harder to save.

</comment><comment><USER11> I hope the author is correct, at least for more advanced forms of payment by plastic. I'm skeptical for the US though. Contactless cards are hard to come by and several issuers stopped sending them even though the terminals are present. We don't even have widely-available chip-and-PIN (and the few issuers of those cards almost entirely make them default to chip-and-signature).

<comment><USER8> Most cards in the UK are contactless now but I can genuinely say I've never seen anyone ever make a contactless transaction. I have tried myself a few times but it failed the first time (operator had no idea why) and the other times it felt more awkward and took more time than doing it the "standard" way.

I think we'll get there with it, but socially it's currently about as popular as the self-check out line back in 2001 or the self-ordering kiosk at KFC now (which I always use to jump the line while everyone else is standing around.. seriously, why?)

<comment><USER12> It's definitely a matter of getting used to it. I moved from Canada where contactless has been standard for a couple of years now and being unable to get a contactless UK card for now I'm super annoyed every time I have to punch in PINs to pay £1 - £3 for something small. The extra 30 seconds just feel totally unnecessary.

<comment><USER13> Why not just use cash? Unlike in the US and Canada, the coinage is actually worth enough to buy real things with pocket change.

<comment><USER12> A few coins are much heavier and thicker than a card, for one. Just tapping a card is faster than picking out coins then waiting for change. And when paying with card I know I won't get any two pence coins back.

</comment></comment></comment></comment></comment><comment><USER14> I'm hoping for the return of coins. In Norway, I hear that they are planning to introduce a 50kr (equivalent of €5) coin.

</comment><comment><USER15> That would be one way to push the poor, the homeless and those who want privacy even further to the margins of society.

</comment><comment><USER12> The headline sounds great - I much prefer dollar coins

<comment><USER16> From financial perspective dollar coins make sense, as they are cheaper. But from the weight in my pocket, I much prefer dollar bills.

</comment></comment><comment><USER17> Is it true that 1% of people (by any large geographic measure) have used Bitcoin in the last month?</comment>


Author of LuaJIT explains why compilers can't beat hand-coded assembly - <AUTHOR> http://article.gmane.org/gmane.comp.lang.lua.general/75426 <comment><AUTHOR> In the "BareMetal is a 64-bit OS for x86-64 based computers" comment thread (<http://news.ycombinator.com/item?id=2587640>) haberman linked to this email (<http://news.ycombinator.com/item?id=2588170>), and I thought it was interesting enough to garner its own submission.

<comment><USER2> Definitely. Thanks for posting this, I wish HN still regularly had more hard tech content like this.

<comment><AUTHOR> You're welcome. The thanks should go to haberman, though, who not only linked to the email but apparently also asked the questions that led to such a great explanation.

Reading it really made me want to do some assembly programming.

<comment><USER3> I used this info as the basis of writing some assembly of my own: [http://blog.reverberate.org/2011/04/25/upb-status-and- prelim...](http://blog.reverberate.org/2011/04/25/upb-status-and-preliminary- performance-numbers/)

DynASM, which I used to write my JIT (also written by LuaJIT's author Mike Pall) is also an impressive piece of work. It makes writing a JIT so pleasant that I meant to write an article walking through the process of building a simple JIT. I was going to implement a JIT for the "Universal Machine" from ICFP 2006 as the example (<http://www.boundvariable.org/>), which is an absolutely delightful problem, but unfortunately discovered that it is heavily self-modifying, and thus would be hard to get a big performance boost from. But it looks like someone did it anyway: <http://paul.bluereboot.net/um/>

<comment><USER4> _It makes writing a JIT so pleasant that I meant to write an article walking through the process of building a simple JIT._

Please do, if you have the time. DynASM really is a stroke of genius.

The way it works is that the JIT contains bits of pre-generated machine code, and then a little bit of bytecode stitches these bits together and emits them. This means the actual runtime piece of the JIT doesn't need to know all the wacky details of x86 instruction encoding. All that is done in a preprocessor written in Lua.

I never got around to finding out how to do dynamic register allocation though. It seemed to me that there was no way to parameterize the pre- generated machine code, but presumably I'm missing something.

</comment></comment></comment></comment></comment><comment><USER5> I enjoyed this article, I always find the argument that "these days you can't beat the compiler" fishy. I remember more than 25 years ago I wrote an 8080 machine code interpreter in 8088/86 assembly. My innermost interpreter comprised 3 lines of asm;

      lodsb     ;al = *si++
      mov ah,al ;eg if al=3eh (=mvi a,dd), then ax=0x3e3e
      jmp ax    ;eg goto 0x3e3e. Preloader puts the emulation
                ; code for mvi a,dd followed by another copy
                ; of the inner interpreter at cs:0x3e3e.

This approach uses 64K to sparsely store the emulation code for each of the 256 opcodes at 0x0000,0x0101,0x0202 ... etc. which seemed a pretty outrageous waste of memory at the time. But it was worth it because none of the 3 inner interpreter instructions modified the flags, which meant I could use the native 8086 flags to directly emulate the 8080 flags, a huge speed win. Remember that the 8086 had LAHF and SAHF instructions to help solve this very problem, yet my approach avoiding the whole flags issue was still twice as quick as commercial equivalents.

Okay admittedly this is all irrelevant dinosaur nostalgia from the perspective of modern technology, but it reinforces the point that there's no substitute for human creativity when it comes to real hardcore optimization tricks.

<comment><USER6> > I enjoyed this article, I always find the argument that "these days you > can't beat the compiler" fishy.

I enjoyed your example; very clever. I think the "these days you can't beat the compiler" should really be understood to carry a qualifier like "for compiling things that we understand well enough and that occur often enough that it's worth teaching the compiler how to optimize them". Knuth's maxim "Science is what we understand well enough to explain to a computer. Art is everything else we do." seems particularly apt here.

Problems where humans can apply very domain-specific knowledge to come up with clever solutions are not the sort of problems that a general-purpose compiler is going to generate good code for. Knowing when you're dealing with one of those problems (e.g. writing inner loops for multi-precision arithmetic or vectorizing multimedia codecs) is part of being a good programmer. But to a first or even second approximation, you are not dealing with one of those problems, so you might as well just sit back and let the compiler do its job.

<comment><USER5> I was with you until "but to a first or even second approximation, you are not dealing with one of those problems, so ...". Perhaps there is a typo there and it should have been something like "but _when_ to a first or even...."

</comment></comment></comment><comment><USER7> A great find. And some of this argument is very simple even if you don't do a lot of assembly programming; for instance:

* that C-code interpreters force the compiler to do register allocation across the slow paths of all instructions, even though the interpreter dev can in this case definitely predict which things need to be in registers;

* and, the optimizer in the compiler makes it hard to tune the code for the 20% that does 80% of the work, where an assembly program can hoist the slow path code out of the I-cache entirely.

<comment><USER8> Dunno. I've found with __builtin_expect() in GCC, the code corresponding to an unlikely branch is placed at the end of the interpreter body, even after all the other instructions, which is probably as good as you can do icache-wise. I'd have to check again, but such annotations could inform register allocation as well.

<comment><USER9> I was thinking about this, and I think you're mostly right. There's a bit more that needs to be done, but some sort of expect builtin (name varies based on compiler) would help.

What I'm thinking is, he needs to factor all the instructions out into separate functions so that he stops hitting the optimization boundaries (or he may be able to override them with parameters to the compiler). Then, mark the hot paths with the expect builting.

One that's in, use profile guided optimization (again name varies based on compiler), to feed the compiler with more frequency information on the branches. Partial inlining (not sure if GCC has this opt yet) should take care of the rest.

As for the register assignment, well that's hard to get right while getting the instruction scheduling right at the same time. Profile guided optimization will help guide the global register allocator in the right direction, but it likely won't be perfect. Thankfully, on modern architectures, register to register moves are pretty much NOPs thanks to register files, short circuting, and out of order.

</comment></comment></comment><comment><USER10> To be clear, he qualifies this with the statement

 _Final words: A modern compiler contains a huge set of heuristics that interact with each other in strange ways. They have been tuned for 'average' code. But an interpreter is a very different beast, so you'll inevitably get disappointing results._

More info in this reddit comment: [http://www.reddit.com/r/programming/comments/badl2/luajit_2_...](http://www.reddit.com/r/programming/comments/badl2/luajit_2_beta_3_is_out_support_both_x32_x64/c0lrus0)

<comment><USER11> What if someone developed pragmas and a compiler that could use that information with a different set of heuristics?

<comment><USER12> I wonder how close you'd get to that by running the interpreter with gcc's or llvm's profiling on, and then recompiling with profile-guided optimization. If it's just a matter of compilers having heuristics tuned towards the wrong kind of workload, PGO ought to re-tune them appropriately, if enough of the heuristics in question actually look at the profiling data.

<comment><USER13> I was thinking this too - you would think that profile guided optimisation ought to be able to identify the slow paths.

</comment></comment></comment></comment><comment><USER14> A couple of years ago I wrote a direct-threaded .NET interpreter. This article made me go and have a look at the assembly that is being generated for the simple 'add' instruction case.

The .NET IL is stack-based which the interpreter runs directly, so it will never be as efficient as LuaJits register-based approach, but I was somewhat disappointed to see this as the output of the Windows VS2008 C compiler in release mode (enables all optimizations):

      JIT_ADD_I32I32_start:
      	BINARY_OP(I32, I32, I32, +);
      0041164D  mov         eax,dword ptr [pCurEvalStack]
      00411650  sub         eax,4
      00411653  mov         dword ptr [pCurEvalStack],eax
      00411656  mov         ecx,dword ptr [pCurEvalStack]
      00411659  mov         edx,dword ptr [ecx-4]
      0041165C  mov         eax,dword ptr [pCurEvalStack]
      0041165F  add         edx,dword ptr [eax]
      00411661  mov         ecx,dword ptr [pCurEvalStack]
      00411664  mov         dword ptr [ecx-4],edx
      JIT_ADD_I32I32_end:
      	GO_NEXT();
      00411667  mov         edi,dword ptr [pCurOp]
      0041166A  add         edi,4
      0041166D  mov         dword ptr [pCurOp],edi
      00411670  jmp         dword ptr [edi-4]

The assembly after 'BINARY_OP(...)' is the integer add code. pCurEvalStack points to the top of the current evaluation stack. Binary ops are performed on the top two items on the stack, and pushing the result back onto the stack.

The assembly after 'GO_NEXT()' is the standard epilogue, which just dispatches the next instruction (see <http://en.wikipedia.org/wiki/Threaded_code#Direct_threading> for description of direct-threading).

There are 3 seperate loads of the same memory address, which surely can be done better.

I'll probably have a go at altering the definition of the BINARY_OP(...) macro to see if I can persuade it to generate better code. I'm not keen to hand- write the assembly for this as it can compile to multiple processors.

The macro is currently defined as:

      #define BINARY_OP(returnType, type1, type2, op) \
      	pCurEvalStack -= sizeof(type1) + sizeof(type2) - sizeof(returnType); \
      	*(returnType*)(pCurEvalStack - sizeof(returnType)) = \
      	*(type1*)(pCurEvalStack - sizeof(returnType)) op \
      	*(type2*)(pCurEvalStack - sizeof(returnType) + sizeof(type1))

...which can probably be improved, or at least specialised to give better results for binary ops where the operand and result types are all the same - e.g. integer addition.

<comment><USER14> Frustratingly, I just changed the macro to this:

      #define BINARY_OP(returnType, type1, type2, op) \
      { \
      	register PTR pRet = pCurEvalStack - sizeof(type1) - sizeof(type2); \
      	pCurEvalStack = pRet + sizeof(returnType); \
      	*(returnType*)pRet = *(type1*)pRet op *(type2*)(pRet + sizeof(type1)); \
      }

And the assembly generated is this:

      	BINARY_OP(I32, I32, I32, +);
      004117BA  mov         eax,dword ptr [pCurEvalStack]
      004117BD  sub         eax,8
      004117C0  mov         dword ptr [pRet],eax
      004117C6  mov         ecx,dword ptr [pRet]
      004117CC  add         ecx,4
      004117CF  mov         dword ptr [pCurEvalStack],ecx
      004117D2  mov         edx,dword ptr [pRet]
      004117D8  mov         eax,dword ptr [edx]
      004117DA  mov         ecx,dword ptr [pRet]
      004117E0  add         eax,dword ptr [ecx+4]
      004117E3  mov         edx,dword ptr [pRet]
      004117E9  mov         dword ptr [edx],eax

which is still unimaginably terrible. Why isn't it re-using values that have already been loaded into registers? Why isn't it using a register for pRet? I've even told it to! Although I think it's documented that the MS compiler ignores the 'register' keyword.

And this is with all optimisations turned on. How depressing.

<comment><USER15> Most compilers completely ignore "register" these days. I believe also that aliasing the stack when you're performing the actual operation is greatly hindering the compiler's ability to optimize.

<comment><USER14> It does look as though no optimisation is being performed at all.

I just isolated the use of the BINARY_OP macro, and put it in a simple-ish test function. now it's being optimized excellently.

The function that contains the apparently unoptimisable code is in a hugely long and complex function, and I wonder if something in it is preventing all optimisation from occuring within that function. I've quickly looked through the assembly produced in the function and all of it appears unoptimised; whereas code in other functions is optimised ok.

What can prevent all optimisation from occuring in a function?

<comment><USER16> How long is long? Is this a code gened function? I've seen in some compilers that they sometimes have limits where they turn off optimization due to throughput issues.

If you could break the function up in to pieces, and see if it still doesn't optimize.

<comment><USER14> The function is just over 2900 lines long. Every line crafted lovingly by hand.

The whole source file is here: <http://pastebin.com/9L8N3AVF>

The function starts at line 232, and the disassembly I was looking at is from line 1852.

This is the function that implements the direct-threaded interpreter. Direct- threading works by using goto's (jmp's) to dispatch the next instruction to be interpreted, which means that I don't think it can be broken up into multiple smaller functions.

Please let me know if you think I'm wrong :)

<comment><USER17> What if you made BINARY_OP a __forceinline (<http://msdn.microsoft.com/en- us/library/z8y1yy88.aspx>) function instead of a macro?

Things that the function contains that might disable optimization: a lot of gotos, inline assembly in GO_NEXT (this _does_ affect optimization: <http://msdn.microsoft.com/en-us/library/5hd5ywk0.aspx>)...

It's also not immediately obvious to me that pCurEvalStack is initialized.

Having a look at the architecture optimization manual re: the redundant loads; it's not immediately obvious to me that modern processors won't handle this fine (<http://www.intel.com/Assets/PDF/manual/248966.pdf>).

<comment><USER14> Thanks for the idea. Unfortunately it doesn't improve things:

      	pCurEvalStack = TestAdd(pCurEvalStack);
      0040BFFA  mov         eax,dword ptr [pCurEvalStack]
      0040BFFD  mov         dword ptr [ebp-694h],eax
      0040C003  mov         ecx,dword ptr [ebp-694h]
      0040C009  sub         ecx,8
      0040C00C  mov         dword ptr [ebp-690h],ecx
      0040C012  mov         edx,dword ptr [ebp-694h]
      0040C018  sub         edx,4
      0040C01B  mov         dword ptr [ebp-694h],edx
      0040C021  mov         eax,dword ptr [ebp-690h]
      0040C027  mov         ecx,dword ptr [eax]
      0040C029  mov         edx,dword ptr [ebp-694h]
      0040C02F  add         ecx,dword ptr [edx]
      0040C031  mov         eax,dword ptr [ebp-690h]
      0040C037  mov         dword ptr [eax],ecx
      0040C039  mov         ecx,dword ptr [ebp-694h]
      0040C03F  mov         dword ptr [pCurEvalStack],ecx

And also the nice thing about the BINARY_OP() macro is that it can take types and the operator as arguments (e.g. BINARY_OP(I32, I32, I32, +), BINARY_OP(I64, I64, I32, <<) ) meaning it can be used for many operations on many types. I would have to write seperate functions for each case if using in inline function.

And pCurEvalStack is initialised in the LOAD_METHOD_STATE macro, referenced first on line 590, before interpretation begins.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER18> This is a very interesting post that fits nicetly with many other interesting statements Mike Pall made (such as his reference to the LuaJIT2 interpreter being faster than the LuaJIT1 jit compiler on some cases [<http://lambda-the- ultimate.org/node/3851#comment-57646>].)

On a related note, a similar problem is highlighted in a paper by Anton Ertl and David Gregg ([1]), where they claim that the slowdowns of an efficient interpreter to an optimizing native code compiler is 1:10, whereas the slowdown between an efficient interpreter and an ineffcient one is 1:100. Consequently, there seems to be a lot of optimization potential (though I admit that depending on the kind of the interpreter, it might be unlikely to stay within a slowdown of 1:10)

 _[1]_ : Ertl, Gregg. "The Structure and Performance of Efficient Interpreters", 2001. ([http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.69....](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.69.1513&rep=rep1&type=pdf))

</comment><comment><USER19> While this is true, it's important to note that you need to know a lot about how a modern process works to do this.

I once looked at the possibility of optimizing the byte code dispatcher for a javascript engine (Opera's old one), and my 386 assembly skills were far outdated: Removing one seemingly useless instruction of the ~10 core assembly instructions actually slowed the whole thing down. (Unfortunately, I have forgotten the details)

</comment><comment><USER20> It seems to me that this is more of a complaint leveled at C-ish compilers. One could envision a language that makes the sort of structures that baffle the normal compilerore visible and amenable to optimization. I'm not sure if some syntax and some clever semantics could address all these problems, but...

When I was first learning C formally, I had this idea that C was the "fastest language". My instructor said, day 1, that the problem with C was that it was not amenable to optimization. It was pretty surprising, but as time goes on and languages, compilers, and interpreters get better I think we'll see this sort of problem more and more.

<comment><USER21> // One could envision a language that makes the sort of structures that baffle the normal compilerore visible and amenable to optimization. //

Yes, the problem is that it starts to look like assembly at that point. With a whole bunch of notes on each variable and operation to let the compiler know about whether you care about specific aspects of the result of the operation. (See the 'restrict' keyword in C for an example of what I mean)

<comment><USER9> That sounds like PL/X[1] to me. That language will let you do things that REALLY aren't advisable on most systems like specify pointer sizes (per variable), specify address mode (arbitrary blocks, functions, nested functions, per module, whatever), specify registers for any variable. Oh an my favourite, when calling a function, you can specify an arbitrary global label for it to return to.

Or of course you can write as cleanly as possible, let the compiler do the lion's share of the work, and fix up what you need to.

[1]<http://en.wikipedia.org/wiki/PL/X> and <http://c2.com/cgi/wiki?PlsLanguage>

</comment></comment></comment><comment><USER22> There was an attempt to comperss code for VLIW processors using stack machines.

[http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.37.3...](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.37.3758)

"On a TriMedia VLIW with a load latency of three cycles and a jump latency of four cycles, the interpreter achieves a peak performance of four cycles per instruction and a sustained performance of 6.27 cycles per instruction. Experiments are described that demonstrate the compression quality of the system and the execution speed of the pipelined interpreter; these were found to be about five times more compact than native TriMedia code and a slowdown of about eight times, respectively."

They used pragmas. They used loop pipelining: <http://en.wikipedia.org/wiki/Software_pipelining> and compiler optimizations. Their results aren't that bad, they achieved CISC code density for 8 times slowdown.

</comment><comment><USER23> I've always thought that the standard wisdom is that _most_ developers can't beat the compiler by hand-writing _most_ of their code in ASM, but in things like game development, things start higher level Lua, C++, C and work their way down to increase performance as bottlenecks are found.

Compilers are like Google or Wikipedia, 99% of the time they know more than 99% of developers.

</comment><comment><USER24> LuaJIT is freakishly fast for a certain (not small) set of problems [0].

[0]: <http://luajit.org/performance_x86.html>

</comment><comment><USER25> Related paper: Context threading

<http://www.cs.toronto.edu/syslab/pubs/demkea_context.ps>

The gist is you can structure your interpreter to improve branch prediction by keeping your vm & hardware in similar states.

</comment><comment><USER26> For anyone recently trying LuaJIT2:

If you are wondering why readline (history) support is missing from the interpreter and how to add it: [http://dhllndr.posterous.com/adding-readline- support-to-luaj...](http://dhllndr.posterous.com/adding-readline-support-to- luajit2)

Also, I just put a small functional programming library for Lua on GitHub about an hour before reading this: <https://github.com/davidhollander/fn> Here's an example of how to create a sum function with Fold Left and an addition function:

        require'fn'
        sum = fn.partial( fn.foldl, function(a, b) return a+b end)
        x = sum(1, 2, 3, 4, 5, 6, 7, 8, 9)
        print(x)
        -- 45

...and for anyone who hasn't tried LuaJIT2: <http://luajit.org/download.html>

</comment><comment><USER27> The referenced page is more interesting than what the title promises. The title is trivially true, the referenced page makes the stronger claim that a compiler can not _tie_ with hand-coded assembly.

<comment><AUTHOR> All I have to say for the title is that I did my best to not make it linkbait-y.

</comment></comment><comment><USER28> Would runtime optimization or profile based optimization help here?

</comment><comment><USER29> It's a very old story: Hand coded assembler can commonly get a factor of several faster execution time on compiled code and more when using special instructions on special data. But on large programs, no, because some of what compilers and runtime libraries do for large programs is too difficult for hand coded assembler. Indeed, some of what compilers do in register allocation (and maybe cache usage) will be too difficult for hand coded assembler.

More generally, now the execution time of a small, single threaded program is nearly never of much concern. E.g., when have a processor with dozens of cores, it's easy to let a small, single threaded program have a core and then f'get about it.

Execution time remains important but now mostly just for large, multi-threaded programs.</comment>


Ink – Decentralized reputation and payments for p2p marketplaces - <AUTHOR> https://paywithink.com <comment><AUTHOR> Hi, I'm a co-founder at Listia. We are building Ink using our 8 years of experience with all the difficult aspects of building a marketplace.

Also we think it's time to shift the ownership of reputation away from companies, even us. eBay and others have had a monopoly over your feedback and reputation for too long :)

Feel free to ask me any questions and I'll do my best to answer them, thanks!</comment>