Modern garbage collection - <AUTHOR> https://medium.com/@octskyward/modern-garbage-collection-911ef4f8bd8e#.674yqu7mr <comment><USER2> An interesting shift in the wind for modern garbage collection comes from microservices.

The work on JVM garbage collectors over the last ten years has increasingly been about supporting bigger and bigger heaps efficiently, as physical machines have got bigger, and programs' (or at least application servers') heaps have grown to fill them. Even Shenandoah, a current prototype collector, is designed around large heaps.

But with microservices, we use our cavernous machines to house many small heaps instead. That leaves us with rather different garbage collection challenges.

So, it's possible that the Go developers will get to ignore decades of progress, as they love to do, and still be competitive, because that progress was towards a different future to the one we arrived at.

<comment><USER3> I'd love to know about other GC schemes. Between no GC and whole VM GC. By type GC, localized sub tree GC.

edit: thank you all for your answers, much appreciatedly

<comment><USER4> Erlang has a per-process GC - each Erlang process has it's own heap and it's own garbage collector. There's no VM-wide GC. This usually means the GC pauses apply only to a small part of your system, while the other parts are running completely normally. There's a good article that describes it in depth: [https://www.erlang-solutions.com/blog/erlang-19-0-garbage- co...](https://www.erlang-solutions.com/blog/erlang-19-0-garbage- collector.html)

<comment><USER5> Of course, the cost is extra heap fragmentation.

<comment><USER6> In Erlang case, that is not so much of a problem as most allocated objects have essentially same size. On the other hand this causes another problem that GC's and allocators in Erlang VMs tend to have enormous overhead because most most GC'd objects are very small with sizes comparable to per-object GC overhead.

<comment><USER3> I joked once that in lisp, a vast majority of memory allocation are basically malloc(2). I wondered if it made fragmentation irrelevant.

</comment></comment></comment></comment></comment></comment><comment><USER7> > It has been known since 1984 that most allocations "die young" i.e. become > garbage very soon after being allocated.

1983: A Real-Time Garbage Collector Based on the Lifetimes of Objects, Lieberman/Hewitt

[http://web.media.mit.edu/~lieber/Lieberary/GC/Realtime/Realt...](http://web.media.mit.edu/~lieber/Lieberary/GC/Realtime/Realtime.html)

      Since objects with short lifetimes account for a large
      portion of storage use, it is worth optimizing a garbage
      collector to reclaim storage for these objects
      more quickly.

<comment><USER8> I'd be interesting to know to what degree Go's aggressive stack-based allocation/escaping makes this hypothesis less strong than other languages.

For example, Java a few generations ago was very poor at escaping heap allocation to the stack and so much the opposite skewing towards heap allocation over stack allocation, so I'm sure the JVMs of this era were an extremely good fit for the generational hypothesis.

</comment></comment><comment><USER9> One thing I wish people in the PL/GC community would take more seriously is the idea of explicitly separate heaps or heap partitions. If it's a problem that pause times can be proportional to heap size with some approaches, then the problem can be alleviated by having more small heaps instead of one large. I've used this approach in my own code, including kernel code, to very good effect. Yet somehow the language features to support this properly have never existed in any language that I know of. At best we get heuristics to approximate it; more often, nothing.

<comment><USER10> Research is done. Here is a tech talk on a form of GC integrated into the JVM where the user can annotate regions:

[http://www.oracle.com/technetwork/java/jvmls2015-wimmer-2637...](http://www.oracle.com/technetwork/java/jvmls2015-wimmer-2637907.pdf)

[https://www.youtube.com/watch?v=3JphI1Z0MTk&list=PLX8CzqL3Ar...](https://www.youtube.com/watch?v=3JphI1Z0MTk&list=PLX8CzqL3ArzUo2dtMurvpUTAaujPMeuuU&index=12)

The region markings are used as hints. Allocations are put into the region when possible and then the region is discarded at the end in one go, which is of course a well explored idea but this one is interesting in that you only scope out the regions and don't need to manually assign each allocation. Also allocations that naturally exist beyond the end of the region work properly without additional effort.

One issue is that it's often quite hard to beat a properly tuned generational collector even with techniques that intuitively 'feel' right. As one example, Azul experimented with aggressive hardware-assisted stack allocation on their custom CPU and abandoned it because they found it didn't really beat well tuned generational collection.

<comment><USER9> Awesome, thanks!

</comment></comment></comment><comment><USER11> Tangent:

In databases, it's common to do arena based allocation. You allocate objects of similar lifetime (life of a query, life of a transaction, life of a procedure execution, etc.), and free them all at once when the lifetime is up.

When hacking postgresql, for instance, using pfree() is fairly uncommon. You just allocate, (which goes to the current arena by default), and then the arena is wiped out all at once later.

Of course, sometimes you might use a lot of memory temporarily in a loop, and you need to pfree() it each iteration. But that is the exception.

I think of each arena as a tiny heap (though of course they can grow large).

<comment><USER12> I've seen this strategy appear before in real-time situations where new allocations remain in a near-constant band: As long as you have enough memory to make it through one loop, you could use it as the defining strategy of your game engine or signal processor, at least w/r to temporary allocations.

I've also heard of GC algorithms using exactly this behavior for the nursery stage.

<comment><USER10> Arena allocation in C/C++ is doing something similar to generational GC. I was about to type "simulating it" but that's not quite right ... they exploit similar observations to similar effects in different ways.

In C/C++ you have to manually decide how short lived allocations are ahead of time and either stack or arena allocate them. Freeing all the allocations on the stack or the arena is then very fast.

In a generational GC the programmer doesn't annotate allocations like that ahead of time. Instead it is assumed that all allocations might be short lived. How long they actually live is then tracked and eventually they're moved out of the "arena" (young gen space) by copying them. But again, freeing the contents of the young gen is very fast because you don't actually touch dead data at all, you only copy the live objects.

I've heard of GCs that can actually do "pre-tenuring": when the VM knows an allocation will definitely live for a long time based on profiling data it can be allocated directly into the old gen space. That reduces your minor GC pause times further.

All this adds overhead of course, but makes automatic something C/C++ devs would do themselves. Given the potentially catastrophic security consequences of screwing up in the C/C++ model I've become a lot less sympathetic to that approach over the years. How much extra performance is a security advisory and possible resulting botnet worth to you?

<comment><USER11> Arenas have fast allocation (because you often have no freelist and just increment the end). They also allow you to quickly reclaim old data, which is important for a database because a query might run long enough for the allocated memory to be considered old (for one example). And they don't require compaction or copying, because objects of the same lifetime are physically near each other in memory.

It's fair to draw an analogy to generational GC, but they aren't the same.

Rust is an example of a language that can do arena allocation without the dangers of C++.

EDIT: Based on my experience with postgresql, and what I've learned about rust so far, I am starting to believe that object lifetime is a fundamental concept in CS. It's possible that sophisticated GCs have, to some degree, hidden the importance.

<comment><USER13> _Arenas have fast allocation (because you often have no freelist and just increment the end). They also allow you to quickly reclaim old data, which is important for a database because a query might run long enough for the allocated memory to be considered old (for one example). And they don 't require compaction or copying, because objects of the same lifetime are physically near each other in memory._

This is exactly how the Gen0 nursery works. The surviving objects get copied to Gen1 before the Gen0 bump pointer is reset. Of course, in C++ arenas there cannot be surviving objects or that is implemented manually.

The main difference here is that what is in the nursery is decided by the GC, whereas a manual arena requires you to pre-decide what is in the arena or not.

</comment></comment></comment></comment></comment><comment><USER14> Mike Hearn criticizes the Go implementers for making misleading claims about building a "garbage collector for the next decade", but I don't see them making claims that they are solving garbage collection for any language other than Go?

There's an underlying assumption (both in the article and in comments here) that the results of garbage collection research are generally applicable across languages. It's not obvious that this is true. While you can certainly get a lot of ideas from previous research, memory allocation and access patterns can differ significantly between languages (due to different language features and common idioms), and what seemed like a win for one language might not actually apply for a different one.

<comment><USER15> Yeah. I find it a haughty article because he's not acknowledging the elephant in the room, which is that GC is not a solved problem despite all the whiz- bang techniques ... and that the Go people aren't being dishonest or ignorant of these techniques, but rather, judging those techniques not to be so useful for the class of programs they care about.

<comment><USER13> The following statement from the article addresses this:

 _As Go is a relatively ordinary imperative language with value types, its memory access patterns are probably comparable to C# where the generational hypothesis certainly holds and thus .NET uses a generational collector._

I think that's reasonable. I would say it is on the Go developers to point out why they think Go would have seriously different performance characteristics from similar C# network applications using Task-based asynchrony (similar to goroutine + channel based asynchrony in Go).

<comment><USER16> I don't think this is an actual claim made by go developers. Their arguments for choosing the algorithm they did was (at least as far as I remember, feel free to point me to a source) a) go for implementation simplicity, readability and clarity first then b) once you figure out where it needs improvement, do that.

They didn't choose mark-and-sweep because they thought other algorithms are worse, go has special needs or anything else. At least this would be the first time I've heard about it. The same goes, from what I've seen, for most of the arrogance ascribed to them (in this article and elsewhere); it usually hinges on making up or misrepresenting things they've allegedly said.

<comment><USER13> I think the only point being made by the article is that their focus seems to be on pause times above all else.

<comment><USER16> a) I wasn't replying to the article, but to a particular comment about it.

b) I disagree with that summary of the article. I believe it's first paragraph should be an adequate indication of what it's point is, that's how you usually write articles, and it's distinctively not making that point.

And lastly, c) I also don't believe that this statement, if it were the point, is at all true. At best it's hyperbole, but even on the surface, of the list of 15 design criteria for a GC mentioned in the article, at least 7 are explicitly taken into account and considered an optimization goal for the go GC (not all on equal footing and some in a different way than the author might prefer, but claiming they were ignored is just false). I can definitely say that for pause times, pause distribution, compaction, concurrency, scaling, tuning and portability.

<comment><USER17> I keep seeing papers with the lede paragraph and sometimes the conclusions written in a different voice that the rest of the paper. I suspect "men of good faith but limited understanding" are contributing to the editing (;-))

</comment></comment></comment></comment></comment></comment></comment><comment><USER18> > It has been known since 1984 that most allocations "die young" i.e. become > garbage very soon after being allocated. This observation is called the > generational hypothesis and is one of the strongest empirical observations > in the entire PL engineering space. It has been consistently true across > very different kinds of programming languages and across decades of change > in the software industry: it is true for functional languages, imperative > languages, languages that don't have value types and languages that do.

Is it really? The linked paper is about Smalltalk. A lot of the research is about Java. Functional probably talks about Haskell and ML. All those languages rely on GC completely.

Is this hypothesis still true, when a language allows the programmer to mix different memory management techniques? For example, if you have smart pointer and stack allocation, do objects still "die young"? It might be that these young objects never reach the GC, because they are now allocated elsewhere, which means we have mostly long living objects under GC management.

I guess a selective use of the Boehm GC would be the only reasonable way to falsify the hypothesis. Did anybody try that?

<comment><USER19> > It might be that these young objects never reach the GC, because they are > now allocated elsewhere, which means we have mostly long living objects > under GC management.

Where elsewhere? The stack? One of the things people always get wrong about GC challenges is "what about stack allocation?". Modern servers come with up to 6 _TB_ of RAM. Applications that are interesting from a GC perspective (i.e., lots of concurrency, data structures etc.) commonly employ heaps that are between 10s and 100s of GBs today. The amount of memory available on stacks is 0.5-10% (thousands of threads are required for the high amount) at most. In other words, stack allocation does not remove the needle by much (although it may affect the frequency of young-gen collections). The Java team discovered that when they added escape analysis and stack allocation, and the effect was not dramatic. The stack simply does not matter that much in the grand scheme of things (arrays-of-structs, OTOH, matter a great deal).

<comment><USER18> Ok, then array-of-structs. Maybe obstacks (region-based allocation). All that stuff that cannot be done in Java, Smalltalk, etc. Does that invalidate the generational hypothesis or not?

<comment><USER20> > All that stuff that cannot be done in Java

Maybe not on the standard implementation, but just like C and C++, Java also enjoys some extensions.

Packed Objects in IBM J9

[http://www.ibm.com/support/knowledgecenter/SSYKE2_8.0.0/com....](http://www.ibm.com/support/knowledgecenter/SSYKE2_8.0.0/com.ibm.java.lnx.80.doc/user/packed_intro.html)

ObjectLayout in Azul

[http://www.azulsystems.com/content/goto-conference- chicago-2...](http://www.azulsystems.com/content/goto-conference- chicago-2015-2)

And in some standard form in Java 10

[http://openjdk.java.net/jeps/169](http://openjdk.java.net/jeps/169)

</comment></comment></comment></comment><comment><USER21> The title of Go's GC announcement is "Go GC: Prioritizing low latency and simplicity", and I think that a variation of that would be a better title for this. Some users may prefer a different tradeoff, and I think it's important to know that there _is_ a tradeoff. But as this article says "I have no doubt that many Go users are very happy with the new runtime.", and that seems like a worthwhile metric.

[https://blog.golang.org/go15gc](https://blog.golang.org/go15gc)

<comment><USER22> That there are tradeoffs is the whole point, the messaging makes it sound like there aren't.

<comment><USER23> The title says "prioritizing"; if that's not synonymous with "making tradeoffs", I don't know what is.

</comment></comment></comment><comment><USER24> Interesting article.

I haven't internalized all the parameters of the Go GC, but I wonder if there's a way to tune it on the fly. For example, if I want to run at 60 FPS (16ms/frame), and I've measured that actually doing all the CPU side work took 10ms, can I let the GC have ~5ms of GC time (rather than the <1ms advertised) at the end of my frame to increase throughput (read: less cycles burned overtime & better power efficiency)?

If not in Go, can I do this elsewhere like on the JVM?

Hopefully this isn't too off topic (since the article is about trade-offs of throughput vs latency).

<comment><USER10> That's a realtime GC and yes, you can buy a JVM from IBM that has one, they call it the Metronome collector:

[http://researcher.watson.ibm.com/researcher/view_group_subpa...](http://researcher.watson.ibm.com/researcher/view_group_subpage.php?id=175)

Metronome has pause time in the hundreds of microseconds range. The tradeoff it makes is that, amongst other things, it requires read barriers i.e. every time you read a pointer from the heap you have to execute some extra code to interact with the GC subsystem. If you think about how often you read pointers from the heap in a typical program, well, that turns into a lot of barriers and a corresponding execution performance hit.

<comment><USER25> Per the linked article, if their claims are to be trusted, the barrier overhead is only 4%, which is amazingly low. Low enough that it's inconsequential for any application actually needing low pause times, if the claims are correct, which I unfortunately do not know.

Funnny, I remember years ago reading one of the patents the GC is probably based on, and thinking the entire thing was brilliant.

</comment></comment></comment><comment><USER26> But why does Go prioritise low latency? I would think that low latency is not super important for web-servers since there are various other factors adding up to the total latency (what i mean here: time spent between call of the http request to finished receiving all bytes).

<comment><USER27> It prioritizes latency because it's written by Googlers, and Google prioritizes latency. I don't think we need any more complicated explanation.

<comment><USER17> From thew point of view of a capacity planner and performance engineer, one goes for low latency first, because for many algorythms, the best throughput is seen when latency is lowest. Some cases will be different, but starting with removing time wasted sitting in a queue is A Universal Good (;-))

</comment></comment></comment><comment><USER28> When processes are short-lived, sometimes the best GC is exit(2).

</comment><comment><USER29> One thing I don't miss now writing in Swift is a garbage collector. Are there other languages that do something similar to Swift? It seems most languages either use a GC of some kind or make you do your own heap management.

<comment><USER20> Swift has a garbage collector.

[https://www.amazon.com/Garbage-Collection-Algorithms- Automat...](https://www.amazon.com/Garbage-Collection-Algorithms-Automatic- Management/dp/0471941484)

Check chapter 5.

<comment><USER30> I think it's both fair to say that reference counting is a form of garbage collection (dynamic lifetime determination is dynamic lifetime determination), but also remark that reference counting has notably distinct performance characteristics (some good, some bad) compared to a tracing collector.

<comment><USER20> Both being GC algorithms, what one should discuss is implementation advantages and disadvantages of each algorithm.

Just like sorting or graph navigation algorithms get discussed, for example.

However somehow we get this pop culture of GC vs RC instead.

</comment></comment></comment></comment><comment><USER11> Are GCs scaling well to new hardware? Memory sizes are increasing, as are core counts. But if those extra cores are being used to GC the larger memory, are we effectively losing those extra cores?

<comment><USER10> That's partly what the article is about.

In a pure mark/sweep GC, as the heap size goes up, the work needed to complete a GC cycle and free some memory goes up too. Thus the more memory you use the more cores you need.

In more complicated collectors that isn't necessarily the case: collection is incremental and done only as much as needed to keep up with the creation of garbage.

As I say in the article, there are reports of people running JVM/G1 on terabyte sized heaps. The problem they have is that if they experience a concurrent mode failure (which G1 is not designed to optimise because in a properly tuned system it's meant to never happen) you're looking at, like, a 5-10 minute pause time whilst the entire heap is mark/sweep and cleaned out as much as possible. So with such huge heaps you'd better make sure the collector doesn't fall behind!

</comment></comment><comment><USER31> I can't wait to see how ROC (Request Oriented Collector) [1] will benefit Go GC. Go is mostly used in web services which means you could collect garbage from whole request when it's over. ROC will try to do that. You can read more about it in the link that I've provided.

1\. [https://docs.google.com/document/d/1gCsFxXamW8RRvOe5hECz98Ft...](https://docs.google.com/document/d/1gCsFxXamW8RRvOe5hECz98Ftk- tcRRJcDFANj2VwCB0/edit)

<comment><USER32> It's addressed in the article. Basically, you can mimick that with a generational GC as long as the youg generation space is large enough to hold all objects from a request.

<comment><USER10> Note that advanced collectors like G1 auto-adjust the young gen size as the program runs. This means pause times can take a little while to settle, but after the GC warms up you should find it's more or less like the ROC:

[https://software.intel.com/en- us/blogs/2014/06/18/part-1-tun...](https://software.intel.com/en- us/blogs/2014/06/18/part-1-tuning-java-garbage-collection-for-hbase)

(note: that article is about a rather old version but the graphs illustrate the concept)

This may not help you if you can't tolerate warmup latency spikes. In that case you need to do some GC tuning to tell the collector enough that it can get it right from the start.

</comment></comment></comment><comment><USER6> > implementing a generational collector requires the ability to move things > around in memory

While this is true for traditional implementations with separate heap regions, it is not necessary. One can easily represent new and old generations as linked lists in per-object overhead and just change pointers in such overhead to move objects between generations. There is at least one Smalltalk VM that works in this way and IIRC traditional BEAM (ie. non-HiPE) Erlang VM has GC that works in essentially same way.

<comment><USER33> The Boehm collector claims to be generational, and doesn't move memory, but I haven't found the details of its generationality.

<comment><USER6> The relevant paper describes it as "mostly-parallel, mostly-generational". The idea is that it works like traditional tri-color mark and sweep collector, but white objects stay white between partial collections unless they are modified (ie. reside on memory page that was modified and thus caused memory access trap). In essence it seems to me that difference between generational and incremental/parallel mode lies only in what causes white objects to revert to black (ie. if it happens after each sweep (incremental) or only when sweep does not produce enough free memory (generational)).

</comment></comment></comment><comment><USER34> Why use garbage collection at all when value semantics and smart pointers cover virtually all the use cases for GC, much more efficiently?

<comment><USER35> A generational GC is way more efficient than `std::shared_ptr`. `std::shared_ptr` is implementing a reference counting garbage collection, which is like the most inefficient way of implementing a GC.

FYI: \- Allocating memory in a generational GC is as fast as allocating something on the stack. It's basically changing the value in a register. It's order of magnitude faster than calling `malloc()`. \- A generational garbage collection is slower than freeing a stack frame, but has a lower per item overhead than calling `free()`.

tl;dr: for most programs, generational GCs are the fastest at allocating dynamic memory.

<comment><USER36> > A generational GC is way more efficient than `std::shared_ptr`

that's true, but it's also a moot point because std::shared_ptr is almost never the semantics you want. You want std::unique_ptr.

Also, you can get away without allocating memory dynamically in a lot of cases (thus not incurring overhead of malloc).

After C++11 and Rust I'm starting to think garbage collection is holding us back from doing research into better approaches to automatic memory management.

<comment><USER37> > that's true, but it's also a moot point because std::shared_ptr is almost > never the semantics you want. You want std::unique_ptr.

unique_ptr still has the malloc()/free()[1] overhead, I'm assuming the parent poster just misspoke. One major issue with shared_ptr (which unique_ptr doesn't have) is that it must work properly when different threads have different shared_ptr's (copies of each other) pointing to the same object. This adds overhead.

> Also, you can get away without allocating memory dynamically in a lot of > cases (thus not incurring overhead of malloc).

Ok, but we're specifically talking about GC and RC here, so I don't see why you'd bring that up.

> After C++11 and Rust I'm starting to think garbage collection is holding us > back from doing research into better approaches to automatic memory > management.

GC literally _is_ automatic memory management. (And as others have pointed out, so RC is a form of GC.)

Also, there's lots and lots of research into various different ways of doing automatic memory management.

[1] Alright, they don't _literally_ use malloc/free, but you know what I mean.

<comment><USER36> > Ok, but we're specifically talking about GC and RC here, so I don't see why > you'd bring that up.

well, my point was, in garbage-collected languages you don't have the option to never even engage the garbage collector by creating variables on the stack, while in languages like C++ you can avoid the overhead of malloc/free by using stack variables.

> GC literally is automatic memory management.

It is, but there are other approaches to automatic memory management that don't introduce another entity that does stuff to your program at runtime. C++'s smart pointers is one such approach, see also Automatic Reference Counting and Rust's borrow checking. I wish there was more research done into compile-time techniques to prevent resource leaks.

<comment><USER37> > well, my point was, in garbage-collected languages you don't have the option > to never even engage the garbage collector by creating variables on the > stack, while in languages like C++ you can avoid the overhead of malloc/free > by using stack variables.

Oh, I see. I can see what you mean, but there _are_ solid techniques for avoiding GC in e.g. JVM-based langauges -- for an example see e.g. Disruptor/Aeron.

I'll happily grant that it's a problem that it can be _quite_ hard to see if you're _actually_ avoiding GC when using these patterns. It'd be interesting if there were a "@no-gc" annotation which could enforce such patterns on the source code. (So, essentially you'd have a compile-time "GC by default, but you can opt-out and the compiler will tell you when you violate that". One can dream.)

</comment></comment></comment></comment></comment></comment><comment><USER38> Alas, having no garbage collector at all is a blissful state of simplicity when performance matters. It's not very difficult to clean up one's objects manually in a well-thought-out codebase.

<comment><USER39> Experience (i.e. the number of memory-related bugs) suggests otherwise.

<comment><USER38> What about smart pointers? It gives the flexibility and benefits of garbage- collection without the performance degradation. It's not a silver bullet but it helps.

What I'm getting at is, Rust is the only modern language (in vogue at the moment) that does not use garbage collection. It would be nice if more languages didn't _require_ a garbage collector but gave the option to use one.

If there are some more of such languages please chime in and provide a link to them! :)

<comment><USER40> Smart pointers are as hard to manage as pointers.

There are different kind of smart pointers and you gotta think what you want to do with them. A GC is easier.

<comment><USER38> GC is certainly easier. However, it is important to understand the life-cycle of objects regardless. To be able to control them (if desired) is good because in many cases one would prefer to have that control instead of being forced to use garbage collector.

In an ideal world, garbage collector makes sense because we don't want to have to worry about life-cycle management when we are solving other problems. Memory management can indeed be an annoying implementation detail, but as it stands there are certainly limiting factors in hardware which are impacted by generic object-management for specific application domains.

</comment></comment></comment></comment></comment><comment><USER41> I read "typical worst-case pause time" in go and don't know whether to laugh or cry.

We are seeing massively longer pauses in production and the suggestion is to go to 1.8, which is not ready.

Reminds me of ruby - religious people practicing on production to get the nirvana.

Yes, it's getting better, or at least less bad, but yeesh. We'll just waste resources architecting around long pauses, and rewrite critical stuff in C, for a few more years...

</comment><comment><USER42> We should focus on multiple small heaps instead - like erlang does it. Low latency, high capacity, no stop the world, simple implementation.

</comment><comment><USER43> I thought this was going to be about actual garbage collection and municipal infrastructure.

That's what I get for being non-tech and browsing HN!

</comment><comment><USER5> So, why GC instead of region memory allocation like in Rust? (and some other less known languages)

<comment><USER30> Rust doesn't use region-based memory management, though the borrow checker does somewhat resemble it. And the reason to use a GC is that it often works well enough for tasks that aren't especially demanding.

</comment></comment><comment><USER44> I would love to see an implementation of this [http://wiki.luajit.org/New- Garbage-Collector](http://wiki.luajit.org/New-Garbage-Collector)

<comment><USER45> Why? It's still designed to use the worst of the three possible strategies, which Mike tend to ignore. A modern GC is compacting, with mark&copy or copying, if you have enough memory. Mark&sweep is only good for small devices with not enough memory, or a FFI or bad ABI/VM. Walking the heap is always at least 10-100x slower than walking the stack.

<comment><USER44> _Why?_

Because I don't know a lot about garbage collectors, but I am familiar with Mikes amazing work on LuaJIT and I trust that he knows enough about what he's talking about (based on the quality of his luajit work) for enough of his claims to be true for this to be interesting, useful or both.

<comment><USER46> The thing you have to understand about Mike's work on the new LuaJIT GC design is that it's _heavily_ constrained by the Lua API. It's not a very interesting GC, though it does improve slightly on the standard tricolor incremental marking GC. See my reply to your other post.

</comment></comment></comment></comment><comment><USER47> >But if you take one thing away, let it be this: garbage collection is a hard problem, really hard, one that has been studied by an army of computer scientists for decades. So be very suspicious of supposed breakthroughs that everyone else missed. They are more likely to just be strange or unusual tradeoffs in disguise, avoided by others for reasons that may only become apparent later.

I really take issue with this. Imagine all of the progress that would have been delayed or quite possibly not achieved had people not persevered against the traditional thinking. Yes, GC is a hard problem but just because something's been researched for a long period of time does not mean there is no room left for innovation. Breakthroughs come some people that think differently not from people that think it's all been done before.

</comment><comment><USER22> Excellent post, thank you.</comment>


Cassandra at Twitter Today - <AUTHOR> http://engineering.twitter.com/2010/07/cassandra-at-twitter-today.html

<comment><USER2> I don't know about you, but to me this seems like a big blow to the reputation of Cassandra. I know a bunch of people (including myself) whose primary motivation for investigating Cassandra as a large-scale data store was "Twitter is using it, therefore it's got to be fine for what we want".

In my experience this was not the case -- after quite a lot of prototyping, we moved away from Cassandra. It scales brilliantly, but its query time has a pretty solid floor that I was unable to shift to speeds approaching what I needed: it never went up, but nor did it go down.

Of course, I am a total noob at distributed systems -- perhaps it's simple to get it to go faster, but Twitter's move away suggests not. (Facebook's use- case is quite different, and in any case they still use an awful lot of MySQL too) The other systems they cite as currently using Cassandra in production are nowhere near the scale of the primary tweet store.

<comment><USER3> I don't think they're avoiding Cassandra as their tweet store because it's not capable of performing well. It's probably more cause it would be a really long process migrating every tweet into Cassandra. I'm sure if they could, they would go back to the first day Twitter launched, and use Cassandra as the tweets store.

As for query time, maybe your problem was I/O throughput? I've read Cassandra can be really slow if you have poor I/O throughput. This happens a lot in the cloud, where I/O is usually shared.

<comment><USER4> Don't they not show / hide & archive old tweets anyway? Seems like they could really simple transition Cassandra in if they wanted to.

<comment><USER5> On a small scale, yes. On Twitter scale (in February they were averaging 50 million tweets/day [1]), it's probably not so simple.

[1] <http://blog.twitter.com/2010/02/measuring-tweets.html>

Edit: As of mid June, it was about 65 million per day.

[http://blog.twitter.com/2010/06/big-goals-big-game-big- recor...](http://blog.twitter.com/2010/06/big-goals-big-game-big-records.html)

<comment><USER4> My point was that they aren't importing legacy data at all if they aren't showing old stuff. Just start writing the new ones to the new DB, right?

</comment></comment></comment></comment></comment><comment><USER6> I think that apart from the speculations about why twitter does not want to move to a new store for tweets, the important "take off" here is that what they (and other companies) do is experimenting with different platforms, and then using what they feel it's right for them.

Don't use what others are using or suggesting. Download a few stores, take your time to understand how they work, and then do your pick. Time consuming? Not as time consuming as starting a big project with a system that is not a good fit.

<comment><USER4> Or you could just use MySQL.

</comment></comment><comment><USER7> 1\. Cassandra is very young! Especially, the design and implementation of local storage and local indexing are junior and not good.

2\. Pool read-performance is also due to the poor local storage implementation.

3\. The local storage, indexing and persistence structures are not stable. They need to be re-designed /re-implemented. If Twitter move data to current Cassandra, they should do another move later for a new local storage, indexing and persistence structure.

4\. There are many good techniques in Cassandra and other open-sourced projects (such as Hadoop, HBase) etc. But, they are not ready for production. Understand the detail of these techniques and implement them in your projects/products.

Schubert Zhang

</comment><comment><USER8> Every time I use Twitter, it is malfunctioning in a very noticeable and bizarre new way. I don't care about scale, how difficult it is to be twitter, etc - I would not take their word about any technical matter without a tsp of salt.

<comment><USER8> Ha ha. Everything I say on this site is down voted recently. As if I fucking care.

Twitter :

\- can't find my @ replies. Shows zero. Not correct.

\- has been showing 2-5 copies of each message in my own history for the past week

\- gave me 30 error screens in a row, then an hour later on their status blog "we've noticed elevated rates of errors". Like a constant stream of errors is normal for them, but elevated rates - time to do something!

\- can't get their widgets/external code straight, redirects people to twitter.com/undefined and twitter.com/null

Etc, etc. Give me a break. Twitter is hopeless, and has always been hopeless in the tech department.

</comment></comment><comment><USER9> At the middle of Hype cycle, they are now getting real.

</comment><comment><USER10> Interesting to see that they use two different db systems. I'm a bit worried about this. I believe it is just postponing an important decision. In the future they will have to choose for one db system. With the growth of Twitter wouldn't it be better to make a decision now and focus all resources on optimizing one system?

<comment><USER11> They're also using their own FlockDB (<http://github.com/twitter/flockdb>) to store their social graph.

<comment><USER3> FlockDB isn't a data store itself. They use FlockDB on top of MySQL AFAIK.</comment></comment></comment>


The Anti-Startup Manifesto - <AUTHOR> http://pastebin.com/HR16SHfW

<comment><USER3> This looks like the work of someone very deep inside the tech bubble. I strongly suspect that this kind of "live off of rounds of funding without making any profit" mentality doesn't really exist outside of the tech startups in the bay area.

<comment><USER4> The mentality does, however due to lack of funding they tend to fizzle out fairly quickly.

I would argue this is actually a bad thing and it's why the Bay Area tends to produce lots of successful tech companies and other areas have trouble with it. Many would-be profitable businesses die out because they lack the funding necessary to get over the initial hurdles of starting a company. The willingness of investors in the valley to throw money at anything is a huge boon to startups - even if it wastes a ton of money in the process.

On the other hand profitability should still be a top goal of a startup - <USER2> investment. Many startups seem to have confused the two.

<comment><USER5> You make a good argument, and I'd only add that during an uptick in the tech sector, it becomes easier to get funding and so a series of effects flow from there. More people get funding, and more of it, and the risk profile and spread changes - probably to include more players that have less chance of success.

For my anecdotal experience I can say I have friends in 2015 who are working for good salary at funded startups that have quite literally no idea what mission or product they are making right now, whereas in 2010 my friends were mostly contributing sweat equity in return for actual equity.

</comment></comment></comment><comment><USER6> Makes me want to write "anti-manifesto manifesto". If you want to write a manifesto work on it, make it clear, concise, well written. This is drunken mumbling rant, even if the author is right in some parts.

<comment><USER7> or maybe a song: [http://youtu.be/jXoQsS1Bu80](http://youtu.be/jXoQsS1Bu80) ? :)

</comment></comment><comment><USER8> Why would we want to return to "the late XIX's century" [sic]? The pain of the Civil War and the invention of cattle drives are tough to connect to the birth of the Internet.

Perhaps this is a form of performance art...

<comment><USER9> > the late XIX's century

There's a reason it was called the "gilded age" and not the "golden age".

</comment></comment><comment><USER10> This confirms my (admittedly biased) outsider's perspective of startups, where the goal is not to do something useful or even profitable, but rather to fleece investors or get acquired by some bigger company. A result of this possibly-to-pessimistic perspective is my belief that it is a tragedy that so many smart, talented people, who could be solving real problems, end up in this shell game.</comment>


Ask HN: What are your favorite Arduino and Raspberry Pi projects and sites? - <AUTHOR> Just getting my feet wet with research and reading. I recently subscribed to Hacker Boxes. I'm a photographer, lighting guy and color corrector by trade, but I'm interested in fun stuff to do in general and for kids projects too. <comment><USER2> The official forums are nice:

[http://forum.arduino.cc/](http://forum.arduino.cc/)

Reddit has a good community too, it mostly has people posting their finished results with varying degrees of implementation detail.

I've only recently finished working my way through the official starter-kit project-list - so I'm only a beginner myself - right now I'm just waiting for a bunch of new sensors/hardware to ship from AliExpress. Hopefully once it does I'll be able to build an MP3-player, NTP-based clock, etc. Nothing too novel, but fun regardless I hope!

Finally [https://www.hackster.io/arduino](https://www.hackster.io/arduino) has a few nice projects, as does instructables. As with any of those kind of sites the projects/clarity vary.</comment>


Game Off 2017 winners - <AUTHOR> https://github.com/blog/2483-game-off-2017-winners <comment><USER2> It is interesting to see no games were developed using C, C++ or Java and outside of JS for web games it is all C# and Unity.

I don't really follow game dev, is C# popular in the industry then? I know a couple of popular indie games were developed using C# and Unity but has it made its way into the bigger studios?

<comment><USER3> As I understand it, Unity is crazy gargantuan huge, and C# basically came along for the ride. Many extremely popular recent games were made in Unity, including Cuphead, Kerbal Space Program, Cities Skylines, Firewatch, Monument Valley, Ori and the Blind Forest, Superhot, and more.

<comment><USER4> Yeah, I remember playing around with Unity when their main selling point was being a game engine for the Mac. It's insane how they went from niche novelty to utterly dominating an industry.

</comment></comment></comment><comment><USER5> I'm guessing that the time-crunch of being a "game jam" drove most of the participant's decisions to use Unity and C#.</comment>


Pinebook: My First Few Surprising Hours with a $99 Linux Laptop - <AUTHOR> https://www.forbes.com/sites/jasonevangelho/2018/11/21/pinebook-my-first-few-surprising-hours-with-a-99-linux-laptop/amp/ <comment><USER2> Looks awesome, but I put my name on the build-to-order notification list /months/ ago, and am still waiting to be contacted. Does anyone know how regularly they _actually_ ship?

<comment><USER3> I just got mine after being on the list for about a year. It's a wonderful little machine. It's how laptops should be - they should be concerned with less heat and longer battery life, not on how many cores we can run at super speed for six seconds at a time.

I'm slowly compiling all the software for it that I think I might ever need in preparation for making it my primary machine. With NetBSD, the laptop feels nimble and quick.

The keyboard layout takes a while to get used to, like the article says. For those of us who are on the command like 95% of the time, the pipe really takes some time. But aside from the layout, the keyboard feels darned good for an inexpensive laptop.

Within a month I think this will become my main portable.

<comment><USER4> Report back in a month, please! I'm interested

</comment></comment></comment><comment><USER5> A recertified Chromebook that will accept a Seabios rom upgrade is another cheap linux laptop approach: [https://johnlewis.ie/custom-chromebook- firmware/rom-download...](https://johnlewis.ie/custom-chromebook-firmware/rom- download/)

</comment><comment><USER6> I really wish they could commit to shipping the 1080p screen on the 11" model. I'd pull the trigger tomorrow. But as far as I know, there's still no guarantee if you'll get the 1920x1080 or the 1366x768 display. That's a dealbreaker for me.

</comment><comment><USER7> Disappointing to see that the keyboard layout is so nasty. Does anybody have a favorite laptop that's netbook size (<=13") with a focus on writing?

It seems the perfect "write comfortably on train, so light you forget you have it" device remains unsolved.

<comment><USER8> A ThinkPad x1 14" model weighs the same as the Pinebook, and is roughly similar in size. Even an older model x1 would run circles around a Pinebook in most every respect. ThinkPad x2x0 models weigh in heftier, but generally can be found at prices aren't much higher than a Pinebook. For heavy typing there's practically nothing to recommend but ThinkPads.

<comment><USER9> I have a first gen X1 carbon that I recently purchased on ebay for a little less than $200 and I haven't ran into any hiccups with general usage on W10. The keyboard is really nice, but I don't think as nice as the keyboard on the x200/220, but better than the x230 from when I remember using one.

If you have $200 to spend I think the 1st gen x1 is easily the best ultraportable for the buck.

</comment></comment></comment><comment><USER10> Pretty surprising that KDE Neon would be the default distro for this. Why wouldn't they choose something lighter?

<comment><USER11> I can see a few reasons:

1\. It's a full-featured, widely tested desktop environment. If a user can't figure out how to enable Bluetooth in LXQt or there's screen tearing in Xfce, then they might blame your hardware. You especially also need someone to rely on, who does ARM builds, which many distros do, but fixing the build of the desktop environment often is something you need the help of the DE devs for.

2\. It looks modern and polished. You can theme the more lightweight alternatives, but they often still have UI elements that are not correctly aligned and such.

3\. The KDE Community is big, and many of them got the news of this Pinebook. If some buy it for themselves, or gift it to someone, or just tell their friends about it, that might very well be your first year of funding secured.

4\. It is kind of news-worthy on its own that one of the most feature-rich OSs ships on one of the cheapest laptops.</comment></comment>


Overall enrollment at universities is down for the seventh year in a row - <AUTHOR> https://www.wsj.com/articles/college-enrollments-down-again-this-year-11559217601 <comment><USER2> Who is surprised by this?

While education is highly inelastic, we've basically moved the cost to the point where you would have to be a madperson not to think twice.

In the next 10 years, when people are more normalized to the pricing, you'll be seeing these clickbait articles about how no one is going to school anymore, and the middle class is shrinking, and everyone in gen +1 is so lazy unlike their parents who earned their jobs.</comment>


3D Books - Google Books Now Offers Digital Books in a New Dimension - <AUTHOR> http://booksearch.blogspot.com/2010/03/3d-books-google-books-now-offers.html

<comment><USER2> I smell April fools...</comment>


Jails are replacing visits with video calls–inmates and families hate it - <AUTHOR> https://arstechnica.com/tech-policy/2018/05/jails-are-replacing-in-person-visits-with-video-calling-services-theyre-awful/ <comment><USER2> This is American "justice" culture. We can do anything we want to "bad people" because they are intrinsically "bad". You can tell they're bad because they're in jail. If they were good people, like me, they wouldn't be in jail.

Anyone caught suggesting something is too much punishment is not "tough on crime". They want the bad people to win (so don't vote for them!). Reducing sentence length, improving prison conditions, trying to actually help those on prison, that's all just weakness, that's not being tough on crime.

Who cares that recividism is more expensive than rehabilitation- the point isn't to save money or reduce crime, it's to punish the bad people. I mean, if we stop thinking of them as bad people, how am I supposed to know that I'm a good person?

<comment><USER3> Something happened over the years that caused, at least in my experience living in America, people to incredibly deeply associate legality with morality. I suspect this is universal to some degree in societies around the world and in history. But how prevalent it is in my lived experience feels deeply Orwellian. As long as Big Brother approves your in the right, right?

<comment><USER4> If the law is not at least an approximation of morality, then what is its purpose?

Or to put it another way, assuming that I share a common moral sense with my fellow citizens, I would much prefer a legal system that is strongly associated with morality than one that is not.

If your objection is that the law feels to retributive, then that is a separate issue.

<comment><USER5> > If the law is not at least an approximation of morality, then what is its > purpose?

To maintain existing power structures

<comment> V-2 The more the law diverges from the morality (as commonly understood in a given society), the more oppressive power structures you need in order to maintain it.

</comment></comment></comment></comment></comment><comment><USER6> There is a callousness in the attitudes the general populace has toward prisoners. The American desire for retribution and revenge toward prisoners is insatiable. This overall attitude affects our views and reactions in other areas. For instance there is a lingering fear in a large segment of American society that poor people get something they don't deserve through the welfare system. The dehumanizing way we view prisoners and their concerns is toxic. As a people we have lost our way.

<comment><USER7> Why has it become so hard to convince people that they should care about others.

<comment><USER8> It's confusing because the people making these decisions surely have families and are not made of metal. Surely.

</comment></comment></comment><comment><USER9> > In the early years, companies selling video-visitation products would ask > jails to sign contracts requiring them to phase out in-person visits.

This seems evil. Actually evil. Inhuman, sick, psychopathically twisted, demonically evil. This is a contract that literally obligates its signatories to destroy other human souls: to deprive them of all touch, eye contact, the natural timbre of a loved one's actual voice. I cannot imagine how a person, peddling such a contract, could possibly sleep at night.

<comment><USER10> >I cannot imagine how a person, peddling such a contract, could possibly sleep at night.

Simple: because they're evil, psychopathically twisted and demonically evil, just like you said.

And the government officials who agree to these contracts are just as demonically evil.

I really think that estimates by mental health professionals that psychopaths are only 5% of the population are probably far, far too low.

<comment><USER11> The stats I saw were actually lower. 2% for gen pop, 5% for corporate env.

</comment></comment></comment><comment><USER12> Prison sentences really ruins people's lives. They lose their job, their house (can't pay mortgage or rent) and many times their relationship. I wonder how using ancient punishment would work nowadays. When someone commits a crime the judge and a doctor can decide how many time you should get whipped, shocked, or waterboarded and then after a few days of pain and humiliation the criminal can continue on with their life. Obviously for repeat offenders this may not work, but for first timers, it may work.

<comment><USER13> This is wrong on so many levels...

<comment><USER14> I think you're under-estimating how disruptive prison is to one's life. I'd definitely choose lashes over prison. Heck, I'd probably take lashes + prison in order to get a felony reduced to a misdemeanor. It would suck less than the effects on my life from losing my job.

<comment><USER13> Then the problem is with how prisons are. Those should be changed. Introducing some extreme measures and a new way to punish is not the way.

<comment><USER8> The counter argument here is that long term prison sentences are the more extreme punishment

</comment></comment></comment></comment></comment><comment><USER15> This doesn't have to be a political issue. No matter how "tough on crime" you are, bringing down recividism rates should be a priority for everyone. If nothing else, then at least for practical reasons.

When you deprive prisoners of meaningful interactions with people on the outside, you are isolating them more and more from the "civilian" population, and civilian life. Which invariably means you're pushing them closer to other prisoners and prison gangs. As a direct result of this, when they leave prison, they won't have anyone else to turn to, in order to get their life back on track. The only option they know, and can easily access, is a life of crime together with their prison-peers.

By not helping prisoners integrate into normal society, you are directly contributing to robbery, gang violence and homicide. That's the very opposite of being "tough on crime".

<comment><USER16> I think you didn't notice the connection between "bringing down recividism rates should be a priority for everyone" and "when they leave prison". Make that "if they leave prison". Recividism is zero if they never leave.

<comment><USER15> I can't tell if you're being sarcastic or serious. Are you advocating life imprisonment for crimes like robbery?

<comment><USER16> I can't tell either. It's just an observation of something that is factually correct. I'm pointing out the coldly rational conclusion. I don't think I intend to advocate either way.

Most robbers will reoffend. That really isn't OK. People get badly hurt and even killed in many cases, often including the robber. The robber may be safer in prison.

</comment></comment></comment></comment><comment><USER17> For profit jail systems are incentivizing some of the worst things in America. Speaking as someone who has a relative in prison, anything that can be charged - will be. Everything from email, food, hygenics, time, visitation... etc. Not to mention the imbalance of sentencing and recidivism rates.

</comment><comment><USER18> Land of the free.

Making money off keeping people in prison, you have to be a cold-hearted SOB in this industry.

</comment><comment><USER14> If the misdeeds of politicians, beurocrats, cops, etc landed them in prison the way the misdeeds of "commoners" did the problem would solve itself.

This is yet another problem that would be solved or mostly mitigated if the groups of people who shape, administrate and enforce the rules of society had to take their own medicine. It's much harder perpetuate an inhumane system for the purpose of screwing over people who screwed up when you and the people around you would have to face that same system if you screwed up.

</comment><comment><USER19> From my limited knowledge about this, I think a big issue about contact visits is smuggling of drugs by the visitors (for profit or under gang-pressure with threat of violence on the inmate). This might help with that. However, thats not an issue for non-contact (behind a glass visits) so not sure why that shouldn't be allowed.

<comment><USER6> I assume almost all contraband comes from employees. Visits are too infrequent and the monitoring of visits makes it too difficult to smuggle at scale via visitation without employee involvement.

</comment></comment><comment><USER20> How are the US even complaining about human rights violations when so many serious ones happen right at home?! It's appalling to see all this

<comment><USER21> Human rights violations are wrong 100% of the time. Whataboutism doesn't change that.

<comment><USER22> Judge not, that ye be not judged. First cast out the beam out of thine own eye; and then shalt thou see clearly to cast out the mote out of thy brother's eye.

Criticism is a lot more effective when it's not hypocritical.

<comment><USER6> You both have a point. All societies that are sufficiently large have quirks and contradictions. You can find inequities and injustices in every country that is large enough. But people should still be free to criticize policies of other countries. Of course, when the hypocrisy is too great said criticisms falls on deaf ears.

Let us strive to be less hypocritical but still vigilant in pointing out human rights violations elsewhere.

<comment><USER21> > Let us strive to be less hypocritical but still vigilant in pointing out > human rights violations elsewhere.

This is better put. Hypocrisy should not blind us all.

</comment></comment></comment></comment></comment><comment><USER23> I'm not gonna comment on the substitution of video calls for in-person visitation nor the kickbacks that prisons get for the offsite calls. We all know that it's garbage, and that companies like Securus are disgusting.

I will say that the silver lining in all of this is that incarcerated peoples can finally still have visitation if someone is unable to physically come to them. Video calls have been commonplace for a quite a while, so maybe (though I'm not holding my breath) some jails and prisons will actually put them to good use.

Sometimes I want to believe that common decency will win.

<comment><USER24> It's great to have video calls but as one comment pointed out, some of the companies selling these systems require the prisons to make them the only option.

</comment></comment><comment><USER25> I'd rather see augmentation than replacement — I bet we could reduce recidivism rates if we gave inmates more time with their families.

</comment><comment><USER26> 1\. This is not human 2\. Any pubishment (In civiliaded world is meant for correction) 3\. If Law permitted human interaction , there ahould not be any reason to stop it. 4\. WRT : Private prisons , this means more surveillance , less policing( need to hire )

Since this is about the inmates, they do it without much resistance , for heavy pockets

</comment><comment><USER27> How can you certain that the video you're watching isn't procedurally generated fake, or there is a gun pointed at the prisoner from behind for not talking about human right violation?

</comment><comment><USER28> Internet access should be one of inalienable human rights.

And prisioners should be able to call home any time in addition to normal visitations.

</comment><comment><USER29> From a nordic european perspective the very idea of privitized jail is a bit too much. Like, really?

This article makes me think about and why I will never leave the north: [http://www.businessinsider.com/an-american-warden- visited-a-...](http://www.businessinsider.com/an-american-warden-visited-a- norwegian-prison--and-he-couldnt-believe-what-he- saw-2014-10?op=1&r=US&IR=T&IR=T)

<comment><USER30> Pretty sure the jail in the article is run by the county and is not privatized. Private prisons are bad and I'm categorically against them, but US public run jails are overwhelmingly more common and they aren't so great either.

</comment></comment><comment><USER31> Did anyone think they would not hate it?

</comment><comment><USER32> if you are an engineer working for one of these companies, you should get a different job as soon as possible.

<comment><USER33> And if you are an unemployed engineer, you should seek out these companies in anticipation that their employees heed your comment.

<comment><USER34> I understand you are a student, and you really need a job, but working for an unethical company is not worth it, for your sanity and the world.

You didn't study for years just to end up making the world a worse place.

<comment><USER33> > You didn't study for years just to end up making the world a worse place.

You are correct, I didn't study for years _just_ to make the world a worse place. I'm not some cartoon villain.

But I also didn't study for years _just_ to make the world a better place.

What I _did_ study for years, was to obtain a valuable skill and a credential thereof. I expect that my lifetime earnings will be significantly higher on account of my education, compared to if I had not perused any undergraduate education; else I would have not spent so much time+tuition+effort.

And if my skill becomes more valuable and rare for certain companies, on account of nothing more than the irrational bias of my peers? Not taking advantage of such an opportunity, is leaving money on the table. It's as irrational as being biased against them in the first place.

> your sanity

I don't understand. Do people actually go crazy whenever their company isn't Robin Hood reincarnate?

<comment><USER35> It's so refreshing to hear today's youth recognize that the profit is the highest ethic we can aspire to.

After years of difficult math and physics, it can be easy to forget that the most important equation: More Money = Good!

<comment><USER33> More Money = Good! Yes! Exactly! How do people fail to grasp this!?

<comment><USER36> I believe the parent was missing a `/s`, and was in fact pointing out how a profit driven ethic is actually quite ruinous.

<comment><USER33> I don't see how that comment was indicating how optimal action is ruinous; but more importantly its words ring true at face value.

<comment><USER36> I can't speak for the parent. But to me it's obvious at face value that the best choice ethically is not necessarily the choice with the most monetary gain as you seem to argue. Exploiting the incarcerated for personal gain falls into that category IMO.

In my experience, there are many opportunities to make a quick buck in life, but taking those opportunities doesn't necessarily lead to a better the world, and can very often make it worse.

If your only concern in life is getting mo' money, don't be surprised when you have plenty of money but a lack of meaningful relationships and personal fulfillment.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER37> These video calls charge around $2/minute as well.

</comment><comment><USER38> Inhumane</comment>


Official Google Research Blog: Market Algorithms and Optimization Meeting - <AUTHOR> http://googleresearch.blogspot.com/2009/01/market-algorithms-and-optimization.html <comment><USER2> Up-voted because it links to this:

<http://research.google.com/pubs/papers.html>

Looks like a lot of interesting reading in there. If you're of a conspiratorial mind, might also give guesses as to what Google is up to (or wants to be up to in the future).</comment>


Time to Say: good bye WordPress? - <AUTHOR> http://ewebdesign.com/ghost-review/

<comment><USER2> This seems a lot like linkbait to me. Most of the post is just reiteration of Ghost's features, and the title intentionally asks an inflammatory question.

<comment><USER3> To be fair, the article's actual title seems slightly less link-bait-y.

</comment></comment><comment><USER4> The most recent episode of In Beta with Gina Trapani and Kevin Purdy ([http://5by5.tv/inbeta/70](http://5by5.tv/inbeta/70)) discusses the current state of WP and specifically addresses Ghost. While they make a sound argument that WP has become a bloated mess compared to its humble origins, they point out that Ghost isn't terribly accessible to the general public because it's node.js based and can be complex to configure. Probably not a huge deal if you know what you're doing, but even as a full time web dev, I don't have a node.js server of my own to screw around with.

</comment><comment><USER5> I don't see anything indicating why someone would migrate from WP to this new platform. WP grew among other things because it was easy to install and 'anyone' could create plug-ins, and now besides there's _a lot_ of themes.

I'm toying with the idea of reusing those WP themes, as in a platform with a drop-in for the themes.</comment>


How To Uninstall McAfee Antivirus [video] - <AUTHOR> http://www.youtube.com/watch?v=bKgf5PaBzyg&feature=player_embedded#!

<comment><AUTHOR> a strange comeback to the public</comment>


Judge Orders FCC to Hand over IP Addresses Linked to Fake Net Neutrality Comment - <AUTHOR> https://gizmodo.com/judge-orders-fcc-to-hand-over-ip-addresses-linked-to-fa-1843202071 <comment><USER2> From the Judge: releasing the logs may help clarify whether fraudulent activity interfered with the comment period, as well as whether the agency's decision-making process is "vulnerable to corruption."

My take: I was one of the flesh-and-blood people who commented. when I did that, I also looked up every comment made by a person with my name. Result: every comment by someone with my name was obvious cut-n-paste.

From my POV, it's clear: the FCC site was swamped with fake comments. Their unwillingness to publicly say so is an abomination in a democracy.</comment>


FunnyBizz:Conference on Tech Startups Using Humor and Storytelling - <AUTHOR>

<comment><USER2> there's a whole conference dedicated to this?

<comment><USER3> Yep. One whole day at least. Should be more! :)</comment></comment>


Man makes money buying his own pizza on DoorDash app - <AUTHOR> https://www.bbc.co.uk/news/technology-52724062 <comment><USER2> BBC summary of Monday's [https://news.ycombinator.com/item?id=23216852](https://news.ycombinator.com/item?id=23216852)

</comment><comment><USER3> From reading the blog post linked from the article, DoorDash doesn't actually make money. Someone who understands finance better, what is the longer term play here? Is it to optimize to grow and then raise prices in the future?

<comment><USER4> Same as Uber &co. Kill competitors by selling at loss and achieve monopoly. Which will never work because there is almost no barrier to entry in these businesses.

<comment><USER5> How is there no barrier of entry?

Very high marketing, legal, etc costs and highly optimized/intelligent routing, scheduling system within an app.

</comment></comment></comment><comment><USER6> Softbank is a money-wasting machine. I will never give them even a penny. They are the worst investors on the entire planet.

<comment><USER7> I am not sure if they are the "worst investors" or just "the ones left holding the bag at the end".

Lots of super smart VC people in the valley just exited a little bit sooner. That's quite literally the only difference. This whole investment ecosystem was built around this, gotta exit ASAP

<comment><USER8> The difference is that VCs exited when they realized that the business model was probably never going to work, while Softbank happily took their place despite likely knowing that the other investors were getting out.

It's like walking into a house party while everyone is running out the back yelling "cops are coming!" and thinking, "this is gonna be a great party once all these screaming people leave!"

<comment><USER7> No, the VCs exit when their mountains of data say to exit. It has nothing to do with the business, just valuations.

If you get in at or before Series A you are going to make money nearly every time. If you get in after that you are going to lose money nearly every time. This has nothing to do with the industry or the business model or smart people making savvy choices.</comment></comment></comment></comment>


Ask HN: What web ads did pleasantly surprise you? - <AUTHOR> <comment><USER2> I never minded web ads from The Deck Ad Network because they were always small, not intrusive, and well-designed. They looked nearly identical to the ads you see on the left column of Daring Fireball these days (a small thumbnail + text blurb)

Of course, The Deck is no more, so go figure. Almost all other web ads I cannot stand. They're always in terrible positions, there are too many on a page, they move content around or cover it, and these are just the aesthetic problems. Security/tracking, battery, and browser performance is a whole different beast, and an arguably far more important one.

Even if all ads were to begin behaving, at this point in my life I would still block them. Ad firms / publishers had their chance and they squandered it. Even now, the issues remain and many of them are not cleaning up their act.

Pi-hole for blocking at network level is a great solution.</comment>


Creating an iPhone Application - Tutorial 2 - <AUTHOR> http://experimentgarden.blogspot.com/2009/07/creating-iphone-application-tutorial-2.html <comment><AUTHOR> This is a sequel to Experiment Garden's wildly popular beginner's iPhone application programming tutorial.

It explains how to create controls and link them to code methods within the iPhone view controller.</comment>


Mpd + ncmpc - perfect combo for iTunes replacement - Tuan Anh Tran - <AUTHOR> http://tuananh.us/articles/mpd-and-ncmpc-perfect-combo-for-itunes-replacement/ <comment><USER2> I wrote my own mpd client with dmenu and a few lines of shell:

[https://github.com/technomancy/dotfiles/blob/master/bin/musi...](https://github.com/technomancy/dotfiles/blob/master/bin/music- choose)

This lets you just type just enough letters to uniquely identify the album you want to play, then hit enter to queue it up. Yay Unix!

<comment><USER3> Nice. I did too: <https://github.com/dbro/muss> . It's a shell script that coordinates mpc and dmenu. It's got two modes of operation, one for interactive browsing and another for a simple command line search. Some examples:

>muss elvis [queues up all songs that have "elvis" in the artist, album, or track name]

>muss elv pres not jailhouse

>muss lion rich -a -r ["Hello, is it me you're looking for?"]

-a means append to existing playlist, -r means randomize the playlist

>muss

when no arguments are provided, it opens up a very fast search/filter/browse interface using dmenu

Inspiration for this project were plait and the then-new instant search on Google.

</comment></comment><comment><USER4> I used to use MPD on Linux and really liked it, and just recently realized that it's possible to use on OSX as well. I somewhat prefer the ncmpcpp client to ncmpc, but ncmpc is fine too.

On that note, I also like that I can switch interfaces (even CLI to GUI) without having to switch to a completely different piece of music-library- management software, thanks to the client/server design. I even use different clients for different things; for example, the simple CLI client 'mpc' is good for using in scripts.

<comment><USER5> ncmpcpp is an awesome player, though sometimes I do wish that I could have a full featured GUI brower like amarok plug into the mpd backend. Are there any good Qt based interfaces?

<comment><USER6> I'm a huge fan of Sonata ( <http://sonata.berlios.de/> ). While it isn't a Qt based interface, I've used it a bunch over the last couple of years and it functions great as a simple GUI frontend for mpd. Anyways, just my 2cents.

</comment></comment></comment><comment><USER7> I've been using MPD (+ mpc, ncmpcpp, ario) for a long time now (several years). There is one thing that I miss from my iTunes usage: the party mix. It was an awesome tool (at least back in iTunes v4 to v6) and I was really always in there.

Basically what I want is to be able to tell my player "randomly select songs in this playlist/folder/…" and then that it shows me at some place the last 5 or 10 songs played and the next 20 to play, and let me remove songs I don't wanna listen to right now from this "songs-to-play" list, and then refill the list up to 20 songs. This way I can repeat the operation a few times and know that I have at least an hour of aweome music to come.

I know I could script that up for MPD, even with just MPC and Bash (and Zenity for the cozyness), but I'm a lazy guy and I can almost reproduce the behavior with the shuffle command of MPD. You know, the worst ennemy of "great" is "good enough".

<comment><USER8> I agree, I'll sound like a freakshow but I actually like iTunes for playlists and its ability to share music libraries easily on a LAN.

<comment><USER9>
      its ability to share music libraries *only with other
      iTunes users* easily on a LAN

Fixed that for you.

</comment></comment></comment><comment> Arcanum-XIII The killer feature of itunes for me is the library management — I drag & drop the mp3's, if they're tagged correctly it will handle them at ease, auto organizing the directory and files and copying them at the right place. If some other tool can do this, even CLI ones, I would gladly hear about them !

<comment><USER10> My iTunes does that just fine.

<comment> Arcanum-XIII Yes, but library sharing with non iTunes/Apple box is no easy pie. Backup and transfer too are a pain in the ass - but well it is not like it happen everyday. And that without even speaking of his weight, which seem to grow with each minor version for no good reason. So while it is (for me) a good tool to organise and browse my library, it's limitation can be irritating, more often than not.

</comment></comment></comment><comment><USER11> Love MPD/ncmpc (it is probably my favorite music player outside of Foobar2000). I use it on my Arch Linux box and it runs like a champ. I also use plain old mpc as well, with some shortcuts defined in my .ratpoisonrc

</comment><comment><USER12> This is fantastic. Thank you for posting it. I plan to use MPD as an alternative to iTunes for the foreseeable future. The number of people clamoring for an alternative to iTunes seems to be growing. I hope Apple gets the message, I'm sick of needing three music players to handle my library.

</comment><comment><USER13> Is there an non-iTunes equivalent to iTunes' Genius? Most of my music listening tends to be of the "I feel like listening to song X and a bunch more like it" and Genius is perfect for that.

If I could get that in a console app, I'd switch in a minute.

<comment> guard-of-terra amarok did that since forever using last.fm similarity data, but it isn't console app (scriptable via dbus but meh)

</comment></comment><comment><USER14> I am using MoC (Music on Console - <http://moc.daper.net>) and really liking it (I'm on Linux). Thought of sharing so the people might like to see another possible option get to see it.

</comment><comment><USER15> I've used MoC, Mpd + ncmpc, and I am currently using cmus (<http://cmus.sourceforge.net/>).

Currently cmus has been my favorite. Just another option.

</comment><comment><USER16> I have been happy with cmus[1] for years, both in Linux and OSX.

[1] <http://cmus.sourceforge.net/>

</comment><comment><USER17> Short of writing your own scripts to control mplayer, mpd+ncmpc is the most simple and most reliable (most difficult to crash) open source UNIX-friendly music player solution I've ever used. And over the years I've systematically tried every player I could find inside the package/ports repositories and outside of them.

These days I just use scripts and mplayer. But for a ready-made solution, MPD fills the "simple and reliable" niche completely, in my opinion. On those two criteria it is without parallel.

On Windows, foobar2000 was great but the source was never available.</comment>


The Digital Marshallese Sovereign - <AUTHOR> https://avc.com/2020/03/the-digital-marshallese-sovereign/ <comment><USER2> I would also talk up idiotic concepts on my blog for his kind of money.</comment>


Milabot: A Deep Reinforcement Learning Chatbot - <AUTHOR> https://arxiv.org/abs/1709.02349 <comment><USER2> Really throwing the whole kitchen sink at the problem here. They're basically running a RL algo over an ensemble of every successful NLP algo of the last decade.

<comment><USER3> Your summary of the paper is both succint and accurate.

What's really interesting to me is that this approach yields state-of-the-art chatbot results, significantly outperforming other, non-ensemble approaches.

It makes me wonder if the subconscious part of the human mind does something similar, i.e., when faced with the task of articulating a response, does the brain subconsciously evaluate a range of possible responses, each generated by different subsystems, and then choose the best of these responses or some mixture of them, to respond?

<comment><USER4> Maybe that, or maybe you have a large number of strategies for generating replies, and you can pick a good one ahead of time, so you don't need to do an exhaustive search.

Compare it to how a human chess player looks at several possible moves as opposed to a computer program which will consider thousands.

<comment><USER5> AI is still very much a search problem -- leveraging what computers are actually good at doing to try to mimic doing things that they aren't good at.

</comment></comment></comment></comment><comment><USER6> If I ask it a question and then ask it the same question 5 minutes later will I get the same answer? If I tell it something about myself and then ask about it later, will it remember? Even the 'smartest' chat bots I have ever 'spoken' with function like someone with no functioning short-term memory or a totally ephemeral identity that alters with every statement.

</comment><comment><USER7> Is it implemented somewhere as a real code I can run?

<comment><USER8> A machine learning paper with actual code? Hah! Not likely.

<comment><USER9> Get thee to github, there's a bunch. Not least the whole cannon of Tensorflow, but in terms of research all of the GAN stuff last year was on git as the papers hit.

</comment></comment></comment><comment><USER10> Slighlty OT: Did run many chatsbots in various fields. Chatbots won't be the next big thing, not at all. Chats are a specific channel for more specific use cases, none of them created any killer app.

Maybe we need more time but chatbots got hyped for too long without any significant traction. Most are web shops/ agencies that build mediocre chatbots for brands nobody is using.

Not bashing chatbots just wondering why this topic gets upvoted again and again.

<comment><USER11> Amazon Echo and Google Home are chatbots, but with a voice interface. They seem relatively well accepted. Your thoughts?

<comment><USER9> I don't think they are - I think they are narrow assistants - order things, search for things, turn things on and off. They aren't there to have a discussion about your brothers emotional response to the Aphex Twin.

</comment></comment></comment><comment><USER12> What's really interesting is how they designed their reward function. At the end of each chat the user can give each conversation a rating. The bot then tries to achieve a higher level of human interaction satisfaction, neat!

</comment><comment><USER13> They say their approach relies on a minimal amount of rules, however the most prominent method in their ensemble is a rule based bot.</comment>


Startup Depression - <AUTHOR> http://arcware.net/startup-depression/

<comment><USER2> Man, I hear you. I've been going through a lot of the same, although without the additional (major) challenge of supporting a family. I find that sometimes it's good to have someone tell me the stuff that I already sort of know, but aren't willing to face:

\- If your wife and kids are the most important part of your life, then you must do what's best for them. I grew up with a parent for whom that wasn't quite true, and our relationship suffered later on. Your kids know when you're stressed out or angry or depressed, and they won't understand why you're making the decisions that you are. So, if you have an opportunity to bring stability and happiness into your life by taking a regular job, _take it_ for pete'ssake. Please. Really, you'll be happier, and if you still have an itch to scratch, do it in your spare time.

\- With your skill set, it is entirely possible that you could take a regular job, and then pay someone else to maintain your projects (bug fixes, basic upgrades, etc.), _and still come out ahead_. I know I consider doing this all the time: I could probably move, get a job paying significantly more than I'm making now, and pay someone to manage my business in its current location, and still have more money than I have now. I am an idiot for not doing this; the world doesn't need two of us. :-)

\- Your customers will understand, or they aren't people that you need to be prioritizing over your own family anyway. And again: just because you're taking a regular job, doesn't mean that your products or services need to disappear overnight.

\- There is nothing glamorous or heroic or glorious about what you're doing now. It's a shit-show. If you have your ego tied up in "pulling through this despite all the odds", _let go of that_.

And finally: on your site, I think you should showcase your products immediately below the header, instead of "Latest Blog Posts" and "Get Our Newsletter". I was a little confused about what it was you actually did when I visited your site.

Best of luck to you. My email's in my profile.</comment>