Dynomite – Making Non-Distributed Databases Distributed - <AUTHOR> http://techblog.netflix.com/2014/11/introducing-dynomite.html

<comment><USER2> It's a little unfortunate that consistency is only mentioned as a footnote. If this is being used internally at Netflix, how are they dealing with consistency issues in the face of failed or conflicting writes?

<comment><USER3> This currently looks like the Dynamo-alike sharding without the read-repair part making values eventually convergent in all the replicas. This latter feature is much needed, but they are already planning it apparently.

<comment><USER2> Except no mention of vector clocks, so it's unclear how one would detect or repair a conflict. Maybe that is an implementation detail that is intended to be ironed out later...

<comment><USER4> With a replication factor of 3 or more you don't need a vector clock, you can just choose majority wins.

That being said, I believe it is using a vector clock, but I'm not sure.

<comment><USER5> You still need ordering amongst writes. Without that you will get into a situation where there is no majority, all three nodes have different data.

<comment><USER3> Converging to a single value (which is a liveness property) and what the values will contain (which are usually safety properties) are different stories... when the partition heals, if the nodes detect they have a different version for a given piece of data, they can just remove it, and the system is convergent, which means it is eventually consistent (a liveness property). _What_ happens in the merge function instead provides additional safety properties. For example in the case of a CRDT that models a Set type, with the merge function that is "union of all the values in the nodes", you can state the safety property that if a Set element X was added into a node, at no time will happen that the nodes will be able to talk again, and the system will converge into a Set which excludes such value X.

</comment></comment></comment></comment></comment></comment><comment><USER6> Netflix is a Java-heavy shop (the client talked up in the blog post is the Java client). I was pleasantly surprised to see that dynomite is implemented in C with an eye towards both BSD + Linux. I'm curious if this was because a GC'ed language like Java or Go was tried and proven not up to the task, or what.

<comment><USER7> My guess would be that the ecosystem in their (seed) target set of databases and protocols is written in C. In one case, they're using Twitters twemproxy to provide connection counts for redis/memcached.

<comment><USER8> Not just connection counts; when I first looked at the github I was surprised at how close the code organization was to twemproxy, and when I looked further a huge amount of the code is built on top of existing twemproxy code.

Netflix probably saw an opportunity to built the Dynamo layer on top of what Twitter had already written, and decided to just keep building on in C instead of rewriting a fairly large codebase.

</comment></comment></comment><comment><USER9> Netflix always makes for an interesting case, but what always strikes me is how incredibly "trivial" their application really is: It's a simple application at a very large scale. This doesn't dismiss their accomplishments, of course, but often it does seem like Netflix spends an enormous engineering effort on fairly low value aspects of the service.

<comment><USER10> Well, they make it seem trivial, while dealing with every watch and every pause, every rating updating complex recommendation engine algorithms. They deal with licensing and region awareness and proxying by ISP and running their own CDNs and...

And that's the magic.

To boot, they do it at _enormous_ scale on a multitude of platforms all while their own destructive code (Chaos Monkey) runs around destroying servers.

It's true, some things like "stream me this" are maybe not rocket science, but at Netflix scale, everything becomes a bit more difficult.

<comment><USER11> Netflix is good at providing the service on any device, and having 100% uptime.

Their requirements are 100% availability, and an API's that never breaks, but consistency rarely matters.

The engineering effort is cool, but it is solving a problem that _I_ don't have. The only thing that matters to netflix is they list the catalog somewhat correctly, and always deliver a stream.

I have a different problem, thus I am uninterested in their _magic_. If I had a similar problem or they signed my paycheck, I'd care.

<comment><USER12> Not to deride what you are saying, but when I see comments that dismiss Netflix as a site that just simply streams movies consistently; it tells me that the commenters haven't seen and appreciated the difficulty of the problems they are actually solving under the hood as evidenced by their github page.

[http://netflix.github.io/#repo](http://netflix.github.io/#repo)

<comment><USER9> Neither that or the number of open engineering positions tells you the difficulty of the problems they are solving. It might just tell you the difficulty of the problems they are _making_ : If Netflix has a technology side with a history and the executive ability to overcomplicate, they might be making little problems into big problems. My perspective as an outsider is to look at the complexity of their app platform (again, separating the video streams which happen from systems that never seem to be the foundation of these tech blog entries), and it is unarguably not a complex system.

Early on I remember the heroics that Netflix went through running on EC2, building numbers of complex tools for varied performance, reliability issues, etc. At the time many much larger sites just quietly _worked_ 24/7, minus the heroics, and minus the effort, often by using purpose-suited dedicated servers. Later, once Amazon rolled out SSDs, Netflix triumphantly announced how much of an improvement it was to their product, again demonstrating that they were creating a problem (huge numbers of horrible I/O machines) that they then solved with gusto, albeit unnecessarily.

I'm not trying to be overly down on Netflix, but it is a company that seemed to make tech blog entries a product of the company years ago, and the result is that people have bought into this notion that they're doing some hugely complex task. They aren't.

<comment><USER12> Sorry but you are just flat out wrong. Operating at the scale that they are is in fact complex. They aren't creating all of this software just because they are having fun in their free time.

<comment><USER9> _They aren 't creating all of this software just because they are having fun in their free time._

Sorry, but this is an invention of yours. No one said they're doing anything flippantly.

But they may be doing it _ignorantly_ : Success in one field (in this case turning a mail subscription business into a stream service) doesn't imply technical innovation or leadership, and often is despite it.

This industry is absolutely rife with people creating solutions to problems they themselves invented and caused. In the case of Netflix, an enormous amount of their solutions have been founded around the notion of deploying on huge numbers of miserable Amazon EC2 instances, and then dealing with the problems related to that. Others simply deployed distributed data centers hosting their own purpose suited, reliable hardware with big fat storage arrays, and the problem is solved. It's like trying to build a car out of toothpicks and then detailing the innovations you created in toothpick redundancy and robustness.

<comment><USER12> Sorry but this just reeks of naiveté. When the few areas of tech that I have actually been involved with in the real world come up on HN there are always comments stating that things are just so simple and why so and so is doing it wrong and why it so much easier if they just do something simplistic...

That just isn't the case in the real world when working with tens of millions of customers. This will become apparent when you actually get out there in the field and realize systems involving tens of thousands of machines hosting dozens of different services to millions of people is actually not simple.

<comment><USER9> Starting with a patronizing "sorry", looking down from heights, doesn't somehow give your comment more credibility.

There are a large number of services of the user scale of Netflix, with _dramatically_ more complexity, that don't seem to have the heroic issues that Netflix does. Facebook and Reddit both are of a complexity scale multiple- magnitudes greater than Netflix. Someone else absurdly tried to draw Apple and Google in, but again, what they do absolutely dwarfs the entirety of Netflix's operation, to the point that it becomes almost laughable.

 _Netflix is a very simple application_. Scaling it can be difficult, of course, but this is hardly some new grounds. Further, the particularly make-up of Netflix makes it one of the most profoundly scalable platforms going (silos, little to no need for transactional integrity. It is profoundly simple). You don't have to believe this, but your comically patronizing responses just sound...laughably unskilled.

<comment><USER12> > Netflix is a very simple application. Scaling it can be difficult, of > course, but this is hardly some new grounds. Further, the particularly make- > up of Netflix makes it one of the most profoundly scalable platforms going > (silos, little to no need for transactional integrity. It is profoundly > simple). You don't have to believe this, but your comically patronizing > responses just sound...laughably unskilled.

Have you actually work with any of their backend engineers, seen any of their infrastructure and why they made the design decisions they did, or actually talked to anyone at Netflix, or is this all just speculation?

I'm sure they'd love to hire you to to greatly simplify their overly complicated architecture otherwise...

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER4> When I saw this internally a few months ago, I said that I haven't been more excited about a new project in a long time. I personally think this will solve a lot of scalability problems people have while letting them keep their favorite database.

<comment><AUTHOR> It looks very promising. I like the fact that it doesn't only work for AWS environments.

</comment></comment><comment><USER13> Funnily enough Dynomite was also the name of an Erlang implementation ([https://github.com/moonpolysoft/dynomite](https://github.com/moonpolysoft/dynomite)) of Dynamo that seemed to get off to a very promising start before it was abandoned a few years ago.

</comment><comment><USER14> Cool, it's something that Microsoft SQL Server (Enterprise edition) offers out of the box (Peer-2-Peer replication). The only problem with P2P replication is that, at large scale, requires massive amounts of bandwidth to keep everything synced, and it gets worse the more nodes you add.

<comment><USER15> Depends how things are replicated. In Dynamo style replication everything isn't replicated everywhere. Based on consistent hashing, and how vnodes etc are configured, a key lands itself in more than one nodes so that things are generally available. So scale-out in a dynamo-like system happens more gracefully.

</comment></comment><comment><USER16> Any ideas if this supports Postgres?

Also, I noticed comments about an architecture document here: [https://github.com/Netflix/dynomite](https://github.com/Netflix/dynomite)

Anyone know where the doc they are referencing is located?

<comment><USER4> It doesn't support Postgres yet but that is on the (longer term) roadmap.

However, since it is open source, contributions are welcome! :)

</comment></comment><comment><USER17> I don't know if it was intentional but the name reminds me of the character JJ in the TV series Good Times. Awesome show that was. It sounds solid overall but one thing caught my eye, "A client can connect to any node to read the data. Similar to writes, a node serves the read request if it owns the data, otherwise it forwards the read request to the data owning node in the same rack.". I wonder if that could lead to an owning node arbitration storm when more than one node in a rack owns the data?

</comment><comment><USER18> For projects with similar goals, see also Twitter's Gizzard (abandoned) and LinkedIn's Voldemort.

<comment><USER19> Also Riak. However, one interesting aspect of Dynomite seems to be offer default support for clustering across multiple data centers with pluggable stores.

<comment><USER20> Riak (enterprise version) does that, and has pluggable stores (albeit limited to in-memory, bitcask + eleveldb)

</comment></comment></comment><comment><USER21> Perhaps I missed it, but is there any discussion on how their sharding mechanism works? How do they define a shard and where is ownership of various shards managed so a "non-owning node can redirect to an owning node"?

<comment><USER22> I questioned tbat as well, could be a property of the underlying store (memcache hash). How would mysql sharding work without being application managed?

</comment></comment><comment><USER23> It's interesting that Netflix decided to write this rather than using Amazon's own DynamoDB. I wonder specifically if DynamoDB was too expensive (as I have found), or was there some other reason?

<comment><USER24> This is completely different than DynamoDB. Essentially this is a framework for sharding or horizontally scaling multiple datastores (MySQL, Memcache, Redis, etc).

<comment><USER25> Why would you use multiple datastores? Isn't it much more difficult to _atomically_ store data this way? Assume a transaction succeeds in store 1, and fails in store 2, you'd need to roll back the transaction in store 1 after it has committed (!)

<comment><USER26> Because there is no such thing as a one size fits all datasotre. Some data may make way more sense stored in a relational database, while some data makes more sense as key:value pairs. Some data may be infrequently accessed and stored to disk while others are stored in memory for quick access. Trying to make a single datastore work for many use cases will cause more pain than it is worth.

</comment></comment></comment></comment><comment><USER27> nitpick, the code check for NULLs at a low level but misses checking NULLs at higher levels like here:

[https://github.com/Netflix/dynomite/blob/master/src/dyn_dnod...](https://github.com/Netflix/dynomite/blob/master/src/dyn_dnode_msg.c#L601)

</comment><comment><USER28> Are there any deployment notes/examples?

<comment><USER29> Setup and Installation Wiki - [https://github.com/Netflix/dynomite/wiki/Getting+Started#set...](https://github.com/Netflix/dynomite/wiki/Getting+Started#setup- and-installation)</comment></comment>


Painting a better life through side SaaS hustles - <AUTHOR> https://getmakerlog.com/stories/micah-iverson-founder-and-ceo-of-retrospect <comment><USER2> Interesting to hear another story of someone finding space in a crowded market segment.

> I was working for a company a couple of years ago and we needed a tool to do > retrospectives and the existing solutions we found were not designed well > and didn't work all that great, not to mention the bigger solutions costing > a lot of money to use for a relatively small need that we had. I set out to > build a simple, affordable solution that looked good too.

...

> Retrospect has come a long way since the initial launch about two years ago, > we have provided more permissions, more functionality, and improved the > experience. The future has a lot of potential for new features and growth.

More features and more integration... Seem like most products work towards becoming the next complex, do-everything software, only to be replaced by yet another "simple" solution without the endless bells and whistles. They both have their place, but sometimes it's hard to find a simple software solution whose owners don't aspire to be the next big complicated solution. Perhaps I'm reading too much into interview...

<comment><USER3> Hey, this is Micah the creator of Retrospect.

I 100% agree one "bloat" risk of adding more and more functionality. We have been very careful not to build anything that is complex or clutters the experience. One potential area that we need to do better on is the "team" experience though. Right now every user is siloed into their own area, with the tool being so "company" focused it makes sense to bring some company based features. (Shared/Bookmarked Boards, Grouping, Manage Users, etc.) With that said, we may not either, it might not make sense bloat wise.

We have no interest in trying to "beat" our competitors (Trello essentially) or match all their features, we simply want to compliment other tools that people use.

Thanks for the feedback! Micah</comment></comment>


Terahertz quantum cascade lasers could lead to data transfer at 100 Gbit/s - <AUTHOR> https://nuadox.com/post/190776789317/terahertz-quantum-cascade-lasers-100git-s <comment><USER2> ?

In laboratory one-terabit over single fiber was achieved in 2009, multi-modal cable records are something like 150 Tb/s.

In the field single-carrier terabit-per-second transmission on a deployed fiber network trials have already been done. [https://www.nokia.com/about- us/news/releases/2019/09/30/noki...](https://www.nokia.com/about- us/news/releases/2019/09/30/nokia-completes-worlds-first-single-carrier- terabit-per-second-field-trial-sets-optical-transmission-capacity-record-over- etisalats-fibre-network-in-uae/)</comment>


Google Chrome In-Browser Advertising - <AUTHOR> http://imgur.com/gi55w

<comment><AUTHOR> Has anyone else seen this? I just found it in my browser. The link leads to:

      http://www.google.com/chromebook/index.html#utm_campaign=en&utm_source=en-ntp-holidays-na-us-bkws&utm_medium=ntp-holidays

Personally, I'm appalled that Google would reach into the browser for advertising. To me it's analogous to Microsoft bringing me pop ups in Windows to show me their new phones. I understand that the ad is small and unobtrusive, but it really feels like an invasion of sacred ground to me. I expect advertising when I venture out into the web, through the pages I request. I don't expect it injected directly into the programs I use.

<comment><USER2> Google makes the lion's share of its profits from ad revenue. Chrome is a big billboard. A shiny, speedy billboard, but a billboard nonetheless. It's no big surprise for them to want to monetize it eventually.

I would imagine the reason Microsoft never entertained that idea with IE is because Microsoft's core business is selling software, not selling ads. If you're running IE, you already gave MS money, either directly or indirectly.

</comment></comment><comment><USER3> That's what you get when you use a browser from a company that makes ~97% of it's revenue from advertising.

When people say "the user is the product" in relation to Google, they aren't joking.

<comment><AUTHOR> I understand that. That's why I'm perfectly happy to see advertising in my search results. I would not have minded if the ad was placed on the Google home page or at the top of my search results, but this ad is showing up in my software, on my own computer. I haven't seen any other reputable browser do that (without annoying toolbars installed, that is).

<comment><USER3> And no other browser maker makes the vast majority of its revenue from advertising.

Google aren't in the business of making web browsers, or tablet and phone OS' out of the goodness of their heart. it's all about pushing users to use their services, so they can mine that usage for data, so they can serve more ads, more effectively.

</comment></comment></comment><comment><USER4> Reminds me of the altavista toolbar that came with their free dial up</comment>


Joe Armstrong: Solving the wrong problem - <AUTHOR> http://joearms.github.com/2013/03/28/solving-the-wrong-problem.html <comment><USER2> I worked in Cray's compiler department for seven years. If we couldn't dramatically parallelize someone's code, we couldn't sell a vector supercomputer. Period.

Automatic parallelization is very possible. The problem is tends to be less efficient. A decent developer can often do a better job than the compiler by performing manual code restructuring. The compiler cannot always determine which changes are safe without pragmas to guide it. With that said, our top compiler devs did some amazing work adding automatic parallelization to some awful code.

We inevitably sold our supercomputers because we had application experts who would manually restructure the most mission-critical code to fit cache lines and fill the vectors. Most other problems would perform quite adequately with the automatically-generated code.

What this article lacks is a description of why Erlang is more uniquely suited to writing parallel code than all the other natively parallel languages like Unified Parallel C, Fortran2008, Chapel, Golang, etc. There are so many choices and many have been around for a long, long time.

<comment><USER3> Erlang is not designed for parallel programming; it is designed for concurrent programming. These are two very different programming domains with different problems.

Every time someone conflates parallelism with concurrency...everyone gets very confused.

<comment><USER4> Isn't it really fair to say that it's designed for both? The way it uses immutable state and something-similar-to-s-expressions to express data make it very straightforward (or even transparent) to distribute work between multiple processes and separate computers, in addition to how it makes it practical and simple to break work into small chunks that can be interleaved easily within the same thread. It's really designed for doing both very well, wouldn't you say?

<comment><USER3> Not at all. Erlang isn't useful for modern parallel computing as we know it, which is usually done as some kind of SIMD program; say MapReduce or GPGPU using something like CUDA. The benefit doesn't just come from operating on data all at once, but these systems (or the programmer) also do a lot of work to optimize the I/O and cache characteristics of the computation.

Actor architectures are only useful for task parallelism which no one really knows how to get much out of; definitely not the close-to-linear performance benefits we can get from data parallelism. Task parallelism is much better for when you have to do multiple things at once (more efficient concurrency), not for when you want to make a sequential task faster.

Maybe this will help

[http://jlouisramblings.blogspot.com/2011/07/erlangs- parallel...](http://jlouisramblings.blogspot.com/2011/07/erlangs-parallelism- is-not-parallelism.html)

and

<https://news.ycombinator.com/item?id=2726661>

<comment><USER5> "modern parallel computing" ... well not everything that can run parallel on multi core CPU's can run very well on a GPU.

I'm using Erlang and GPU programming each for its area of expertise. FWIW I even use both together via <https://github.com/tonyrog/cl>

Erlang is great at asynchronous concurrency which happens to be able to run in parallel well because of how the VM is built.

GPU's solve totally different problems

<comment><USER3> Yes, erlang is great for concurrency, GPUs are great for significant scalable parallelism. They both solve different problems, I agree, and that's my point.

</comment></comment></comment></comment></comment></comment><comment><USER6> > At this point in time, sequential programs started getting slower, year on > year, and parallel programs started getting faster.

The first part of this statement is plain wrong. Single thread performance has improved a lot due to better CPU architecture. Look at <http://www.cpubenchmark.net/singleThread.html> and compare CPUs with the same clock rate, where a 2.5 GHz. An April 2012 Intel Core i7-3770T scores 1971 points while a July 2008 Intel Core2 Duo T9400 scores 1005 points. This is almost double the score in less than four years. Of course, one factor is the larger cache that the quad core has, but this refutes Armstrong's point that the multicore age is bad for single thread performance even more.

For exposure to a more balanced point of view, I would highly recommend Martin Thompson's blog mechanical-sympathy.blogspot.com. It is a good a starting point on how far single threaded programs can be pushed and where multi- threading can even be detrimental.

Also, I think that fault tolerance is where Erlang really shines. More than a decade after OTP, projects like Akka and Hysterix are finally venturing in the right direction.

<comment><USER7> Your point is absolutely correct, but your example could be better IMO. It's not fair to compare a desktop chip with 45W TDP with a laptop chip rated at 35W. Not to mention that the newer i7 actually goes up to 3.7Ghz turbo (vs. 2.5Ghz constant for the C2D) for single threaded loads, so the clock rate is not really comparable in that benchmark (even though base clocks are the same).

A better example would be C2D E8600 @ 3.33Ghz and i5 3470S @ 2.90GHz (3.6Ghz turbo). They are both 65W desktop parts, and the single threaded clock speed is similar. You can see that the C2D gets 1,376 in the single threaded benchmark, while the i5 gets 1,874. The difference is not as drastic (the C2D launched at a significantly higher price point as an enthusiast level chip, while the i5 is a budget chip) but definitely still significant. There are probably even better comparisons but I didn't spend too much time picking out comparable CPUs from different generations.

<comment><USER6> True, good point. I tried to find something with a similar nominal clock speed and forgot about Turbo. But then, Turbo is a good example why single threaded performance is getting better even in the age of multicores.

</comment></comment></comment><comment><USER8> I cant help but read a lot of irony in this.

Erlang solved a problem really well over 20 years ago, its the sanest language by far that I have used when dealing with concurrent programming. (I havent tried go or dart yet) and I owe a lot of what I know to the very smart people building erlang.

However it has barely evolved in the last 10 years, will 2013 be the year of the structs? (I doubt it), every new release comes with some nice sounding benchmark about how much faster your programs will run in parallel and there is never a mention of whats actually important to programmers, a vibrant ecosystem and community, language improvements that doesnt make it feel like you are programming in the 80's. Better constructs for reusing and packaging code in a sane way.

Its fairly trivial in most languages to get the concurrency you need, I think erlang is solving the wrong problem in 2013.

<comment><USER9> I was following you until your last sentence. I've never done concurrency in a FP language before, but I do know that writing it in Java makes it hard to get right.

<comment><USER10> What prevents you from implementing an actor/message passing system in Java?

Erlang's core concept of concurrency seems like something that'd be better suited as a library and app server than a whole language and runtime.

I've yet to hear of any Erlang-specific magic that cannot be implemented inside another language.

<comment><USER11> How would you get per-actor heaps that cannot be violated by other actors? That is critical to Erlang's ability to recover from processes dying. I spent a lot of time doing Java and can't think how you could (you could in the JVM if you had language constructs for it, but then we are back to a new language).

There's a reason Stackless Python's actors aren't just a library on top of Python.

<comment><USER12> TLABs are a partial step in that direction at the JVM level. You could also use object pools/factories keyed to the thread.

Those are the first two "ghetto" hack solutions I can think of that wouldn't require significant code changes on a going-forward basis.

<comment><USER11> But those hacks wouldn't provide the same guarantees that language-level changes provide. Sure, you can try not to impact other thread's heaps, but nothing is stopping me, which means a simple programming error has the potential to impact multiple threads. As a result, you can't just "reboot" that thread (a critical piece of what makes Erlang interesting), because you have no guarantees its errors didn't impact other threads. You also have no guarantees that the underlying libraries aren't mucking up all of your carefully crafted memory management.

It's like the kernel protecting memory so applications can't overwrite each other. Sure, applications could just write to their own memory, but nobody actually trusts that model[1]. Instead, they want something below that level enforcing good behavior.

1\. Obviously, virtual memory adds a wrinkle to this that kind of forces kernel protection, but even if we had literally unlimited RAM, we would _still_ implement kernel protections on memory.

<comment><USER12> Unfortunately, with any native code interaction, whether explicit FFI/NIF or implicit (RNG, clock, etc.), your threads can still mingle state.

In a single process, you're still limited by the memory virtualization capabilities of your underlying processor and MMU.

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> Same old hype. Erlang is good I guess, and I've used it in production a couple of times. But it's just a language that solves 3 problems but creates another 30. Just like C++11, Dart, Go, etc.

This kind of belligerent rhetoric (we're solving the right problems, everyone else is dumb) is the kind of drivel that gives momentum to language zealots that think language X is better than language Y.

I've contributed to Google Go in the early phases and I was naïve and really believed that Go was the "next big thing." But it turned out to be yet another general-purpose language with some things that were really interesting (goroutines, garbage collection, etc.) but some things that were just same-old same-old. Now, I'm editing a book about Dart and I've since lost my enthusiasm for new languages; I can already see that Dart solves some problems but often creates new ones.

And in a lot of ways Erlang sucks, too. The syntax is outdated and stupid (Prolog lol), it has weird type coercion, memory management isn't handled that well (and many more). Of course, since Facebook uses it, people think it's a magic bullet (Erlang is to Facebook like Python is to Google).

The article also forces readers to attack a straw man. Often times, algorithms simply _cannot_ be parallelized. The Fibonacci sequence is a popular example (unless you use something called a prefix sum -- but that's a special case). So in many ways, the rhetorical question posed by the article -- "but will your servers with 24 cores be 24 times faster?" -- is just silly.

<comment><USER14> > The syntax is outdated and stupid (Prolog lol),

How is it any more or less stupid than curly bracket.

Show me another production ready language that has the same level of pattern matching as Erlang.

> Often times, algorithms simply cannot be parallelized.

Who cares. How many people here have implemented individual algorithms and delivered them as units of execution. Sure middleware companies maybe sell a cool implementation of A* or some patented sort algorithm.

You can think of parallelization at system level. Can you handle client connections concurrently (and in parallel?). If yes, that covers a large chunk of the usage domain for platforms these days.

> memory management isn't handled that well (and many more).

Can you expand on that?

<comment><USER15> Haskell has nicer pattern matching capabilities.

<comment><USER5> No it hasn't :-) Just to stay on your level of detail

</comment></comment></comment></comment><comment><USER16> We used Erlang several years ago. The code base has ~100k lines of code so it should be representative. We abandoned it later and switched to C++ because of performance (mostly in mnesia) and quality issues (some drivers in OTP). We didn't expect too much from performance considering it is functional (which seldom does in place update) but it is still below expectation.

It is understandable though. Just think about how much resources have been put into development of Erlang VM and the runtime/libraries(OTP), and compare it with JVM/JDK. There is just no magic in software development. When talking about high concurrency and performance, the essential things are data layout, cache locality and CPU scheduling etc for your business scenario, not the language.

<comment><USER17> The tests here confirm your experience, Erlang is for many algorithms significantly slower, even when more cores are used:

<http://benchmarksgame.alioth.debian.org/u64q/erlang.php>

<comment><USER18> Do not use Erlang for number performance. I tried with EulerProject and create some prime number generators in Erlang. It is slow as hell.

Use it for what it is built for!

</comment></comment></comment><comment><USER19> If zlib could be rewritten in Erlang to be lock-free, why not just rewrite it in C to be lock-free instead of porting it? AFAIK Erlang isn't some magical language that allows traditionally-locked data structures to become lock-free.

<comment><USER20> Well, zlib is fairly trivial and probably not a good example due to overheads. However, an example such as a torrent server this would make much more sense. That being said, Erlang is basically a scripting language for building fault- tolerant and parallel applications.

Using C, you might be able to get parallel, but it'll be a lot of work to make it distributed and fault tolerant.

The underlying data structures have little to nothing to do with what's being said in the article.

<comment><USER19> I've looked at Erlang before, and I would certainly agree that it's far simpler to write a concurrent application in Erlang than it would be in C.

I'm just taking issue with the bit at the end, where they're bragging about removing a serial bottleneck by rewriting zlib in Erlang in order to remove a lock. Rewriting it in Erlang really doesn't have anything at all to do with switching to a lock-free data structure.

<comment><USER20> Ah yeah, I had to read it a second time to realize what you meant. That's true, that it's a bad example and doesn't make much sense.

I _think_ what they meant to say was that they parallelized the image processing mechanism of the application as a whole.

</comment></comment></comment></comment><comment><USER21> There's one big problem Erlang couldn't solve that I live with to this day :

Unlike another general purpose language (like say, C++ or C#) allow me to grasp what's happening after staring at it for 30 seconds. This is the same problem, I have with Lisp.

Maybe I'm just dyslexic, but these rhetoric pieces for one language or another that says it's concurrent (which it is), fast (obviously), more C than C, will bring the dead to life, create unicorns and other wonderful, fantastic things that I'm sure are all true, just don't seem to be capable of passing into my grey matter.

You know another thing all these amazing super power languages haven't been able to do that even a crappy, broken, in many ways outright _wrong_ , carcinogenic etc... etc... language like even PHP has allowed me to do? Ship in 48 hours.

Before, I get flamed, I already tried that with Nitrogen (<http://nitrogenproject.com>). It didn't end well, but maybe it will work for someone already familiar with Erlang.

It's like you've written the Mahabharata; it'a a masterpiece and it's one of the greatest epics of all time. Unfortunately, it's written in Sanskrit.

<comment><USER22> > Unlike another general purpose language (like say, C++ or C#) allow me to > grasp what's happening after staring at it for 30 seconds. This is the same > problem, I have with Lisp.

I had the same problem with Lisp (Scheme, to be more specific) and I thought that it'd be impossible to reason about run-times and such. That is, until I learned the language and the standard libraries. I've never looked at Erlang, but I'd bet it's the same issue.

A C++ programmer can look at C# code and figure out what it's doing because they have similar syntax and vocabulary. Just because Erlang isn't immediately accessible to you, it doesn't mean it isn't any good for shipping in 48 hours.

Perhaps if you spend 24 hours sharpening your axe, you'll chop that tree down in another 4 hours instead of using the full 48.

<comment><USER21> I used hyperbole of course, but in fairness, I've spent on the order of several months getting into Lisp. Mabye it's because I didn't work with it exclusively (I've been told here on HN that you need to be immersed in it completely and continously) that I still didn't become _OK_ in any semblance of the term.

But here's the thing: I switched to C# from VB.Net. Before that, it was VBScript (ASP pages) and even before that it was PHP and JavaScript. At no point did I _stop_ in the language I'm currently working in to start learning another.

Until that changes, I don't see how it will help me.

<comment><USER23> Basically, I would say that what you are suffering from is a kind of mental block syndrome: you think in a procedural/imperative paradigm. All your listed languages operate in that paradigm. It's a very transferable paradigm, as it so happens. I can come up to (some approximation of) speed in an imperative language in under 2 weeks. In order to ship Lisp(Prolog, Haskell...), you _have_ to break out of that paradigm. I am not condemning you, mind. It is what it is. Rewiring your head is hard, and often doesn't have direct results.

I can, however, ship with Common Lisp, because I've spent on the order of 5 years learning it and writing it most evenings. I am learning Clojure and am preparing to ship a (excruciatingly minor) product with that after only maybe two months of dabbling. This is possible because I've bent my head around into Lisp shapes.

It's also been said that some people have the shape of Lisp in their head, and when they learn Lisp, their heads fit it by nature, and other people don't have that innate meshing. I certainly found Lisp to mesh with my head well.

Oh yes. It can be hard to get started with Common Lisp, just in terms of getting an environment working. I have a tutorial site to help with that( _plug plug plug_ ): <http://articulate-lisp.com/>

<comment><USER24> > you have to break out of that paradigm.

That's why I never regretted trying to learn Common Lisp some time ago, even though I didn't ship anything into production, and why I really do enjoy doing the same thing with Erlang right now, i.e. trying to understand it and getting as comfortable with it as I can get (and preferably this time maybe putting something out there).

Both these experiences helped me see programming differently, a change of "paradigm" as you very well put it, so now even when I get back to Python or PHP I feel like I'm a better programmer. Plus, there's something to be said about the fact that always trying to learn new and interesting stuff and not only focusing on "shipping code" is what keeps one's passion at higher levels. After almost 10 years in this trade I've found that passion at what you're doing is a very valuable and in the same time very volatile resource.

</comment></comment></comment></comment></comment><comment><USER25> zlib is fine as long as you don't give an non-threadsafe memory allocator - see <http://www.gzip.org/zlib/zlib_faq.html#faq21>. As far as I can tell, it either means that the summary was imprecise and the slowdown was in the image processing code and not zlib or that they chose to rewrite (and debug) a big chunk of code rather than read the zlib documentation.

Ignoring that point, this seems like a poor point for comparison as it's a trivially parallelized task because zlib operates on streams and shouldn't have any thread contention. There's very little information in the description but unless there are key details missing, this doesn't sound like a problem where Erlang has much interesting to add. The most interesting aspect would be the relative measures for implementation complexity and debugging.

</comment><comment><USER26> Because the lock-free Erlang meme doesn't die:

      1. Erlang has locks and semaphores [1], receive is a lock, actors are semaphore. Erlang chose a 1 semaphore/ 1 lock per process model
      2. Erlang scales better not because of being lock-free (see above), but because it easily uses async compared to other languages
      3. Async prevents deadlocks not Erlang being lock-free (see above)

Some 4year old reading [http://james-iry.blogspot.de/2009/04/erlang-style- actors-are...](http://james-iry.blogspot.de/2009/04/erlang-style-actors-are- all-about_16.html)

[1] <http://en.wikipedia.org/wiki/Semaphore_(programming)>

</comment><comment><USER27> > _We're right and the rest of the word is wrong. We (that is Erlang folks) > are solving the right problem, the rest of the world (non Erlang people) are > solving the wrong problem._

> _The problem that the rest of the world is solving is how to parallelise > legacy code._

As member of the rest of the world, I can assure you that I'm not trying to solve either of these problems. :p

</comment><comment><USER28> People have been thinking this, that it's vastly better to design for concurrency upfront, for literally decades. And every single time there has been a big sea change in processor technology it's always been the _next_ generation which will see things like VLIW or Erlang and so forth come to the fore while what I will call "iterative advancements" and "patched solutions" turn out to have too many weaknesses to be competitive. In reality the reverse has happened, and new specialized languages and instruction sets have been relegated to niches.

It'll be the same over the next 20 years as well.

I predict that we'll see a lot of technological leaps which will serve as much to maintain the ability to run "old code" in new and interesting ways as to enable a brave new world of purpose-built languages.

In the next few decades we'll see advances in micro-chip fabrication and design as well as memory and storage technology (such as memristors) which will result in even handheld battery powered devices having vastly more processing power than high-end workstations do today.

Is that an environment in which one seeks to trade programmer effort and training in order to squeeze out the maximum possible efficiency from hugely abundant resources? Seems unlikely to me, to be honest.

Indeed, it seems like the trend of relying on even bloatier languages (like Java) will continue. Do you think anyone is going to seriously consider rewriting the code for a self-service point-of-sale terminal in Erlang in order to improve performance? That's not the long pole, it never has been, and it's becoming a shorter and shorter pole over time.

In the future we'll be drowning in processor cycles. The most important factor will very much not be figuring out how to use them most efficiently, it'll be figuring out how to maximize the value of programmer time and figuring out how to use any amount of cycles to provide value to customers effectively.

(I think that advancements in core, fundamental language design and efficiency will happen and take hold in the industry, but mostly via back-door means and blue sky research, rather than being forced into it through some impending limitation due to architecture.)

<comment><USER29> Can I rephrase slightly (and take the odd strawman liberty)

The "mainstream" has been relying on incremental improvements for decades, and in doing so avoided rewriting legacy code until last possible moment

Some people have taken concurrency upfront and anecdotally seen cost / performance benefits plus more modern codebases and have anecdotally enjoyed competitive advantages in areas where concurrency makes a difference

We will never see the average, user interface bother with concurrency and legacy rewrites because the competitive advantages are low.

There are likely to be areas where the concurrency advantage is great enough - if you like erlang look for those niches

<comment><USER28> Yes, precisely.

It's like designing a race car, or a fighter jet. Sure, they are amazing things. But are people ever going to commute to work in anything resembling a Bugatti Veyron or an F-22? Of course not. Neither maximum automotive performance nor air combat effectiveness are the sorts of things that are normally necessary to optimize for in daily life. Some time in the far future we're going to have both the tools to write amazingly efficient programs and to do so with a minimal amount of fuss from the programmer's perspective, but it'll be a long time getting there. And in the meantime there are going to be plenty of cycles of figuring out how to produce performance gains with the least disruption to existing ways of doing things.

<comment><USER29> Puuuuhhhleeeeaaaseeee can I commute to work in a Bugatti Veyron??

Please please please :-)

Edit: sorry unable to resist. However I am on Joe Armstrongs side - I would far rather make a decent living doing fun Erlang work than be in a java shop making the next generation of POS

Added to that I think _not_ using Erlang or some STM based concurrency language _must_ be an informed decision - if the CTO of big bank says we have tried two pilot projects rewriting the ATM network in Erlang and the projected costs do not add up, fine. If he says "I have two hundred java coders, we aren't moving". I don't think that's valid

<comment><USER28> Of course, most people would like to be working on race cars, or spacecraft, or fighter jets, but that just isn't an option for every body. And it's not as though there's no in between. The choice isn't just between some soul sucking blub-job in the enterprise trenches or using Erlang, there are lots of languages, lots of development patterns, lots of products.

<comment><USER29> I would agree, but with the proviso that the spectrum between soul-sucking and cool-space-tech is not a nice linear graph - in my experience its step- gradients, some companies are entirely on one level, and then they have to make a real effort to climb to the next (i.e. From manual deploys to CI)

Its actually a consultancy opportunity (I hope :-0)

</comment></comment></comment></comment></comment></comment><comment><USER30> The lack of understanding is amazingly widespread. I often have to explain to people that when they look at their CPU utilization and it is at 10% it means "you are throwing money way", not "you are efficient".

<comment><USER31> That's not really true though, or at least not on all workloads: much as you are not "throwing money away" by not pegging your car engine in the red zone 100% of the time, you're not throwing money away by not being at 100% CPU all the time, there are other metrics, values and issues to take in account e.g. a pegged CPU but an unresponsive computer is useless for a desktop, a pegged CPU which can't serve requests because the CPU is pegged _because_ it's swapping like mad is useless for a server, so is a server at 100% CPU when there's no load on it which will just keel over when people start trying to actually interact with it.

<comment><USER32> It sounds like you missed the point here. If an eight-core server is at 10% utilization, it effective has a single processor nearly pegged and the process doing it is thus CPU bound (and maybe serving responses at a high latency) while you have other cores sitting idle. Conserving CPU resources and running under capacity is wise, but has nothing at all to do with this comment.

</comment></comment></comment><comment><USER33> This blog post shows everything that is wrong with languages like Lisp and Erlang. This is total disregard for that the rest of the world considers valuable to them.

The problem with these languages remain unchanged. The syntax is so strange and esoteric, learning and doing anything basic things with them will likely require months of learning and practice. This lone fact will make it impractical for 99% for all programmers in the world.

No serious company until its absolutely unavoidable(and situation gets completely unworkable without it) will ever use a language like Erlang or Lisp. Because every one knows the number of skilled people in market who know Erlang, are close to zero. And those who can work for you are going to be crazy expensive. And not to mention the night mare of maintaining the code in this kind of a language for years. There is no friendly documentation or a easy way a ordinary programmer can use to learn these languages. And there is no way the level of reusable solutions available for these languages as they are for other mainstream C based languages.

In short using these languages attracts massive maintenance nightmares.

The concurrency/parallelisation problem today is very similar to what memory management was in the 80's and 90's. Programmers hate to do it themselves. These are sort of things that the underlying technologies(Compilers/VM's) are supposed to do it for us.

I bet most of these super power languages will watch other pragmatic languages like Perl/Python/Ruby/Php etc eat their lunch over the next decade or so when they figure out more pragmatic means of achieving these goals.

<comment><USER34> >> I bet most of these super power languages will watch other pragmatic languages like Perl/Python/Ruby/Php etc eat their lunch over the next decade or so when they figure out more pragmatic means of achieving these goals.

You know, Lisp's syntax is weird but it is exactly this what makes it so flexible. It's easy to manipulate code as data, because the syntax is very regular. Try to do that with C's syntax...

So, unless someone knows how to solve this in a easy way, I'd say that the lot's of parentheses are actually a pragmatic decision (i.e. you want easy macros... so you have to use this uncommon syntax).

If popularity is the goal, then maybe those languages were not pragmatic. However, It seems the language designers of such powerfull languages (e.g. Lisp, Erlang, Haskell) were looking to solve other problems where popularity is really not a concern.

<comment><USER35> > _You know, Lisp's syntax is weird but it is exactly this what makes it so > flexible. It's easy to manipulate code as data, because the syntax is very > regular. Try to do that with C's syntax..._

Why you'd "manipulated code as data"? To write macros? A good template system can help with that (if you need it) without homoiconicity.

For me, the level of manipulation of "code as data" (and vice versa) you get with JSON/JS is enough for a lot of use cases.

<comment><USER34> > Why you'd "manipulated code as data"?

To write DSLs? You cannot use JSON to create new syntax for your language. The whole idea of DSLs is to extend the language for the problem at hand. How would you do that with JSON? Say... how would you write something like CLOS, for instance, using the alternative mentioned by you?

Maybe your option is good enough for a lot of use cases. But what when it is not good enough? Then you're stuck and there's nothing you can do except waiting for the language desingers to release a new version of your language with, hopefully, the changes you need.

<comment><USER35> > _To write DSLs? You cannot use JSON to create new syntax for your language. > The whole idea of DSLs is to extend the language for the problem at hand._

I'm not that sold on DSLs. I have a language (the base language) that people know, has certain semantics, etc.

Now I suddenly go on and add a new mini-language on top of it, with my ad-hoc semantics for the "problem domain"? Why multiply the languages used, so that someone would have to reason and understand both, instead of just one?

I could just use the functionality of the base language, AND it's syntax/semantics, to model the problem. I.e with objects in an OO design, with functions, in a procedural design, data and first class function in a functional design etc.

I don't really like all those Ruby DSLs for example, like for testing, were you have to learn each one ON TOP or knowing the core language.

</comment></comment></comment></comment></comment><comment><USER36> > The road to automatic parallelisation of sequential programs is littered > with corpses. It can't be done. (not quite true, in some specific > circumstances it can, but this is by no means easy).

vs three paragraphs later

> Alexander's talk gave us a glimpse of the future. His company concurix is > showing us where the future leads. They have tools to automate the detection > of sequential bottlenecks in Erlang code.

why is that not a contradiction? because an erlang program isn't "sequential" to start with?

<comment><USER37> I think you missed the "automatic" part. Completely rewriting a program in a new language is certainly not automatic.

<comment><USER36> both phrases feature the term "automate"...but yes, one is detection, one is resolution

</comment></comment></comment><comment><USER38> Have you even used zlib in c++? The largest ecommerce site out there uses zlib in a multitreaded c++ application(24 cores, 100s of threads, 1000s requests/sec/server) and it works just fine! Bet you erlang can't come within a tenth of the performance of c++...

<comment><USER35> > _1000s requests/sec/server_

That's not particularly impressive you know.

<comment><USER38> yes, because it also does other computation. Poiint was to illustrate that zlib can be used in a concurrent computing setting with high performance. The blog writer had claimed that zlib doesn't work in a multithreaded setting.

</comment></comment></comment><comment><USER39> This completely misses the fact that many network services are not compute bound, and multi-tenancy (as we get from "the cloud") lets "legacy" code make very efficient use of CPU resources, even a larger number of relatively slow cores.

</comment><comment><USER40> A very very interesting article, but I was extremely disappointed with the example they gave.

So, there was an error in someone's code which you rewrote without the error and it ran faster? Well done detective...

We need more parallel programs, no doubt, but we need more, better programmers, who are willing to write in compiled languages with low-overhead.

</comment><comment><USER41> Excel solves this right problem, and Erlang does not.

Erlang allows you to create concurrent programs, i.e.: programs where the result is schedule dependent.

One right problem is allowing people to write deterministic parallel programs. This gives you the speed (from parallel) with the reliability (from deterministic).

</comment><comment><USER42> In conventional blocking languages, you can get a start on parallelizing your programs this way:

      - break program into function calls that match the steps that can happen in parallel

      - wrap the function calls in messages passed over the network
         + i.e. process(thing) -> post(thing)/poll_for_things()

      - split the sender and receiver into different processes

OF COURSE there are big advantages to using a language (Erlang) or a heavyweight framework (map/reduce) designed for concurrency. Rolling your own process-centric concurrency is a different set of tradeoffs, not a panacea. But it's worth considering for some problems.

</comment><comment><USER43> To say that Erlang fails to deliver what most programmers need misses the point. If you have a mainstream problem, use a mainstream language!

I've spent many years developing and reviewing products in the telecoms realm, and have found that failing to realize _when_ something like Erlang brings life-saving concepts to your project may well make the difference between delivering on time and disappearing into a black hole of endless complexity. It's not for everyone, but when it fits, boy does it help!

</comment><comment><USER44> I would hazard a guess that 90%+ of the worlds programmers are working on projects that don't really need to use parallelism to get the job done.

<comment><USER45> The question is, will that percentage go up or down as time goes on?

</comment></comment><comment><USER46> Joe Armstrong: "The problem that the rest of the world is solving is how to parallelise legacy code."

Donald Knuth: "During the past 50 years, I've written well over a thousand programs, many of which have substantial size. I can't think of even five of those programs that would have been enhanced noticeably by parallelism or multithreading."

</comment><comment><USER47> Which frameworks can you use?

This? <http://nitrogenproject.com/>

<comment><USER48> I suppose you mean web framework. There's Chicago Boss: <http://www.chicagoboss.org/>

Also a recent book using other Erlang web technology: [http://www.amazon.com/Building-Web-Applications-Erlang- Worki...](http://www.amazon.com/Building-Web-Applications-Erlang- Working/dp/1449309968)

</comment></comment><comment><USER49> all true. to add' we are in a local optimum where we have a lot of fast non- parallel solutions and the parallel way needs >10 x parallelism. we'll see if we ever get to ubiquitous 100-way parallelism with no need for backward compatibility.</comment>


The Army of Silicon Valley Activists Trying to Elect Democrats - <AUTHOR> https://www.wired.com/story/the-army-of-silicon-valley-activists-trying-to-elect-dems/ <comment><USER2> This page loads between 15 and 17 trackers (per Ghostery).</comment>


Ask HN: Alternatives to Atlassian Confluence? - <AUTHOR> My team currently uses Confluence (an Atlassian product) for requirements, designs, procedures, and for various other engineering and product-related documents. However, I find it slow and difficult to navigate.<p>Has anyone found a better alternative? <comment><USER2> In these situations, I find that the conversation always revolves around the tool, and not the end goal. Documentation is a lot like code, but with a lot less people willing to write it.

Confluence and it's ilk are only as good as the effort / ownership of those that are willing to put in the time. If your org is not willing, than no other tool is really going to solve the core product.

<comment><USER3> Respectfully disagree. We may not be representative of all teams, but we did devote considerable time and energy into Confluence. No amount of love from the users can resolve application issues like slow load times, buggy draft- saving, inability to copy-paste, clunky commenting UI, etc.

</comment></comment><comment><USER4> I played around with: [https://www.notion.so/](https://www.notion.so/) it didn't stick. Our team is using Tapd (free, in Chinese, made by Tencent) for kanban boards, document sharing and other kinds of collaboration.

<comment><USER3> I'm curious, why didn't Notion stick?

We started using Notion as an alternative to Confluence, but I'm not a fan. It feels too fluffy (as do other tools like Monday). Confluence can be a pain and expensive, but the UI/UX is professional.

</comment></comment><comment><USER5> Navigation is about how you've structured things as team/company. Perhaps creating additional directory pages would help to get different views on all the pages/data in the system?

Speed: Does the server have enough RAM? Is it running with the recommended database for fastest use? Atlassian offer some guidance: [https://confluence.atlassian.com/doc/performance- tuning-1302...](https://confluence.atlassian.com/doc/performance- tuning-130289.html)

</comment><comment><USER6> I create SimpleDocServer for one of my customers as a quick-and-dirty Confluence replacement based entirely on Markdown. So if your point is information delivery and you are OK to collaborate on content creation via Git (or any other similar tool) then it might fit your needs. [https://www.helpinator.com/simpledocserver.html](https://www.helpinator.com/simpledocserver.html)

</comment><comment><USER7> I could live with Confluence if it played better with Markdown

<comment><USER8> When you create a new page, click the Try the New Editor option. It handles MD inline. I still haven't figured out how to open existing pages with the new editor.

</comment></comment><comment><USER9> At the last place I worked -- and at a volunteer-run event I'm involved with -- we use DokuWiki.

The most difficult problem is getting people to use the damn thing, whatever you choose. WYSIWYG editing might help for less-experienced users (or those who don't want to learn the markup language), but I haven't looked for a plugin for that. Yet. :)

</comment><comment><USER10> [https://wiki.js.org](https://wiki.js.org) seems to be pretty neat. It's all file based and only uses CouchDB for user administration.

</comment><comment><USER11> You can try [https://www.getoutline.com/](https://www.getoutline.com/).

<comment><USER12> I tried. Couldn't install it on my vanilla Ubuntu server because of infinite dependency hell. Then I tried wiki.js which installed and worked fine, aside from a few minor bugs. I was actually rather surprised by absence of decent open source wiki options.

</comment></comment><comment><USER13> recently saw [https://www.nuclino.com/](https://www.nuclino.com/)

</comment><comment><USER14> Maybe try Slab: [https://slab.com/](https://slab.com/)

</comment><comment><USER15> tl;dr, check these sites:

[https://www.wikimatrix.org](https://www.wikimatrix.org)

[http://wiki.c2.com/?TopTenWikiEngines](http://wiki.c2.com/?TopTenWikiEngines) <\- origin of the word "wiki" ( [http://wiki.c2.com/?WikiWikiWeb](http://wiki.c2.com/?WikiWikiWeb) )

Having setup the whole Atlassian on-prem SSO suite of apps on Linux with Git and LDAP talking to AD on Windows, it depends on your needs, as there are a variety of different wiki/collaboration solutions with different capabilities. I've also used these:

\- Dokuwiki (very simple, text file-based)

\- Mediawiki (LAMP / database-backed)

\- SharePoint (MS $$)

\- Quip

Others to look at (including sometimes CMSes)

\- OneNote

\- Liferay

\- Drupal

\- Statamic

\- WordPress

Also, Confluence has tons of plugins for customization and can be made to be fast with proper configuration.

</comment><comment><USER16> try helpjuice.com</comment>


What One 14-Year-Old is Doing This Summer - <AUTHOR> http://mplanner.co/blog/what-one-14-year-old-is-doing-this-summer/

<comment><USER2> Nice work! I wish I had access to all this tech when I was on school. Organization has always been a major problem for me.

If I had help earlier I think things would be different.

<comment><AUTHOR> Thanks, my inspiration actually came from me not being able to organize effectively.

</comment></comment><comment><USER3> Emerson, I have two questions:

(a) What was the last thing other students said at the movies, or say before you stop talking to any one of them about this project.

(b) Do you or other students have ideas about homework when you are not supposed to be studying? If so, what do you do then?

Great work with your project!

<comment><AUTHOR> (a) They usually said something like "see you later, nice talking to you" :)

(b) I actually had the idea during the school year but i've definitely been thinking about homework a lot more this summer.

<comment><USER3> (a) What I meant was some of the most important things people say are often the last thing they say. If there were some feature people wanted they might have mentioned or hinted it right before you part. It was to a product feature I was referring to, not to goodbye.

(b) I was also asking about what you do with planning when the solution to a homework unexpectedly comes to you, either when you are working on something else or even not working at all. It can throw off the planning if you are working on something else. At which point you might decide to switch back to that other subject for which you now have a solution. And now you are starting to multi-task. I was asking how you handle that. I am asking because that's something I have personally found to be a problem with planning. I am not sure if the solution to this is planning software.

I enjoyed reading your blog post and this is a wonderful project to work on.

<comment><AUTHOR> (a) Many people have said to gamify the application, or make a cool interface since high school students will be using it. (b) There is no real solution to this problem, this is just your problem that you didn't think about this before, or you could go into the app and alter your planning so that it could fit in somewhere else.

</comment></comment></comment></comment><comment><USER4> Great write up, Emerson. You're destined for great things. Please tone down the music on your kickstarter video.

</comment><comment><USER5> Hey Emerson!

It's really cool what you're doing. I'm a 15 year old myself and I'm in a similar situation as you. Shoot me an email at zchlatta (at) gmail.com. I'd love to chat with you.

<comment><AUTHOR> Cool Zach i'll make sure to shoot you an email. It's nice to hear that other 15 year olds are doing the same thing that I am doing.

</comment></comment><comment><USER6> Very cool--beats the hell out of my summer job working at K-Mart when I was your age. Good luck!</comment>


Why I'd Love to Be a College Student Again - <AUTHOR> https://www.gatesnotes.com/Education/CalTech-Visit <comment><USER2> For all those "trapped" on the academic treadmill, take heart.

Once you've been caught in the rather more insidious honeytrap of industry (which may seem to offer enough money to anesthetize yourself in return for what takes out of you... at the cost of, if not outright controlling, severely warping the horizons of what you'll be learning) -- the drudgery of the ivory tower, if still not automatically worth the negative tradeoffs -- definitely merits a second look.</comment>


Y Combinator News: Filter Features Wanted - <AUTHOR>

<comment><AUTHOR> Am I the only one who would like some new filtering features for this site?

Either:

A) A better filtering scheme. Many of the links submitted have nothing to do with startups. This isn't a general service like Digg or Netscape

And/Or

B ) The ability to subscribe to the 'best' list via RSS. This one would be better than just getting all the noise of the 'new' list. Not great, but it's a start.

</comment><comment><USER2> dupe: <http://news.ycombinator.com/comments?id=363></comment>


How to succeed when you have no special skills - <AUTHOR> http://oliveremberton.com/2013/how-to-succeed-when-you-have-no-special-skills/

<comment><USER2> Fair point. About to post something similar based on mediocrity in enough domains, perhaps some specific expertise but mainly systems and persistence. Some kind of self pep talk!</comment>


Soylent raises $750k on Crowdhoster – what's next for open source crowdfunding? - <AUTHOR> http://blog.crowdtilt.com/post/53534200723/soylent-raises-750k-on-crowdhoster-whats-next-for

<comment><USER3> I know you're probably dealing with 2 different systems, but I'd love it if Crowdhoster campaigns could also be hosted on Crowdtilt.com. I get that Crowdhoster is supposed to enable all types of cool group transactions way beyond the controlled scope of Crowdtilt, but even just generating a secondary Crowdtilt.com landing page for a Crowdhoster campaign would be cool.

I also get how this isn't a big issue for now. But as Crowdtilt grows, I would think it becomes a "brand" that people trust to handle transactions, and would tip a certain % of the market to then give up the CC digits. So for example, as a consumer, I might be more inclined to purchase something on crowdtilt.com, as opposed to xanga.com.

Just like how right now, your average consumer might be more inclined to support a Kickstarter project over a Crowdtilt project, due to inherent trust he feels in Kickstarter as a curator/brand. But since I'm bullish on Crowtilt, I think Crowdtilt eventually develops and trusted brand name.

As an campaign director/organizer, in and ideal world, I want both the flexibility of Crowdhoster and that brand stamp of "Crowdtilt". Just a suggest... otherwise, these updates are actually really cool, love where it's going!

</comment><comment><USER4> I'm going to try to raise money to bring an IP-unrestricted anticancer compound up to the point of FDA clinical trials. (i.e. as far as you can go without involving humans). I'm pretty sure that I'll be using crowdhoster as the platform.

<comment><USER5> This is awesome. Are you developing it unfunded at the moment?

<comment><USER4> Work on it was discontinued when my boss quit her job at the university of maryland. The 1 year patenting period has expired, I have no love for patents anyways, so I'm going to generate several analogs and register them as something called "US Statutory Invention Registration" which prevents them from being patented. I have her blessing (not that I would need it from a legal POV) to continue work on this project.

Since then I've been accruing a lot of synthetic biology experience (I'm currently probably the most experienced person in the world at a new technique called "gibson assembly" \- I've done upwards of 100 successful gibson assemblies on a wide range of molecular biology targets), which should help the process go even more smoothly.

Details are at: [http://indysci.org](http://indysci.org),

currently we're waiting for the IRS to approve our application for 501(c)3 status before launching our fundraising. Our target launch date is January 2014

<comment><USER5> That's an awful lot of Gibson! What tools are you using to design your constructs? I briefly joined the developers of Gibthon (FOSS gibson assembly design software; [http://gibthon.org/](http://gibthon.org/)) last year - <USER2> will be thrilled to hear if you use it. In fact, if you would be willing to share some of your experience, no doubt it would be invaluable for improving the design tools. We've also got lots of people in our department doing lots of Gibson assembly. If you ever want to compare notes or anything... my email is in profile.

How are you doing for computational biologists?

<comment><USER4> We use CLC.

Basically the workflow is 1) find sequences that you want to stitch together. 2) build a draft plasmid. 3) make fine tuning changes (making sure RBSes are ok, we aren't stomping on terminators or promoters, etc, add mutations if we want). 4) annotate primer regions. 5) cut and paste into IDT's primer orderer.

The bulk of my gibson assemblies (about 50 of the ~100 that I've done personally, plus 12 that my college intern last year did, and 13 going on 48+ that my intern this summer is doing) are for site-directed mutagenesis; quikchange blows, and the plasmid I'm playing with is 20kb anyways, with 12kb of mission critical DNA. The general scheme is: design 2 mutagenic primers, 2 PCR using primers that flank a pair of unique restriction sites near our site, then gibson the left hand and right hand products with a stock of pre-digested plasmid. Since bacterial plasmid propagation is "error-free", we don't bother sequencing all but 1kb or so of the final product - saves on cost - and our plan is to do shotgun sequencing of all of the plasmids, barcoded, with ionTorrent when we're done.

We're also doing combinatorial installation of promoters that my boss' intern has done (27 constructs) - that I've partially supervised. Among the other things that I've done are gene fusions, leader sequence deletions, moving around restriction sites, swapping plasmid backbones, assembly of complex, highly repetitive sequences, and we attempted (but only partially succeeded at) capturing a eukaryotic chromosome by the telomeres.

We cut a lot of corners, but for NEB's sake (I like them) I'm not going to discuss exactly what those corners are, quite yet. Even with ours sloppiness, it works nearly every time, often with 8/8 correct (and sometimes you have to screen 48 to get one, but that's rare and I haven't had that happen in about a year or so). My general philosophy is the more sloppy you can get away with being, the more robust your procedure is - I like gibson assembly because I can teach it to an undergrad in a week and have them rip roaring on a project.

There are lesser known properties of gibson assembly (and this is in the original paper if you read it carefully, which most people don't) - you can have inexact ends, for example, if you want to have a template that is released using restriction enzymes - the 3' exonuclease activity of the polymerase will remove it and create a seamless construct with no evidence of the former restriction site. I have to warn: This would be hard (but not impossible) to implement in a design suite, and nearly impossible for automated design unless you are using exactly the same RE protocol each time.

Slight disclaimer - I'm a coworker of Dan Gibson's, so take my evangelism with a little bit of a grain of salt. I will also say, that there are plasmids that you absolutely can't gibson clone into and _we don 't know why_. pBR322 tet marker is an unclonable position, for example.

Finally:, we don't really use computational biologists. Actually, we're designing enzyme variations from first principles and previous literature report, without appealing to simulation or model, with quite a bit of success.

</comment></comment></comment></comment></comment><comment> cupcake-unicorn I keep seeing Soylent pop up on blogs and stuff and I don't know what the big deal is. He made some shady claims before about things like "suffering from a severe iron deficiency" in just a few days, which was magically fixed when he remixed the formula. I'm also shocked that he's marketing it to this extent and _still_ doesn't say _anything_ about the ingredients. People have allergies, people have dietary restrictions. How is this at all different from a protein shake at the Health Food store?

<comment><USER6> > I don't know what the big deal is.

It's basically junk science (or a de-facto scam) by a guy who is doing well at marketing himself.

Food replacements already exist. The biggest is a product called Ensure. There are others. It's not clear that there's a need for Soylent at all.

You're right, iron deficiency can't be detected in a few days (3 months would be closer to the time frame required assuming you had healthy iron levels to begin).

Most of the other continual reformulations he discusses also seem pretty ridiculous on the timeframes involved. Basically, this guy doesn't seem to have any understanding of stochastic experimentation, limiting confounding factors or limiting free variables.

<comment><USER7> I did some googling about ensure:

[http://www.naturalnews.com/002698.html](http://www.naturalnews.com/002698.html)

It doesn't seem to me that any product made with high fructose corn syrup as its main ingredient represents a credible attempt to make a healthy food product (just using sugar would be healthier).

<comment><USER6> High fructose corn syrup is just a sugar. Your opposition to HFCS sounds a little skewed. HFCS is not substantially different to other sugars biological effect and is not really any more or less healthy.

From Wikipedia ([http://en.wikipedia.org/wiki/High- fructose_corn_syrup_and_he...](http://en.wikipedia.org/wiki/High- fructose_corn_syrup_and_health)):

> Epidemiological research has suggested that the increase in obesity is > linked to increased consumption of sugars and/or calories in general, and > not due to any special effect of fructose alone.[1]

If you're worried about "not natural" then meal replacements are not for you (unless you've had your stomach or jaws surgically removed). They are heavily processed and artificial by design.

</comment></comment></comment></comment><comment><USER8> I think what's next, and quite frankly, what's already been happening, is that crowdfunding will move away from platforms and onto self-hosted websites.

<comment><USER9> One of the things I like most about Kickstarter is having the "activity feed" of friends who have funded other projects. I'm sure there's a way to recreate that without having the restrictive ToS kickstarter has, though, or it could be solved by publishing "I've funded this" to twitter/fb/etc.

</comment></comment><comment><USER10> Am I the only one surprised by the choice of name "Soylent" for a nutritional drink?

<comment><USER11> It's an apt name. In the film "Soylent Green," soylent was just a line of meal-replacement rations. The green variety was the newest one released, and the one that happened to be made of humans.

<comment><USER12> Additionally, Soylent Green wasn't even in the book that the film was based on ('Make Room! Make Room!'). The book was about the impact of unchecked population growth.

</comment></comment></comment><comment><USER13> Hi Garry,

With all due respect, isn't your headline a bit backwards? Crowdtilt is just the platform. Soylent raised most of this on their own, due to their own efforts and the press they've been receiving. Even the original headline is: "Soylent raises $750k on Crowdhoster." Why switch that around?

I'm not trying to diminish Crowdtilt and their efforts, but your headline makes it seem like Soylent didn't do anything.

<comment><AUTHOR> Fair point! Did not mean to imply that; I love and support the Soylent team too!

<comment><USER13> No problem! :-)</comment></comment></comment>


Weekend Hack: Dead Simple A/B Polling - kind of Facemash for anything - <AUTHOR> http://abt.io/

<comment><USER2> I posted this late in the last thread you made, don't know if you saw it. I think it would help. The [ask a question] -> [get the link] -> [upload mutations] workflow seems very backwards, threw me off at first.

The site would really be helped by a WYSIWYG editor. What you see is what you get.

Basically, a new user creating an A/B test isn't going to understand the flow and terminology of the site. The workflow is quite confusing really.

But a new test taker will understand how the site works immediately upon seeing a pre-created test. A question and two pictures on the same page, simple.

To bridge that gap, make the test creation process look like the final result. Make it feel as if you're filling in the page with your info, rather than uploading separate pieces of info. Something like:

<http://i.imgur.com/BzsNy.png>

<comment><AUTHOR> ouah I love what you say.... this is just a weekend hack... It still requires a lot of work! Please contact me @spolu so that we can make it a better tool!</comment></comment>


Ask HN: Why do we kill and eat other living beings? - happy-go-lucky I've been to places where they take wandering lizards into their hands and leave them in a safe haven.
Do we really have such a scarcity of food?
Fellow HN'rs, it's my genuine question please. <comment><AUTHOR> Because human physiology evolved in an environment in which critical nutrients were sourced predominantly (^exclusively animal in list below) in the flesh of animals.

Examples of key animal-based nutrients: Vitamins: A, B6, B12^, D3^, F, K2^; Amino Acids: Creatine, Carnitine, Carnosine, Taurine; Heme Iron; CoQ10; CLA; Cholesterol

This list does not include nutrients/vitamins which often lead to deficiencies when sourced in plants, due to very low bioavailability or antinutrients which block the absorption of the nutrients.

So, why do we kill and eat other living beings? Because we evolved this way and it is part of our natural diet.

<comment><USER2> Vitamin A: widely available in plants not sure why this is listed.

B6: also widely available in plants

B12: Should be supplemented, most people get this from fortified foods.

B12 deficiency is common across both vegans and those who eat animals.

D3: main source is the sun. again, deficiency is common across both vegans and those who eat animals.

F: This is common outside of the animal kingdom

K2: Found in soy

Amino acids: also found in most non animal foods.

Creatine: For most people your body produces enough of this to function. This can be supplemented.

Carnitine: your body uses the amino acids lysine and methionine to create carnitine.

Taurine: not essential

Heme iron: not essential and actually may be detrimental

CoQ10: commonly found in vegan diets

CLA: True, not found outside animals. CLA is controversial as there isn't clear evidence on wether it is good or bad. It is not essential however.

Cholesterol: Your body makes it.

The most common deficiency among vegans is B12 I believe.

There are millions of people thriving on a vegan diet. They consistently test with better biomarkers than non vegans. Many of your favorite celebrities are vegan, many athletes as well. The largest associations for both the U.S and E.U recognize vegan diets as safe and adequate for a healthy life.

Yes we evolved to be able to eat many things and thrive. That doesn't mean we need to.

<comment><USER3> I lived the first 18 years of my life as a healthy child that ate no meat. I now eat meat. Why? It's tasty and everyone else does it. Should we stop mass murdering sentient beings? Probably[1]. Is hunting and killing on an individual basis okay? I think that should be left to every individual to decide for themselves. If you don't want to kill animals, then don't. But don't stop me, a fellow animal, from doing it.

[1]: Sadly, our economic system does not agree.

<comment><AUTHOR> Perhaps you did not eat meat, but you almost certainly ate animal products of some form (eggs, butter, milk, cheese, yogurt, human breast milk or formula when you were an infant, etc.) which is all the animal-based foods you need to obtain the animal nutrients your body needs.

All human diets come at the expense of other animals and creatures. This is a natural part of life and shall always be so. It is false to beleive that there is a choice to make between slaughtering animals for food and peaceful coexistence with nature/animals. That's not an objective accounting and not reality-based.

For plant-based agriculture, first you take a plot of land away from the wilderness and native animals. Then you clear the land, purging its native plants (which are inedible and toxic to humans) and ground-cover, insects and any pest animals (e.g., birds, woodchucks, racoon, fox, rabbits, mice, to name just a few). Then you plant monoculture agriculture crop, which over time depletes the natural soil, worms and microbes of natural nutrients and requires increasing amounts of chemical fertilizers and pesticides. Animal pests continue to be killed as the crops mature, and then also by harvesting machinery at the end of the season.

A dramatized question: Would you rather humanely shoot a single steer (or goat/sheep or similar ruminant) which you could then consume over the course of a year, assuming you make use of the entire animal? Or would you choose to avoid meat, but in the process of eating an all-plant-based diet kill dozens or more wild animals and countless smaller pests and insects while disrupting local flora/fauna/ecosystems and depleting the soil?

</comment></comment></comment></comment><comment><USER4> Plants and fungi are living beings. We eat them for complementary reasons - what nutrition we don't get from one, we get from the others. I suspect you wish to engage in a veganist conversation, but until the "rules" are considered and applied with equal determination for fungi life forms, I have no interest in the debate that is most likely to emerge from this composed type of questioning.

Verbal cherry picking is annoying to all except those in the echo chamber. Nobody has ever said they eat hamburgers only because carrots are so hard to come by. Scarcity is a clever ruse, but doesn't work here.

<comment><USER2> Fungi aren't animals. They also don't have a central nervous system and they don't feel pain. also in most cases not killing the organism by eating a mushroom.

<comment><USER4> From below: A fungus is able to search through a maze and apparently remember the route. Is it sentient?

<comment><USER2> It can react to its environment. That's not the argument. The "rules" are it isn't an animal.

<comment><USER4> If you are sticking to the argument so cleanly, then perhaps you can stick to my answer, which is at the beginning and not the end of the response you are speaking of.

As I predicted, the purpose of this post was vegan debate, and my answer has been completely ignored.

<comment><USER2> Fungi aren't animals. Vegans don't eat animal products. Your answer was ignored because it is not relevant.

If there's a diet you'd like to discuss that involves not eating fungi then go ahead and make a post for it.

<comment><USER4> My answer wasn't irrelevant. Go back and re-read it.

It is clear the question asked is a ruse to goad people into a vegan debate.

If you take OP as being 2 questions, then the first answer is at the beginning, and the answer to the goading question of food scarcity is the end.

But the answer is clearly not what OP was after. Only vegan preaching was intended.

</comment></comment></comment></comment></comment></comment></comment><comment><USER5> Because of biology. Also because the planet could not sustain its current population on plants alone. Finally, because they're delicious.

The idea that animals should be spared from the dinner table is a quasi- religious viewpoint not shared by most people.

</comment><comment><USER6> Because they taste good.</comment>


Ask HN: Must Read Books for a Product Manager - <AUTHOR> What are the best book that you would recommend to be read by a product manager?
I'm currently doing (probably) what a product manager does for 3 small technical teams trying to implement some ideas.
I would like to document more on what a product manager does, what are the best ways to do the job and what are some best practices. <comment><USER2> I was a PM for an enterprise product for several years (I took a sabbatical from software development for a little while to do something more business-y).

The original Ries "Positioning" book was good, and is probably more valuable than Porter's Competitive Strategy for tech PMs.

"Selling The Wheel" is a very light read and, if you've never watched a sales team get managed, is a good introduction to how products get sold (in B2B situations) at different stages of maturity.

Most of Joel Spolsky's blog posts are effectively describing the job of a PM; they're more valuable than any book I can think of.

I like "Four Steps To The Epiphany" as a sort of roadmap to bringing enterprise-type products to market. The book itself is kind of funny; it has the appearance of being typeset entirely in Microsoft Word, which gives it a sort of samizdat charm.

</comment><comment><USER3> Surprisingly, the book that had the largest impact on how I think about (software) products is "The Design of Everyday Things" by Donald Norman. You won't find expressions like 'creating value' nor the word 'software' in it but IMHO its ideas are at the beating heart of any product-market fit.

</comment><comment><USER4> Two good books for anyone interviewing PMs or being interviewed for a PM position are Decode & Conquer and Cracking the PM interview.

Whilst both are focused on interviews rather than doing the job, they each contain some nuggets about how to think.

As others have said, Inspired, Lean Startup and Four Steps are essential reading.

</comment><comment><USER5> Lean Startup is a great book. When I first started, I found "Inspired, How to Create Products that Customers Love" by Marty Cagan to be a great resource.

</comment><comment><USER6> Lean Startup - Eric Ries, Business Model Generation, The Design of Business - Roger Martin</comment>


Ask HN: Post PRISM - Any companies considering moving their data overseas? - <AUTHOR>

<comment><USER2> To where? Anything overseas is stated as fair game for the NSA. They at least _pretend_ to not watch American activity. Any other country will cave to US pressure if they want your data, and of the countries that don't like the US and still have a decent internet connection, they'll probably do worse spying of their own.

</comment><comment><USER3> I work with companies that do this pretty regularly due to other regulatory issues. The problem involves how poor data privacy laws are in other countries, in some places data is more illegal than others, and then how its accessed from the US. Sometimes you're not better off in other countries anyways.

Right now the problem is "the cloud" not the country for the US anyways...

</comment><comment><USER4> The main issue is latency and cost. Try browsing the internet from New Zealand or Australia for a few weeks: you really notice the difference of being another 150-200ms away from servers in the U.S. If you'd like to host in those countries directly you'll be subject to higher bandwidth costs.

</comment><comment><USER5> I've always been surprised so many companies have their servers based in the USA.

Iceland seems a safer place to me from a democracy standpoint.</comment>


Anthem's emergency room coverage denials draw scrutiny - <AUTHOR> https://www.ajc.com/news/state--regional-govt--politics/anthem-emergency-room-coverage-denials-draw-scrutiny/sWT8ts3TYv6vNjrEN99kdO/ <comment><AUTHOR> Traditionally, policy states that ER utilization is covered by a prudent layman's standard: if a reasonable person would think their condition is an emergency, then the ER visit is covered.

Anthem/BCBS is now changing it to a retroactive review. If one of their medical staff now thinks, retroactively, that a condition didn't merit an ER visit - even if there's _no reasonable way_ a layman would know that - they're denying coverage. They're basically arguing that their post-hoc review is a reasonable interpretation of the prudent layperson standard, where "prudent layperson" means "health insurance company algo."

It's a ridiculous policy stance, and one that is going to cost lives. There's no reasonable way for a layman to know whether their chest pain is musculoskeletal or ischemic in nature. Going to the ER with a suspected heart attack is _appropriate_. Anthem now says, hey, if that doesn't turn out to have been a heart attack - even if you had no way of knowing that - your ER visit will not be covered. They're actively encouraging people with heart attacks not to present to the emergency room.

This is the most repulsive move I've ever seen in the private insurance industry, and I've seen some stuff in my career.</comment>


Rustboot: a 32 bit kernel written in Rust - <AUTHOR> https://github.com/charliesome/rustboot <comment><USER2> I saw zero.rs and said, "Hey, I know! I'm going to make a kernel!" Then after showering... came back and saw this.

Well, in any case, great work! (Perhaps I'll use yours as a starting point.)

EDIT: One question. When I try to compile it, rustc gives me these errors:

        error: no item found for `modulo`
        error: no item found for `durable`
        error: aborting due to 2 previous errors

If I had to guess, it would be due to the fact that I'm trying to compile it on OS X but have i386-intel-linux set as the target, but your screenshot shows (at least it being emulated) on OS X. Did you have to build a special cross compiler version of rustc?

<comment><USER3> I use ELF as the object file format, but all the development was done on OS X.

Maybe try updating your version of Rust. I had trouble using zero.rs on 0.6, but the latest from Git works well.

<comment><USER2> Ah, that would explain it -- I'm trying it with 0.6. Building master now, have a feeling that it'll be a while though.

<comment><AUTHOR> Beware, "the latest" usually means incoming, not master. That said, I'm not sure which is actually meant above.

</comment></comment></comment></comment><comment><USER4> If anybody is looking for something to do with this, consider writing a KVM- like virtual machine host which is barebones enough to boot from coreboot.

I've dreamed of a checkpointed VM which could be used to press a key and fork the running system, or instantly reboot to a usable state when a fault occured, but was small enough to fit in a flash ROM and boot in milliseconds.

The last attempt was linux+QEMU and was to large for most boards supported by coreboot.

<comment><AUTHOR> Yup, this kind of thing is one of the reasons I like exokernels so much.

</comment></comment><comment><USER5> This is great, but I'm confused. Not that long ago, I remember reading that it was explicitly not a goal of Rust to be able to write an OS kernel in it. Is this done in spite of Rust not being designed/suited for it?

<comment><AUTHOR> I'm not sure that I've ever heard that as an implicit goal, but a 'runtime- less Rust' hasn't existed yet. Now, you can't access arbitrary memory without an unsafe block, that may be what you're thinking of.

Recently, zero.rs was released, which allows you to write some Rust programs without the runtime. This is because Rust basically expects certain functions to exist, so they have to be defined somewhere. That said, things like 'a GC' aren't implementd, they just throw an error. Please see: <https://news.ycombinator.com/item?id=5771173>

This is just one step further in demonstrating that these things work.

<comment><USER6> A link to the HN thread on zero.rs

<https://news.ycombinator.com/item?id=5770212>

</comment></comment></comment><comment><USER7> Great to see new Rust news every day.

At first I was most interested in Go: But now it seems that Rust is much more close to the metal than Go is, which I find more interesting.

</comment><comment><USER8> >It paints the screen bright red and then hangs.

Well, it's a start, I guess. :) (The project is literally only half an hour old.)

<comment><USER7> Sounds better than Vista!

<comment><USER7> Obviously didn't get the joke: <http://xkcd.com/528/>

<comment> qu4z-2 Or didn't consider the joke appropriate for Hacker News, I guess.

</comment></comment></comment></comment><comment><USER9> Wow, if people like this, then maybe I should post my tiny 32-bit kernel in C++ somewhere. It has a physical, virtual, and kernel memory manager and support for text-mode I/O. And you can use pretty much anything C++ (even Boost) that doesn't rely on exception handling...

<comment><USER10> I'd like to see it if you don't mind.

<comment><USER9> Hmm... since I'm not sure I want to share the code yet, here's a class diagram instead that should still give you an idea what state it's currently in:

[http://picpaste.com/pics/21616b59902f2992e1e488af35acba20.13...](http://picpaste.com/pics/21616b59902f2992e1e488af35acba20.1369637934.png)

The Process class is just a dummy at the moment, it doesn't do anything.

</comment></comment></comment><comment><USER11> Something similar I did with Clojure:

<https://news.ycombinator.com/item?id=5772632>

</comment><comment><USER12> It's really impressive what people have done with rust. I don't know how they do it because when I write short rust programs I get internal compiler errors.

<comment><AUTHOR> You should open up issues for those: <https://github.com/mozilla/rust/issues>

There's no reason for an internal compiler error, especially in a language with safety as its focus.

<comment><USER12> Have done. Here are the ones I found in 300 loc.

<https://github.com/mozilla/rust/issues/6699> <https://github.com/mozilla/rust/issues/6698>

Don't get me wrong. I'm super excited to learn rust once it stabilizes. But right now I don't spend enough time on fun programming to deal with a language that has this many errors.

<comment><AUTHOR> Great! Thank you. I've been mostly sticking to documentation myself, but you can't fix what you don't know...

</comment></comment></comment></comment><comment><USER13> I find this a very exciting project. Rust is always in my peripheral vision. To see a language operating at such a high level able to get down so low is very compelling.

</comment><comment><USER14> I am confused: if it's a kernel, why do we have the mac user interface? Is this the screenshot from a VM or am I missing something else?

<comment><USER15> Yes, it's a VM. Note the title, "QEMU".

</comment></comment><comment><USER16> I guess this is kind of a factious question to ask, but why 32 bit and not 64 bit?

Are there any technical reasons for this?

<comment><AUTHOR> 32 bit is waaaaaaay easier to write.

</comment></comment><comment><USER17> This is cool!

One request, Can you detail out the steps, for noobies to be able to re- produce this effort? Thanks!

<comment><AUTHOR> Reproduce as in 'build a tiny kernel in something' or reproduce as in 'build and run this example?'

If it's the first, OSDev has a great community, and some decent tutorials: <http://wiki.osdev.org/Bare_Bones>

If it's the second, the answer is 'have edge Rust installed, have qemu installed, then run `make`.'

</comment></comment><comment><USER18> Wow, cool! Great job :)

</comment><comment><USER19> That isn't a kernel

<comment><USER20> Don´t know why this was downvoted, you hit the nail on its head. The fact that it runs directly on hardware doesn't make it a kernel, it doesn't provide any abstractions for applications to run unmodified without specific hardware provisions.

<comment><AUTHOR> Exokernels provide no abstractions, just a few safety guarantees. So you could argue that this is just an unsafe exokernel. ;)

That said, "is it really a kernel as per Wikipedia" is not particularly interesting. It's a damn cool hack that could be the starting point for something interesting and useful, that's all I care about.

<comment><USER19> I was just a little disappointed Steve. Interesting post for sure. Would be interested in seeing part 2 ;)

<comment><AUTHOR> My friends have been working on an exokernel in D for years, so that's been our go-to for this kind of thing. It's only like 10 kloc, maybe we should port it for fun...</comment></comment></comment></comment></comment>


BMW Cars Vulnerable To Blank Key Attack - <AUTHOR> http://www.bbc.co.uk/programmes/b006mg74/features/bmw-car-theft-technology

<comment><USER2> That's 8 long weeks to wait, compared to the 3 minutes it takes for thieves to get away with the car...

<http://www.youtube.com/watch?v=DshK4ZXPU9o>

</comment><comment><USER3> "For obvious security reasons we cannot say what these measures are."

Rubbish. Anyone know that the measures are?

<comment><USER4> Blank keys now cost £50,000 each, so you might as well just buy a BMW to start with.

<comment><USER5> Or buy the $50k key and steal BMWs to break even.

edit: [http://www.alibaba.com/product- gs/466366659/New_BMW_blank_ke...](http://www.alibaba.com/product- gs/466366659/New_BMW_blank_key_3_5.html) is this key different than what the article mentioned? Even if you but the minimum order, it will only cost, $6k, sell the rest for a mark up.</comment></comment></comment>


Ask HN: Should I start learning Deep Learning in JavaScript or Python? - <AUTHOR> I'm new to Deep Learning and have had limited exposure to traditional machine learning algorithms. Since JS is the lingua franca of the web, should I start learning DL is JS, or should I stick to Python? <comment><USER2> Python.</comment>