A tiny static full-text search engine using Rust and WebAssembly (2019) - <AUTHOR> https://endler.dev/2019/tinysearch/ <comment><USER2> I've been a fan of Matthias' project for a while. I learned about it soon after starting Stork: [https://stork-search.net](https://stork-search.net)

They're very similar and share a lot of principles, though Matthias went full- on towards the algorithmic aspect and I focused on the experience of including the UI (copy-pastable from the code on the home page) and building a search index.

I think WASM-aided in-browser search is really exciting! There are clear benefits for content creators (embedding search on a Jamstack site has historically been tricky) and for users alike (Caching & offline support is pretty rad if your users are doing a lot of searching). I'm excited to see Matthias' project get attention here!

</comment><comment><USER3> Does anyone else begin to feel like their role as a software developer is to maintain a mental search index of available techniques, languages, libraries, and metadata properties about each of them?

It's becoming so easy to compose software from available open source components, and migrate functionality (like full-text search) to different layers of the stack (and that's fantastic!).

It's just tricky to keep all the requirements and constraints (and implications) in mind when selecting the appropriate libraries :)

<comment><USER4> It's sad that we still don't have automatic interoperability between languages.

Someone should define a common API, and every language should adhere to it (or risk not be taken seriously). This is not trivial, since some languages have garbage collection, but it should be possible.

<comment><USER5> Like ImprobableTruth said, this isn't really possible without restricting the expressivity of the interop API or the set of supported languages. At least not on the function-call level.

A more flexible – though less efficient – approach would be a service-oriented protocol. You'd send requests in the form of messages (binary or text) over a byte-oriented bidirectional channel and receive the replies on the same channel. Unfortunately this approach would require more code to set up than primitive [1] function calls, and fine-grained interaction with the library would be harder.

[1] "primitive" as in _lower-level_ , not as in _dumb_.

<comment><USER5> Dammit, now I can't think about anything else but how to design such a protocol, and how to generate adapters which translate between this protocol and the API of a library...

Edit: From the perspective of the interop protocol, it wouldn't make much difference if the library runs in the same address space or in a different process. Large blobs of data, like an picture or a long string, could be passed via pointers (in the same process) or via shared memory (in different processes).

<comment><USER6> If you're trying to make an API for all programming languages, aren't you essentially just recreating something like the Java virtual machine but with your own biases and assumptions inserted?

<comment><USER5> You're misunderstanding my idea. Don't think "C ABI with higher-level types and objects", think "HTTP with more structure".

<comment><USER6> But it seems that kind of protocol would just be a way of telling a computer _what_ to do, not _how_ to do it. How would that be better than any other messaging format that exists?

Genuinely curious, because I don't fully understand this myself but the idea is interesting.

<comment><USER5> To be honest, I don't know. It was just a quick idea, and I'm increasingly less sure, whether it makes sense at all. Sorry to disappoint you :-(

</comment></comment></comment></comment></comment></comment></comment></comment><comment> krut-patel I was looking for something similar (client side text search) and landed upon MiniSearch[0]. While it doesn't support some of the advanced features of lunr (like wildcard search), it was perfect for my needs. The accompanying blog post[1] explains the trade-offs pretty well.

0 - [https://github.com/lucaong/minisearch](https://github.com/lucaong/minisearch)

1 - [https://lucaongaro.eu/blog/2019/01/30/minisearch-client- side...](https://lucaongaro.eu/blog/2019/01/30/minisearch-client-side- fulltext-search-engine.html)

</comment><comment><USER7> Really cool. Reading through your incremental discoveries (aka going down the rabbit hole!) reminded me of my own adventure with building a typo tolerant search engine (you can see it here: [https://github.com/typesense/typesense](https://github.com/typesense/typesense)).

What began as a simple side project 4 years ago has consumed a significant part of my free time over the last couple of years.

Web assembly is certainly going to open a lot of new avenues for doing interesting things on the browser.

</comment><comment><USER8> Really neat project and fantastic write-up. I always enjoy following the journeys of people who forge into the untamed lands of WASM.

I always find myself wishing I had a good excuse to use WASM for something, but never being able to find one, so it's exciting to see that you did! The fact is that JavaScript logic is rarely the bottleneck in web apps. And when it is, it's usually tangled up in UI rendering code that would be hard to tease out into WASM. You do bring up an interesting point, though, which I hadn't considered: WASM isn't just faster, it's _smaller_. That alone could make it useful in some cases where the speed may not be needed!

<comment><USER9> Thanks! On top of the size benefits, I love that I can finally use languages other than JavaScript on the frontend. I couldn't have done it in JS because I'd have to write a BloomFilter implementation in it (which I would not be capable of) or bundle an existing library, which would have increased code- size (hence, defeating the point of the project). Portability is the other big feature of wasm.

</comment></comment><comment><USER10> Great post! Doesn't it make sense to load the index separately, instead of a single bundle? RN the client would bust it's cache every time the content changes?

<comment><USER9> This was requested before and there even was work on a prototype that has since stalled. If you (or anyone else) is interested, please check out [https://github.com/mre/tinysearch/pull/37](https://github.com/mre/tinysearch/pull/37). Maybe we can get this done in a future version. :)

</comment></comment><comment><USER11> I've always been curious about this. What's the best practice for loading a large JSON file for large sets of search results? I believe when working with lunr in the past, I ended up making large network requests to load the entire JSON file at once. What's the proper way to deal with this?

<comment><USER12> Once your website reaches a certain size, the JSON will be too big to load. Then you'll have to offload the search request to a server. Either self- hosted, or a service like Algolia.

<comment><USER13> Push the corpus into SQLite, it has built-in FTS engines[^1]. Then serve it with anything. Unfortunately this needs server side code, but like 30 lines of PHP.

[^1]: [https://www.sqlite.org/fts5.html](https://www.sqlite.org/fts5.html)

<comment><USER14> You can do SQLite in the browser but it'll have to download the entire dB file instead of only opening the pages it needs (because the naive JS port can't convert page requests to range requests).

<comment><USER15> It should be possible to support loading only the required pages on the browser with SQLite compiled to WASM along with a custom VFS implementation. Here's a project[1] which does something similar (selectively load the SQLite DB on demand), albeit with the SQLite file in a torrent.

[1]: [https://github.com/bittorrent/sqltorrent](https://github.com/bittorrent/sqltorrent)

</comment></comment></comment></comment></comment><comment><USER16> So once this loads, it sounds like it could be made to work offline. That might open some interesting possibilities.

<comment><USER17> Wasm can be cached!

</comment></comment><comment><USER18> Couldn't find the search on mobile :(

<comment><USER19> I found it, but couldn't make it work. Pixel 3a running Android 10 and the stock Chrome browser. Hitting enter on the search field did nothing and I can't see any other submit button. Then again, 10% of the search field is also missing.

<comment><USER19> On second look, it's real-time. No need to submit. The results just blend into the page so I thought it was broken.

<comment><USER9> Sorry to hear that. Not an expert, but if you have any ideas on how to improve the UX I'd be thankful.

</comment></comment></comment></comment><comment><USER20> How about putting this index and search logic into a CloudFlare worker?

[https://developers.cloudflare.com/workers/templates/pages/he...](https://developers.cloudflare.com/workers/templates/pages/hello_world_rust/#resources)

<comment><USER20> Then you can upload index up to 1MB and still have decent performance [https://developers.cloudflare.com/workers/about/limits/#numb...](https://developers.cloudflare.com/workers/about/limits/#number- of-scripts)

<comment><USER9> That's a good idea. In my case, I wanted a static search that I could deploy next to my content. Cloudflare workers would require a (free) account, but most importantly they wouldn't work full offline. For bigger indices, that would be a great trade-off, though. If you like, you can try pushing tinysearch to a worker using wasm-pack. It's all Rust in the end, so you'd only need to add a `/search` route e.g. with hyper ([https://github.com/hyperium/hyper](https://github.com/hyperium/hyper)). If you're willing to experiment with this, don't hesitate to open a pr/issue on Github and we can add that feature.

<comment><USER21> It would be interesting to see a hybrid approach:

* server side WASM such as cloudflare workers and kv to build and maintain the index

* streaming copy of the simplified index to be pulled in by a browser-side wasm

* queries that go beyond the simple index forwarded to the worker

One way of simplifying would be to limit search terms to a certain length, or only expose the most popular results.

By sharing wasm code the format can be optimized and not require a compatibility layer or serdes.

</comment></comment></comment></comment><comment><USER22> How does it compare with flexsearch? It claims to be the fastest, smallest, prettiest search library in town. [https://github.com/nextapps- de/flexsearch](https://github.com/nextapps-de/flexsearch)

<comment><USER23> this one is 100% client side, flexsearch is client-server.

I guess for bigger indexes not gonna work out, as the payload will be huge and it pushes all the work to the client.

<comment><USER24> Nope.. Was looking into flex search today.. is all client side

</comment></comment></comment><comment><USER25> Today's thread on the other search engine: [https://news.ycombinator.com/item?id=23473365](https://news.ycombinator.com/item?id=23473365)

</comment><comment><USER26> Thanks for describing your process as well as the tool, jaden! I appreciate your pursuit of efficiency in the download and implementation. This is inspiring me to add Wasm to my Rust usage.

<comment><USER26> That is, thanks to Matthias. But thanks for the post, jaden

</comment></comment><comment><USER27> this should be smaller than sth like lunr.js?

[https://lunrjs.com/guides/getting_started.html](https://lunrjs.com/guides/getting_started.html)

<comment><USER28> potentially _much_ smaller, since you don't need to bundle the full content of all articles to be able to search them.

</comment></comment><comment><USER29> How does this compare to a full reverse index? I would expect a full index to be much simpler to implement and would compress better.

Still very impressive work, and gives me a new reason to learn Rust.

<comment><USER9> Author here. Tried a full reverse index first but it's much bigger in size - think around two orders of magnitude, if I remember correctly.

</comment></comment><comment><USER30> I'm also curious, how does zola compare to jekyll?

<comment><USER31> jekyll:

\+ plugin support

\+ large community with lots of themes/plugins

\- need to install ruby and dependencies

\- slow to build large sites

zola:

\+ easy install (precompiled binary)

\+ fast

\- smaller feature set and community

\- no plugins

</comment></comment><comment><USER32> this search does not work, but I enjoy your enthusiasm.

<comment><USER33> In terms of not performing what the user might expect from search behaviour, an example I found was the following:

A word "elasticlunr", appears in the linked article, and the linked article appears in search results, but searching any partial string such as "elastic", "elasticl" "elasticlu" and "elasticlun" will not result in finding the linked article. Perhaps this behaviour is intended by the author, but it may not be intended by the various users of the site.

Oddly,

> elastic* and elasticl*

does find the linked article, but

> elasticlu* and elasticlun*

do not.

<comment><USER34> Also the search index has not been updated in 8 months so it doesn't include the several recent articles. Which can be confusing, since those articles are right next to the search box when you're at the homepage. I opened a github issue for him.

<comment><USER9> Thanks for the heads-up; will fix.

The reason is, that I'm working on decoupling the search frontend from the JSON search blobs. Want to make the frontend-part installable through npm as well (and not just cargo as it is now). Didn't get around to adding the search index generation to Github actions yet due to limited time. Here's the pipeline if you want to give me a hand and add the tinysearch build: [https://github.com/mre/mre.github.io/blob/source/.github/wor...](https://github.com/mre/mre.github.io/blob/source/.github/workflows/ci.yml)

</comment></comment></comment></comment><comment><USER35> Interesting use of zola, been thinking about trying gatsby.js perhaps Zola as well now. Has anyone used either?

<comment><USER26> Just started using Zola recently. Early, but after comparing with several other engines it seemed the best suited to my application. So far I'm happy with it.

<comment><USER36> I'm a big fan of Zola. When I need more features, I'd reach for Hugo before Jekyll. But for most simple static sites, Zola is my favorite.

<comment><USER37> Which features are you missing the most?

</comment></comment></comment></comment><comment><USER38> Great stuff, but it doesn't seem to search titles

<comment><USER9> Yeah, that's a bug. XD I was ingesting the title into the bloomfilter without making it lowercase like the rest. Then when searching, I lowercase the user input and guess what... the title can't be matched. Whoops. ;) Will fix.

</comment></comment><comment><USER39> Thanks Matthias! I learned a lot from your YouTube channel on Rust. One of my favorite tech channels ever.

<comment><USER9> Awww. Thanks so much. I suffer from extreme impostor syndrome, which is one reason why I didn't continue making shows. Hearing that people actually learned something is heart-warming. If anyone is interested, the old episodes are here: hello-rust.show

<comment><USER39> I truly mean it. Your channel is one of the best real tech channels I've ever seen.

<comment><USER40> Link to said channel please?

<comment><USER9> [https://www.youtube.com/hellorust](https://www.youtube.com/hellorust)</comment></comment></comment></comment></comment>


How Long Can YouTube Survive?   - <AUTHOR> http://www.independent.ie/business/technology/how-can-youtube-survive-1809620.html

<comment><USER2> That Credit Suisse report about how much it costs to run YouTube has been _widely_ debunked.

Here's a newsflash: Google pays a lot less for bandwidth than you do. They've got peering agreements everywhere.

<comment><USER3> I figure they pay $2.50 / M / month or less, the best deals that I can find when I go shop for high volume is about $4 / Meg, that's when you're committing to 10G or thereabouts.

</comment></comment><comment><USER4> If its Google we're talking about even if the youtube portion of the company is losing money, does it matter? I think its pretty safe to assume that youtube will survive.

I mean, I'd assume that the idea of a loss leader somewhat applies here as well. Youtube is the leading brand for video on the internet, and that's worth something to the bigger Google brand.

Don't get me wrong, I think the debate about "free" is a debate worth having, I just don't like the misleading/sensational headline.

</comment><comment><USER3> They'll survive as long as google is footing the bill. Note that they've started to be more aggressive with their advertising on youtube lately, especially ads overlaying the video in the beginning.

I'm pretty sure they'll find a way sooner or later. It took them a while to come up with adsense as well.

</comment><comment><USER5> I am adding what I said on a previous thread. it is not direct profits but it is indirect profits and strategic intentions as the reason google bought it. google wants you to use its services, the more you do the more you depend on them. video, voice, data, etc. they make profit as a whole, so you cant look at one unit and say it doesnt bring in profit. the longer you associate google with your video, voice and data needs they win. brand recognition and dominance.

</comment><comment><USER6> Youtube definately will survive, once Google introduces the YouTube Marketplace, it will allow producers to sell products directly from the video. Call to action, buy now videos. Google is positioning itself to become a serious platform (better & bigger then TV), think iTunes but not limited to just creative content.

<comment><USER7> I don't know if I should be sad or grateful that Billy Mays did not live to see this day.

</comment></comment><comment><USER8> OP doesn't even mention the most likely scenario: Google has something else up it's sleeve.

With their technical prowess, it could be anything. Perhaps embedding ad placements within existing videos. Or assembling sellable demographics about who watches what, a distinct possibility once Chrome reaches critical mass. Or acquiring copyrights. Who knows?

One thing's for sure, Google was always one step ahead of mainstream media. Probably still is.</comment>


US Open Government Initiative Enters Phase Three - <AUTHOR> http://news.slashdot.org/story/09/06/23/0019219/US-Open-Government-Initiative-Enters-Phase-Three?from=rss

<comment><USER2> Three of the top 10 most popular ideas called for legalizing marijuana, and two featured conspiracy theories about Mr. Obama's true place of birth.'"</comment>


What I've learned about scaling OSPF in Datacenters - <AUTHOR> https://elegantnetwork.github.io/posts/What-Ive-learned-about-OSPF/ <comment><USER3> This article challenges partisans in the BGP vs OSPF debate but it's deeper than just pointing out that the wire protocol is quite superficial. Obviously, at scale you have to control OSPF to avoid update floods. But BGP also needs to be controlled so bad entries aren't trusted and propagated; for example, [https://blog.cloudflare.com/how-verizon-and-a-bgp- optimizer-...](https://blog.cloudflare.com/how-verizon-and-a-bgp-optimizer- knocked-large-parts-of-the-internet-offline-today/)

The deeper message in the article is linked [https://elegantnetwork.github.io/posts/Network-Validation- wi...](https://elegantnetwork.github.io/posts/Network-Validation-with- Vagrant/) and it's about simulating infrastructure. This is so important for large scale systems as complex aggregate behaviour easily emerges from myopic error handling strategies that only take into account the local view from each actor. This is true for any large scale system not just networks!

The wire protocol is the surface; what information is propagated is essential and what lies underneath. Simulation illuminates these depths. Whiteboard discussions are great for debugging and communicating specific cases - <USER2> example, aggregate responses to retry backoff rules that are locally sensible can cascade into catastrophes if many actors follow the same process. However, it's hard as human beings to find these cases without the help of tools. Tools are so important!

<comment><USER4> The BGP problem you're linking to owes to it being run at Internet scale by diverse teams, not as an interior routing protocol.

<comment><USER3> Yes.

I linked to it because it touched on BGP optimisation techniques, and illustrates the dangers of automatically passing on each update.

Is there a better link to a public postmortem that illustrates these problems in a datacenter context? When we were operating the Facebook Moonshot project we found that even internally in controlled datacenters, at a scale less than Amazon's, there is plenty of diversity

</comment></comment></comment><comment><USER5> >I think whiteboards are the most important tool for network design (...)I can't even tell you the number of disasters averted by 2-3 great network engineers arguing over a whiteboard.

I strongly agree with this statement and think it holds true for a lot of IT/dev/ops fields.

That's something I miss when working remotely. Has anyone found a tool that is seamless enough to allow for true interactivity, as much as be in the same room and sharing a whiteboard?

<comment><USER6> Absolutely agree as well.

I've solved a lot if IT problems with a whiteboard, from networking engineering to server engineering to software engineering

</comment></comment><comment> m-app > If you are running a small Clos network and don't have IPv6 or EVPN [..]

... seem like two relevant qualifiers to choose BGP over OSPF.

Other than that, being religious about a certain technology seems like a bad mindset to hold on to indeed. When you have a certain scale it starts to make sense to question the efficiency of the primary technology in the market for your specific use case.

Edit: essential qualifier for the mentioned mindset added

</comment><comment><USER7> I would suggest the author also left out the consideration of skillset.

In the sort of environment described, its pretty much guaranteed you'll be already managing EBGP sessions to the outside world.

So why not capitalise on the BGP knowledge you have and use it internally too, instead of bringing another protocol (OSPF) into the mix.

Jack of all trades, master of none. As the old saying goes.

<comment><USER8> Most network engineers are very well versed in OSPF. Having come up through the ranks of Cisco's early NetAcad curriculum in the early 2000s, when BGP was very rarely used in internal networks, thr content was 80% focused on IGP and large L2 design and troubleshooting. It seems to me the IGP has fallen out of favor partially due to simplicity over the right tool in the right context. I'm sure, as you state, part of that has to do with the responsible engineers lacking experience and shying away. But in many cases BGP can be slower and less forgiving where a focused IGP can add value and, most importantly control.

I was part of a team running a multi-state network for an ISP around a decade ago. Even then we had a lot of design and architecture reviews of spreading BGP as far and wide as possible. But the IGPs, at regional node levels (HFC type networks) always had functionality for end user provisioning that allowed us to more easily provision and troubleshoot.

While I don't agree or disagree that BGP can't be the right solution what it really comes down to is understanding the use cases compared to the protocols. I find that most engineers today only loosely understand BGP and OSPF. Fully vetted network engineers seem to be fewer and further between. I moved away from that as a role years ago but my experience in large networks and understanding those protocols, although admittedly not as well as when I was studying for the CCIE labs, has been a fundamental skill I still use day in and out. And as we move more to software defined overlays I find it coming up more and more these days.

</comment></comment><comment><USER9> Is this why AWS doesn't support IPv6? Because they're running OSPF (v2) across their datacenters?

After all the work they (apparently) had to do to get OSPF to work, I can understnd why they'd not want to repeat all of that to test and validate OSPFv3 too.

\---

For the uninitiated, OSPFv2 _only_ supports IPv4. IPv6 would also require deployment of OSPFv3. BGP, which is typically used in these situations, supports both IPv4 and IPv6 (and many other features which are often desired).

<comment><USER10> AWS does support IPv6.

<comment><USER9> That's a relatively recent change, no?</comment></comment></comment>


Ask HN: What would you learn/do if you had an entire summer free? - <AUTHOR> <comment><USER2> Welding and turning. I watched a competent metal worker on a lathe, its thinking in 3D writ large. welding is making one thing from two. its increadibly hard to do well. (gas tight)

Baking. I worked briefly in a bakery in the 1980s and there is something about the midnight shift, making the bread for the morning. I know its an unhealthy lifestyle.

I know a gypsy brewer who now runs his own microbrewery. I'd haul sacks all day to be part of the cycle of a complete brew to keg.

We spend far too much time in our heads online in the ICT business. I'd use my free summer to learn how to integrate body and mind.

I'm at the back-end of my working career btw.

</comment><comment><USER3> Approximately 50% of what I would like to get done. Happens every time I go on vacation too.</comment>


PHP Developers - Wake up - <AUTHOR> http://www.reddit.com/r/PHP/comments/1m64m0/phpdev_wake_up/

<comment><USER2> why post a link to lebbit when you could have just posted the link to the actual post...

this is bait

<comment><USER3> Linking to a Reddit submission that links to a blog that links to a tweet that links to a blog that talks about mailing list drama.

...and it all boils down to "Fuck you, I'm leaving & I think I'm important enough to cry about it for 5 pages".

<comment><AUTHOR> The link from reddit links to the PHP mailing list.

Anyhow, it seems like that's the fundamental issue with PHP internals.

<comment><USER3> ...and it shows.</comment></comment></comment></comment>


Show HN: Portrit.com - Finding the best photos between your friends. - <AUTHOR> http://www.portrit.com/

<comment><AUTHOR> Portrit started as a Facebook photo viewer. As my co-founder and I worked, we realized that people have way too many photos and most are not that interesting to look at. We wanted a way to find our friend's best photos quickly, so we created what we are showing today.

Portrit is all about filtering you and your friend's photos. This happens through a method of nomination. You look through your friend's photos and nominate one that you think is above the rest. We have 10 categories that we call "trophies", including Hot, WTF, LOL, and Fail. Once nominated, the photo goes into the active stream where you can see all you and your friend's nominations. At this point it's up to your social graph to determine what photo is the Hottest, funniest, etc. This is done through a simple voting system similar to HN, Digg, or Reddit. At the end of a 24 hour period (11PM PST), whoever has the highest votes in that category wins the trophy and the losers are cleared out for the next day. The trophy is added to the user's profile so their friends can easily find the best photos.

For the non-facebook people, see screenshots in our chrome app: [https://chrome.google.com/webstore/detail/hbbhgkmmpcjnmijojo...](https://chrome.google.com/webstore/detail/hbbhgkmmpcjnmijojodbgebcklobaijh?hl=en- US)

We would love any feedback!

Quick technical info for those that are inclined: * Standard backend: Django/Apache/Nginx/Memcached * frontend is a single page with dynamic views. * Around 90% of the code base is on the frontend. We love JavaScript! * Using Node.js for notifications/real-time content updates.</comment>


The Hitchhiker's Guide to Ethereum - <AUTHOR> https://www.wslyvh.com/ethereum-guide/ <comment><AUTHOR> Over the past few years, I've been helping and on-boarding new team members and developers into the blockchain space. This helped me to collect blogs, links, materials, and resources and see which worked well for everyone's learning journey. It's time to share that with a broader audience.

The guide provides a starting point for developers who're keen to learn more about blockchain and development on top of Ethereum.

<comment><USER2> Thanks so much for putting this together!

<comment><AUTHOR> My pleasure. Hope you find it useful

</comment></comment></comment><comment><USER3> > Blockchain has emerged from the once shadowy world of cryptocurrency to > become the most in-demand skill in 2020, according to LinkedIn. The promise > of blockchain is huge. And large companies are continuing to hire and expand > their blockchain teams, incl. Facebook, Amazon, Microsft, EY, Deloitte, IBM, > and Oracle, to name a few. So, it should be well worth the effort to become > familiar with how blockchain works, what its perceived benefits are, and as > a developer starting building on top of them.

This is the problem with "Blockchain" in one paragraph.

Notice how there's no statement of a problem? Notice how it leads with hype and technology? Notice how it tries to FOMO you into action?

The Bitcoin block chain has been tried in many different situations. So far there is very little show beyond electronic cash. Not inventory control, not voting, not land registration.

The one other application that might be considered mostly viable is timestamping. Bitcoin is a timestamping server, after all.

The good news is that the problems electronic cash solves are getting worse every year. Privacy. Deplatforming. Weaponization of the dollar and other local currencies.

The bad news is that Bitcoin is also good at running itself, which means there isn't a lot of room for startups in the traditional sense. There may be room for sponsored non-profits such as Blockstream and Lightning Labs. There the funder model seems to be "improve Bitcoin and the value of your holdings will increase."

<comment><USER4> The article lost me at "According to LinkedIn" :) I think this perfectly captures how Blockchain is something you put on your LinkedIn profile purely for signaling to recruiters and non-technical PR and marketing people that use LinkedIn, which is full of spam and pyramid schemes.

</comment></comment><comment><USER5> I see these "Hitchhiker's Guide" a lot, but other than a reference to the book "hitchikers guide to the galaxy", is there a better definition here?

How should I picture "Hitchhiker" \- in the book, isn't the main character an unintentional observer/participant? So is this a beginner?

Or perhaps a hitchhiker is a hardened, obstinate person who enjoys the frontier and moving around a lot?

I don't know; it never made sense to me.

<comment><USER6> In the books, there is an electronic guide to the combined knowledge and wisdom of all the various species that inhabit the universe. The guide is itself called "the hitchhiker's guide to the galaxy."

I have always interpreted such titles to be a reference to this in-universe guide rather then to the novel or its characters.

We could split hairs over the fact the in-universe guide is more analogous to Wikipedia in scope and scale than a more condensed tutorial or primer, or whether the reference is obscure enough that it confuses, alienates, or distracts readers who aren't part of the intersection of Hackernews Readers and Hitchikers Guide fans.

It never really bothered me but I was a huge fan of the series when I was younger.

Edit to add: as I am a technical writer on the side, it's a timely reminder to avoid any analogies or cultural references that require knowledge others in my target audience might not share. So, interesting point!

<comment><USER7> Before those books, there was a non-fiction book called _The Hitch-hiker 's Guide to Europe_ [0]. It was a reference for travellers in Europe on a budget, something similar to the niche the _Lonely Planet_ books fill today. It was a big tome, updated regularly with new editions from 1971 through the mid-90s, and well enough known that Douglas Adams' readers would have got the joke. My parents had a copy.

That's the original inspiration for the in-universe _Hitch-hiker 's Guide to the Galaxy_ and perhaps the better point of comparison for other guides that are named after it.

[0] [https://en.wikipedia.org/wiki/Hitch- hiker's_Guide_to_Europe](https://en.wikipedia.org/wiki/Hitch- hiker's_Guide_to_Europe)

<comment><USER6> Thanks for the insight, I never heard of this until today and this adds another dimension to explore.

I read some excerpts here [https://h2g2.com/edited_entry/A4158164](https://h2g2.com/edited_entry/A4158164) and it definitely adds context that only really improves my interpretation of the Douglas Adams series.

"Always carry a towel" is one such reference that I can now replay and re- imagine as a parody of the original guide. I'd assumed them to just be a random amalgamation that parodies the overall genre of various guides and travel literature, it's fun to know there's a solid reference point.

As for the relevance of the "Hitchhiker's Guide to <subject>" title format in light of this information, I agree with you that the original source is a better comparison point; while there will be a lot of fans of the Adams work who can relate on that level, there will be others who either owned or knew about the original source of the parody who will interpret the format more closely to its intention. And ultimately it's a format that's so frequently used that there will be a broader spectrum of people for whom "Hitchiker's Guide..." is just synonymous with "down and dirty advice to get you started quickly, without pretension."

This has turned out to be an interesting rabbit hole, personally at least.

</comment></comment></comment></comment><comment><USER8> Is blockchain still a thing ?

<comment><USER9> Nobody has ever been able to explain to me why it has any value for our society.

[https://louwrentius.com/cryptocurrencies-are-detrimental- to-...](https://louwrentius.com/cryptocurrencies-are-detrimental-to- society.html)

<comment><USER2> Reading through your post, I want to point out that all your insightful reasons why cryptocurrency should be shut down are actually acknowledged problems that have been worked on by many in the space leading to solutions that address different pain points you've brought up. Examples include stablecoins (this name is meant to be as ironic as you think it is), private blockchains (here's a hint, nobody is using these to buy drugs or demand ransom), blockchains protected by something other than "energy wasting" Proof of Work.

I do have a few points for you to consider that might help you see the value of all this experimentation for society:

\- Imagine someone living in a country such as Cuba or Venezuela or Iran that wants to buy things online (no, not heroin) and why they might find the existence of something like Bitcoin or Ethereum useful.

\- You call for it all to be shut down, but how exactly would that be accomplished? (Most) cryptocurrencies are open source software and in a fringe scenario where experimentation here only serves to improve the anti-fraud precautions surrounding traditional currencies, isn't that still of some value to our society?

Some might argue that these supposed benefits don't outweigh the existing harm and therefore there isn't net value to society and there I'd just say we need to agree to disagree.

Have I perhaps changed your mind that cryptocurrencies don't have any value for our global society?

</comment></comment></comment><comment><USER10> Ethereum seems to be taking a lot of time to release their "2.0" version that provides scalability and doesn't require using huge amounts of energy - something that is pretty much inexcusable in this age of global warming.

In the meantime other contenders seem to have more momentum.

Case in point: CryptoKitties is leaving Ethereum because it lacked capacity and had high transaction fees.

<comment><USER11> Ethereum is scalable already. Layer 2 solutions like [https://zksync.io/](https://zksync.io/) and [https://loopring.org/#/](https://loopring.org/#/) are already live and allow 2000 transactions per second. This is based on ZK-rollups and provides the same level of security as the base layer. This technology can be used for trades or transfers.

Another exciting technology that is coming this year is optimistic rollups, which will provide the same level of scalability for arbitrary smart contracts.

<comment><USER12> From the zkSync website:

"WARNING: _zkSync v1.0 is in alpha. Blockchains and zero-knowledge proofs are still experimental technologies with rapidly evolving attack vectors. zkSync relies on cutting-edge cryptography that has never been used in production before. While Matter Labs is rigorously following scientific and engineering best practices with regard to security, we can not provide 100% fault-free guarantee. Use zkSync at your own risk and do not put more money into it than you can afford to lose._ "

That doesn't sound like 'scalable already', it sounds more like be prepared to lose your money.

Further, on the ZK-Rollups website, we have:

" _The initial set up of ZK-Rollups promotes a centralized scheme_ _The security scheme assumes a level of unverifiable trust_

 _The initial setup of ZK-Rollups is assumed to be a trusted state, when this trust cannot be proven. A small group of developers will be subject matter experts on the initial trusted state. This undermines decentralization and opens the risk of social engineering hacking attacks by convincing a developer to manipulate code or provide vulnerability information._ "

This is not reliable scalable technology. It's centralised so it takes all the supposed advantages of blockchains away, at which point you are better off just using a single trusted database, instead of thousands of machines duplicating the same work.

</comment></comment></comment><comment><USER13> The world's most expensive tech demo / self-assembling pyramid scheme continues apace.

Has anyone found a non-criminal use case for blockchain yet? Preferably one that isn't better addressed by a single Postgres instance?

<comment><USER11> Yes, here's a use case - an open, decentralized, neutral, borderless and censorship resistant prediction market like Augur.

<comment><USER13> Why does anyone want that? Aren't you describing the supposed proposed assassination market?

[https://reason.com/2018/07/27/markets-in-assassination- every...](https://reason.com/2018/07/27/markets-in-assassination-everybody- panic/)

<comment><USER11> Why do people instantly jump to the extreme negative? Any technology can be used for to do bad things. The community is aware of this possibility and have taken precautions. Also any financial market can also be viewed as an assassination market. You can short a stock and assassinate the CEO and profit, but people don't actually do that in practice. Also, Augur v1 (V2 is coming out this month) has existed for about 2 years and there were no assassination markets on it. The community would resolve them as "invalid", so it'a not possible to guarantee profit unlike in the regular stock market.

<comment><USER14> If people tout anonymity and 'censorship resistance' as a market's main advantage over other markets, it's hardly surprising that people focus on the use cases - hypothetical and otherwise - which wouldn't be possible on prediction market platforms where the platform operator knows their customer and rules out certain types of trade. If your USP is the ability to do 'bad things' to manipulate outcomes [manipulating the 'reporting' is probably a likelier outcome than assassination, admittedly] with little chance of your trades being discovered or reversed then people are going to focus on that.

</comment></comment></comment></comment></comment><comment><USER15> My guide to Ethereum: don't get involved in shady, unregulated industries unless you're aware of the possible consequences are are willing to accept them gracefully

<comment><USER11> What are the consequences that you're talking about?

<comment><USER16> Ever heard of the DAO? [1]

Seriously, crypto is one of the shadiest things you can get into. Avoid it, and any exchanges, like the plague. It's interesting tech, but nothing more than that. Hugely speculative and not worth wasting hard earned money over.

[1] [https://www.coindesk.com/understanding-dao-hack- journalists](https://www.coindesk.com/understanding-dao-hack-journalists)</comment></comment></comment>


D.J. Bernstein proposes an almost zero-effort DNSSEC alternative (Presentation) - <AUTHOR> http://cr.yp.to/talks/2008.08.22/slides.pdf

<comment><USER2> It's not quite that simple: I think DJB is proposing that we solve a different problem than DNSSEC tries to solve.

Compare DNS security to HTTP security. SSL/TLS secures HTTP connections, but not their underlying content. DNSSEC is like trying to encrypt and sign every web page, instead of the secure connections. Which is why it's unlikely to ever happen.

DNSCurve is a design that uses modern high-speed ECC crypto to do packet- transaction-speed security. It looks like its to DNS what TLS is to the web.

That said, since DNSSEC is a crazy idea (obligatory self-link: <http://www.matasano.com/log/case-against-dnssec/>) and because DJB got a lot of (well deserved) attention after this summer's DNS debacle, this could get some serious attention.

</comment><comment><USER3> I'd argue that slides are more on DNSSEC weaknesses than on DNSCurve [1] strengths. Actually he only mentions advantages of his solution as opposed to giving quite detailed criticism of DNSSEC.

However, great presentation and I highly recommend it to all network/security geeks.

[1] <http://dnscurve.org/>

</comment><comment><USER4> This proposes to encode an encryption key into the NS delegation for a zone, and backing it up with an encryption scheme touted to be fast enough to handle high loads.

The idea of encoding a key into the NS name sounds like something djb would dream up, seeing the names as useless anyway (you'd get that impression after using his software and reading his notes on dns). The part that pushes the solution over the top is providing a fast enough encryption algorithm to work for dns servers.

What can I say, I'm impressed.</comment>


GTUG Campout: Free hacking event at Googleplex - <AUTHOR> http://gtug-campout-sv-fbnews.eventbrite.com/ <comment><USER2> I had tons of fun last time at Googleplex when they had an OpenSocial event, and going there (even though I had to fly from Finland) actually ended up saving me tons of money with all the stuff I learned about how to use less resources when making profile boxes and how to make things scale. Highly recommended!

</comment><comment><USER3> I'm setting up a similar event in Los Angeles, perhaps a barcamp-type event. Sign up for it if you're interested: <http://www.meetup.com/speed-dating-for- hackers>

</comment><comment><USER4> Kinda reminds me of a Startup Weekend type event (disclaimer: current startup came outta a Startup Weekend). Pretty neat to see Google try something like this. Whoever's going will have a blast! oh and hey Phil. :-)

</comment><comment><USER5> Sweet deal. Just signed up. Anyone else going? Will be new to the Bay Area, would be great to meet up with some HN folks.

<comment><AUTHOR> I can't go for all 3 days, but I'll be there for the Sunday night demo. Looks like fun.</comment></comment>


GraphLab: A New Parallel Framework For Machine Learning - <AUTHOR> http://graphlab.org/

<comment><USER2> In short, GraphLab is a parallel programming abstraction. It is slightly low- level than MapReduce. Many algorithms can be written in a scalable way in the GraphLab framework that, were they written in MapReduce, they would not scale correctly. Which is to say that MapReduce is too high-level to implement many useful data processing algorithms, but GraphLab is at the appropriate level of abstraction.

I asked about GraphLab on MetaOptimize, and the authors of GraphLab gave some high-level information:

[http://metaoptimize.com/qa/questions/285/when-should-you- pre...](http://metaoptimize.com/qa/questions/285/when-should-you-prefer- graphlab-over-mapreduce-and-vice-versa)

Note that this Q+A was done before the paper was published (but was based upon the camera-ready). I believe that the code has matured since then, but it might not yet be ready for production.

Joseph Gonzalez: "MapReduce is good for single-iteration and embarassingly parallel distributed tasks like feature processing, while GraphLab is good for iterative algorithms with computational dependencies or complex asynchronous schedules. For instance, we find that GraphLab is highly suitable for Gibbs sampling, EM-style algorithms and some classes of optimization algorithms. Programs which fit well on a systolic abstraction (such as PageRank on Pregel) will also work well with GraphLab. There are probably a lot more algorithms that will fit well in the GraphLab and we are still exploring the capabilities and implications of the abstraction (and whether further extensions will be needed)."

<comment><AUTHOR> Awesome, thanks for the pointer to MetaOptimize. I always forget to check there for info.

</comment></comment><comment><AUTHOR> Posted to see if anyone has experience working with GraphLab. Alternatively, do any GL contributors frequent HN?

<comment><USER3> I sadly missed a talk by its creator last week :( It looks interesting as a framework for parallelised algorithms where the dataflow isn't quite as straightforward as MapReduce (quite common in machine learning). I may try and use it for my dissertation actually. But no experience yet to speak of.</comment></comment>


Work until your empty not dry - <AUTHOR> https://medium.com/personal-growth/work-till-youre-empty-but-not-till-you-re-dry-e79c463dbec6#.cbkqtxuly <comment><USER2> Note sure if the author or the poster got the since-fixed "Your" wrong (which is what I think HoopleHead was getting at), but the author definitely botched "'Till" (sic).

The eBay companies must have a similar botch right in their style guide or something. They use "'til" _everywhere_ instead of "till" or "until".</comment>


How the GNU coreutils are tested (2017) - <AUTHOR> http://www.pixelbeat.org/docs/coreutils-testing.html <comment><USER2> Really nice overview.

</comment><comment><USER3> Have you ever heard that old phrase "you can either make a program so simple it obviously has no bugs, or you can make a program so complex that it has no obvious bugs"?

GNU programs are the latter. That's why they need to go to such lengths to test "cat".

<comment><USER4> Trite, and seems to ignore evolution. Most of the complexity in these utilities came from emergent behaviour.

<comment><USER3> Wrong, most of the complexity was designed in. Have you ever read the source code for GNU coreutils? And maybe compared it to the POSIX specification for those tools?

<comment><USER5> GNU true/false in particular is like one of those jokes that takes two minutes to build up to a cheesy pun at the end:

Overly elaborate setup: [https://github.com/coreutils/coreutils/blob/master/src/true....](https://github.com/coreutils/coreutils/blob/master/src/true.c)

Snappy punchline: [https://github.com/coreutils/coreutils/blob/master/src/false...](https://github.com/coreutils/coreutils/blob/master/src/false.c)</comment></comment></comment></comment>


GitHub Down : 404 Not Found - <AUTHOR> https://github.com/ <comment><USER2> One minute after posting time, it's up for me.

<comment><USER3> 11 mins after this, 404ing here

</comment></comment><comment><USER4> 15 minutes later, still 404'd</comment>


Ask HN: What equivalents of Erlang/OTP exist for other languages? - <AUTHOR> <comment><USER2> I know I missed this but I do want to make some (general) comments:

\- Erlang, the language and system, was designed for highly concurrent and fault tolerant systems. In many ways these were more important than raw throughput in that if it could not handle the massive concurrency and build truly fault-tolerant systems then it wasn't interesting. At all.

\- Same with pre-emptive scheduling, the system had to be non-blocking. Again if it wasn't then it wasn't interesting. Yes, you can provide primitives to allow the programmers to do it but this does make things difficult and not- reliable.

\- The language design was focussed on these types of systems and and implementing the types of architectures which OTP supports.

\- From Erlang's point of view using OS threads as a base for Erlang processes is not an option. They are way too heavy and you can't have enough of them to be interesting. 10k processes are child's play, 100k processes is starting to get interesting and 1M process production systems exist, for example WhatsApp.

\- For Erlang processes are the basic building blocks in a similar way to objects in an OO language. What would a Java programmer say if they were told that they couldn't have more than 1000 objects?

\- Having the erlang VM handle using all the cores, or as many as you want, by default is just the natural way to do things. If I had to do any form of restructuring of my system because I was running on 4, 8, 16 or 32 cores I would consider that to be intolerable and so primitive I would wonder what the implementors were thinking of.

\- While the Erlang syntax is different (which functional language doesn't have a different syntax?) it is actually very concise and consistent, this by design. The elixir syntax is "Ruby influenced" and more complex and feature filled. Which you prefer is up to you.

\- I am very fond of lisp so there is at least one native implementation of lisp on the Erlang VM, LFE (Lisp Flavoured Erlang), [http://lfe.io/](http://lfe.io/) and [https://github.com/rvirding/lfe](https://github.com/rvirding/lfe).

\- There is nothing which you can do in one which you can't do in the others, after all they run on the same VM and you can easily combine them and use them together.

That's about all for the moment,

Robert

</comment><comment><USER3> The real question is: Why don't you just use Erlang or Elixir?

Can you find the magic sauce in another language?

I believe the honest answer is no. You can't do it in Java because the JVM doesn't support it. You can't really build this on top of something that was not designed with parallelism and concurrency in mind, and designed correctly. For that reason Go doesn't do it either. Scripting languages aren't even thinking about it and to do it in something like C or C++ you'd basically be recreating a poorly implemented half working version of erlang.

So more interesting is--- why do people have this aversion to Erlang and Elixir?

It doesn't take but a couple weeks to get used to erlang programming, and Elixir is even faster I think.

For the record I've been taking interns thru Programming Elixir in about 2 weeks and getting them to where they can program in the language (just at that point now with a new group so we'll see how much they struggle... but I've already seen code written by them.)

Erlang did it right and has put in several decades of effort getting it even more right. Other languages, like Go, aren't even trying (though of course advertising "concurrency" is very popular, none of them really do it.)

All these people choosing things like node.js or go for problems where erlang is the correct solution make me think that too many of the people in our industry aren't really engineers, but more scripters following the herd.

And engineer knows why erlang is the correct solution for distributed systems.

<comment><USER4> My aversion comes from relatively poor language syntax - it's more verbose than it should be. Lack of powerful type system. And less-than-stellar performance. Out of these, the language is the most annoying.

Can you state which language constructs Erlang has that other languages don't, that are critical to concurrency? Cause it looks like someone built a great VM and cluster library, then tossed in a language for fun.

And please explain why Erlang's features are not _possible_ in other languages. Not just unavailable at the moment. Or do you mean the guarantees are different? As in other languages will let you have corrupted shared state? (I view that as a positive; let me escape guarantees when I want too.)

Or perhaps point me to a decent intro that'd cover all this? I looked at it before and didn't get the real point why we need Erlang the language or why in theory, you can't have OTP with the CLR or JVM.

<comment><USER5> Syntax does not matter much; Erlang's focus is not to be a good general- purpose language. It is good for distributed computing _because_ it lacks many useful and pleasant features.

Erlang has built-in immutability done right (more so than in Scala, OCaml, Clojure(?)) - if it's not built in the language from the start, there's nothing you can do about it.

Erlang has built-in binary serialization/deserialization that makes messages across network easy and natural. Please show me a language that can serialize a closure and send it over network. A rich type system and general-purpose language features are obstacles here, types of values have to be _dumb_ to serialize seamlessly.

Other than that, it has a lot of convenient and coherent features like good pattern-matching (useful for parsing and routing messages), proper tail-call optimization (just look at Scala, again), the built-in actor model, lightweight processes, process monitoring, the large library of battle-tested distributed practices, OTP, – these things _can_ be recreated in another language/VM, but it will probably take huge resources and efforts and _self- limitation_ of the platform, not features.

</comment></comment></comment><comment><USER6> JVM: Quasar [http://www.paralleluniverse.co/quasar/](http://www.paralleluniverse.co/quasar/)

Scala/Java: Akka [http://akka.io](http://akka.io)

Haskell: Cloud Haskell [http://haskell-distributed.github.io/](http://haskell- distributed.github.io/)

<comment><USER7> FWIW, one of the above is widely used in production systems.

and none of the above have solved preserving types across machine boundaries (or at least not in a stable/non-experimental let's-bet-the-bank-on-this release).

Quasar looks pretty insane at first blush; if Java 9 or 10 delivers on the missing piece [1] for this library, watch out.

[1] "Quasar fibers are implemented by creating and scheduling continuation tasks and since the JVM doesn't (yet) support native continuations, Quasar implements them through selective bytecode instrumentation: methods that can block a fiber currently need to be explicitly marked through annotations so that Quasar can insert the continuation suspension and resumption hooks."

<comment><USER8> I'm not familiar with Erlang: what does preserving types across machine boundaries mean in this context?

<comment><USER9> When you build highly distributed systems, making sure a value is properly transferred across different types of machine is actually not a trivial task because sender and receiver may encode values differently at the bit level (for example, because different endianess system is used). Erlang VM take care of this naturally, and the language offers built in functions to encode / decode binary representations that can be sent over basic transmission channels (like TCP / IP sockets).

If Erlang is not your cup of tea, look at htons() and nstoh() functions to have another example.

My trade is in real time distributed systems applied to the automatic control of trains. You wouldn't believe the number of times I have had to fix this issue in multiple codes, both new or legacy.

And my example is limited to individual values, not data structures. Think about how ugly things can turn if you have things like lists or dictionaries.

When you consider that Erlang adds on top of this built-in marshalling and demarshalling of data structure. As of today, I haven't found a more efficent environment to do this kind of stuff safely.

<comment><USER4> How's that different than using a well defined binary serialization format (protobufs?) and having a controller daemon that makes sure everyone reports the same version or has the same serialization contracts loaded before adding them to discovery?

Or is it just that Erlang has this all batteries included, the way Python's popular with scientific programming?

<comment><USER9> You are spot on. You can do what Erlang does in many ways. It's the combination of all this, built into a coherent architecture, that makes OTP unique.

<comment><USER10> You call it coherent, I call it a small ecosystem with limited choices, limited development resources behind it, and disappointing performance.

<comment><USER11> The reason this thread exists is because other options with strong ecosystems and better performance which accomplish the above mentioned stuff just don't seem to exist.

I'd actually love to see some performance numbers comparing performance on Erlang/OTP vs the alternatives, though. I can't find anything quite like that, maybe someone knows a link.

<comment><USER10> Similar solutions exist everywhere, which is partly why the Erlang ecosystem is so small!

But I agree that Erlang's design is beautiful, which is why I've decided to port it to the JVM with Quasar. I wanted to bring that great design to a more powerful runtime and a much bigger ecosystem. Again, the problem isn't with the Erlang language but with the runtime that, while really nice, simply does not have nearly enough resources to compete with the bigger runtimes like the JVM and .NET (or even V8).

We will have some benchmarks eventually, I promise, but those take a long time and aren't too critical, because, like I said, the Erlang ecosystem is very small.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> Apache Storm[1] isn't exactly like Erlang/OTP, but it definitely aims to achieve similar goals. Though Storm is written in Java and Clojure, the infrastructure it provides is usable across languages thanks to something it calls the "multi-lang protocol".

I work on a Storm library for Python that's called streamparse[2], and the goal of the project is to allow us to easily achieve Erlang OTP-style reliable processing atop open source infrastructure while still writing pure Python code.

I also gave a PyCon talk about streamparse which you can find on YouTube[3]. It describes the motivation for the project -- which is to solve a large-scale real-time analytics problem with Python and do so in a reliable way, while beating Python's multi-core limitations (the GIL, etc.)

[1]: [https://storm.apache.org/](https://storm.apache.org/)

[2]: [https://github.com/Parsely/streamparse](https://github.com/Parsely/streamparse)

[3]: [https://www.youtube.com/watch?v=ja4Qj9-l6WQ](https://www.youtube.com/watch?v=ja4Qj9-l6WQ)

<comment><USER13> not sure if this exists right now, but i would love to see a good tutorial/article on storm showing how to use it to solve real life problems instead of "hey here's a way to split words and count them".

<comment><USER12> It is a fair criticism of the state of Storm documentation and examples. The book Storm Applied by Manning details tons of Storm usage examples, ranging from the technically advanced to the mundane. I recommend that book to learn more about real world usage (disclosure: I wrote the foreword).

Also, my team is working on an open source example project called birding that uses streamparse and pykafka together to build filtered firehose tweet streams from Twitter's public API. We think this will illustrate both Storm and Kafka really well, and be beyond the typical word count examples. See [https://github.com/Parsely/birding](https://github.com/Parsely/birding) to track that effort.

</comment></comment></comment><comment><USER14> I suggest you take a look at Elixir, it's an incredibly well designed and powerful languages. FWIW, I am pretty sure that it will take off exponentially in the coming years... the support for concurrency is _amazing_ , and it also allows for simpler software to be developed easily.

Definitely worth taking a look at it.

<comment><USER15> For those unaware, Elixir is a language that runs on the Erlang VM, so its _amazing_ concurrency is the same as Erlang's.

<comment><USER16> Yup, syntax wise it's like if Ruby was mixed with Erlang.

<comment><USER17> I was deeply disappointed with the syntax… there are so many compromises because of underlying semantics that the 'rubyish' layer on top is very clunky.

It does cut down on a lot of Erlang boilerplate, so it's still a better option… but I'm hoping someone will give it a third try.

<comment><USER18> Totally agree. What they have improved on is a nice macro system, and a coherent (so far) standard library. Both, which Erlang lacks.

The syntax itself is slightly disappointing though. Optional parentheses around argument calls doesn't really add much except confusion, and the extra cruft around blocks and functions make things like pattern matching quite a bit clunkier.

</comment></comment></comment></comment></comment><comment><USER19> My claim is that Erlang/OTP is a cluster OS, not a language. It supports two languages: Erlang and Elixir.

<comment><USER20> The idea that Erlang/OTP is an OS is something promoted by Garrett Smith in this talk ([https://www.youtube.com/watch?v=0ZGHzI9F5YE](https://www.youtube.com/watch?v=0ZGHzI9F5YE)), and the idea that it is intentionally OS-like has been promoted by the Robert Virding and Joe Armstrong ([https://twitter.com/rvirding/status/386612140242915328](https://twitter.com/rvirding/status/386612140242915328))

And two different lisps:

* Lisp Flavored Erlang ([http://lfe.io/](http://lfe.io/)) * Joxa ([http://joxa.org/](http://joxa.org/))

</comment></comment><comment><USER21> Erlang was designed from the ground up to do what it does, so it's... not easy to replicate that in other systems.

<comment><USER10> ... unless those other systems are more general. Erlang itself is written in C. The JVM (or, rather, any JVM) is more low-level and therefore more general than an Erlang VM, and includes much of the work already, so replicating Erlang on the JVM is much easier than replicating it in C.

There is nothing Erlang VMs provide that the JVM doesn't, except for some guarantees that are simply different on the JVM, and I think most applications would prefer the JVM's advantages anyway.

<comment><USER3> OTP is written in erlang. The JVM is not "more low-level" and not "more general". In fact it is impossible to do on the JVM in any JVM language what the erlang VM gives you. You literally cannot replicate erlang on the JVM. You can fake it, but you will not have the guarantees and reliability that the erlang VM gives you.

To get erlang/OTP you would have to replicate it from the beginning in C, and there's no point in that, when you can just use erlang.

<comment><USER22> >and there's no point in that, when you can just use erlang

That is a strange comment, ofc the Erlang VM is not the single best VM that will ever be created. Some day, someone will create a better one.

<comment><USER23> I think the point was more that Erlang was produced at considerable expense, so the cost of recreating Erlang for your own project is probably less than the cost of simply producing your project in Erlang.

<comment><USER24> > I think the point was more that Erlang was produced at considerable expense, > so the cost of recreating Erlang for your own project is probably _more_ > than the cost of simply producing your project in Erlang.

Fixed! Read your comment, understood it, but I think you probably meant the opposite

</comment></comment></comment></comment></comment></comment><comment><USER25> Celluloid: [http://celluloid.io](http://celluloid.io)

<comment><USER26> Do you know why this comment is awesome? I've used celluloid for so long with Ruby.

But recently I've been trying to understand OTP/Erl and I'v had no luck.

Now seeing these two as an alternative makes a lot of sense for diff langs.

<comment><USER25> Yeah, we get people saying we're not 'true Erlang' but that was kind of the whole idea. It's an alternative to Erlang and Akka, pulling the best from both. And it's so active and involved that it's almost becoming more of an RVM than a library only.

With `jRuby` and more and more with `Rubinius`, the concept of actor-based concurrency itself is coming into range of being language-unspecific, with Erlang not being the default authority on that.

Akka gets a lot of respect, and of course Elixr, Go, Rust, etc get a lot of concurrency attention... But Celluloid with Ruby is far superior in my opinion, because it is not only performant and functional, it actually allows you to get into the internals and change how your actor system behaves. It's as simple or as complex as you want it to be. Oh and you don't feel like killing yourself while you're writing code.

<comment><USER26> Indeed, thanks!

One quick question:

Aside of the docs, any suggestions for learning on further Celluloid & Ruby /actor based patterns?

<comment><USER25> We're doing much better documentation as we approach 1.0 ... possibly writing and publishing a book as well.

For now..

Google Groups: [https://groups.google.com/forum/#!forum/celluloid- ruby](https://groups.google.com/forum/#!forum/celluloid-ruby)

IRC: #celluloid on Freenode

Or drop me a message with your question. @digitalextremist on github

<comment><USER26> Thanks a bunch!

</comment></comment></comment></comment></comment></comment><comment><USER27> I'm not an expert in comparing these but there's also:

Orbit for the JVM / Scala by EA (BioWare actually)

[https://github.com/electronicarts/orbit](https://github.com/electronicarts/orbit)

Then there's Orleans by Microsoft for .NET:

[https://github.com/dotnet/orleans](https://github.com/dotnet/orleans)

They feature a concept called Virtual Actors as opposed to other frameworks like Akka for the JVM.

<comment><USER3> Erlang/OTP are not just "actors", and Actors on the JVM aren't actually actors. They're poser actors.

</comment></comment><comment><USER28> Something like Erlang/OTP not really... Although here are some bits/pieces I've seen in Go community:

* Project Iris [http://iris.karalabe.com/](http://iris.karalabe.com/) * Go Circuit [https://github.com/gocircuit/circuit](https://github.com/gocircuit/circuit) * NSQ [http://nsq.io/](http://nsq.io/) * Consul [https://www.consul.io/](https://www.consul.io/) * SkyNet [https://github.com/skynetservices/skynet](https://github.com/skynetservices/skynet) * Grace [https://github.com/facebookgo/grace](https://github.com/facebookgo/grace)

Each having their pros/cons with varying quality.

Replicating Erlang processes -> usually you use goroutines. When that level fails you fail the whole process and restart it using some supervisor. You may need computer level failover as well, depending on the requirements.

For the servers/protocols part of OTP, the stdlib is usually sufficient to get things running; although may need some additions for rarer encodings.

For machine level deployment there are cluster managers i.e. kubernetes, aws, google cloud etc.

Regarding debugging/monitoring I haven't seen anything that is close to Erlang.

Basically, I have no idea what you are doing and cannot recommend anything particular. For the full feature set there is no single replacement. If you need OTP then use OTP.

</comment><comment><USER29> There are similar attempts but OTP is a unique and very powerful approach, the deep integration between the language, core libraries and the framework is hard to match. Why not stick with it? Elixir is fresh and nice, but even Erlang is easy to grab.

<comment><USER3> I think it's more binary. Either the platform takes care of it for you, the way Erlang and OTP do, and thus you do your code and it just works... or the platform hides it from you and you think you're doing ok until you get bitten in the ass in production-- and then you have a Very Bad Day, and possibly years of Fail Whales.

So, Erlang/OTP are correct. The rest, so far, are not correct.

</comment></comment><comment><USER30> Elixir/OTP. :)

</comment><comment><USER31> Ah. Yes... This is mostly how I began my long awkward journey from the JVM, Ruby, Node, et. al to Elixir. Let me give you the one tip that'll make this way easier:

Use the right tool for the job. Not the one you like best. Just because you can use the blunt end of a drill to hit a nail, doesn't mean you should give up hammers... You might even break your drill!

If your problem's solution requires extremely concurrent, high availability, low latency, throughput then you must use BeamVM. The JVM will be too slow and largely memory inefficient compared to Beam.

If concurrency isn't needed, or isnt applicable (parallel, or synchronous, problems), then it's not the answer nor is OTP. E.g. I wouldn't dream of writing and image processor in Elixir-- Beam isn't the right tool to solve that problem.

Also, checkout dialzyer if you think you need type safety. It'll catch plenty of stupid mistakes... I had that urge (for types) coming from the JVM, or a non-functional non-dynamic language, but it's actually a straw man. I'd highly encourage you to get past classes, and state, so you can really enjoy the platform for what it offers-- it's pretty eye opening.

<comment><USER24> I hear frequently that BEAM languages shouldn't be used for numerical computing or any CPU-bound tasks. Why not, though?

If I take a high resolution image and spawn N*M BEAM processes which each deal with performing processing on some subcomponent of the image, and then use some scheduling algorithm to reassemble the image after the processing is done, wouldn't you have substantial speedups compared to using some other image processing library in a language like Python?

In the same vein, things like matrix multiplications or other numerical algorithms that are known to parallelize well seem to lend themselves really well to BEAM processes, although the libraries aren't written yet since you can trivially call Python from BEAM anyways.

<comment><USER5> NxM BEAM processes will give you a real boost only if you have NxM physical cores to run code in parallel. Yes, Erlang makes clusters painless, but it's still more reasonable to use a single machine with a faster VM/native code.

</comment></comment></comment><comment><USER32> Celluloid for Ruby implements the actor concurrency model. It's not exactly the same, but running on JRuby it's a reasonable platform for writing concurrent applications in a highly productive language and style.

<comment><USER3> Or you can use Elixir and get the real thing with a ruby like syntax (plus a lot more.)

<comment><USER25> Can you expound on this?

<comment><USER32> Elixir is a Ruby like language that runs on the BEAM VM, the bytecode virtual machine that Erlang runs on.

Elixir is thus able to execute Erlang code directly including OTP. This means you can use OTP as you would in Erlang but without the somewhat arcane syntax. Thus it can be said, you can use the "real" thing.

</comment></comment></comment></comment><comment><USER33> Not OTP, but the Erlang message passing model has been available for Python since 2002 or so, in a package called Candygram[0].

[0] [https://launchpad.net/candygram](https://launchpad.net/candygram) \- main page is on SourceForge which is down at the moment, and should be avoided even when it is up.

</comment><comment><USER34> Preemption. Erlang VM counts each instruction per process and uses it's own scheduler to switch processes, all other implementations of lightweight processes can't do this without implementing their own erlang.

<comment><USER4> So every OS is it's own Erlang?

<comment><USER34> Yes, but "modern" OS'es have heavy processes, in other words they save registers, stack pointers, other process related things and restore them on every process switch.

I'm not sure what makes real difference to Erlang's processes here, I hope someone can explain this with more details.

</comment></comment></comment><comment><USER35> Why not just use Elixir or Erlang? I never understood why programmers would spend years re-writing something in their language of choice if it already existed.

</comment><comment><USER36> VS C++: Generic Parallel Computing (GPC) API [http://parallect.codeplex.com](http://parallect.codeplex.com)

<comment><USER3> You could implement a half written, buggy version of erlang in C++... but GPC isn't even close to erlang/OTP.

What I'm learning here is that people don't understand what erlang/OTP is.

</comment></comment><comment><USER22> Akka for .NET, [http://getakka.net/](http://getakka.net/)

</comment><comment><USER37> Clojure/JVM: core.async, but moreso pulsar: [https://github.com/puniverse/pulsar](https://github.com/puniverse/pulsar)

<comment><USER3> Not even close. And that your claim seems to be widely believed shows me that people don't understand what erlang/OTP are and what makes them unique (so far.)

<comment><USER38> Maybe you could enumerate some of the things that are unique or impossible to replicate instead of just saying "nope that's not it"

</comment></comment></comment><comment><USER39> pulsar for clojure

</comment><comment><USER40> I have not worked in Erlang and and am not super familiar with OTP, so my answer might be only quasi on-topic.

Java concurrency primitives like ExecutorService and BlockingQueue with support from libraries like Google Guava (ListenableFuture & ListeningExecutorService and, more loosely related, EventBus), as well as libraries like Netty form an excellent basis for concurrent software development in Java. I've developed a number of event processing type applications in Java (including our email delivery system) using message passing styles and future/promise styles and feel like Java does a good job accommodating them, helping them perform well, and allowing them to be easy to operate and troubleshoot. You can make this work all the way down to asynchronous IO at the operating system level if you wish, although in practice we get acceptable performance allowing that lowest level to use blocking IO for simplicity.

It's really easy to set up a "pipeline" of message processors in a Java application that produce and consume from BlockingQueues; or alternatively, submit work to an ExecutorService or publish an event to an EventBus. Maybe not as easy as other languages or frameworks, since there is some scaffolding, but it feels pretty minimal to me. (Again I say this as someone who has not worked in Erlang or other concurrent languages / frameworks extensively.) Unlike some other concurrent code, pipeline-type code is pretty easy to write, understand, and debug. The hardest part is usually orchestrating safe controlled shutdown.

One area where I understand that Java probably does not compete with Erlang is in the reliability of individual processes and threads and whatnot within a machine. There is no simple way in Java, for example, to continue processing requests in the application while you shutdown and restart with a new version. However, we typically accommodate problems like this at the next level up by routing traffic away from a machine in preparation for it to receive a software updated.

Java's exception handling is also robust. There is not much need to worry about "termination" of individual processes. A top level try/catch block in your message processing system goes a really long way to making the app itself immune to any kind of lower level failure. It might be difficult to convey the supreme confidence of error handling in Java if you've only worked with other languages, but a subjective feeling is: "There is _absolutely nothing_ that can go wrong in _any Java code that I might execute_ that will not unwind in a controlled way through my try/catch blocks and give me a nice, clear stack trace and error message." Whatever thread was handling that work can move onto the next message nicely and cleanly. And beyond this, "There is _absolutely no way_ that any Java code I execute can interact with any part of the application except the objects being passed to it."

People credit garbage collection in Java with making applications and libraries much easier to compose. I personally believe that Java's error handling and behavior containment is also a big part of it. A library that I call simply cannot crash me or interact with anything except what it's passed. (OK, there are some exceptions to these rules, such as code that calls into native code, or running out of heap space, or weird dynamic/reflective stuff, but (i) you can avoid most of them in practice (ii) they don't come up much anyway (iii) exceptions to the rules don't tend to be a problem.)

Application crashes like OOM are also problems that we solve in other layers: if they happen, it means the software is not correct or not tuned correctly, and the crash of an application at this level is handled by the routing layer on top. We don't consider application crashes as something that we need to worry about on an ongoing basis; it's more of a QA issue during development. We do not tend to have systems though where "this _particular_ machine must be available at all times".

None of this specifically supports distributed application development along with concurrency, beyond making message passing easy within the app. That's where client libraries like Netty or frameworks like Akka go much further.

<comment><USER10> > There is no simple way in Java, for example, to continue processing requests > in the application while you shutdown and restart with a new version

It's not trivial, but Java certainly supports hot code swapping. How hard it is to use it depends on how much you want it to do, like preserve state etc. For stateless services its not hard to load the new version using a new class loader to live side-by-side with the old one, and then kill the old one (automatically) once all its requests have completed, and let the old code be collected by the GC.

<comment><USER40> It's something you could arrange if you wanted to, but it could be pretty complicated! Swapping individual classes within a VM and having multiple classloaders could be very nuanced to debug. I suspect it would also complicate usage of concurrency and primitives -- for example, there will be a separate EventBus for each classloader, rather than one for the application. The app will need to embrace subtlety like that to be correct. I'd almost be tempted to consider the two classloader roots to be two separate apps in the same process, but they'd need to share the same file descriptors for network communication and logging. Urk.

It's a neat idea though. Do you know of any frameworks that would make the idea easier to implement? It sounds loosely similar to the Unix availability concept of passing off the listening socket to a new instance of the process, while allowing the old processes to linger until they finish serving their requests.

All other things equal, I favor architectural approaches where I can take any machine offline safely and effortlessly since I tend to need to handle that case anyway for availability reasons.

<comment><USER10> > for example, there will be a separate EventBus for each classloader

Not if you leave the message classes alone.

> Do you know of any frameworks that would make the idea easier to implement?

No, just careful design (JRebel is for development rather than production, as I understand, but I could be wrong) Tooting my own horn (again) but Quasar does hot code swapping for actors (including state preservation)[1]. This is possible (as it is in Erlang) because actors encapsulate their own state.

[1]: [http://docs.paralleluniverse.co/quasar/#hot-code- swapping](http://docs.paralleluniverse.co/quasar/#hot-code-swapping)

</comment></comment></comment></comment><comment><USER41> Hugh

</comment><comment><USER42> Go [https://golang.org/](https://golang.org/)

<comment><USER3> Unfortunately, I think you're serious. And you're probably one of many who think this way. Which shows that people really don't understand what it means to do concurrency (as go doesn't support it really.)

Look the erlang syntax is not that hard.

<comment><USER43> Could you explain what Go is missing, then?

<comment><USER44> These posts cover some of that:

[http://blog.erlware.org/some-thoughts-on-go-and- erlang/](http://blog.erlware.org/some-thoughts-on-go-and-erlang/)

[http://vagabond.github.io/rants/2015/06/05/a-year-with- go/](http://vagabond.github.io/rants/2015/06/05/a-year-with-go/)</comment></comment></comment></comment>


DHS: 'Pre-crime' detector shows promise - <AUTHOR> http://www.newscientist.com/blogs/shortsharpscience/2008/09/precrime-detector-is-showing-p.html <comment><USER2> Not sure how any of this will help anything.

Let's think about 9/11. Say a hijacker's mood was detected to be "will cause a disturbance". Well, now they have to prove it. He isn't carrying anything illegal, he has a valid visa, etc., etc.

</comment><comment><USER3> > Some subjects were told to act shifty, be evasive, deceptive and hostile.

How do they know they are not catching on to fake aggression as opposed to real aggression?</comment>


In Solidarity with Jacob Appelbaum and on the Side of Justice for All - <AUTHOR> https://ourresponse.org/ <comment><USER2> Which billionaire is behind this this time?

<comment><USER3> If that's in reference to that statement, I'd say don't explain with billions what can be _very easily_ explained with intellectual integrity.

[https://github.com/Enegnei/JacobAppelbaumLeavesTor/blob/mast...](https://github.com/Enegnei/JacobAppelbaumLeavesTor/blob/master/JacobAppelbaumLeavesTor.md)</comment></comment>


It Was Never Going to Work, So Let's Have Some Tea - <AUTHOR> https://vimeo.com/146524997 <comment><USER2> Highly amusing, if you have 30 mins to spend listening to the life of a sysop or how screwed up some of the startup culture is, this would be a good choice.</comment>


Ask HN: Are tech companies too powerful to get broken up? - <AUTHOR> <comment><AUTHOR> Should they be? or does competition take over and the value gets passed on to the greater economy. or some other alternative.</comment>


Hundreds of surveillance experiments along the U.S.-Mexico border - <AUTHOR> https://github.com/marcdacosta/border-radio-investigation <comment><USER2> Using github for what amounts to reporting from investigative journalism is a fascinating use case.

<comment><USER3> These are some of my favorites!

If anyone has anymore Githubs like this, please link them :)

<comment><USER4> [https://github.com/fivethirtyeight/data](https://github.com/fivethirtyeight/data)

</comment></comment></comment><comment><USER5> With something as important as data at the border being farmed out to so many parties, I can hardly trust information can be kept securely with so many 3rd party relationships, especially with foreign firms.

<comment><USER6> Note that it looks like this technology is being deployed at the border _not_ at border crossings.

What threats are you worried about here?

Privacy of people in the process of illegally immigrating? I don't think the people deploying these experiments share your concern.

Privacy of people who happen to live near the border? It seems unlikely that they are collecting much data on them, just because it's not useful to collect... I can see it being a real concern if they are collecting data on them.

Privacy of troop movements? Are there serious adversaries who are more likely to be aided by this data leaking than they are to be hindered by the increased surveillance? If a real war broke out on the US border I'm sure we would quickly become more selective about the providers, and where the data from the instruments is going.

<comment><USER7> > Privacy of people who happen to live near the border? It seems unlikely that > they are collecting much data on them, just because it's not useful to > collect... I can see it being a real concern if they are collecting data on > them.

I could easily see an agency argue that it needs to collect data on everyone near the border in order to track coyotes and others inside the country assisting with border crossings, etc.

After Snowden, it's hard to envision any agency with surveillance powers opting not to collect data for not being "useful". The justification is definitely there for those who are adamant about collecting the data.

</comment></comment></comment><comment><USER8> Fascinating. The article is talking about surveillance blimps and I can't help but wonder if someone is working on repurposing that tech to detect fires at their onset.

<comment><USER9> Well, there are satellites that have far larger fields of view, so the question is how much smaller fires you can see with a blimp compared to a satellite.

<comment><USER10> How many satelites are devoted to reporting fires on the other hand?

</comment></comment></comment><comment><USER11> [https://github.com/marcdacosta/border-radio- investigation/bl...](https://github.com/marcdacosta/border-radio- investigation/blob/master/jan2020-exports/USMX-Border-Experiment-Cos.csv)

Anduril doesnt even show up, which probably means that it's overstating its progress.

</comment><comment><USER12> A company called Ticom Geomatics has applied for the most experimental licenses.

From their homepage: "we offer challenging projects where the results make a difference globally"

Note: "Make a difference" can be double-speak.

They also mention their "diverse and collaborative environment". I wonder if there are Mexicans on their staff.

<comment><USER13> This is a picture of their staff. It looks reasonably diverse: [http://www.ticom-geo.com/wp-content/uploads/2016/07/tgi- grou...](http://www.ticom-geo.com/wp-content/uploads/2016/07/tgi-group2.jpg)

Remember, legal immigrants (from Mexico or anywhere else) tend to be among the strongest opponents of illegal immigration.

<comment><USER14> I don't understand how you look at that staff picture and say that it's "reasonably diverse".

<comment><USER15> Do you know their backgrounds? Do you know whether they were raised rich or poor? Perhaps some come from the Eastern states, some from the West, others from the south. Some are young and some are old. Maybe some like classical music while others prefer rap. Some like long walks on the beach while others like spending time hiking in the forest.

Lots of diversity is probably contained in that photo, just not the superficial type you're most fixated on.

<comment><USER14> The comment I replied to said that they look diverse. So how do they look diverse based on that picture alone?

<comment><USER15> Based on that picture alone? Some tall, some short, some old, some young, some men, some women, some skinny, some fat, etc. Lots of superficial diversity in that picture.

</comment></comment></comment></comment></comment></comment><comment><USER16> Why are we spending presumably billions of dollars restricting movement of people? It would be by far cheaper to open the border and allow working age migrants to pay into our tax system legally....

<comment><USER16> Follow up to this: First it's tested on the illegal citizens, then it's tested on the legal ones.

<comment><USER17> What's an "illegal citizen"?

<comment><USER16> I think I might need to clarify my point here. I'm not placing any blame on migrant workers who come to the US to make a living, my biggest issue is with companies that take advantage of this labor without taxes. Having an easier system for migrant workers to have legal status here (i.e. able to bank in the US, pay taxes, etc) would raise their income, and benefit the US as a whole.

<comment><USER17> How would a massively increased supply of unskilled labor (which is already abundant) raise anyone's wages? Say, for instance, tomorrow it became really, really easy for companies to hire programmers. Do you think the wages would go up or down?

</comment></comment></comment></comment></comment><comment><USER18> Defense paying for a lot of R&D that will be materially useful in a lot of areas.

Now we just need a ton of legislation to protect us from entities using this tech when they shouldn't.</comment>


New Apps for the Bottom Billion - <AUTHOR> http://www.technologyreview.com/communications/40364/

<comment><USER2> > this form of payment for a person working just two hours a day every day > could equate to about $21 a month—12 percent of the average monthly wage in > the region.

Unless the average person in the region works 2 / .12 = 16.67 hours per day every day (do they?) this is a pretty raw deal.

</comment><comment><USER3> Not long ago this was a big new feature being advertised for new phones. "Look! Picture messaging! Isn't that amazing" and already it's so old it's been forgotten. The world moves so fast!

</comment><comment><USER4> I'm pretty sure the bottom billion still don't own cellphones. But this is a smart idea for the bottom billion of cellphone owners.

<comment> mkr-hn It was 5 billion in 2010: <http://reviews.cnet.com/8301-13970_7-10454065-78.html>

Life for the average person in the bottom billion isn't great, but it's getting better.

edit: [http://en.wikipedia.org/wiki/List_of_countries_by_number_of_...](http://en.wikipedia.org/wiki/List_of_countries_by_number_of_mobile_phones_in_use)</comment></comment>


Adding more taxis in New York City would slow service for everyone - <AUTHOR> http://www.npr.org/blogs/money/2012/07/31/157477611/does-new-york-city-need-more-taxis

<comment><AUTHOR> New York City is considering licensing more taxi medallions. But critics argue more taxis would slow down taxi service for everyone, and of course also increase congestion and travel time for all other vehicles too. Unofficial taxi services like Uber and Lyft discussed recently on HN would have similar effects.</comment>


Opal: Ruby runtime and library on top of Javascript - <AUTHOR> https://github.com/adambeynon/opal

<comment><USER2> I have been using a similar project (Red) for a year in production.

Here is why we wanted to use it:

\- Development speed (ruby can be typed so much faster and has a class system)

\- Packaging system (we want one minified js-file)

\- Code reuse (we use Ruby in the backend)

Here is why we stopped using it in favour of native JS:

\- Slow runtime (try to read some big JSON in your favourite js-ruby, we had time diffs of 20secs to 0.2secs native js speed)

\- Debugging (this is the most annoying thing, which you will not realize in the beginning)

\- Development speed (recompiling time increases the more accurate the ruby implementation gets - my fork actually used multiple cpu cores)

\- Red was immature (had my own totally rewritten fork going...)

I still love projects like this, but the mentioned problems will not all go away any time soon (a few of them might).

By the way, CoffeeScript shares some of these downsides, but is far less of a pain, because the whole language was designed with JS in mind (fast compilation, fast runtime etc.).

<comment><USER3> Actually, as stated on Coffescript's wiki[1], there are many other implementations of ruby on JS:

        * HotRuby Runs opcode, compiled by YARV on Ruby inside a web browser or in Flash.

        * rb2js Converts Ruby to JavaScript.

        * Red Writes like Ruby and runs like JavaScript

        * Quby Used for game coding site, not open source.

        * Opal Ruby to Javascript compiler.

It seems interesting that nobody has tried to use Rubinius (which tries to implement _Ruby in Ruby_ ) to leverage any of these implementations, possibly reusing a JIT mechanism.

[1] [https://github.com/jashkenas/coffee-script/wiki/List-of- lang...](https://github.com/jashkenas/coffee-script/wiki/List-of-languages- that-compile-to-JS)

</comment></comment><comment><USER4> Looks like a cool project, but it would be nice if the docs were a little more complete or there was a try it out feature like on <http://jashkenas.github.com/coffee-script/>

<comment><USER5> Docs are definitely being worked on, kinda the top priority now that the language and runtime are pretty stable.

</comment></comment><comment><USER6> What does it mean when something is 'on top of' another thing in computing?

<comment><USER7> In cases like this it means that the language was built to compile down into the other language. So you can write ruby code andnthat spits out JavaScript for thte browser to read.

Same thing with coffee script / objective-j

</comment></comment><comment><USER8> I can't tell what .rb is intended to run on the ruby interpreter, what .rb is intended to run with Opal, and what .rb is intended to run on both.

<comment><USER9> Opal seems to be compiling the rb file to a js file. This js along with the supplied opal.js will run in the browser.

Check out [https://github.com/adambeynon/opal/tree/master/examples/brow...](https://github.com/adambeynon/opal/tree/master/examples/browser)

</comment></comment><comment><USER10> is there an associated product or project that this was developed along with? I am curious as to how this is being used

<comment><USER5> Im using it for an internal project. There is an associated `vienna` gem, that is for building rich web apps (think sproutcore, cappuccino). It is not open source yet, but will be once the project its been written for is finished.

</comment></comment><comment><USER11> Thought this was gonna be this: <http://www.opalvoip.org/>

<comment><USER12> Thought this was gonna be this: <http://en.wikipedia.org/wiki/Opal_(programming_language)>

</comment></comment><comment><USER13> Interesting that it has partial node.js compatibility</comment>