Simple recommendation system written in Ruby - <AUTHOR> http://otobrglez.opalab.com/ruby/2014/03/23/simple-ruby-recommendation-system.html Nothing fancy, just simple tag/words based recommendation algorithm implemented in Ruby. <comment><USER2> Programming Collective Intelligence is an excellent book for learning these sort of things. First chapter is a recommendation engine! :)

<comment><USER3> I second this. And if you decide you want to go really in-depth on recommender systems, I suggest taking a look at "Recommender Systems Handbook" ([http://www.springer.com/computer/ai/book/978-0-387-85819-7](http://www.springer.com/computer/ai/book/978-0-387-85819-7)). It's basically a collection of scholarly articles on on the topic, so the approach is academic. But it's also the best resource I'm aware of for understanding what's state-of-the art across a range of aspects of recommender systems. (Also, although the price is a somewhat hair-raising $179, there are PDF copies of the whole thing floating around that are easy to find with a google search.)

</comment></comment><comment><USER4> I did something very similar in the past except I used Cosine Similarity ([http://en.wikipedia.org/wiki/Cosine_similarity](http://en.wikipedia.org/wiki/Cosine_similarity)). It allowed me to give each tag a "weight" and when comparing the tag clouds, I would zero out any that aren't found. It works really really well.

</comment><comment><USER5> Good stuff, and a nice writeup/explanation!

To impudently hijack the thread: for a very similar approach (jaccard similarity coefficient, ruby) which has a nice abstracted implementation for background workers, take a look at David Celis 'recommendable' \- here's him introducing the same system: [http://davidcel.is/blog/2012/02/07/collaborative- filtering-w...](http://davidcel.is/blog/2012/02/07/collaborative-filtering- with-likes-and-dislikes/) and the gem itself: [http://davidcel.is/recommendable/](http://davidcel.is/recommendable/) I believe it's been discussed on HN before.

Redis is used to store the binary votes, and to compute similarity coefficients. Since redis is very good with set operations (intersections on multi-million-member sets (and more) are crazy fast), it's quite the natural choice for the db backend. One of the cases where a NoSQL solution seems to be the right tool for the job, as a matter of fact!

I've used recommendable (incl. in production code) in the past, it works very well, is reliable, robust, and easily hackable for whatever needs. (e.g. it's meant to integrate with Rails, but it's quite simple to make it work on barebones ruby, with (e.g.) Sinatra as a lightweight web app exposing vote functionality, and so on.)

<comment><AUTHOR> Thanks for the reference. David's post looks awesome!

</comment></comment><comment><USER6> You may use Levenshtein Distance to get better results by taking word variations into account.

And also you can enhance it by using semantic similarity scores for strings.

<comment><USER3> If you're willing to get into actual NLP, then semantic similarity would certainly be one way to go. Is there any equivalent to Stanford (Java) or NLTK (Python) in Ruby land? But I'm not sure that Levenshtein will necessarily get you better results than the bag-of-words approach the author is taking with Jaccard distance, if all you're doing is document classification.

<comment><USER7> As far as NLP libraries in Ruby land, there is both [treat]([https://github.com/louismullie/treat](https://github.com/louismullie/treat)) and [ruby bindings to the Stanford Core NLP]([https://github.com/louismullie/stanford-core- nlp](https://github.com/louismullie/stanford-core-nlp)).

<comment><AUTHOR> I've used OpenNLP with jRuby for my NLP experiment. Check it out [https://github.com/otobrglez/politiki- ner](https://github.com/otobrglez/politiki-ner) to get an idea how to mix it.

</comment></comment></comment></comment><comment><USER8> Here is a cool approach to the subject by Linked In. [http://engineering.linkedin.com/open-source/cleo-open- source...](http://engineering.linkedin.com/open-source/cleo-open-source- technology-behind-linkedins-typeahead-search)

Here is my HN-obligatory, self-written golang version: [https://github.com/jamra/gocleo](https://github.com/jamra/gocleo)

I went with this author's approach to use Jaccard to rank the results, however, I like this approach better: [https://neil.fraser.name/writing/patch/](https://neil.fraser.name/writing/patch/) They basically take the distance to the beginning of the text into account.

</comment><comment><USER9> I recommend reading [http://nlp.stanford.edu/IR- book/](http://nlp.stanford.edu/IR-book/) on this topic.

</comment><comment><USER10> if you want something that scales better use minhash. You get a similarity that is approx jaccard but with a lower footprint memory and cpu wise.

</comment><comment><USER11> How well does this perform compared to your postgreSQL example?

Also - We are looking for a Ruby / Backbone.js developer drop me an email firstname.lastname@example.com :)

<comment><AUTHOR> Thanks for you comment Josh. In production I actually implemented PG version and it's still in use today. I thought it will scale better and last longer. I didn't do benchmarking or anything like that - it think it would really depend on size of you dataset. For something serious - compute recommendation in background and store them in database. I believe that is how "big boys" do it. :)

</comment></comment><comment><USER12> for an approach using neo4j, check out cadet! (my project) cadet is more just a jruby wrapper around neo4j, but one can use it to interact with neo4j (and thus come up with recommendations without touching a line of java, or even cypher )

still in progress, and id love any input! [http://github.com/karabijavad/cadet](http://github.com/karabijavad/cadet)

[http://github.com/karabijavad/congress- graph](http://github.com/karabijavad/congress-graph)

</comment><comment><USER13> I remember seeing a nice little system that involved taking the square root of something, but I can't remember what it was. Anybody know it?</comment>


New React DevTools Beta - <AUTHOR> http://facebook.github.io/react/blog/2015/08/03/new-react-devtools-beta.html

<comment><USER2> I am super stoked they now have a Firefox compatible version. The React team have been working very hard and it shows. I've been working with React for about a year now, building an application using React Native and the libraries and tools are absolutely solid, something other libraries and frameworks are definitely missing.

</comment><comment><USER3> Gotta admit, one of the things the React team is handing better than Angular is UX, usability, and general newbie-adoption. This looks very polished and useful.

</comment><comment><USER4> This is really nice, and it's great to see a non-Chrome version, too (working great in Firefox so far). Chrome often gets all the love (and is a great choice) for new dev tool extensions, but many of our clients -- for one reason or another -- can't/won't use Google's browser on intranet-based solutions. It's nice to be able to test/debug on more than one browser (Some of us still have to party like it's 1999).

</comment><comment><USER5> Does this version support a better debug view of Immutable.js props/state?

<comment><USER6> We'd like to add optional support for this in the future. It should be much easier to do with the rewritten architecture.

<comment><USER7> Awesome. Any idea what sort of timeline we're looking at with this? If weeks, I'll wait. If we're thinking months, I'll look at patching that in myself.

</comment></comment></comment><comment><USER7> If anyone is wondering how to get it working, unpack it into a folder with your favourite unpacking thing, enable developer mode in chrome://extensions, and use Load unpacked extension to install.

Should fix the "not a chrome webstore whatever nonsense" notice you get.

</comment><comment><USER8> > The Side Pane.. > Right-click to store a prop/state value as a global > variable

I could see this being pretty useful. Can't wait to try it out :)

</comment><comment><USER9> Chrome seems to uninstall the extension on boot 'cause it's not on the web store.

</comment><comment><USER10> Both github.io and github.com are down here, least in Michigan and that's a first for me.</comment>


YC: Canadian experiences with TN status for startup? - <AUTHOR>

Can anyone comment on experiences with obtaining TN status to work on their startups?<p>Did you put your own new company as the sponsoring entity?  Any suggestions of lawyers to work with? <comment><USER2> Do you require TN status? It is relatively easy to get, however I understand you can only receive it once. Getting legal status after that year is considerably harder, so be careful when you use it!

Offtopic: Why do you need to leave Canada to start it?

There are a few good entrepreneurial hubs in Canada, like Ottawa for telecom and Waterloo for Engineering/Software.

<comment><AUTHOR> It's actually possible to renew TN status indefinitely, as long as one maintains temporary intent. I've been living in Boston a few years, working on a TN for a slightly larger company, and haven't had problems at the border.

However, the situation potentially gets a little trickier when a start-up is involved, as one is not supposed to be self-supporting - you need to have a paying job. However, I believe that in principle, there isn't any reason that one can't hold an equity stake in a company that employs you.

I am already in Boston and working with a co-founder here, which is the biggest reason I need to be in the US. Moreover, the access to smart capital and to our target market is significantly greater here (Boston, SF, and NYC). Personally, I have found the opportunities for inspiration, advice, and serendipity are much richer in Cambridge than they were in Toronto. That isn't to say it can't be done in Canada, but at this stage, my bets are best in the US.</comment></comment>


Is Unicode Safe? - <AUTHOR> http://www.jefftk.com/news/2013-06-14

<comment><USER2> Another post along the same lines, this one from Microsoft detailing the apparent presence of two hosts files in the same directory:

"Can we believe our eyes?" [http://blogs.technet.com/b/mmpc/archive/2011/08/10/can-we- be...](http://blogs.technet.com/b/mmpc/archive/2011/08/10/can-we-believe-our- eyes.aspx)</comment>


Show HN: Llama Logs – Visualize Your Distributed System Through Logging - <AUTHOR> https://llamalogs.com/docs/what <comment><AUTHOR> Hello everyone, this is a solo project that I have been working on during quarantine. The link above has a good overview of how the system works.

It is ready for user accounts now if you want to start visualizing your system. But there is also a demo that does not require signing up.

</comment><comment><USER2> Does this use OpenTelemetry/OpenTracing if not, why not?

<comment><AUTHOR> I have not heard of that project before, so that is the main reason. It looks very cool!

From some brief review of their docs it looks like the implementation requires an agent on the host. Llama Logs does not, making it as simple as possible to get started.</comment></comment>


Visual Studio Code "Insiders" release now supports tabs - <AUTHOR> https://github.com/Microsoft/vscode-docs/blob/vnext/release-notes/June_2016.md#tabs <comment><USER2> Though it's taken some getting used to, I now actually prefer the open files list on the left... Though, wouldn't mind if it included open diff panels in the list. The saved screen estate is especially appreciated, tabs always gets bad after you've got a half dozen or so, and the ux gets weird too.

Been a pretty big fan of VS code, as it's taken some pragmatic approaches that have worked very well.

<comment><USER3> I heard on a podcast that the VS code guys were anti tabs as it usually quickly ends up in that pile of tab clutter with tab labels like "Be.."

I've also got used to this new method and think it is generally better.

Glad to see they are listening to their users though! :)

I hope tabs supports splits - sometimes I have a three way split where each file I'm working on is in the 'right' place. Then I need to look at a few files for reference and I usually wreck my layout. Tabs could be a good solution for this.

<comment><USER2> It looks like vertical splitting will work in addition to tabs on each split... I wouldn't mind seeing VS-style horizontal split within a file... Although I try to keep my files under 100 or so lines, I often have to work with existing code that's far larger in a single file.</comment></comment></comment>


Ask HN: Would it be feasible to install dark fiber while building a home? - <AUTHOR>

With Google rolling out 1 Gbps connections to Kansas City, SSDs easily saturating 1 Gbps LANs (~500 MB/s vs 125 MB/s), and 4k/8k video on the way would it be prudent to outfit a new home with a bunch of dark fiber to all of the rooms? Equipment for 10 GbE is still pricey but falling fast. You could always wait for the non-fiber equipment cost to go down before you switch from, say, using CAT6/1 GbE. Any idea what the dark fiber alone would cost? I would assume that multi-mode would be sufficient but would single-mode fiber offer better upgrade paths? <comment><USER2> Conventional wisdom would be to install conduit in the walls and leave installing the actual fiber until you are planning on using it.

</comment><comment><USER3> In my opinion, Nope. Speed-wise, for short distances like in the home, copper kicks fiber's butt (when you talk price/speed). Spend the extra money on Cat6e, and running multiple runs (as many as possible) to each room/jack - this leaves you with the possibility of link aggregation in the future (you can get switches capable of link aggregation cheap on ebay).

</comment><comment><USER4> Not sure it's worth it. I was just talking to someone who spent good time and money wiring their new house with CAT5 10 years ago and now laughs about it. Everything is going wireless.</comment>


IBM's supercomputer will power an online, anime VR game - <AUTHOR> http://www.engadget.com/2016/02/23/ibm-watson-sword-art-online-vrmmo/ <comment><USER2> truly, this is a golden age we live in</comment>


Ask HN: JavaScript framework that plays nice with bootstrap and jquery - <AUTHOR> I'm currently developing a typical CRUD app with bootstrap and various jquery libraries (datatables, select2...).  My backend is ASP.Net MVC, which I love, and my frontend is pure jquery, which i don't.  I'm using the revealing module pattern to organise the jquery and i'm following best practices, so the code is fairly clean.  The only issue i'm having is maintaining a list of cached selectors and binding them all, creating lots of boilerplate, which is messy to modify.
I've worked with angular and i really like the data-driven approach, though angular doesn't play nice with jquery and you need another library on top of it to work with bootstrap, which is apparently feature incomplete.  I've looked at vuejs, and because of the virtual DOM, it doesn't go well with libraries that manipulate the real DOM directly.  The solution to this is to get modified vue-friendly versions of libraries, which is a road im not keen to go down.  React seems like it has similar problems.
Is there any js framework that plays nice with bootstrap and jquery and would be better to work with than pure jquery? <comment><USER2> I have had success combining old jQuery based components with a Mithril based UI (Mithril is similar to Vue, but simpler). But this only worked because I defined clear boundaries between areas managed by Mithril and areas managed by jQuery. Nesting such areas was possible. It was possible to make them both notify the other when they were about to remove a nested area.

Deleting an element created by the other framework however is never a good idea. They can manage their own event handlers and metadata only themselves. Instead, you need to ask the other framework to detach itself. You can use jQuery's special events to detect elements about to be removed from the DOM [0]. Mithril lets you define an onremove lifecycle callback on any component. From there, it is cakewalk.

[0] [https://stackoverflow.com/a/10172676/2492826](https://stackoverflow.com/a/10172676/2492826)

</comment><comment><USER3> I'm looking at Aurelia and really think it has potential</comment>


Web Programming in SWI Prolog (2019) - <AUTHOR> https://www.monolune.com/web-programming-in-swi-prolog/ <comment><USER2> Also check out Anne Ogborn's tutorial "Creating Web Applications in SWI- Prolog" [http://www.pathwayslms.com/swipltuts/html/](http://www.pathwayslms.com/swipltuts/html/)

Following that I made a very simple ("hello world" simple) server that serves Tau Prolog ( [http://tau-prolog.org/](http://tau-prolog.org/) ) to the client and establishes a websocket connection between them.

It's here FWIW: [https://git.sr.ht/~sforman/yrad- nettles](https://git.sr.ht/~sforman/yrad-nettles) (WARNING: it's not meant for public consumption, I think it might be broken at the moment because I was experimenting with serving prolog scripts like js scripts when I stopped working on it... I'm putting it out here in case you want to try to glean some insight from it. I was looking over it myself just now and I forget a lot about how it's supposed to work, so read it alongside Anne's tutorial, if at all. Cheers! Also, I should mention it depends on a whole different project, an implementation of Joy programming language, to actually do anything, because it's meant to act as a remote shell/UI for that language hosted in a Prolog server. So, yeah... you're gonna want to comment that out.)

</comment><comment><USER3> While web programming in Prolog has been insightful, I think there are more expedient areas of application for the language.

Prolog is especially underutilized in databases.

<comment><USER4> This is a database written in Prolog: [https://github.com/terminusdb/terminus- server](https://github.com/terminusdb/terminus-server) \- albeit with a RUST storage engine - prolog's lack of types means there are better storage solutions, but for constraint logic, it's unbeatable.

<comment><USER3> A statically typed logic language would be ideal for both

<comment><USER5> [https://mercurylang.org/](https://mercurylang.org/)

<comment><USER6> Mercury is cool. I translated a simple Prolog program to it once, and it was a head trip. Hard to get my mind around, but also helped me see the connection between functions and relations.

</comment></comment></comment></comment></comment><comment><USER7> Learning web Prolog is my current programming side project. My ultimate goal is to create an online board gaming platform, as writing board game rules in Prolog has turned out to be a smooth and pleasant experience :)

Not to mention, SWI Prolog already has http, websockets, rpcs, and pubsub built in, so there isn't really a need to download any external libraries.

</comment><comment><USER8> What could be good uses of a prolog-based web service? Calendar availability? Scheduling appointments? Issue diagnosis?

<comment><USER9> It's a turing complete language... so anything you want!</comment></comment>


Hash Tables Aren't a Magic Bullet - <AUTHOR> http://notjustburritos.tumblr.com/post/21221997397

<comment><USER2> I agree that hash tables are not a magic bullet, but you're presenting them as if they're a poor option in general, which is not the case.

> _Instead, they try to cheat by "knowing something" about the expected data. > For instance, they may only run their hash function on the first six > characters of a string instead of the whole thing. But what happens if I am > attempting to insert ten-thousand lines that all begin with "ROBERT:"? O(n) > look-up time. That's what happens._

> _The above example is a bit contrived, but the sentiment is valid._

No, it's not. You either need to provide a sane hash function or you need to use keys that are sane in combination with an existing hash function. If your keys are massive, then the most likely case is that you're being lazy and using an inappropriate object as a key because it's convenvient. If you can't or won't choose a reasonable key for a hash table, how likely are you to pick a reasonable key for a BST-based dictionary (or any other dictionary)?

> _Hash tables waste space proportional to the amount of data you want to > store. Hash tables rely on a lack of collisions. In a rosy situation where > your hash function maps elements randomly (but deterministically of course) > into your hash table, you still need to keep plenty of empty space lying > around to avoid collisions._

A significantly-overloaded hashtable (with chaining) can still be faster than alternative data structures in some cases. Collisions in a hash table are not really a problem unless they are approaching O(N). If you shove O(N x M) objects into an O(N)-sized hash table, you'll need to churn through O(M) entries on every lookup (amortized). Assuming a reasonable M and an efficient comparison operation, that can be a pretty efficient approach.

> _So where does that leave us? Hash tables require very large data sets to be > useful, yet they waste a lot of space for large data sets and aren't great > at storing large elements._

Neither of these claims is really true. Hash tables are not always the best data structure (in fact, they often are not), but moderately-sized data sets or large objects don't rule out hash tables as a good choice. (Small data sets don't either. If you've only got a dozen elements, it probably doesn't really matter how you store them.)

> _The only time hash tables seem useful is when we know something about the > data ahead of time, so that we can vastly improve our hash function._

I should hope you know something about your data, or you are not in a position to choose a good data structure.

> _But if we know that a certain part of the data is most likely to > differentiate it, why not simply use that portion of the data as the key for > the binary search?_

Because a BST isn't a magic bullet, either?

</comment><comment><AUTHOR> I wonder how conflated the ideas are a "dictionary" versus a "hash table" are in a lot of developer minds (a dictionary is an abstract data type that makes no guarantees about how it's implemented, while a hash table is a specific way to implement a dictionary).

For a savvy developer, is it worth it to know the differences? Do you know how a dictionary is implemented in your language of choice?

</comment><comment><USER3> "Hash tables aren't a magic bullet data type. They have plenty of issues, they're not really constant time, and in most cases there are other data structures that are much more appropriate."

Be sure to not tell us anything about said data structures though, we ain't too keen on knowing stuff anyway.</comment>


Using Haveibeenpwned (HIBP) in a Corporate Setting? - who-knows95 Hello there, thank you for reading this,
i work in a small to median sized IT support business.
i'm the cyber sec junior, and i have been wondering about if i can implement HIBP in a corporate setting.
i currently use it to educate staff members on how passwords are breached, and why emails need to be secure, and i know i can use it to scan a domain and find emails linked to breaches and that i can use it as a blacklist for passwords.
is there anything else i should think about or look into?
thank you. <comment><AUTHOR> I have a script running weekly that dumps the NTLM hashes from our domain controller and compares them against the HIBP hash list. It will then automatically force a password reset on the user's account if found in the compromised hash list.

You can also subscribe your domain to get email alerts when one of your email addresses pops up in a new breach.

<comment> who-knows95 AHH that is a brilliant idea, do you have this script posted? i guess, if i added the password databases to a blacklist it would do the same thing?

i did see that, and i am tempted to test it on some of our clients to see what the feedback is. i know when i run a few of their emails through they do pop up in some of the breaches.

kind regards

</comment></comment><comment><USER2> You can find the password dumps if you are persistent enough with google and crack the hashes yourself. You can also look into hashes.org. I'd just enforce strong password rules all around in your domain and for the applications you build.

<comment> who-knows95 you can download the full password database on the HIBP website on the passwords page!

[https://haveibeenpwned.com/Passwords](https://haveibeenpwned.com/Passwords)

we do enforce password policies that match best practice, just wondering if there is more i could do.</comment></comment>


MAGI Synthavision Demo Reel (1980) [video] - <AUTHOR> https://www.youtube.com/watch?v=lAYaX6NuI4M

<comment><AUTHOR> Some context, from [https://en.wikipedia.org/wiki/Mathematical_Applications_Grou...](https://en.wikipedia.org/wiki/Mathematical_Applications_Group):

> Mathematical Applications Group, Inc. (a.k.a. MAGI or MAGI/Synthavision) was > an early computer technology company founded in 1966 by Dr. Philip Mittelman > and located in Elmsford, New York, where it was evaluating nuclear radiation > exposure. In 1972, the graphics group called MAGI/SynthaVision was formed at > MAGI by Robert Goldstein.

> It was one of four companies hired to create the CGI animation for the film > Tron. MAGI was responsible for the most of the CGI animation in the first > half of Tron, while Triple-I work mainly on the second half of the film. > MAGI modeled and animated the light cycles, recognizers and tanks.</comment>


HTC sides with consumers: No more locked bootloaders in Android phones - <AUTHOR> https://www.facebook.com/HTC/posts/10150307320018084 <comment><USER2> This is exactly what I've been talking about -- customers want unlocked phones, and because with Android competition is tough, sooner or later unlocked phones will be the norm. Maybe not just yet, as I don't believe HTC or Motorola until I see it, but soon.

I just bought a Galaxy because SG phones can be easily unlocked. That's a lost sale for Motorola - as my first option was a Defy (those phones are cheap and solid and I tend to mistreat my phones, so Defy seemed like a good option, until I found out that unlocking them is a bitch)

<comment><USER3> +1 for unlocked phones being the norm. This has been the case in asia for years now. It's about time we catch up.

<comment><USER4> Carrier lockdown only happens when you buy a phone under contract. You can buy unlocked phones, easy.

 _But they are so expensive!!!!_

Well, yeah, that's because you're paying the real price, not a subsidized price. When you're paying the subsidized price, IMHO you have less leeway to complain about carrier lockdown.

<comment><USER2> Well, just for the record, I bought a heavily subsidized Galaxy S sim-unlocked with a 2 year contract (the cost was something like $30). Gave it to my wife, then bought myself a SG II. And I'm happily paying for my contract btw.

The carrier I'm talking about is a smaller one from Europe. It's unthinkable to ask this of Vodafone or Orange (the 2 big ones) since their heads are really up their buts. And the unlocking prices they charge are freakin` insane! It's not as if carriers can't do this, they just don't want to.

I really don't understand what's their problem. If I'm terminating my contract I have to pay the phone's subsidized price for the months remaining, their only loss is me as their customer, so WTF is the deal with sim-locked phones? I hate it when companies, instead of pleasing customers, are innovating in better lock-in methods.

But yes, I was talking about bootloader-unlock in my previous message. Really, Nexus S is the best Android sold right now, unfortunately it was only available from Vodafone and I'll never make a contract with Vodafone again.

<comment><USER5> I suspect part of the subsidy is the expectation that you'll make some calls/data/texts which they miss out on if you switch networks.

<comment><USER2> I know that's what they say, but that's bullshit and you know it. If they don't want me to leave, then they should keep me happy.

<comment><USER5> It's a company they are going to do what helps their bottom line the most, if keeping people happy helps they will do that, if not they won't.

</comment></comment></comment></comment></comment></comment></comment><comment><USER6> _"There are four major stakeholders in the Android smartphone market: the cell carriers, the consumers, the handset makers, and Google. The carriers want smartphones to be locked-down carrier-controlled devices through which customers obediently buy services on carrier terms. All the other stakeholders, by contrast, gain from unlocked devices. Consumers win because they're not limited to what the carriers choose to provide them; handset makers win because an unlocked phone is more valuable to a consumer than a locked one (and a little less expensive to ship, too); Google wins because its long-term strategy requires it to commoditize the carriers into a passive channel between the customers and Google."_

More at <http://esr.ibiblio.org/?p=3285>

<comment><USER7> where is the f __king downvote button? ESR-hole knows nearly nothing about the smartphone market.

And until last week, he didn't know that the iPod Touch would run apps.

<comment><USER8> We value polite conversation here.

</comment></comment></comment><comment><USER9> I'm glad they're doing this, but it's sad that it's news. We ought to be able to take for granted the freedom to run whatever software we want on the computers we buy. (Don't tell me a cellphone running Android isn't a computer. You know better.)

<comment><USER10> You know it's a computer, and I know it's a computer, but to the "average person," it's an appliance. It's kind of like the electric motor. Someone (sorry, I don't recall who) recently posted an analogy here on HN between electric motors and any other technology.

At first, factories would install a single giant motor with belts running everywhere, and Sears sold a general-purpose home motor. Eventually, motors were miniaturized and specialized into all kinds of devices. Now nobody thinks about using a motor; they think about using a lathe, blender, or sewing machine.

The same trend is happening with computers. Initially, people had generic computers (ignoring the anomaly of the standalone word processor). Increasingly powerful computers are gradually getting built into everything. You and I see computers everywhere, but everyone else just sees smarter devices.

<comment><USER11> Motors cannot control computers. Computers can control motors.

Motors do not require an operating system.

There is something else that bothers me with this analogy...

The computer (as the 'average person' uses them) is a synergistic experience between both the hardware AND the software.

Without the software, it would be useless. Without the hardware, the software would be useless.

For the motor analogy, I assume the motor is the hardware, and the belts are the OS?

Motive power and computer power are 2 different things.

<comment><USER10> Yes, I wanted to point out the critical difference that software makes, but my post was already getting long enough. The average person doesn't (yet) seem to understand that difference though.

</comment></comment></comment></comment><comment><USER12> More reasons HTC is my favorite phone manufacturer right now. After a bike accident left my Nexus One screen broken, $100 and a 4-day turnaround got me back on my feet. Thrilled. Maybe they'll drop the Sense UI from the stock phones. One can dream.

<comment><USER13> I doubt they'll drop Sense. Many people like it (as you can see by looking at the demand for custom ROMs with it).

<comment><USER14> As long as the bootloaders are unlocked, who cares what the default software on there is? Just load up CM7 and you're good to go.

</comment></comment></comment><comment><USER15> Now that HTC has committed to removing locked bootloaders, it's only a matter if time before all other android handset makers have to follow suit. This is a very good thing for the cellphone industry, and tech in general.

</comment><comment><USER16> One of the things that I find really cool about this is this happened without using a copyright license (i.e. GPLv3) as a bludgeon to force HTC to do the right thing. This was done using simple market forces, and not the anti-Tivo clause of the GPLv3. Which has been the Linux kernel developers' objection all along to the GPLv3....

</comment><comment><USER17> This is great news. The HTC CEO made a pretty quick and sudden decision.

Meanwhile, Motorola vaguely semi-committed months ago and nothing has happened.

I don't really buy that carriers are demanding this as Samsung currently doesn't do this.

<comment><USER18> I remember Paul Obrien tweeting that the GSII was really open, seemed to actively encourage modding. Twitter search is horibble so I can't find the tweet now though.

<comment><USER19> Just so you know, twitter's search only goes back a limited amount of time because its targeted at real-time results, not historical. Google with site:twitter.com might get better results.

</comment></comment></comment><comment><USER20> Well, now I know who I'm buying my next phone from.

<comment><USER21> A million times this. But really, I probably wouldn't even consider it if the phones didn't have extra programs on them that ran pointlessly and/or I couldn't uninstall.

<comment><USER22> well the whole point of having an unlocked bootloader is so it makes it easier to run custom roms. Most roms are aosp or stock that remove all the crap.

<comment><USER21> Oh I get it. I just mean that, if it didn't have that stuff, I wouldn't be too concerned with running a custom rom, and the bootloader wouldn't be as big an issue.

</comment></comment></comment></comment><comment><USER23> Why was this announced via a post on Facebook? Don't they have a corporate blog or something?

<comment><USER24> HTC is a Chinese company, so even if there is a corporate blog, it's not directly hitting English speaking users. HTC probably saw this as its most direct path to that audience.

<comment><USER25> Taiwanese, to be precise.

</comment></comment></comment><comment><USER26> I once had an HTC HD2, and while WM6.5 wasn't very good, the xda-developers community _loves_ this device. I was able to try out all sorts of Android builds on this device, as well as getting WP7 sort-of working on it. Truly amazing.

My only gripe was that it was built with TMO in mind, and didn't have the right radio to talk 3G on AT&T. The battery didn't last too long either, but it was user-replaceable, so not that big of a deal.

<comment><USER27> The HD2 came out 18 months ago, and is still near par to current hardware (1GHz, WVGA, etc.). I've been testing these XDA Android releases: <http://forum.xda-developers.com/showthread.php?t=957652> ; I get standby battery drain of 1% / hour, which is about the same as the original WM6.5.

</comment></comment><comment><USER24> Unlocked bootloader is awesome. I would have settled for just being able to turn off Sense at this point.

</comment><comment><USER28> The unlocked and unbranded HTC Android phones that I bought so far did not have any locked bootloader. E.g., on my HTC Wildfire there was some kind of engineering bootloader preinstalled.

Carrier-branded HTC phones are locked down, but IMO that's OK and I have no problem with that as long as I have the choice to buy and use an unbranded phone.

</comment><comment><USER29> Too late for me - I bought a Samsung Galaxy S2 instead of an HTC Sensation precisely because of the locked bootloader.

</comment><comment><USER30> That's great. HTC is going to be shipping unlocked bootloaders, SE has committed to allowing users to request an unlock code for all new/current devices and the Samsung Galaxy S 2 comes unlocked. Motorola probably won't be able to resist the pressure.

Seems this will be standard for android devices very soon.

</comment><comment><USER31> I just hope they will allow the Sensation to be unlocked -- or release a SKU that comes with the ability to unlock it.

</comment><comment><USER32> Perhaps it's time to upgrade my HTC Hero (rooted, of course)

<comment><USER33> Hero user here: You mean that particular phone that they threw on the market and neglected ever since, right?

I do like these steps in the right direction, but HTC won't win me back for a long while after receiving my (pre-ordered, one of the first in DE) Hero and waiting ages to receive any updates/support on it afterwards. It was more or less dead on arrival and really only usable because of the fine folks over at VillainRom.

</comment></comment><comment><USER34> Wow, it's about time!!

</comment><comment><AUTHOR> After much consumer push back against locked bootloaders in new devices, HTC has (apparently) decided to discontinue the policy. This arrives just after Motorola makes some unsubstantiated comment about reconsidering the policy.

For some history, the Droid 1 was very open and easy to flash with custom ROMs. It's European brother, the Milestone, featured on-chip security that verified the signature of the bootloader which in turn verified the signature of various boot files including the linux kernel at the heart of Android. Subsequent Motorola devices in the US featured similar security and thus do not have official CM7 builds.

HTC recently adopted a similar strategy with respect to some of their more recent phones. The Thunderbolt had a buggy implementation though, and was still hacked quite quickly. The Evo 3d (And another? maybe?) featured a fixed version that more closely represented the impenetrable Motorola style bootloader.

Anyway, it appears now, that they maybe embracing the community more, which makes me happy. My mom won't stop talking about how fast her Droid OG is. I put CM7 on it which includes Gingerbread... a treat her phone would have never otherwise seen. Meaning it also includes a fix for an awful race condition that can make it impossible to answer phone calls. Good stuff all around.

Also, for context, this is the statement Motorola made a month ago: [http://ausdroid.net/2011/04/27/confirmation-motorola-to- unlo...](http://ausdroid.net/2011/04/27/confirmation-motorola-to-unlock- bootloaders-for-future-android-devices/) They make very specific caveats for "where carriers allow it".

<comment><USER35> I'm a Milestone user and long-term customer of Motorola... I can't recommend against them enough. Their problems extend far beyond locked bootloaders, far more than following HTC on this issue would be required for Motorola to be worth dealing with.</comment></comment>


How we built a web controlled robot with SSL tunnels and perl - <AUTHOR> http://cam.ly/blog/2010/10/how-the-pinata-buster-works/

<comment><USER2> I find the "and perl" part particularly dubious. Here's an excerpt:

      system("echo '$mydata' > /home/rhett/usernick");
      system('/home/rhett/music_player.rb &');
      system('/home/rhett/servos/phidgets-examples/AdvancedServo-simple &');

As a former perl hacker, this kind of stuff was infuriating to me before, but it just seems silly now.

<comment><USER3> Hi, I helped to make this. It actually uses a smattering of different technologies. Namely:

Perl, Ruby, C, Javascript, Apache, SSL tunnels, an electric wheelchair, and hobby servos from phidgets.

We didn't mean to be misleading to the perl lovers, it was just too long to post all of that in the title.

Not more than a few lines of each, and it truly is more of a hack than anything else.

BTW, we just changed the 1 line of system("echo...") to have perl print the file to avoid any potential security vulnerability.

</comment></comment><comment><USER4>
      Serving live video over the internet

      The Cam.ly security camera and interface system is super
      simple to set up. Plug power into the camera and connect
      it to the internet. Point and shoot. We slightly changed
      the interface for the pinata buster. We won't talk more
      about this part because how it works exactly is a trade
      secret.

It's seriously disappointing that they refuse to talk about the streaming code. Last year I tried to build a web-enabled robot. I never found a way to stream the live video from the web-cam to a flash widget. There were lots of online services and commercial products, but no open-source stack.

It looks like Cam.ly will become unusable when the manufacturer's online web service goes down.

<comment><AUTHOR> We're actually working on making the cameras run a rtmp server locally, so soon you'll be able to stream it to your own flash widget. The Cam.ly is also a completely open source product. We encourage hacking: [http://cam.ly/blog/2010/08/different-ways-to-modify-your- cam...](http://cam.ly/blog/2010/08/different-ways-to-modify-your-cam-ly/).</comment></comment>


Extracting passwords from Chrome - <AUTHOR> http://usvsth3m.com/post/57516016693/has-your-chum-left-their-computer-logged-in-do-they <comment><USER2> It's the same with every other browser, for instance...

[https://support.mozilla.org/en-US/kb/password-manager- rememb...](https://support.mozilla.org/en-US/kb/password-manager-remember- delete-change-passwords)

<comment><USER3> It's actually not the same with every other browser.

As far as I'm aware, Safari pulls the passwords from the keychain when necessary and does not provide plaintext access to them from inside the app.

</comment></comment><comment><USER4> It's obviously deliberate. Are you not familiar with OS level profiles? in the case of OSX: [http://support.apple.com/kb/PH11468](http://support.apple.com/kb/PH11468) there is your master password.

Also the post you mention was already submitted and discussed: [https://news.ycombinator.com/item?id=6165708](https://news.ycombinator.com/item?id=6165708)

<comment><USER3> Are you suggesting making a separate user account for each person who uses Chrome, and then switching user accounts before allowing a new person to touch Chrome on your computer?

I often browse for things on my coworkers computers (their laptops), and they do on mine. An application shouldn't be pulling passwords out of the keychain and then making the plaintext available.

<comment><USER5> Yes, users should have separate accounts. This whole thing is like complaining that a user can browse your My Documents folder if you allow them to use your account. Of course they can, they're logged in to your account!

<comment><USER6> It's a good idea to have separate accounts. The issue is that many people aren't aware of this, and should be. Chrome is going with the assumption that people know about it.

I'd rather someone browsed My Documents than saw my password in plain text. I don't even like seeing my password in plain text.

It came as an unpleasant surprise to many of us, so I'm trying to point out to as many people as possible that this is the way Chrome does it.

<comment><USER7> Eh, you think that's scary - buy a brand new machine and log in to your Chrome account and see all those passwords (and bookmarks and browsing history) get slurped down from who knows where onto that machine.</comment></comment></comment></comment></comment>


Show HN: Zeroqode – no-code app templates almost for any use case - <AUTHOR> https://zeroqode.com <comment><AUTHOR> Hi, I'm Levon, founder of Zeroqode. Our vision is to make building and launching complex web applications many times faster and less costly than it currently is. To achieve that we use the most advanced no-code web development platform called Bubble and build app templates which are basically ready-made fully functional apps with great responsive design. So anyone with an idea can simply choose a template which is the closest match, modify it without code and launch it within weeks or even days! So what we are trying to do for complex web apps is something similar to what Themeforest/Wordpress have done to simple websites. I know many of hackers here are developers and are very skeptical about "no-code" idea but please check out what we've built before posting something like "no-code will never be as powerful as coding". Think about it as the next level of abstraction, something similar happened when we went from assembler to javascript or from MS-DOS to Windows. It's still programming where you can define algorithms, conditions, workflows, actions, API connections and much more although all that is done without coding, but rather dragging and dropping and configuring. And it makes development/debugging/iterations much easier and faster. Would love to hear what you think. Thanks!</comment>


Coinbase is indefinitely suspending its business in Wyoming - <AUTHOR> https://support.coinbase.com/customer/portal/articles/1999937-coinbase-accounts <comment><USER2> Slightly OT, but I have an amusing story about Montana-based regulation. I work for an insurance company, and when you price life insurance or annuities, you have two separate mortality tables: male and female. Montana decided that that's discriminatory, so they mandate only using one table. So we use whichever table is more onerous (i.e. male for life insurance, female for annuities). So if Montanan women notice they pay more life insurance, or men pay more for annuities, they can thank their local regulator.

<comment><USER3> I don't think you can blame the regulator for that. If you wanted you could calculate tables for "people" regardless of gender. Those tables would more accurately reflect the population than purposely choosing the wrong subset of data to use.

<comment><USER2> That leaves you open to anti-selection bias. It would make men's life insurance cheaper than in any other state, and you'd likely get more men buying in Montana.

<comment><USER4> I do t think he's saying remove gender from the equation. Just don't have two tables based solely on gender.

</comment></comment></comment></comment><comment><USER5> How does this even work? Meaning if I live in WY, but I bank at "Small Union Bank of Delaware" Does that mean I'm using anything with in the realm of the Wyoming Division of Banking? Would just using an ATM put me under their jurisdiction? Because if I by something from Amazon, the transaction is basically from my bank in DE to WA if there are no Physical Amazon locations in WY. Correct?

<comment><USER6> > if I by something from Amazon, the transaction is basically from my bank in > DE to WA if there are no Physical Amazon locations in WY. Correct?

Incorrect. What matters is the state (or states) in which the company is "doing business". The exact definition varies from state to state, but in general having customers in WY means that you're doing business there and will (usually) need to register with the state, follow its rules and pay its taxes for your sales in that state.

Your bank is also doing business in DE and will need to follow their laws too, as the other end of the transaction is there.

</comment></comment><comment><USER7> Wyoming has something like 0.2% of the US population. I wonder if the number of Wyoming-based active accounts at Coinbase even reaches into the triple digits.

<comment> at-fates-hands They must have something substantial enough to warrant a press release. Makes you wonder, doesn't it?

<comment><USER8> I mean possibly yes. But they also might hope to someday have 4 digit numbers of users and know they can shame states, especially reddish states, by claiming they have too much regulation. Worked pretty successfully for zenefits in Utah.

</comment></comment></comment><comment><USER9> Both people in Wyoming could not be reached for comment.

</comment><comment><USER10> What does this mean for Wyoming customers who already have bitcoin in Coinbase-hosted wallets?

<comment><USER11> Even on account suspension CB let's you remove your current funds. I'm assuming the customers will have a time-period where they can remove those funds.

</comment></comment><comment><USER12> Huh. I'm about to go read the the books, but that sounds like a wild interpretation. That sounds like some odd parity reserve, rather than the fractional reserve system common in almost any other domain.

<comment><USER13> No, it's standard for "money transmitters" to have to maintain 100% reserves. Coinbase isn't a bank. They're not a lender. "Fractional reserve" means that a fraction of a bank's assets are in cash, and the rest are in loans owed them. That's fine unless a lot of loans go bad all at once, as in 2008. Even then, it eventually worked out when the economy came back.

Here's Wyoming's law:

 _" Each application shall be accompanied by a surety bond, irrevocable letter of credit or other similar security device acceptable to the commissioner in the amount of ten thousand dollars ($10,000.00) or two and one-half (2) times the outstanding payment instruments, whichever is greater. The commissioner may increase the required amount of the bond or security device to a maximum of five hundred thousand dollars ($500,000.00) upon the basis of the impaired financial condition of a licensee as evidenced by a reduction in net worth, financial losses or other relevant criteria."_[1]

For most money transmitters, this isn't a big deal. They don't hold the money for long. The whole point of the business is to get funds from A to B quickly. So they don't have a large obligation to their customers at any one time.

Coinbase's "hosted wallet" service makes them a depository institution - customers have funds on deposit with them. Given the track record of Bitcoin "hosted wallet" companies, most of which have failed, requiring reserves as a consumer protection measure makes good sense. Coinbase could post a bond from an insurance/bonding company, but they'd have to convince the insurance company of their financial soundness. That they can't easily do so is a bad sign.

Coinbase should still be able to do their business of converting Bitcoins to dollars for merchants. They only hold the money for a few hours, and should be able to have reserves for that float.

If Coinbase became a bank or a broker, they'd be subject to US Federal regulation and audits, but would no longer need state licenses. They're trying to sleaze by as a "money transmitter", while acting as a depository institution, and it's not working out for them.

[1] [http://legisweb.state.wy.us/statutes/statutes.aspx?file=titl...](http://legisweb.state.wy.us/statutes/statutes.aspx?file=titles/Title40/T40CH22.htm)

<comment><USER14> Untrue. It's standard for money transmitters to obtain surety bonds, but never have I seen a 100% reserve requirement flat out.

That being said, given Bitcoin's history, I don't think Wyoming's interpretation is unreasonable.

<comment><USER13> California requires 100% reserves up to $7 million.[1] For a real "money transmitter", this isn't a big deal, because money stays in their system for about a day, often less. For Coinbase, which is trying to be a broker on a money transmitter license, it's a problem. They hold customer assets for long periods.

Coinbase ought to register with the SEC as a broker/dealer. That would free them from state regulation. Then their customers would have SIPC insurance. Of course, Coinbase would have to accept FINRA audits and regulation. Right now, they don't even publish financial statements.

[1] [http://www.leginfo.ca.gov/cgi- bin/displaycode?section=fin&gr...](http://www.leginfo.ca.gov/cgi- bin/displaycode?section=fin&group=01001-02000&file=2000-2003)

<comment><USER14> That's not what it says, and that's not any of the sections where it says it. §2037(e) says you need a surety bond.

[http://www.leginfo.ca.gov/cgi- bin/displaycode?section=fin&gr...](http://www.leginfo.ca.gov/cgi- bin/displaycode?section=fin&group=02001-03000&file=2030-2043)

Surety bonds are not 1:1 reserves.

<comment><USER13> CA Fin. Code sec. 2037(e): _" A licensee that engages in receiving money for transmission shall maintain securities on deposit or a bond of a surety company in an amount greater than the average daily outstanding obligations for money received for transmission in California, provided that such amount shall not be less than two hundred fifty thousand dollars ($250,000) nor more than seven million dollars ($7,000,000)._"

A surety bond is an insurance policy good for the full amount of the bond. A bonding company[2] will have a lot of questions to ask a Bitcoin business before taking on that risk.

[1] [http://www.leginfo.ca.gov/cgi- bin/displaycode?section=fin&gr...](http://www.leginfo.ca.gov/cgi- bin/displaycode?section=fin&group=02001-03000&file=2030-2043) [2] [https://suretygroup.com/surety-bond/money-transmitter- bond](https://suretygroup.com/surety-bond/money-transmitter-bond)

</comment></comment></comment></comment></comment></comment><comment><USER15> This would only make sense if when you moved bitcoins into coinbase they immediately converted them to USD. In that case, sure they would need to have USD on hand to cover your funds.

In the case where you are storing the bitcoins with coinbase and not converting it they just need to make sure they have the bitcoins on hand. Forcing coinbase to both hold bitcoin and USD to cover the value of those bitcoins makes no sense.

<comment><AUTHOR> Explanation here: [https://news.ycombinator.com/item?id=9661614](https://news.ycombinator.com/item?id=9661614)

> If Coinbase became a bank or a broker, they'd be subject to US Federal > regulation and audits, but would no longer need state licenses. They're > trying to sleaze by as a "money transmitter", while acting as a depository > institution, and it's not working out for them.

</comment></comment><comment><USER16> > licensees must maintain dedicated fiat currency reserves in amount equal to > the aggregate face value of all bitcoin held on behalf of customers

Does bitcoin have a "face value"? I'm not sure what the legal definition of that term is, but I'd be surprised if it was the same as "market value".

<comment><USER17> > Does bitcoin have a "face value"?

No, clearly not. Which is one thing that makes either the interpretation or Coinbase's understanding of it ludicrous; given that we have no source for the actual interpretation by the Division of Banking, its kind of hard to tell _which_ , though.

</comment></comment><comment><USER18> Indefinitely is a very strong word and goes beyond the initial implications for the (small) population of Wyoming.

Under this interpretation of Bitcoin companies like Coinbase need to hold the equivalent value of their state holdings in a fiat currency (deriving it's value from government regulation), but fiat currency, aka USD, is no longer on the gold standard (a commodity currency) and thus only holds value because our government tells us it does and our collective agreement that it does hold value.

The interpretation of this law would be the equivalent of telling the native americans they need to hold USD reserves to trade in shell money regardless if all people collectively agree that shells hold value as a currency for trade. It's an attempt to continually establish USD being the standard for global currency value.

If more states follow suit set by this precedent than companies like Coinbase can't do business and the US Government crushes legitimate bitcoin uses.

This requires Bitcoin to continually evolved into an increasingly decentralized currency where both individual humans and computers agree on it's value without the inefficiencies of marketplace or government middlemen.

<comment><USER19> "... thus only holds value because ... our collective agreement that it does hold value."

What exactly do you think the value of a given collection of bits is?

<comment><USER18> The value of a given collection of bits is whatever you and I agree the value of those collection of bits is. It's a one to one relationship.

The value of USD is not what you and I agree the value is, it's what the government tells us what the value should be. This is not a one to one relationship.

One is based on a transparent mathematical equation between you and I and the other is based on 7 non-elected and appointed board of governors between you and I. That's the difference.

<comment><USER20> How does the US government dictate how much value you assign to the USD? You could trade me a piece of gum for $1 or a car for $1. In both cases you've agreed with me on the value of $1 and in both cases that value is different.

You have no more control over the market price of Bitcoin than you do the forex pricing for USD.

</comment></comment></comment></comment><comment><USER14> Maybe now some will take my comments about the need for federal reform more seriously.

[http://www.aarongreenspan.com/writing/20131118.hsgacstatemen...](http://www.aarongreenspan.com/writing/20131118.hsgacstatement.pdf)</comment>


Facebook enlists conservative help to resist privacy rules - <AUTHOR> https://www.politico.com/story/2018/04/17/facebook-conservatives-privacy-rules-489242 <comment><USER2> Other tech companies have PACs that also make questionable alliances: [https://donationsfrom.tech/](https://donationsfrom.tech/)

</comment><comment><USER3> Politico? Oh come on, that is a garbage partisan left site for garbage people. This is hacker news, and one would hope still propaganda free.....

<comment><USER4> Since you appear to be trying to defend HN, which is laudable, could you please not post comments that break its rules? That damages the site, arguably worse than propaganda does.

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)</comment></comment>


Why Isn't Smalltalk Popular? - <AUTHOR> http://stackoverflow.com/questions/711140/why-isnt-smalltalk-popular

<comment><USER2> I'd say the main problem with Smalltalk is that it tries to pretend the OS doesn't really exist. Which is fine as long as you stay in Smalltalkland, but around the edges of the application where it has to talk to databases, the filesystem, the network, etc it gets very messy. I dare say if you could work in a 100% Smalltalk environment it would be very slick, but Smalltalk very much doesn't play nicely with others. Where as (just as an example) Python does.

<comment><USER3> That was one of the original problems, but it doesn't apply to all implementations. Look at Dolphin Smalltalk for example. At its time, it is easily the best windows development tool, having very nice integration with native controls _and_ ActiveX support. I would go as far as saying using it for _only_ exploring and poking at ActiveX controls alone is well worth the license fees. And look where it went. Quite sad really.

Oh, and the free (Community) version is still available for download at <http://www.object-arts.com/>.

Oh wait a minute. I just accessed the web site to make sure it's correct. And it looks like it's alive again. Woohoo!!</comment></comment>


Adult Cat Finder - <AUTHOR> http://adultcatfinder.com/

<comment><USER2> why not just make this a real thing? adopt a cat from your local animal shelter.

<comment><USER3> After reading the byline, that is what I expected this to be. I was a bit disappointed when I clicked the button and all I got was an animated gif.

<comment><USER4> Me too. They should link to adoption shelters and such.

Just for fun, I typed WOOF WOOF, WOOOOOOOF, WOOF WOOF! into the chat box. There is a Kevin Smith joke that follows, but that's left as an exercise to the reader.

They missed a few joke opportunities. Here's hoping they keep adding to it.

</comment></comment></comment><comment><USER5> This site shows up on HN front page every 2 days.

</comment><comment><USER6> Previously posted:

<http://news.ycombinator.com/item?id=5328415>

<http://news.ycombinator.com/item?id=5327871>

</comment><comment><USER7> I've been laughing uncontrollably for over a minute now.

This is quite brilliant.

</comment><comment><USER8> I think someone pulled it off the frontpage already.

</comment><comment><USER9> What's next, catroulette?

</comment><comment><USER10> Needs autoplaying sound.</comment>


Where has my disk space gone? Flame graphs for file systems - <AUTHOR> http://www.brendangregg.com/blog/2017-02-05/file-system-flame-graph.html <comment><USER2> A flame graph does not feel like the best visualization for this kind of data. You can't really make anything of the top spikes and there is a lot of blank space that is wasted. I prefer tree maps as they make more efficient use of the available screen space.

My favorite tool for this is SpaceMonger v1.4.0 ([http://www.aplusfreeware.com/categories/LFWV/SpaceMonger.htm...](http://www.aplusfreeware.com/categories/LFWV/SpaceMonger.html)), which has a very neat layout algorithm. It's a Windows app but it works OK using Wine with only a few minor graphic glitches.

<comment><USER3> On Windows, WinDirStat is a great tool. [https://windirstat.net/](https://windirstat.net/)

<comment><USER4> And ADirStat for Android [https://play.google.com/store/apps/details?id=org.flightofst...](https://play.google.com/store/apps/details?id=org.flightofstairs.ADirStat)

<comment><USER5> I'm a sucker for DiskUsage myself.

[https://play.google.com/store/apps/details?id=com.google.and...](https://play.google.com/store/apps/details?id=com.google.android.diskusage)

</comment></comment></comment></comment><comment><USER6> I usually always recommend [https://dev.yorhel.nl/ncdu/scr](https://dev.yorhel.nl/ncdu/scr) for this purpose. Doesn't look as colorful but the output is basically the same. The SVG output is neat though!

<comment><AUTHOR> Nice, but the output isn't the same. A flame graph can show multiple levels of subdirectories at the same time, proportionally sized to their total bytes, up to a maximum of the screen hight (say, 24). Looks like ncdu can only show 1 directory level at a time.

<comment><USER7> The thing with flame graphs, pie charts and all that is that I rarely find it that useful to see multiple directory levels at the same time. Unless it is /var/log/omg-why-dont-you-rotate that occupies hundreds of gigabytes, I always end up navigating through the largest near-the-root folders with baobab just to make room for subdirectories on the screen, which is hardly different from the typical ncdu (or even 'du -hs *, then cd') workflow.

</comment></comment></comment><comment><USER8> I had this exact idea almost a year ago:

[https://github.com/m1el/du-flamegraph](https://github.com/m1el/du-flamegraph)

        find $1 -type f | xargs -n32 -d'\n' -- du -ab \
        | awk -F'\t' '{print $2" "$1;}' | sed 's#/##;s#/#;#g' \
        | flamegraph.pl --title "Disk usage" --countname "bytes" \
          --nametype "file" --width 1900 --minwidth 0.05

... and tested it using linux sources:

[http://m1el.github.io/du-flamegraph/linux- sources.svg](http://m1el.github.io/du-flamegraph/linux-sources.svg)

[http://m1el.github.io/du-flamegraph/illumos- gate.svg](http://m1el.github.io/du-flamegraph/illumos-gate.svg)

[http://m1el.github.io/du-flamegraph/dota2.svg](http://m1el.github.io/du- flamegraph/dota2.svg)

<comment><AUTHOR> I've done this before with awk, but with "find ... -ls". I don't know why you want to use du with xargs (is it really to get du's measure of bytes rather than the stat's count, for when they are different?) -- as that's calling over a thousand du's to walk the linux tree, when find can print bytes built-in with -ls (much more efficient).

<comment><USER8> Well, this could be explained by my ignorance in *nix tools.

</comment></comment></comment><comment><USER9> KDE: Filelight, [https://docs.kde.org/trunk5/en/kdeutils/filelight/introducti...](https://docs.kde.org/trunk5/en/kdeutils/filelight/introduction.html)

Windows: WinDirStat

<comment><USER10> Mac: Disk Inventory X [http://www.derlien.com/](http://www.derlien.com/)

<comment><USER11> Download link is broken.

Alternative ways to download:

a) Use brew: brew install Caskroom/cask/disk-inventory-x

b) Another download link (same link used in brew/cask)[0]: [http://www.derlien.com/diskinventoryx/downloads/dev/DIX1.0Un...](http://www.derlien.com/diskinventoryx/downloads/dev/DIX1.0Universal.dmg)

0: [https://github.com/caskroom/homebrew- cask/blob/master/Casks/...](https://github.com/caskroom/homebrew- cask/blob/master/Casks/disk-inventory-x.rb)

</comment></comment></comment><comment><USER12> For Linux Baobab is a great utility like this. It does the circular equivalent of a flame though, which can be much easier to read as the circumference of deeper directories gets larger by nature.

<comment><USER13> And the name is amazing (a Little Prince reference).

<comment><USER14> I'd love for this to be true. Do you have a reference for this? Baobab trees are spectacular in their own right. Might the usage in the Little Prince be just a coincidence?

<comment><USER13> Sorry, that's just my assumption (though I'd be willing to bet on it). The circular graph reminds me of nothing more than the image of baobabs swallowing up a miniature planet.

</comment></comment></comment></comment><comment><AUTHOR> After all the comments on treemaps and sunburst layouts, I think many readers might enjoy the following article if they haven't seen it already: A "Tour through the Visualization Zoo", by Jeffrey Heer, Michael Bostock, and Vadim Ogievetsky.[1] See the section on Hierarchies.

In that article, the icicle graph (same as the flame graph) has long rectangles suitable for labeling -- although their example has the font rotated 90 degrees and not making the best use of that space.

But as you can see, these are all related. Hierarchy visualizations. If I were serious about building a tool to do disk space visualizations, I'd let the user toggle between all three visualizations.

I should also note that I think both treemaps and sunburst layout have their own cons, which I've mentioned here on HN.

[1] [http://queue.acm.org/detail.cfm?id=1805128](http://queue.acm.org/detail.cfm?id=1805128)

</comment><comment><USER15> Very nice! On the Mac I recommend GrandPerspective: [http://grandperspectiv.sourceforge.net/](http://grandperspectiv.sourceforge.net/)

<comment><USER16> For Windows: [https://windirstat.net/](https://windirstat.net/)

<comment><USER17> I prefer Wiztree (link below), especially on systems with much larger disks. Since Wiztree works by scanning the MFT it's about an order of magnitude faster than Windirstat or Treesize.

[http://antibody- software.com/web/software/software/wiztree-f...](http://antibody- software.com/web/software/software/wiztree-finds-the-files-and-folders-using- the-most-disk-space-on-your-hard-drive/)

<comment><USER18> Hey, thanks for this. Was using WinDirStat. This is really fast! Plus I like the minimalistic interface.

</comment></comment></comment></comment><comment><USER19> When you have a hammer, you hammer out a reimplementation of `ncdu`.

I'm a die-hard fan of `du -kx dir | sort -rn | less`.

<comment><USER20> I prefer "du -h|sort -h" (for new versions of GNU sort, that support sorting "human readable" sizes.

<comment><USER19> Oh nice, time to update the muscle-memory firmware.

</comment></comment></comment><comment><USER21> This tool for windows is not known well enough: [http://www.steffengerlach.de/freeware/](http://www.steffengerlach.de/freeware/)

Screenshot: [http://www.steffengerlach.de/freeware/scnshot.gif](http://www.steffengerlach.de/freeware/scnshot.gif)

</comment><comment><USER22> The height of the flame in this graph does not seem very informative.

In the example, the 3.88% used by the tower of "linux-4.9-rc5/drivers/gpu/drm/amd/include" looks larger than "linux-4.9-rc5/net" over on the right just because the tree is deeper. Similarly, on my Windows machine at work, "C:\Users\LeifCarrotson\AppData\Local\Microsoft\Outlook\firstname.lastname@example.com" (where my email is stored) would take up more screen space than "C:\hiberfil.sys", just because it's taller.

I don't much care about the depth of the filesystem. The various tree views seem more useful.

</comment><comment><USER23> There exists a 4 year old bug in Ubuntu LTS releases (not sure whether mainline Debian is affected too but it is very likely) in which kernel upgrades fail to remove old headers. Because the way headers are structured it is possible to run out of inodes long before free space is exhausted if you don't pay attention to inode use.

[https://bugs.launchpad.net/ubuntu/+source/update- manager/+bu...](https://bugs.launchpad.net/ubuntu/+source/update- manager/+bug/1089195)

</comment><comment><USER24> There's a bunch of other tools[1][2] that do this sort of thing already, although all the ones I've seen display file size as a sort of flame graph/pie chart hybrid (imagine a flame graph wrapped around a circle).

This flat representation is probably better because it doesn't exaggerate the size of deeply nested files, but I find the example in the article a bit harder to read.

[1]: [https://daisydiskapp.com/](https://daisydiskapp.com/) [2]: [http://www.jgoodies.com/freeware/jdiskreport/](http://www.jgoodies.com/freeware/jdiskreport/)

<comment><AUTHOR> You're right, the flame graphs (which is really an adjacency diagram with an inverted icicle layout) doesn't exaggerate subdirectories like a sunburst layout does. I wrote about problems with the sunburst layout before in ACMQ[1]:

> The sunburst layout is equivalent to the icicle layout as used by flame > graphs, but it uses polar coordinates.7 While this can generate interesting > shapes, there are some difficulties: function names are harder to draw and > read from sunburst slices than they are in the rectangular flame-graph > boxes. Also, comparing two functions becomes a matter of comparing two > angles rather than two line lengths, which has been evaluated as a more > difficult perceptual task.10

(I should have mentioned that they visually exaggerate deeper slices, too). I think they are pretty, but, more difficult to read.

The other app has a pie chart and trees. Both can't visually show everything at once, all subdirectories.

[1] [http://queue.acm.org/detail.cfm?id=2927301](http://queue.acm.org/detail.cfm?id=2927301)

<comment><USER25> I agree with that criticism with respect to profiling, but for filesystems I find the increased surfaces of the outer rings makes the evaluation easier as the filesystem reaches smaller amounts of data, especially as FS tend to be _relatively shallow_ in POI terms.

<comment><USER26> The problem is that humans are not very good at comparing angles, which is one of the reasons why pie and donut charts aren't good (and sunburst is kinda like a donut, so the same criticism applies).

</comment></comment></comment></comment><comment><USER27> I recommend Baobab, its been around for ages and is on pretty much every distro's repo already.

Shows you a radial chart of your disk you can explore and zoom into and even better lets you delete directly from its ui.

</comment><comment><USER28> We built Crab so we could run SQL queries over the filesystem. It's for Win and Mac, free for personal use, and lets you slice this anyway you want and plot the data how you like. I like the flexibility.

[http://etia.co.uk/win/about/](http://etia.co.uk/win/about/)

e.g.

      SELECT sum(bytes) FROM files
      GROUP BY extension
      ORDER BY sum(bytes) DESC;

You can GROUP BY any file attributes

</comment><comment><USER29> Another oldie but goldie for Windows:

[http://www.win.tue.nl/sequoiaview/](http://www.win.tue.nl/sequoiaview/)

</comment><comment><USER30> There was a simple X application called "xdu" from the 90s that did this job adequately.

At the other end of the practicality scale was the SGI tool fsn, recreated in open source by this: [https://en.m.wikipedia.org/wiki/File_System_Visualizer](https://en.m.wikipedia.org/wiki/File_System_Visualizer)

<comment><USER31> I use this ... [http://xdiskusage.sourceforge.net/](http://xdiskusage.sourceforge.net/)

<comment><USER32> Me too.

</comment></comment></comment><comment><USER33> A bit off topic, but Windows needs this baked in. And Windows needs hard links to update ALL links when the underlying file changes, because right now if you update a file from one hardlinked file the other hardlinked files don't change their metadata.

Quite ridiculous, it's not like this hasn't been something Unix has been doing since almost the beginning!

Right now we have a ridiculous situation where the winsxs folder gets out of sync with the c:\windows\system32 folder. Nothing treeview or any other utility can you daily do about it either. And until recently that winsxs store was holding gigabytes of old and useless updates, because Microsoft's updates never removes these components (recently - last year some time I think- they updated the disk cleanup GUI to delete this stuff from Windows 7 upwards. IMO they recognised a big stuff up caused by their product management team's decision, which in turn caused this unheralded enhancement).

<comment><USER34> Doesn't Windows still lack the ability to do several basic file-system operations without resorting to goddamn DISKPART? FS management on that platform in general is a disaster.

</comment></comment><comment><AUTHOR> I posted a follow up: [http://www.brendangregg.com/blog/2017-02-06/flamegraphs- vs-t...](http://www.brendangregg.com/blog/2017-02-06/flamegraphs-vs-treemaps- vs-sunburst.html)

</comment><comment><USER35> Can anyone recommend a tool for macOS that monitors disk usage changes over time?

I usually use Disk Inventory X but I'd really like to correlate usage increases to specific dates / app installs, so it'd be nice to see stats over time, e.g:

\- Installed Android Studio on Feb 1st: Usage in /Applications increased by 850MB, usage in User folder increased by 10G (450MB for android- studio-2.x.dmg, 8.4GB in /Users/name/.android, largest leaf in /Users/name/.android/sdk etc)

I tried to do this with the 'du' tools once but simply writing the current output to disk would take ages and diffs would need some heavy lifting to make sense of.

</comment><comment><USER36> For visualizing your Dropbox' contents as a pie chart, a friend and I developed dgraph: [https://github.com/joplapp/dgraph](https://github.com/joplapp/dgraph)

<comment><USER37> wow that looks nice, I've always wondered why there isn't some tool like that

</comment></comment><comment><USER38> This is really cool. I really like the cpu / profiling usage too for flame graphs.

For disk usage I would normally use du and/or filelight which is also great.

This is a nice way to visualise all the sub directories too in one go though.

</comment><comment><USER39> On Windows I like Treesize [http://www.jam- software.de/treesize_free/?language=EN](http://www.jam- software.de/treesize_free/?language=EN)

</comment><comment><USER40> Nice - ideal use of flame graphs. When you are running out of disk space, you want to know which parts are using what proportion of disk space and IMHO this visualization is perfect for that. Kudos!

On a side note, when I'm on a server with disk space problems, I usually debug it like this:

        # cd /
        # du -sm | sort -n | tail
          (lists the biggest space users)
        # cd <unusually big subdir>
          (goto step 2: du -sm...)

Works like charm, but can be a bit slow when applied to large and slow disks.

</comment><comment><USER41> It's nice that you created something on your own.

On Android I use DiskUsage: [https://play.google.com/store/apps/details?id=com.google.and...](https://play.google.com/store/apps/details?id=com.google.android.diskusage)

Screenshot: [http://imgur.com/CrVcdEk](http://imgur.com/CrVcdEk)

</comment><comment><USER42> On Windows, I've always used HDGraph[1], but as no one has suggested it I'm wondering if I'm missing something important as to why not? I've never really liked the appearance of WinDirStat, but that's just a personal niggle, I know it works well.

[1] [http://www.hdgraph.com](http://www.hdgraph.com)

</comment><comment><USER43> For me a tree map is clearer and more screen-space efficient than a flamegraph.

qcachegrind renders profiling output as tree maps, and for filesystems there are many alternatives, e.g: Baobab (aka Disk usage analyzer) does this, also KDirStat.

</comment><comment><USER44> I am partial to DaisyDisk on OSX. Nice UI. It comes in very handy on the lower capacity Flash Disks (64GB) on the early generation Macbook Airs. Needs constant housekeeping on those.

</comment><comment><USER45> KDE has this thing built-in, I think it's called kdirstat and you can get to it by clicking on directory size (bottom right corner) in Dolphin.

<comment><USER46> Wow - I have an add-on to add "open with k4dirstat" to the right-click menu. Have used KDE5 since it came out and did not know about that. There is no affordance there at all; it doesn't even highlight on mouse-over.

I get a menu with options for not-installed apps - filelight, kdiskfree and two partition tools. 'sudo apt install filelight' and it works, but TBH I'm not keen on filelight, would be nice to have k4dirstat as an option there (sounds like a bug report is due).

</comment></comment><comment><USER47> This is beautiful. Great stuff. I've always had tools on desktops to do this, but never anything I could just run on a server.

</comment><comment><USER48> Interesting that "drm" occupies almost all of the "drivers/gpu" space!

<comment><USER49> Well in this case DRM means Direct Rendering Manager ([https://en.wikipedia.org/wiki/Direct_Rendering_Manager](https://en.wikipedia.org/wiki/Direct_Rendering_Manager)) What I find even more interesting is AMD takes over half of that.

</comment></comment><comment><USER50> I always liked WinDirStat.

</comment><comment><USER51> WinDirStat for the win!

</comment><comment><USER34> Conveniently, I no longer need to delve into any of these visualization tools, because the answer to the question in the title is always "Debian's overly-lax default logrotate.d"

</comment><comment><USER52>
        cd / && du -sh *

<comment><AUTHOR> I'd try that first, and if it didn't find my space, then flame graphs.

du's output does the first level of directories, but not subdirectories, and also requires reading of text rather than visually comparing line lengths (easier).

<comment><USER52> You can quickly move toward the problem and continue to run 'du'.

<comment><AUTHOR> If I'm at the command line already, that may well work.

Or, it may not (the initial problem I had, I already knew the high level breakdowns, and was hunting for wasted 1%'s here and there -- the flame graph made it easy to spy everything at once).

Another use of the flame graph approach is with automated build software. Imagine automatically generating one with every linux version, to keep track of where growth is.</comment></comment></comment></comment>


Why does a new Rails app need 106 MB of JavaScript? - <AUTHOR> https://gist.github.com/rileytg/6f2a1a8d982cbcee8cdcab37bfe344bb <comment><USER2> Rails 6 by default includes webpack to bundle assets for production. Aside from webpack, several of the top 10 on your list are libraries that ensure a great user & developer experience across browsers:

\- babel transpiles ES6/7+ JavaScript syntax to the least-common-denominator of the browsers you designate as supported by your application

\- core-js provides implementations of features that are not supported in older browsers -- things like array methods that cannot simply be transpiled

\- caniuse-lite is a database of supported browser features to determine what needs to be transpiled and when

This toolchain allows developers to write code with the latest language features (which usually results in simpler, more elegant code) while seamlessly supporting legacy browsers as needed. If you decide to drop support for an older browser, you can simply remove it from your browserslist instead of going through all your code to figure out which hacks you put in just for that browser. It takes a ton of cycles out of development googling "can I use feature X in browser Y".

Prior to Rails 6, asset precompilation was powered by Sprockets by default. Sprockets still bundled many analogous libraries, just hidden in ruby gems instead of node_modules. For example, if you want to use SASS then you need a library that compiles SASS to CSS -- whether it's node-sass on your list or the sassc gem.

As noted in other comments, very little of this code is actually loaded in the frontend by end users.

<comment><USER3> Regardless of whether you think all those components are necessary, there is still the question of whether the amount of space they require is justified; JavaScript is a high-level language, and in fact one of the higher-level ones in comparison to things like C/C++. 1KB of JS should be able to do far more than 1KB of C or even binary.

Thus, 100MB of JS is _astoundingly huge_. It's roughly the same size as _all the binaries in a full install of Windows 95_ , with all its features and drivers.

In the spirit of [https://prog21.dadgum.com/116.html](https://prog21.dadgum.com/116.html) perhaps we need a list of "Things an empty Rails app is smaller than".

 _It takes a ton of cycles out of development googling "can I use feature X in browser Y"._

...or you could just avoid the trendchasing and use a minimal subset that you are sure will work everywhere, but that's a different discussion.

<comment><USER4> At $50 for a 1TB hard drive, that 100MB is costing you half a cent. It doesn't get sent to users over the network so it doesn't affect performance. Why does that matter again?

<comment><USER5> 106MB of source code is perhaps five million lines of code, if written in a typical way. Some people use longer lines, not so many "{" lines, but anyway.

Storing five million lines of code is cheap, sure, no question. But don't you feel curious about how why you need that much code in the first place?

</comment></comment></comment></comment><comment><USER6> Looks like it is just the asset pipeline, so not anything that is loaded on the frontend.

Little bit click baity. A lot of other frameworks do the same thing.

<comment><USER7> > A lot of other frameworks do the same thing.

God that is an awful justification.

The heading is the opposite of clickbait. 106 MB is not exaggerated, the link provides proof showing that number. I get pretty tired of people handwaving away bloat these days. Its a real problem across programming languages and communities, and it doesnt just go away by pretending it doesnt exist. It goes away by people testing for it, making it a priority for software to be streamlined and pruning bloat where it makes sense.

<comment><USER8> It's build tools. This is like complaining that Xcode is many gigabytes.

<comment><USER9> I never had to install a new copy of xcode for every project.

<comment><USER10> So the problem is not the size, but that you want to have a single cache of all the npm modules?

</comment></comment></comment></comment></comment><comment><USER11> For reference, a full install of Windows 95 is about 50MB.

There is something deeply wrong with this entire industry.

<comment><USER12> But those are compiled binaries, not source code.

I guarantee you the source code for Windows 95 is _vastly_ larger than the 106 MB we're talking about for Rails.

And considering the ridiculously large number of things Rails does that Windows 95 didn't... the sheer number of different technologies, standards, formats and versions it's able to interface with... I don't see what's wrong at all.

<comment><USER3> _I guarantee you the source code for Windows 95 is vastly larger than the 106 MB we 're talking about for Rails._

I wouldn't be so sure. Win95 has a lot of pieces written in Asm (source > binary), mostly the low-level parts, but also a lot of C/C++ (binary > source).

<comment><USER12> I mean, I know that for a small "hello world", you'll get an executable much larger than the source code.

But with a decent-sized program, full of header files and comments and config files and descriptive variable names and all the rest, there quickly comes a point where the executable is vastly smaller, no?

I really can't believe that the C/C++ source code for Windows (or Word or Photoshop or whatever) winds up compiling to a _larger_ file size, can it?

<comment><USER13> It really depends. Small amount of source code doesn't necessarily mean a small binary if you have a ton of template instantiations.

</comment></comment></comment></comment></comment><comment><USER10> Right, what I love about this is that it completely ignores the installed gems because they're installed somewhere else?

      $ bundle --path vendor
      $ du -h -d 1 .
      64M ./vendor

After building the rails site you can delete node_modules because you only need the built bundles. Those shouldn't be very big at all.

<comment><AUTHOR> When building container images, yes. However in heroku I'm not sure how to do this without custom buildpacks (e.g. [https://elements.heroku.com/buildpacks/istrategylabs/heroku-...](https://elements.heroku.com/buildpacks/istrategylabs/heroku- buildpack-node-cleanup)). Custom buildpacks reduce the value add of heroku for my small team.

Ultimately my solution will likely be build the assets in CI and deliver via CDN or check into the repo on some release branch.

Edit: I intentionally omitted the gem sizes as that was a prior cost of using rails

<comment><USER10> I feel your pain, we also had this problem, and resorted to exactly the thing you've done.

Personally I always felt that this was a shortcoming with Heroku's Buildpacks. Which seem unwilling to support modern rails.

</comment></comment></comment><comment><USER14> Webpack-based projects are heavy. A Rails app comes with a Webpack-based Javascript scaffold out-of-the-box.

I wonder if something like Rollup or NCC could be used to create a single-file Javascript dependency in the general case that is only used for vanilla Webpacker builds, and relying on the individual packages could be something left for when you need to customize the Webpacker build.

Judging by some of the Hey developments, it's possible we could see a much more lightweight JS integration in Rails in the future:

[https://twitter.com/sstephenson/status/1272608117604397063](https://twitter.com/sstephenson/status/1272608117604397063)

<comment><AUTHOR> I've worked on the whole loop of ¡server side! to ¡client side! and now back to ¡server side!. Using Hey for a little, I'm really impressed with the "feel" of it. I really hope they will write about how they do this and even contribute stuff upstream.

Going back to primarily server side was about long term projects that aren't very active. I have a rails app I wrote in 2011, I occasionally make fixes, update gems etc. No stress.

I have a client side app from 2016 that I eventually gave up on trying to get the build system running again (grunt?). I just edited the minified build and told the customer this was the last major edit. Only copy changes now.

This isn't an isolated incident. I frequently have to relearn an old project b/c everything changes so fast in js. First we were on prototype, then we went to jQuery (2011?). Homegrown UI components lacked the browser support we needed (IE5/6?) so we added jQuery UI. Our code was a spaghetti mess without components, so eventually we moved to React (2015?).

Those changes took 10+ years. The rest is all in the last 2.5-3.5 years. And personally, I find it way more complicated than previous js ecosystem changes. Ok now we really want to use modern js so lets add a transpiler. Now our components state is getting out of hand. Lets use reflux. Oh no dont use reflux, use redux. Oh but now you need redux-forms. Remember that build system we had? Now everyone uses grunt. Oh no, everyone uses webpack now. Are you still using redux? Use react hooks. And while I value this ecosystem, I'm nervous that we wont "solve" our problems and just keep writing new partial solutions.

It's a bit story-telly (rant?) but the gist is Rails w/ ssr doesnt come with huge mental cost over time. Client side can. We still love and use the client side apps, just only for complex "apps" like an interactive document viewer where we are actively developing year over year.

</comment></comment><comment><USER15> That can be answered in one phrase, coined by Django:

> Batteries included.

It's ultimately up to Rails how they want to do things. It's OK to complain about it but it's a shame when it discourages people from creating and maintaining open source projects.

<comment><USER16> Other use in the Python ecosystem seems to predate Django ([https://github.com/python/peps/commit/53474baa0790912b8a02f1...](https://github.com/python/peps/commit/53474baa0790912b8a02f1468efd782e90dd09eb)) although I'd certainly be interested in more context.

<comment><USER15> Yeah I wonder if they arrived at the phrase independently. I think Jacob Kaplan-Moss was inspired by Python's standard library.

It seems the owner of that PEP could probably provide more context. He wrote "Meditations on the Zen of Python" recently which means he's still thinking about pythonic software design:

[https://orbifold.xyz/zen-of-python.html](https://orbifold.xyz/zen-of- python.html)

</comment></comment></comment><comment><USER17> Webpack.

I always appreciated the design philosophy behind rails, how it was always about "getting things done", and not about being the latest cool tech of the day, but this introduction of the webpack fad into the "rails convention" feels just like going backwards.

</comment><comment><AUTHOR> FWIW 106mb will fit ~11 million chars.

I use many of the features I get with rails webpack. I also use create-react- apps (which I think is even bigger). I think they are valuable to the development process.

I would think we can achieve most if not all of these goals with less code. I don't know exactly how we would get there or if there are any projects underway. This seems like a problem that can't be fixed in one place but must be addressed holistically. This being a decentralized ecosystem of packages, the only way I can see that happening is by publishing stats and creating a culture of reduction. For example: When contributing to the one of the presumably hundreds if not thousands of packages that roll up into CRA or rails, you should see if that overall increases or decreases size. You may have introduced a huge tree of new dependencies thinking your adding one package to your tiny package.

Just a rough idea, does someone more experienced in the community have ideas on what can be or is being done?

<comment><USER2> I think there are two primary reasons why JavaScript projects tend to have much larger dependency trees than in ruby or other ecosystems.

1\. The JavaScript standard library is much more sparse than what you get out- of-the-box with ruby or many other languages. The existence of lodash is evidence of this in itself. The infamous left-pad fiasco would never have happened if JavaScript's native String had an equivalent to ruby's String#rjust. This part of the problem is significantly improving with time -- for example, node.js 8.0 and up do have a native string pad function, padStart. However, especially for tooling libraries, using the latest native JS features means dropping support for semi-recent versions of node, and there's also generally a lot of inertia.

2\. npm and yarn allow you to install multiple versions of the same package. If I have dependencies A and B, which both require different major versions of dependency C, npm and yarn will both happily install multiple copies of C. With ruby/bundler this is an error and you have to work out how to tweak your dependencies so they're all happy with the same version range. I suspect this architectural choice is largely a consequence of #1 -- especially in the earlier days of npm, getting all your dependencies on the same version of a commonly used helper library would have been nightmarish. And there are benefits to this approach; I have certainly spent long hours trying to upgrade ruby gems with version conflicts. But it results in a lot more bloat almost by default.

To answer your original question -- I don't know that anything in particular is intentionally being done. I do think we're heading in the right direction and things will be much better in a few years as packages slowly move towards new native language features rather than helper libraries. I also just don't think this is of critical importance at the end of the day. If you add up the brain cycles that I've ever spent worrying about 100MB of hard drive space, it's almost certainly less than what I've spent typing this comment :)

</comment></comment><comment><USER18> This should also be done for create-react-app

</comment><comment><USER19> Can't you just pass "\--no-webpacker" to the "rails new" command to skip the npm packages? I haven't tried it myself.

<comment><AUTHOR> Yes. Makes a huge difference. I try to stick with defaults as much as I can. I find that helps with long term maintainability. In this case, I don't know if I will (as I chose in the past to always use pg and rspec).

</comment></comment><comment><USER20> "It's not frontend" isn't an answer. Why should a text-packager clock in at 100mb?

<comment><USER21> Because modern web development is a friggin' minefield of browser support matrices, front-end development languages with experimental features, and aging specifications that require the use of preprocessors for simple things like variables and functions. Almost everything in that 106 MB is a preprocessor of some type that's designed to either make developer's lives easier, normalize code to a widely-supported standard, and/or optimize said code to deliver the smallest possible bundle to end users.

<comment><USER20> Why can't it be 50mb? Why shouldn't it be 200mb?

<comment><USER21> I don't understand the point you're trying to make. That's just how much disk space those tools and their dependencies take. You might as well ask why iron is as dense as it is.

<comment><USER20> Web-pack is not an elementary particle. It's a man-made construction. Describing its disk-size as an intrinsic aspect is a categorical mistake.

Webpack is no-doubt a big program with a lot of features. But I have several even-bigger programs on my PC that clock in at just a few mb - so saying it has a lot of features doesn't answer OP's original question, it simply substitutes an easier question and then answers with a "the size is what the size is" tautology.

When a native app links a static library, it performs dead-code elimination. It might have hundreds of megabytes of dependencies, but the overwhelming majority is not copied to the application. When I make a new rails project I don't download the text-source of Firefox or MySQL and all their downstream dependencies into every project root. There's no reason webpack has to do this, except because of arbitrary workflows imposed by NPM.

If no user-complaint about NPM is ever quantified or interrogated, then NPM will fade into irrelevancy.

</comment></comment></comment></comment></comment><comment><USER22> I just tried it for a new Angular app. I chose to include Angular Routing (because everybody will need that) and then to use CSS as my style sheet because hey why not. End result was a folder that was 273MB in size, with the node modules being basically all of it.

</comment><comment><USER23> I guess he hasn't see the size of a create-react-app project.

</comment><comment><USER24> node_modules is insane. JavaScript needs an STL. Then we need an AI to rewrite all of these modules to use the STL.

</comment><comment><USER25> Basically several compilers are bundled (node sass, webpack and Babel) so that you don't have to download them as a user and their version is stable for the app. You can use a tool like pnpm to only ever have one copy of them on your system but these aren't deps that negativity impact the app.

</comment><comment><USER26> Great fan of Rails but man, that's a lot of bloat. Last time I used Rails it only came with jQuery. Live-reloading was an exotic thing, quite difficult to setup. Now it seems to be enabled by default, explaining some of the stuff in there.

<comment><USER21> It's also 99% server-side preprocessors. Only a single package in that list (lodash) is something that could be actually loaded via browser, and even that would only be a small percentage of the full package size. It's also not required for a new rails project, it's simply a convenience workflow that integrates many of the tools used in modern best practices.

<comment><AUTHOR> Your point is valid that a large portion is server side only (which is still a concern for my use case). But note a lot is omitted from my gist, there are 768 packages.</comment></comment></comment>


Why we are going to be like our parents, and why that's ok - <AUTHOR> http://danielflopes.com/why-we-are-going-to-be-like-our-parents-and-why-thats-ok/

<comment><USER2> Seth Godin is far from the first person to refer to the amyglydia as the "lizard brain". I really couldn't tell you who coined that term or when, but I was seeing it when I was reading about the brain as a kid back in the seventies.

Also I am in my early 40s and am not even beginning to turn into my parents, for what that's worth.

</comment><comment><USER3> I believe that the brain changes between the age of 15 and 30, and I also believe that most people's lives changes between those ages. I'm not so convinced that the former cause the latter, especially without any kind of scientific evidence. There are a lot of things that change socially over that time period.

The author's take on the amygdala is also pretty weird. Sleep, hunger, and sex are all functions generally associated with the hypothalamus, not the amygdala.</comment>


Stanford Professor of Medicine Dr Ioannidis: Acting Without Reliable COVD19 Data - <AUTHOR> https://www.youtube.com/watch?v=QUvWaxuurzQ <comment><USER2> Unfortunately, you will never see this level-headed, common sense approach promoted by the main-stream media.</comment>