Announcing Backbone.js: Models, Collections and Views in 2.4kb - <AUTHOR> http://documentcloud.github.com/backbone/

<comment><USER2> Wow I am incredibly impressed by this, great work to the DocumentCloud team! I can't wait to throw this into real-life use... it looks like everything I could ever want in a framework.

I too am a jQuery developer, but not your typical one. I tend to avoid slapping plugins together and write most of my own things from scratch, in my own pythonic classical style. I also tend to use the little jQuery helper methods and CSS selector tools purely as an interface to the DOM. The hard work is all done via classes, which drastically reduces the code that I write. I see other not-so-skilled jQuery script kiddies out there piecing together documents full of $('#blah') this and $('#bloo') that with little regard for chaining or simplifying things. I digress, as that is not the demographic for this wonderful library of code.

Here is an example of what kind of code I have been writing as of late: <http://dpaste.de/pKOi/> \-- as you can see, there is a small bit if boilerplate required and a 'var self = this' inside of each class method that is required. That's more for my own personal style actually, to help with preventing loss of 'this' scope down the road. Anyway... over time this code just sort of piles up and it would be nicer to 1) have a better way to define and organize it all, and 2) all of the great model/collection/validation stuff looks excellent! Evented programming is brilliant to, and the fact that a thousand pieces of code can all stay in sync thru events is just fantastic.

<comment><USER3> I'm a JavaScript newbie. I'm just curious as to how your code (as shown in dpaste.de/pKOi) is better than Douglas Crrockford's module pattern? Please enlighten me,thanks for help.

<comment><USER4> Basically he's just using JavaScript's built-in objects and making the initializer function call the prototype's init. The advantage vs. Crockford's "module pattern" is that if you plan to make many instances, the functions don't all need to be recompiled every time, or take up extra memory.

I would advise the GP poster to pick up some class implementation (or roll his own), and use some bind function instead of all the `self`s (if only because other JavaScript programmers will recognize the style better). One that seems to work reasonably well for simple use cases is John Resig's suggestion, <http://ejohn.org/blog/simple-javascript-inheritance/> though one thing to keep in mind with that implementation (just as with the poster's) is that new instances must be created with `new`.

That one is supposedly inspired by base2: [http://code.google.com/p/base2/source/browse/trunk/lib/src/b...](http://code.google.com/p/base2/source/browse/trunk/lib/src/base2.js)

However you decide to do it, the prerelease versions of Resig's upcoming book are quite helpful in understanding how these designs work and all the tricky corners of JavaScript closures and object inheritance.

</comment></comment></comment><comment><USER5> Key point of Backbone.js: This is for designing web apps, not just a web page. When you have multiples of the anything on a page, it's time to move to separating out your views from your models, and Backbone.js is perfect for this sort of transformation. Even better if you start with Backbone.js. And Backbone.js is a minimal set of functionality to get this done.

We use Backbone.js at DocumentCloud for our document workspace. It took a few weeks to pull out all the pieces, but we rely on the JavaScript MVC that Backbone gives us for the entire workspace. (That too will go open-source, someday soon enough.)

I am personally planning on integrating Backbone into NewsBlur, an RSS feed reader, which currently has an ad-hoc model system. Backbone would give it the ability to update stories and feeds without having to remember where all of the stories and feeds are on the page.

Anyway, Backbone takes a bit of getting used to, but it is fairly easy to read the annotated source to see exactly what's happening. And once you do learn the Backbone.js conventions, hard problems become much easier on the front- end.

<comment><USER6> I'm concerned that the default approach to saving is making a request to the server. Defaults matter, and this default will lead people to ignore the 8 fallacies: [http://en.wikipedia.org/wiki/Fallacies_of_Distributed_Comput...](http://en.wikipedia.org/wiki/Fallacies_of_Distributed_Computing)

N+1 across the internet will suck.

I assume it's early days, but please consider this issue. It's killed a bunch of otherwise interesting libraries.

<comment><AUTHOR> This issue is very much considered. You absolutely don't use this to make a bunch of tiny little Ajax requests to load your application in pieces. Instead, you bootstrap all of the data you need for an initial load onto the page, and then populate the collections directly.

For saving, a single update often needs to be a single request, and the UI needs to know that the update has been applied successfully. If you need to do a bulk update, then use "set", not "save", and make a custom Ajax call for the bulk operation.

Finally, Backbone.sync is the lowest-common-denominator default REST request, which will work for many applications. You can and should override it if you have more specific needs -- using timeouts to aggregate many granular saves into a single HTTP request is a great idea for some applications...

</comment></comment></comment><comment><USER7> I'm a little confused about what the best-case/ideal scenario is for this.

As an example, I am currently working on a project that has a basic profile page. So on the server side, I have my nice OO representation of a Profile object, complete with properly defined public and private variables, a slew of functions, and a save() function that validates saves the data to the database.

When I display that page to the client, I have a PHP view which takes the data from the model and then renders it. In the view, I do currently use jQote2 to do some basic client-side templating where appropriate (for example, displaying all of the friends of the profile's user).

Now, the Profile page features the ability to edit profile fields. There is jQuery involved here. Currently, I just have a function that is called when the "Save" button is clicked that gathers up all the various data from my input fields and whatnot, and sends that off as JSON to be saved on the PHP side of things.

Is this someplace where Backbone.js would make sense? Would I then have to have a representation of my model both in PHP and in Javascript? And my PHP view would contain several Javascript sub-views? And when I do save the model on the Javascript side of things, am I then just executing database queries on the PHP side, or am I re-loading the model, changing the attributes, and then using the PHP model's save() function?

I guess I'm just confused as to where this fits into the grand scheme of things. There's a part of me that certainly likes the idea of data binding and having smart re-rendering of parts of my pages based on when models change, but I'm just unclear on how this fits into my existing PHP MVC stuff.

<comment><AUTHOR> Backbone is intended for JavaScript-driven web applications. If you're doing most of your logic and HTML rendering on the server-side, then it's not appropriate for you.

However, if you have an application with a lot of JavaScript interactivity, or a single-page application where the entire interface is driven from JavaScript (this is the case for us), then it helps to structure your client-side code with a little more oomph than just jQuery provides.

Taken to the extreme, in our case, the DocumentCloud workspace is effectively an empty body tag, and all of the HTML rendering and interesting logic happens in JavaScript models and views -- you never have to refresh the page. The server-side Rails code becomes smaller and less complicated, essentially delegated to performing validations and authentication and serving JSON to the client. Think GMail, or New Twitter, or 280 Slides...

<comment><USER7> Okay, that makes sense. Thanks for the reply.

So basically this is for those folks who are making real "web applications" -- meaning applications where a lot is happening on the client side, typically more of a "single URL/page", with dialog boxes and whatnot as opposed to a bunch of pages in the more traditional "web site" sense.

I have often considered trying out Cappuccino, but for some reason none of the websites I build end up being "web applications" in this sense. But I can't help but wonder if that's because they aren't, or if it's because I'm not thinking about them the right way.

It's outside the scope of this topic, but I can't help but ask: "How do you know when a website becomes a web application"? Is it the scope of the problem (trying to do "one thing", e.g. edit a photo, vs. presenting an entire web site full of information?) Or is it truly just a preference thing? I can only think that as projects like Node.js gain prominence, the lines will continue to be blurred...

<comment><AUTHOR> The line is as blurry as you'd like to make it. A great example of this is Google Instant. Old Google.com was a web page -- and Google Instant is now a web application, the page doesn't refresh, and sends down JSON data which is rendered into your search results by JavaScript...

</comment></comment></comment></comment><comment><USER8> Sorry, I'm a little late to the party with respect to javascript. Could someone please explain to me what this means in english?

 _Backbone supplies structure to JavaScript-heavy applications by providing models with key-value binding and custom events, collections with a rich API of enumerable functions, views with declarative event handling, and connects it all to your existing application over a RESTful JSON interface._

<comment><AUTHOR> Yep -- let's dejargonize that for you:

"key-value binding and custom events": When a model changes its state, other JavaScript objects can listen for that change and be notified. It's the usual inversion-of-control pattern, but especially crucial for models and views. A model has no business knowing about what views may or may not currently be present in the UI. Instead, the views listen to changes in the model, and update themselves accordingly.

"rich API of enumerable functions": JavaScript arrays are pretty feature-poor, at least in terms of things that will work cross-browser. Backbone collections include all of these handy functions for working with your data:

[http://documentcloud.github.com/backbone/#Collection- Undersc...](http://documentcloud.github.com/backbone/#Collection-Underscore- Methods)

"views with declarative event handling": Instead of creating a mess of nested jQuery "bind" or "delegate" calls, it's nice to just declare what elements in a view should be hooked up to specific callbacks:

<http://documentcloud.github.com/backbone/#View-handleEvents>

"RESTful JSON interface": The persistence strategy for Backbone can be swapped out for something different (Websockets, Local Storage, CouchDB), but the default is to fire off a standard JSON Ajax call when you call "model.save()"...

Hope that helps a little.

<comment><USER8> Yes it does. Thank you for the explanation.

For anyone unclear about REST, try this: <http://tomayko.com/writings/rest-to- my-wife>

</comment></comment></comment><comment><USER9> I got a sneak preview of this yesterday. Having worked on a couple of large- ish JavaScript UI projects this is the kind of thing that people invent over and over again without ever distilling a minimal, extensible, and reusable set of functionality, it's either too application specific or heavily embedded in a larger framework (Cappuccino/SproutCore). MooTools+Backbone.js and/or jQuery+Backbone.js sounds like a killer combo to me!

<comment><USER2> Totally! I can't wait to replace my own internal model classification stuff with this (or at least play around with it in that sense) to go along with jQuery. I've become accustomed to the jQuery way of handling the DOM so it's nice to see something agnostic to that. I'd rather avoid a one-size-fits-all js library to be honest ;)

</comment></comment><comment><USER10> It's somewhat hidden on the project page, but I love the annotated source view: <http://documentcloud.github.com/backbone/docs/backbone.html>

Edit: It looks like it's taken straight from the source ([http://github.com/documentcloud/backbone/blob/master/backbon...](http://github.com/documentcloud/backbone/blob/master/backbone.js)) using Docco: <http://jashkenas.github.com/docco/>

</comment><comment><USER11> This looks great. I'm primarily a Flex developer and JavaScript for web apps has always scared the bejesus out of me as I felt I was back to the dark days of ActionScript 2. I still feel I'm faster with strong typing and a good IDE (I hardly type anymore it's all code completion) but micro frameworks like this are a huge step in the right direction.

Is there an IDE for serious JavaScript development? Right now for JS work I use TextMate with a bunch of custom tag triggers.

<comment><USER12> Try using haXe for your JS code. The syntax is extremely close to AS3 and you'll get the static types and classes back, as well as lots of other nifty stuff.

IDE support is still kind of lightweight, but there are several options out there: <http://haxe.org/com/ide>

I use Geany, because I only really need context-sensitivity, highlighting, and search+replace.

</comment></comment><comment><USER13> This looks very nice. I already use underscore an jquery for all of my projects. I'll try this on my current project and report my findings.

<comment><USER14> Looking for underscore and found this interesting testing page: <http://documentcloud.github.com/underscore/test/test.html>

</comment></comment><comment><USER15> I really like the approach here of extracting the core functionality of js mvc and data binding without the bloat of a huge framework. I plan on using this on a project soon. Well done jashkenas and co!

</comment><comment><USER14> Would be nice to have a complete minimal web app using Backbone to get the feeling of what you can do with this.

<comment><USER16> I have made a demo app. See my blog post for code and explanation. [http://afewgoodlines.com/post/1329452279/a-backbone-js- demo-...](http://afewgoodlines.com/post/1329452279/a-backbone-js-demo-app- sinatra-backend)

</comment></comment><comment><USER17> I'd be interested to know how this compares to js-model (<http://benpickles.github.com/js-model/>). Anyone have any insight?

<comment><AUTHOR> Yes. I recently discovered js-model while wrapping up the Backbone documentation. The two libraries are extremely similar, and share the same core idea, but with that said, here are some of the differences:

* js-model is explicitly inspired by Rails' ActiveRecord models, and a lot of these points fall out from that fact...

* In js-model, a collection and a model class (constructor function) are the same object. This is a big problem in client-side code -- you don't always want to have just a single collection of say, notes. That makes sense in a one-database-per-app, one-table-per-model world, but not so much in JavaScript.

* Backbone sets up the prototype chain so that you can continue to extend (subclass) your Models, Collections, and Views.

* Backbone includes a richer set of enumerable functions, based on Underscore.js, so you get native performance in browsers that support them natively.

* js-model's validation and errors API mimics Rails' Errors object, which may or may not be what you want.

* Backbone includes Views, and js-model sticks to models.

That's just a list of things off the top of my head. It's an amazing example of convergent evolution in code -- and I think it's much more widespread than this. In my experience, many folks who work on big JavaScript projects end up with an internal framework that bears an uncanny resemblance to Backbone.

<comment><USER18> js-model has some nice support for serializing to local storage and so forth baked in, which I've found to be really handy.

Backbone looks like it forms a much more cohesive package though, which is nice.

</comment></comment></comment><comment><USER19> Thanks for releasing this DocumentCloud. I've been ranting about bound models with jQuery for months. It's nice to throw away the start I've made and use someone else's code.

</comment><comment><USER14> So if we were designing the Twitter front page with Backbone, we would have models like TrendingTopics, WhatYouDoing, NewMsgs and some more. They would be updating against the server using JSON, and calling some render code so they redraw themselves when there is new data.

If we didn't have Backbone, we would probably call a main JSON sync function repeatedly, and iterate on this new data updating the different parts of the page "manually".

Is that it?

<comment><AUTHOR> A more apt comparison is the "new twitter" homepage. If you load it and keep an eye on your Ajax requests, you'll see something like this:

<http://cl.ly/b043f31cd91dfdac8918>

There, Recommendations, Friends, Followers, and Memberships are the models, and render off the contents of that JSON. Of course, Twitter should really be bootstrapping all of those bits of JSON into the initial page load, instead of firing off six requests immediately...

<comment><USER14> Thanks, and yep I meant newtwitter. (How long until we can call it twitter? :)

</comment></comment></comment><comment><USER20> Finally MVC on the front end as well and no more custom json objects to keep all your data in one massively redundant and unreadable object.

</comment><comment><USER21> Looks great - imagine that it would be a perfect companion to something like sammy.js

<comment><AUTHOR> Yes, they could work together quite well.

However (and this is another conversation altogether), I don't think that you really want to structure your client-side application around faux-routes. Maintaining browser history with "hashchange" is important, but hardly the central aspect of a JS app. Usually you want to reserve history changes for special states that deserve to be bookmarked, not for every single action performed.

For the record, this is the little module that we use to record and listen for "hashchange" events:

<http://gist.github.com/624773>

<comment><USER14> Can you please expand on this? I don't get it: do you think something like newtwitter is badly designed because it uses faux-routes? If you don't have faux-routes then how can you easily insert links which will "go to page X" ?

<comment><AUTHOR> Not at all -- hash-based URLs for Ajax applications are critical. They're just not something that you want to structure your _entire_ application around, and Sammy is a framework that uses faux-URLs as the central abstraction.

All I'm saying is that you don't need to ape the server-side paradigm (one URL, one page) quite so closely. Use a module that gives you URL setting and tracking with "onhashchange", by all means, but you don't have to shoehorn your application into it.

<comment><USER22> Thanks for that example. I was just wondering if you use any library for parsing the hash based URLs (another nice thing that sammy provides) ?

<comment><AUTHOR> We use something along these lines...

<http://gist.github.com/624773>

</comment></comment></comment></comment></comment></comment><comment><USER23> Awesome! Any plans to integrate this with coffeescript? I see someone's already taken a stab at it: <http://gist.github.com/625893>

<comment><AUTHOR> It's just JavaScript, so they should work together fine:

        Todo = Backbone.Model.extend

          done: ->
            @get("status") is "done"

          toggle: ->
            @set status: if @done() then "active" else "done"

Doing a quick Backbone.coffee version would be fun though, hmm...

<comment><USER23> TBH I was a little surprised that you didn't write this in coffee in the first place :-)

Isn't this the kind of thing that coffee would be aimed at?

<comment><AUTHOR> Yes, and the CoffeeScript version would probably perform marginally faster. But pragmatism comes first. Because this is a part of a grant-funded project, it needs to be available and accessible to the widest possible audience -- and that means JavaScript.

</comment></comment></comment></comment><comment><USER24> Great framework, terrifying logo.

</comment><comment><USER25> Currently in the process of refactoring a webapp with a lot of jQuery soup and this looks a lot like how I envisioned my client side models. Looks great!

</comment><comment><USER26> Maybe I missed it but is there any kind of offline support? Ie. What happens when update to server fails? Or it's completely up to the developer?

<comment><AUTHOR> Offline support is up to the developer -- there are many applications where the notion doesn't make sense: imagine an offline Google or and offline Twitter. But having nice logical models certainly helps organize an offline application.

The first thing you would do is override Backbone.sync to save your models to Local Storage instead of the server:

<http://documentcloud.github.com/backbone/#Sync>

</comment></comment><comment><USER27> This looks really nice! I always struggle with my client-side js becoming a tangled DOM mess, and this looks like it could push me towards a much saner path.

</comment><comment><USER28> Another approach, though more alpha in code/docs:

<http://github.com/jvinet/jive>

</comment><comment><USER29> This is not specifically about Backbone. How do you provide content for web crawlers/searchbots with a client framework?

<comment><AUTHOR> Frequently, JavaScript-heavy applications are for individual use: think GMail, Basecamp, or Google Docs -- none of which need to be indexed by web crawlers.

But if you are doing something public facing, you can get it indexed by Google with a little elbow grease:

[http://code.google.com/web/ajaxcrawling/docs/getting- started...](http://code.google.com/web/ajaxcrawling/docs/getting-started.html)

"New Twitter" takes this approach.

</comment></comment><comment><USER30> Any plans for a tutorial down the road?

<comment><AUTHOR> Yes, there is. We're working on one actively.

</comment></comment><comment><USER31> At the risk of making a faux pas, could anyone share the .js and .html files of a very simple example?

</comment><comment><USER32> This appears to be similar to choco.js (sammy.js, jim, js-model) but much more lightweight?

</comment><comment><AUTHOR> Just pushed out a minor update for Backbone 0.1.1: If you define an "initialize" function in your Model, View or Collection class, it will be now invoked when an instance is created.

</comment><comment><USER33> Would it be possible to use this with Google App Engine?

<comment><AUTHOR> It's just a JavaScript library -- I don't see why not.</comment></comment>


Ask HN: Practical aspects of code-reading for study? - <AUTHOR>

I am a noob, and I have read plenty of advice on reading code - read good code and let it sink in like listening to an orchestral recording, read code with bad style and refactor it mentally, print code and annotate it, etc. I know there is tons of code to read, and absent of further guidance I should just dive in and get started. But in the interest of learning: How do you find "good" code to read? How interactive should the reading be? Are there ways to find a progression of stuff to read - simple to complex? Know of any "reading lists" to follow, maybe even a code-reading curricular plan? Thanks in advance. <comment><USER2> In the interactive part it helped me to use debuggers.

It's a bit harder with callback heavy languages, but it helped me big times when I started developing.

With a good IDE and a debugger, you can read while you step through and actually see whats happening.

Also it helped to ask the devs of the code on their architectural design decisions. Especially in the Java sphere where many design patterns get used and you find yourself often "Why didn't she code this more straight forward?!"</comment>


Unsafe Lead Levels in Tap Water Not Limited to Flint - <AUTHOR> http://www.nytimes.com/2016/02/09/us/regulatory-gaps-leave-unsafe-lead-levels-in-water-nationwide.html <comment><USER2> Ancient Romans knew that lead water pipes were unhealthy and so they preferred terracotta pipes. It's hard to imagine why the U.S. ever had a single lead water pipe, let alone huge numbers of them.

Updated with a reference for the downvoters:

 _Sheets of lead were used to line Roman aqueducts (as was cement) and lead pipes to convey water. But lead also was known to be unwholesome and, for that reason, pipes made of clay were preferred—as Vitruvius, who wrote during the time of Augustus, explains.

"Water conducted through earthen pipes is more wholesome than that through lead; indeed that conveyed in lead must be injurious, because from it white lead [ceruse or lead carbonate, PbCO3] is obtained, and this is said to be injurious to the human system. Hence, if what is generated from it is pernicious, there can be no doubt that itself cannot be a wholesome body. This may be verified by observing the workers in lead, who are of a pallid colour; for in casting lead, the fumes from it fixing on the different members, and daily burning them, destroy the vigour of the blood; water should therefore on no account be conducted in leaden pipes if we are desirous that it should be wholesome. That the flavour of that conveyed in earthen pipes is better, is shewn at our daily meals, for all those whose tables are furnished with silver vessels, nevertheless use those made of earth, from the purity of the flavour being preserved in them" (VIII.6.10-11)._

[http://penelope.uchicago.edu/~grout/encyclopaedia_romana/win...](http://penelope.uchicago.edu/~grout/encyclopaedia_romana/wine/leadpoisoning.html)

<comment><USER3> > _Ancient Romans knew that lead water pipes were unhealthy and so they > preferred terracotta pipes._

Do you have a source for this? At the same time, they used lead acetate to sweeten wine and food, which is way more dangerous. If they made the weaker connection between health and water pipes, they should have been able to connect health and this as well.

<comment><USER4> Read Vitrivius's De Architectura, specifically the part on building aqueducts. It's basically a manual for "So the Emperor told you to build a city from scratch…"

As a random aside, he also includes a proof that the water level mechanism he prefers works even though the Earth is a (roughly) spherical object and not a flat plane.

<comment> bjelkeman-again Couldn't find it in the book itself, but it is referenced here. [https://en.wikipedia.org/wiki/De_architectura#Materials](https://en.wikipedia.org/wiki/De_architectura#Materials)

<comment><USER5> Seems health and taste both dissuaded use.

 _For when lead is smelted in casting the fumes from it settle upon their members and day after day burn out and take away all the virtues of the blood from their limbs Hence water ought by no means to be conducted in lead pipes if we want to have it wholesome That the taste is better when it comes from clay pipes may be proved by everyday life for though our tables are loaded with silver vessels yet everybody uses earthenware for the sake of purity of taste._

Google books, pg 246 [https://goo.gl/g19bhQ](https://goo.gl/g19bhQ)

</comment></comment></comment></comment></comment><comment><USER6> Nassim Taleb taught me that the absence of known danger is not the same as safety.

Repeat that to yourself the next time you read your cities water report. It's not "I know my water is safe" it's "I know I don't face these specific dangers. What else could be in the water that they don't test for?"

Look into reverse osmosis.

<comment><USER7> Drinking distilled water is not a good idea, which is basically what reverse osmosis gives you stripping out essential minerals. It's also an incredibly wasteful 4:1 process.

Look into three stage filtering, that's all you need.

<comment><USER8> Carbon filtration will not remove dissolved solids (lead, heavy metals) without added ion-exchange resins. RO is generally the most effective broad spectrum treatment method for drinking water.

The mineral issue is pretty much bunk, if the lack of trace minerals in your water is a concern you need to see a doctor. Food is your body's preferred source of minerals, you have to drink _a lot_ to have much effect on your daily mineral intake.

The waste water is real, 4 to 1 is about average, it can be improved but it's pretty unavoidable in the process. They are water using appliances, we use 20-40 gallons of water to wash a load of clothes and most people don't think twice about it.

<comment><USER9> I'm a water treatment engineer. Carbon filtration will remove dissolved solids and heavy metals, and with high efficiency if done correctly. Most GAC columns designed to remove lead and heavy metals are enriched with sulfide ions to increase the removal efficiency as well. People prefer to use ion exchange resins to remove heavy metals because they can be regenerated more easily and efficiently. Also ion exchange resins allow you to recover any absorbed minerals.

[http://ascelibrary.org/doi/10.1061/%28ASCE%290733-9372%28199...](http://ascelibrary.org/doi/10.1061/%28ASCE%290733-9372%281994%29120%3A2%28416%29)

<comment><USER10> Carbon filtration doesn't remove fluoride, does it? (Or arsenic) The only water filtration system I have found that removes fluoride is the Berkey brand. I've not seen any filtration systems in the stores even mentioning fluoride or arsenic.

Berkey filters [http://bit.ly/1WdBneq](http://bit.ly/1WdBneq) (Edit to add arsenic and Berkey link)

</comment></comment></comment></comment></comment><comment><USER11> If it's a wider problem than just Flint, why isn't it being presented as such by and large in the media and by politicians? Is it because that's where the political shenanigans were caught first, when will it get addressed as the more wide-spread problem it appears to be? Or is it more like an opportunist PR stunt [i.e. "we care a lot about..."] or the issue du jour which they'll abandon after they "address" Flint?

<comment><USER12> Flint was 100% avoidable and motivated by greed.

<comment><USER13> 100% avoidable, yes. _Motivated by greed?_ No, by indifference. No decision- maker here made a decision about the water supply in order to make money off of it.

<comment><USER14> If it was not to make money, then it was to save money due to increasing budget constraints. Allowing the poisoning of the water supply due to budget constraints is some third world shit.

<comment><USER15> They didn't realize what would happen. The Flint River water is safe to drink - it only becomes unsafe when you run it through lead pipes.

<comment><USER14> Then why did they choose to use lead pipes?

<comment><USER15> Because it's cheap, easy to work with, and tends to deform under stress instead of breaking.

These were old pipes - they pre-date this problem by decades.

<comment><USER14> Did they know that they were lead pipes before they decided to start using them?

<comment><USER16> Water companies should be measuring the quality of the water going into the pipes, but they also need to sample the water that comes out of the pipes - they need to collect samples coming out of people's faucets.

<comment><USER14> Sad that such an obvious thing needs to be stated.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> My hardware guy and I are building fun weekend projects.

What raspberry pi sensors can you use to figure out if our water is polluted, etc.

It should be noticeable, I'm just asking for basic testing. Nothing too crazy.

<comment><USER13> You're thinking inside a rather small box here with the Raspberry Pi angle. Usually detecting harmful chemicals dissolved in water involves running some chemical reactions that consume their reagents, not just some reusable electronics.

There's a reason the kits are like this, instead of an electronic device: [http://www.amazon.com/First-Alert-WT1-Drinking- Water/dp/B000...](http://www.amazon.com/First-Alert-WT1-Drinking- Water/dp/B000FBMAVQ/)

</comment></comment><comment><USER18> If I travel to the US (Let's say Los Angeles, Seattle and New York) how do I know if the tap water is safe to drink? In the EU I normally just ask a local. In Germany it's safe to drink but in the Czech Republic you might be better off drinking bottled water (last time I checked was ~10 years ago, sorry if it's changed).

<comment><USER19> Default to not drinking from taps directly. I live in Seattle. We only use tap water after filtering with a Brita Filter for drinking and cooking. When we are outside home, we either bring filtered water with us or buy bottled water. At restaurants, if in doubt we ask if water is directly from tap.

Since the flint lead news, I have been exploring how can I get my tap water tested on a routine.

<comment><USER20> Ridiculous. First off, evidence that Seattle water is unsafe?

Second, brita is useless [http://www.naturalnews.com/046536_water_filters_heavy_metals...](http://www.naturalnews.com/046536_water_filters_heavy_metals_lab_results.html)

<comment><USER21> you're using 'natural news' as a source? this is one of the worst sites on the internet, literally riddled with unproven BS.

</comment></comment></comment></comment><comment><USER22> You can cheap water testing kits on amazon which you can use. Also not water is 100% safe and it makes a lot of sense to diversify the water sources. We use tap water for cooking, bottled water from different brands at home and workplace. That probably is good enough.

I have tested water at home for lead and pesticides and found it to be safe to drink. [Sunnyvale]

<comment><USER23> Google says 45% of bottled water is just tap water. I will never understand this phenomena.

<comment><USER24> As I currently don't have a Soda Stream, I'd pay for sparkling tap water. I don't think sparkling water can explain all those percentage points, but maybe some of them?

<comment> nommm-nommm That is actually just referring to regular old non-carbonated water, not sparkling water (or as we call it around here, seltzer).</comment></comment></comment></comment>


Many Eyes on Rails - <AUTHOR> http://manyeyes.alphaworks.ibm.com/blog/2008/11/21/many-eyes-on-rails/ <comment><USER2> Rails is five years old and based on proven patterns that are much older. I can't believe posts like this still get written.

</comment><comment><USER3> I'd be interested to hear the advantages Rails offers for data visualization compared to, say, a .NET-based site. Are there any Rails features that help deliver client-side-heavy content?

</comment><comment><USER4> Cool but I wished the post gave more details.</comment>


Wall Street Analysts Turn to Clickbait - <AUTHOR> https://www.bloomberg.com/news/articles/2016-12-04/boring-wall-street-analyst-notes-are-out-bold-and-funny-are-in <comment><USER2> I think the title of this HN link is clickbait itself...it's _metaclickbait_</comment>


Redox: A Rust Operating System - <AUTHOR> https://github.com/jackpot51/redox <comment><USER2> Can we just have a really great/secure micro-kernel written in rust first?</comment>


YouTube is down - <AUTHOR> http://www.theverge.com/2013/11/18/5119658/youtube-is-down-hard

<comment><AUTHOR> Here's a screenshot [http://i.imgur.com/1MdZSPB.png](http://i.imgur.com/1MdZSPB.png)

</comment><comment><USER2> Anyone able to decode/make sense of the error string?

</comment><comment><USER3> it's an autopost? How did you have find the time to write immediately this post?

</comment><comment><USER4> its back now</comment>


AWS Ground Station – Ingest and Process Data from Orbiting Satellites - <AUTHOR> https://aws.amazon.com/fr/blogs/aws/aws-ground-station-ingest-and-process-data-from-orbiting-satellites/ <comment><USER2> (It's helpful for the first post to be a _summary,_ right?)

"Instead of building your own ground station or entering in to a long-term contract, you can make use of AWS Ground Station on an as-needed, pay-as-you- go basis... You don't need to build or maintain antennas, and can focus on your work or research.

"We're starting out with a pair of ground stations today, and will have 12 in operation by mid-2019. ... [T]he raw analog data from the satellite is processed ... and routed to an EC2 instance that is responsible for doing the signal processing to turn it into a byte stream."</comment>


China backtracked on almost all aspects of U.S. trade deal – sources - <AUTHOR> https://www.reuters.com/article/us-usa-trade-china-backtracking-exclusiv-idUSKCN1SE0WJ <comment><USER2> Although I don't agree with the way the trade war was started or handled, being firm on these demands with China is one thing I do give the Trump administration credit for. Previous administrations have mostly just turned a blind eye and hoped for the best while ignoring all the downsides of an unfair playing field with China (aside from military containment strategies).</comment>


Wi-Fi security has been breached, say researchers - <AUTHOR> https://www.theverge.com/2017/10/16/16481136/wpa2-wi-fi-krack-vulnerability <comment> n-gauge So - if you notice that the web site your visiting isn't https anymore when you have a login screen over wifi, you have been compromised.

</comment><comment><USER2> Technical details here:

[https://www.krackattacks.com/](https://www.krackattacks.com/)</comment>


Nearly three dozen Android apps infected with malware - <AUTHOR> http://blog.mylookout.com/2011/05/security-alert-droiddreamlight-new-malware-from-the-developers-of-droiddream/

<comment><USER2> Now updated to increase the total number of apps with malware to 34.

<comment><USER3> Users affected matters, apps infected doesn't.

</comment></comment><comment><USER4> Time to buy an iPhone. Curated app store not so bad afterall..</comment>


The Green Light Bundle - <AUTHOR> http://www.thegreenlightbundle.com/

<comment><USER2> At the risk of asking a stupid question - this isn't related to the Humble Bundles is it?

</comment><comment><USER3> The site doesn't seem to let you zoom on ios, and is zoomed in quite far by default.</comment>


Robocalls Flooding Your Cellphone? Here's How to Stop Them - <AUTHOR> https://www.nytimes.com/2017/05/11/smarter-living/stop-robocalls.html <comment><USER2> Well, blocking them, reporting them to FCC, etc will not make them stop. I've done all of that. The only way to stop them is maybe through legal approach. If we could track down those numbers and find out who's calling, it's better for us to file a lawsuit against them. I read this at [http://www.whycall.me/news/my-4500-payday-from-a- telemarkete...](http://www.whycall.me/news/my-4500-payday-from-a- telemarketer/) that we could sue a company who violates the TCPA.</comment>


Designing better file organization around tags, not hierarchies (2017) - <AUTHOR> https://www.nayuki.io/page/designing-better-file-organization-around-tags-not-hierarchies <comment><USER2> Hello everyone, thank you for all the comments. Seeing this on the HN front page caught me by surprise. In the past year I shared this article publicly (Reddit) and privately (with tech-savvy acquaintances) for comment, and the general sentiment I received was that these ideas were not ready to be read by a mass audience. The article is way too long and pulls in many disparate ideas; it explains both why traditional features are problematic and how new features would work better. In the end, it is unclear what a real implementation would look like, and what concrete benefits and annoyances would come out of real-world usage. I was hoping to build an ugly prototype before asking for feedback.

Regarding the comments on this HN thread, it seems the general discussion is around tagging. This is indeed the title of the article and the main idea that motivated my exploration, but I believe the other ideas are just as important. I explored notions like no-filenames, strong preference for hash addressing and references, location independence, immutability, backups and deduplication, preference for external (non-embedded) file metadata, first- class media libraries, and more.

I think the debate about tagging is quite adequate, and would be happy to hear comments about the other features/non-features, and whether all the ideas fit or don't fit cohesively as a system.

<comment><USER3> Hello, I am trying to make an application for my wife to manage embroideries. I encounter almost all your issues. My wife has thousands of embroideries downloaded from internet. There are many duplicates (filenames not unique because of internationalisation and special characters). She needs to add tags to help search. She also needs groups of tags (tiger belongs to animals, ...). She also has metadata (origin of the file, which license applies to which file). Sometimes she modifies an embroidery. She needs to ensure that the original file is not modified and to keep a link between the two files. Sometimes there are groups of embroideries (for example letters) or there is documentation attached to an embroidery. It is a mess and I think that your work would help a lot to handle this kind of use cases. The current paradigm of directory tree is outdated and something smarter could be done.

<comment><USER4> How large are these embroidery files? I feel like this is something that a SQL database might be able to help with. It's probably not an ideal solution, but remember that a filesystem is really nothing more than a database, which organizes files on-disk in a particular way and indexes them so that you can find them. It obviously has serious limitations due to its hierarchical nature, which is why relational databases were invented, so using existing tools it's probably quite feasible to create an application that uses a SQL database to index all the embroidery files and store all this data on them (license, whether it's a derivative of another one, and tags), and then if they're too large to just store in the DB itself, just point to files in the regular filesystem.

</comment></comment></comment><comment><USER5> This is good thinking, and mostly overlaps with what I've tried to do a few times, so I would love to see it finally happen somehow.

For example, the Newton storage system I worked on at Apple 1990-1996 was based on separating organization from storage so we could have multiple (tag and/or hierarchy) organization systems. That eventually became the "soup" system in the shipping Newton OS, where objects were retrieved by content rather than hierarchy.

More recently, I spent some painful years working on Microsoft's WinFS, which had a ton of overlap with the principles here, and demonstrates just how hard it is to go from some nice principles that all seem like the Right Thing to an actual successful adopted implementation of the principles.

<comment><USER6> I think most people would agree a tag filesystem or a similar concept is a great idea. I have wanted one myself for a long time. Yet, for some reason, it doesn't take off. Do you think it is just a problem of implementation?

<comment><USER7> A few things I can see off the top of my head that would need to be solved for tag filesystems to be able to take off:

1\. Inertia. Most software assumes hierarchical filesystems, and assumes it can control some portion of that hierarchy. This includes things ranging from search paths for various things ($PATH for binaries, library paths, etc), temporary files, preferences files for applications, assumptions made about hierarchical filesystems in archive formats like ZIP and TAR, etc.

2\. Permissions. With a hierarchical filesystem, you can apply permissions on higher levels of the hierarchy to control access to lower levels, and you can have various forms of permission inheritance to control permissions on new files. Need a design for how to do that on tag filesystems.

3\. Mounting. Filesystems come and go; some are on your OS drive, some are on removable media, some are network filesystems. Hierarchical filesystems means that each one has a single root, and it's easy to tell where the boundaries are.

4\. Tagging taxonomy. What kinds of tags do you use? What happens if you mount a filesystem in which someone else used a different tagging taxonomy than you used? Who controls different parts of the tag space? What happens if you import an archive of material which uses a different tagging scheme than you use?

5\. Projects. How do you group files of different types with different tags into discrete projects? How do you bundle related files together? How often would you want to see files by arbitrary tag lumped together, rather than looking in particular projects that have a pre-defined structure?

6\. UI. How do you browse tags? How do you refine down? In many cases, rather than a general purpose tags based interface, you actually want media-specific browsers, like ones specialized for music which let you browse by artist, album, playlist, etc, or photo galleries that can show you previews of the photos, or video browsers which can show projects, bins, and sequences (for video editing), or IDEs which can either show you file hierarchy or allow you to browse by class, function, etc.

7\. And finally, why is a tag-based filesystem necessary for this? What is wrong with the current approach, in which there are special purpose applications which can index, tag, and display certain types of media in certain ways? For instance, you can use your text editor or IDE to navigate among files within development projects, iTunes or Play Music or whatever to browse your music, Lightroom or Darktable or iPhoto or Google Photos to manage your pictures, iMovie or Final Cut or Premiere or Avid for browsing and managing your video, and so on. They all frequently have some way of tagging files, but also have specialized UIs for browsing the specific types of files they are defined for without having to do explicit tagging, and the actual files are just stored on a normal hierarchical filesystem.

There are a couple of good thoughts in the original post, but a lot is handwaved away, such as mutability of files, which is an incredibly important use case, for a huge amount of what files are used for today. Lots of people have brought up the idea of making tag based or database based filesystems, such as the failed WinFS effort ([https://en.wikipedia.org/wiki/WinFS](https://en.wikipedia.org/wiki/WinFS)), but it's actually a pretty big problem to solve.

<comment><USER8> I think many of these issues can be addressed with mechanisms proposed by the author.

Mainly, the more complex tags which can themselves refer to other tags.

1\. This is probably the trickiest one. You may be able to do some sort of translation between a hierarchical system and the tag system using tags themselves. You could have a series of tags that refer to each other, such that the hierarchical location is essentially encoded in the tags themselves.

2\. Again, maybe just special tags?

3\. Yeah, again, tags. Just tag the thing with the media it's on.

4\. Aside from the basic UI side of things which should help, there is the idea of shared tagging systems. I don't recall if that came from the author or another commenter on HN. And you can basically ask the same question about hierarchical systems. It's not exactly a solved problem there either.

5\. Again, the complex tags. Just make a tag for the project.

6\. Obviously UI is a big question. I'm not sure how it relates so much to media-specific browsers though. They basically present a different view of a section of a filesystem. You have to do some work to let them do this, or else use a system like iTunes and buy all of your media through them.

7\. Although I feel this is well addressed by the author, one thing I think you aren't considering is that each of these applications requires their own setup in order to provide that view. You often can't just take the directory from one of these programs and use a different program to view it and have it all work properly. If you only have one program for each media type and never want to use anything else that works, sort of. Many years ago I directed iTunes to redo the file layout for my music collection and rendered it effectively useless for direct browsing. I never really recovered from that due to the time involved to sort it out.

And mutability isn't totally handwaved away, again with the complex tag system you could tag mutated works with a reference back to the original. This doesn't cover the case where you don't wish to retain the original, but then you could just do a simple find/replace with the old and new hashes in the simplest case.

<comment><USER7> I think you're missing some of the subtleties of solving these problems using "just more tags."

In a hierarchical system, a lot of these organizational issues are local. If I have one directory that consists of a project organized one way, and another directory that consists of a different project organized a different way, those different organizations don't really interact with each other in any way.

If you are using tags for everything, in order to avoid weird mishmashes of different ways of using tags, you would need to either have a completely standardized tagging system that everything used consistently, or you'd have to always include various contextual information in your queries or in your browsing in order for the queries to make sense. For instance: [mount: my- hd][project: my-project][type: jpeg]

I think you overstate the problem with different applications as well. For a large amount of the metadata that is relevant for these applications, there is a standard tagging system. ID3 for music, EXIF for images, XMP for various image and video formats. It's true that there is some metadata that these applications store in proprietary databases, but that's mostly an issue of it being difficult to come to a consensus on standards that meet everyone's needs, and it's easier to just write some proprietary metadata somewhere. With tagging systems, if there wasn't agreement on the schema of tags, you'd still have the same issue.

I don't think it's a bad idea to consider alternatives that are more general and more flexible than what we're doing now, but I do think that it's pretty easy to handwave about how nice a tag based system would be, but a lot harder to solve all of the little problems that are going to come up and turn it into a real, coherent, working whole, and then getting enough critical mass so that it is used outside of a small niche with a handful of applications.

<comment><USER8> I'm sure you're right that there are a lot of overlooked subtleties. That said I'm not sure some of those problems you mentioned would exist, or at least I'm not sure they would be any worse with a tag system than a hierarchical one.

For example, how is that example query any worse than the current situation? Right now you'd navigate to the project directory (requires specifying more than your example already) and then use some search method depending on OS/WM/etc. And then you still end up with a big list of jpegs to look through. This is sort of a worst-case example for both systems, and still I think the tag system comes out ahead here - by a little - just because it would give you the ability to spread the project across multiple drives without requiring you to do two searches if you don't know which drive the desired image is on. You can improve the situation for either system by manually specifying more information. Put better tags on the images or put them in more specific directories or title them.

As for specific applications, it's not the metadata encoded into the files that I'm talking about. It's as simple as the directory structure itself that is used to store all of this. I can't have one application organize everything and then trivially point another application at the directory and have it work.

With a tag-based system this starts to change. I don't need to tell a new music player where my music is, and then go through whatever process is needed to let it properly work with the current directory organization. At worst I tell it which tags to include or perhaps exclude. From there many options exist. Maybe it pulls in metadata from the files themselves. Maybe I provide an external file in whatever format. Maybe I tell it which tags to associate with which fields. You could do a lot of things here.

I also won't end up telling the application to reorganize things as I did many years ago with iTunes, which promptly made it nearly impossible to wade through my music manually. I had it sort everything into directories based on the artist with subdirectories for albums. It sounded great, until I remembered just how much music I had off OCRemix, where an album is a large collaboration between many people. All of those albums were ripped apart. Ironically, I also had some standardization issues with things like artist names which caused more trouble. Once I stopped using iTunes I basically abandoned that collection because of the work required to fix it.

Yeah, standardization is going to be sort of a problem, but I don't think it's quite as big of a deal as you think. For one, the OS is going to ship with a bunch of standard tags just for itself to work. There will also just be a lot of really standard stuff people are interested in that can be shipped with them. You also have file extentions, for both specific extentions and also generally what kind of information they contain. And finally there is just good old translations. The hierarchical system basically utilizes all these methods and suffers from the same problem - namely you can put directories wherever you want and name them whatever you want. Same problem, different manifestation.

I think the biggest benefit would come from a system that can present itself either hierarchically or tag-based. They both have merits. I've already presented some ideas on how you could store the hierarchical structure in the tags. I'm not so sure how you store the tags in a hierarchical system directly. You could probably fake it with a separate datastore easily enough though.

Finally, when did this discussion of general design goals turn into one of a real-world implementation, much less widespread adoption? I'm not sure how this is relevant.

</comment></comment></comment></comment></comment></comment><comment><USER9> Have you considered a file system organized as a timeline that _also_ supports tagging?

I find one of the key concepts that's not a first-class concept is _when_ the file was modified. Rather than a file-and-folder physical analogy for the file system UI, I think a timeline-oriented UI could present some advantages for the way that humans actually think and work. Tags would be a helpful orthogonal organization scheme, but I don't think they work as a primary UI for navigation.

This is great work, though! I love the compilation of various other works, the references, and the way you've dug into the details!

<comment><USER2> I have thought about file tagging for over a decade, before setting out to write the article. But a timeline-oriented file organization only came to my awareness near the end of writing the first draft.

A year has passed since I wrote the article, and the idea of timeline presentation has grown on me a lot. Especially because I use numerous data systems daily that are already time-oriented: Every chat program, email, Twitter, Facebook personal profile timeline, the "recent documents" view in major popular applications like Microsoft Office or Adobe Reader.

You should find this blurb in my article helpful:

> The Lifestreams Software Architecture > [http://www.cs.yale.edu/homes/freeman/dissertation/etf.pdf](http://www.cs.yale.edu/homes/freeman/dissertation/etf.pdf) > (185 pages)

> Comprehensively designs and tests a system for workflow and archival, based > on chronological presentation plus keyword and attribute filtering.

Thanks for your compliments on the thoroughness of my exposition. I did a lot of research and thinking as preparation for writing. I wanted to see how other people viewed the problem of file organization and what kind of solutions they proposed. I wanted to find weaknesses in my arguments and to avoid repeating unnecessary work, and of course I wanted to move toward the best solution.

<comment><USER10> If all you want is knowledge about when a piece of data was created and/or modified, then tags do that just fine.

</comment></comment></comment><comment><USER11> >But fundamentally, there is a mismatch between the narrowness of hierarchies and the rich structure of human knowledge, and the proposed system will not presuppose the features of HFSes.

This hits the nail on the head !

All the fileSystems I had to work with are fine as engineering tools. By that I mean using them as an engineer works just fine, their own implementation is off topic.

As a user though.

What the hell !

I don't want to go to c:/users/me/documents/talks/stockholm2018/draft3

I just want to open my document !

I really hope that someday we expose a document based filesystem to the user.

The underlying implementation does not matter, we can always add a layer on top of the hierarchical file system.

I just want to be able to display :

-all the games installed on my system .

-all the pictures

-all of my text documents .

-all of my pictures of Paris

etc

<comment><USER12> How do you make sure things are tagged properly though? A file must reside within a folder, even if it's a default location, which forces a user to think about the folder where the file is stored. With tags, a user could very easily forget one tag on a file, and now any filtering on that tag is never going to be aware of the new file existing.

What if you find another picture somewhere from your <Paris> trip, but you forget to add the <Travel> tag? Or you have a tag for "cool architecture pics" or something and you miss tagging one of your Paris pics with this when you upload?

There just seems like so much friction in properly keeping tags organized, despite how much extremely better the "read" UI is for someone browsing or searching file collections.

<comment><USER13> Manual tags require a lot of curation and upkeep, but some "tags" are really just restatements of attributes or facts about a file, like search filters, e.g. ("Pictures downloaded from the web on 2018-04-05", "Files created during installation of World of Warcraft", "Files opened in the last two weeks").

In fact, a tag-based document filesystem is largely useless without powerful search, where tag keys and values can be searched at will.

<comment><USER14> I suppose I should mention that while MacOS search is far from perfect, Apple put quite a lot of working into this kind of autotagging.

Here's one a screenshot showing a tiny portion of the tags available in search [https://imgur.com/a/AYm5V](https://imgur.com/a/AYm5V)

</comment></comment></comment></comment><comment><USER15> Every time I read an article about attempts at non-hierarchical filesystems, I try to figure out how I'd take the huge piles of stuff I generate when I'm drawing (and publishing) a graphic novel and reorganize it under tags. It's never pretty.

Like, okay, sure, I tag everything with the name of the project, that's a no- brainer. But if I just do that then I get the hundreds of files I generate (one per page) mixed up with everything else - web-res renderings of each page, model sheets (and their source files), promotional material, stuff sent to publishers to try and convince them to deal with that part of the process, and the huge mass of files I generate for each book I print (which can be more than one for a single multi-year project): source files tweaked for print, print-res renderings thereof, files for the kickstarter for each book... So I tag all of these attributes too, and imagining putting all these tags _on_ a file as I save it sure is a lot of fun, even if I imagine some sort of save requestor that keeps a list of all my previously-used tags, including ways to filter _those_ \- I don't care about any of the tags attached to my music collection or my collection of cartoon porn or my programming projects when I'm working on my comics projects, for instance, so I'd want to quickly narrow it down to just tags found in my art projects, and...

Ultimately it just starts to look like a hierarchical structure in my mind, except for the fact that I'm interacting with it by some kind of tag-filtering file browser on top of a huge filesystem that mixes everything together in a non-human-browseable structure.

<comment><USER16> I can strain my imagination and optimistically say that with a suitable UI, the tag-based file browser would be no worse than the hierarchical one — using tags to categorize your files in a not-too-coarse, not-too-fine way, and maybe even having more flexibility in how you organize and browse your files. Do you ever put information in the file name that could be a tag, or make up meaningless file names when there are few enough files related to a particular page, for example? Or put information in names that could be attached as some sort of comment or notes metadata instead? But I think you bring up some important issues when it comes to organizing your stuff.

Would it be better if, instead of having kitchen drawers and cabinets, we had a tag-based system, because, you know, the structure of human knowledge and all that? Why should I be forced to put a utensil in zero or one drawers? Actually, the cabinets and drawers system is nice because you have a sense of "place" — you can think intuitively about where an item is, where an item goes; even if where an item goes is a somewhat arbitrary choice, at least you know exactly what decisions have to be made (which drawer and which compartment in the drawer organizer, for example) to put it away. You can also do a traversal through the cabinets and drawers to see what objects are being stored and how they are being organized, and any time you open a cabinet, you are focusing on a different set of objects. Imagine if you had 10 cabinets and 10 objects in each cabinet, but you actually only have 20 objects total. Every time you open a different cabinet, you see a different 10 of those 20 items. Confusing.

I wonder if it would help to have required tags, and exclusive tags. Files with tag X must have tags of type A, B, and C, and may have other nonessential tags. X would be something like, "is a project file for some project," and A could be the type of project name tags, and so on.

</comment></comment><comment><USER17> Tags are great BUT

It's pretty important to realize that a files position is merely it's default tag (and you can tag it further with many different types of systems like extended attributes, as I think both Gnome and KDE have used at times).

Without that default tag you have a mess.

It's also important to note that Tags have a very high maintenance cost of their own.

Duplicate, inconsistently applied and redundant tags are a aggressive cancer in any of these systems.

No you can't just ignore them as they make it more and more difficult to accomplish even basic viewing /scanning over files for the system and the user.

Many many users have trouble even doing basic maintenance on their file locations (that default tag) that makes a tag based system even more prone to failure.

</comment><comment><USER18> I spent last year ruminating on this (Independently. However I find it interesting the article was published around the same time that I was voicing my ideas to a friend on this!) and toying with a few prototypes. This year I committed myself to hacking on a proper implementation (named 'libkoios' and 'koios') of it, using the Extended Filesystem Attributes. What I found interesting is that while there was a lot of prior work systems existing for _tagging_ , none of them use the extended attributes system, which to me feels like a waste. However there are problems with ext(2,3,4)'s implementation of file tags that make it difficult to store a lot of data without compression (I'm storing one bit per tag, which allows fast masking and comparison operations per file), so I guess that is understandable.

I believe that for image-based systems there is 8ch's /hydrus/ (probably the only good thing to come out of the chan-networks). One upshot of there being existing network sharing systems for tags is that it should be possible to scrape them when autotagging things (Nobody. NOBODY, wants to manually tag hundreds of photo memes, which is the main forseeable problem with file tagging).

</comment><comment><USER19> I never personally used it, but I've heard the BeFS was designed to have significant non-hierarchical use cases:

[https://en.wikipedia.org/wiki/Be_File_System](https://en.wikipedia.org/wiki/Be_File_System):

> [BeFS] includes support for extended file attributes (metadata), with > indexing and querying characteristics to provide functionality similar to > that of a relational database.

IIRC, this was pretty hyped at the time, but they had to back away from it. I don't know if it was because if the concept was too unfamiliar to people familiar with the hierarchical paradigm or if it didn't work as well in practice as it was imagined.

There's also a book about it written by its designer and now freely available: [http://www.nobius.org/dbg/practical-file-system- design.pdf](http://www.nobius.org/dbg/practical-file-system-design.pdf)

<comment><USER2> BeFS extended file attributes are a good example to point out. I watched this excellent talk which shows the power of live queries: [https://systemswe.love/archive/minneapolis-2017/ivan- richwal...](https://systemswe.love/archive/minneapolis-2017/ivan-richwalski)

Unfortunately, file attributes is not a feature I want to see. They don't solve the problems with naming files or deduplicating files. Metadata is attached to a file, so when the file is gone the metadata is gone. I instead proposed that metadata can be freestanding, and can exist even if the main file is missing. Relevant section to read: [https://www.nayuki.io/page/designing-better-file- organizatio...](https://www.nayuki.io/page/designing-better-file-organization- around-tags-not-hierarchies#internal-vs-external-metadata)

I did read the entire book "Practical File System Design with the Be File System", but didn't find it helpful for what I was working on. I can skip the low-level bits because I'll probably build on top of a NoSQL database or something.

</comment></comment><comment><USER20> Tagging is the first step, but how do you know if you don't have overlapping or duplicate tags, say country and folk music? If you need something that fits both categories, you eventually start designing taxonomies and eventually ontologies, there's just no end to it. I think tagging is a sensible, lightweight approach, but it has limitations...

<comment><USER18> > country and folk music

Folk music has nothing to do with country.

Country was invented in the 1800s by a businessman who aimed it at primarily white Americans. Folk music has a rich history dating before the 1200s and before much of written language. Many songs of Irish and Welsh descent date before record, still being played today.

Nevertheless, to answer your question:

1) metatags, parent tags, and the like provide ways to structure tag relationships to encompass and describe what you speak of.

2) There are many networks out there that share user's tags based on file hashes. A system could scrape these networks for existing tags and autotag many items without the user's interaction (beyond initiating the autotagger). Users could also get their hands dirty and ask for only tags relating to parent categories, or something like that.

<comment><USER21> "Folk music" possibly rides on the definition of "folk", which brings in a lot of diversity if interpreted broadly.

The word "folk" also denotes recognizeable format in the context of commercial broadcasting and streaming of canned music. It basically refers to a locus roughly centered around someone crooning while strumming chords on an acoustic guitar.

<comment><USER18> > The word "folk" also denotes recognizeable format in the context of > commercial broadcasting and streaming of canned music.

That's a very american-centric view you have there.

<comment><USER21> That isn't what you might call my "view"; I'm just remarking on how it seems that a word happens to be used in a certain culture and context.

</comment></comment></comment></comment></comment><comment><USER22> I find it hard to remember if Google Docs originally had tags instead of folders. Or if I just imagined it. I can't find it through googling.

I would rather use tags than folders, but can't find good support in an operating system.

Google used to be the closest since you could use the search bar as a command line and search queries as tags. There are no folders. But they changed now that try to guess what you're looking for rather than what you type.

OSX has tags, but their search is slow and inaccurate.

The closest is I've been trying to use Gmail as an organizer with inbox infinity rather than inbox zero. Nothing organized other than tags. Using search to find anything.

</comment><comment><USER23> Google Drive used to be based around tags, not hierarchies. It was wonderful. Then as it matured and catered to more and more 'normal' people it introduced the concept of folders. The folders were initially tags 'really' \- the same file could exist in multiple folders at the same time. But they made that harder and harder, and now I think it's hierarchical folders through and through.

I miss the old days.

<comment><USER24> > as it matured and catered to more and more 'normal' people it introduced the > concept of folders

I think it has more to do with capacity growth. Relying in tag+search would be insane if most queries returned hundreds of files,

<comment><USER25> Yeah. Imagine a Google search that returned more than 100 results. Madness!

</comment></comment></comment><comment><USER16> I found this to be a great collection of insights!

My criticisms:

Organizing your files and digital "stuff" has very little to do with the "rich structure of human knowledge," to me, any more than organizing your kitchen or garage is an exercise in philosophy. The goal should be as _usable_ a system as possible, full stop. Now, the actual content of the article is extremely practically oriented, so I have no beef with that. I just think people get carried away with the idea that storing a file is "representing knowledge," and it takes them in weird directions like trying to create elaborate universal ontologies. The question is, is it easier or harder to find your files, and save your files?

Whenever a phrase like "representing knowledge" or "augmenting intelligence" comes up, it's like everyone gets a boner, and then moves on to something unrelated, like (hopefully) usability.

Mutability: Everything changes. The only way to have immutable facts is to have timestamps. Image hosting sites, message boards, etc are misleading examples of file storage because they are really means of _publishing_. When you publish something, and people link to it, there's a case for thinking of immutability as the default, though even then, most things that can be published can be retracted or edited. This comment can be edited after I publish it. I think true immutability as a default, for files as opposed to time-stamped facts, only makes sense in a very narrow domain.

</comment><comment><USER26> I've used a self-hosted Booru for all of my image sorting. It took me roughly 2 months to upload and tag 53,000 images and another week of cleaning up rarely used or redundant tags. Since I used it outside of the Artist/Series/Character hierarchies my Artist/Series/Character hierarchies refer to Topic/Subject/Details.

For example, visualizations of various algorithms would be filed under:

Topic: Computer Science / Subjects: Algorithms, Visualizations / Data: {Algorithm Name}

By personal restriction - something may only be filed under one topic, no more than three subjects, and can have as much data as is relevant. It gets stored under what I believe to be the primary subject.

The #1 problem is I add files to my filesystem without uploading and tagging them to the Booru. Also, since the only open-source Booru software I could find is quite dated/buggy, I'm often fighting the Booru for how I use it. Now that I think about it, this might be a good problem for me to solve myself.

</comment><comment><USER27> ALL available meta data should be exposed and forced into tags, categories and folders.

Move everything file-like into the file system. Make emails into folders with tagged files. Link Torrents to their files and folders. Treat zips like folders. etc

Bit more on date range sliders, colors, files as tags and 3d models here:

[https://steemit.com/filesystem/@gaby-de-wilde/how-a-file- sys...](https://steemit.com/filesystem/@gaby-de-wilde/how-a-file-system- should-be-organized)

</comment><comment><USER28> An approach I find interesting is the [Perkeep]([https://perkeep.org/](https://perkeep.org/)) or Google Drive model for post hierarchy.

Storing all files as objects and then indexing..

An interesting indexer for images would be one that groups objects by faces recognized or exif data(camera model, GPS location, lens, date, etc) Google Drive does this.

Perkeep can deal with tags, span devices, deal with permissions. Check out HN user @bradfitz

<comment><USER29> My main issue with Perkeep is that a lot of the automated tags are very limited, and adding more is not really easy-to-do. Though the last time I used it, it was known as Camlistore. So maybe things have changed (there were some pretty bad UI issues back then as well).

I do like the _idea_ that nothing is deleted and everything is stored using "permanodes" and signatures of objects defining mutations of a "permanode". The downside is that everything is so incredibly dependent on the indexer, and my experience is that if the indexer has a bug you are in a lot of trouble. Also sometimes you don't want to keep everything you made 10 years ago around -- especially if it's burning storage space that costs you money.

</comment></comment><comment><USER30> Every time I use a tagging-based system, I become more convinced that tags are what I want for almost all things, not just files.

<comment><USER31> How do you find an untagged file? Are wrongly tagged files undiscoverable?

<comment><USER2> This is more of a UI/UX question, but I thought about it on and off for months and have a partial answer. Look at how Gmail works - you have All Mail, but you also have Inbox, Sent, and your own custom labels (tags). Every message can always be found in All Mail, and you can restrict your search by date range. Similarly on an image board like Danbooru, even if you don't tag an image, it will appear on the chronological stream of every image ever uploaded to the system. So, I'm hoping the design will end up something like these two examples. You should be able to list every file, preferably in chronological order; you should also be able to exclude files that have at least one tag; and the software might have a special nag section showing you all the files that you left uncategorized.

</comment></comment></comment><comment><USER32> I've wanted something along these lines for a long time as well. I have trouble drawing hard lines and distinctions (this is pervasive; things like having a "favorite" anything, or the desire to debate what genres a song or movie fall into, are rather alien to me). This makes picking "one" place for something difficult. Because these are fine/fuzzy distinctions for me, it's also tricky to reason my way back to where I would have put something.

The biggest directory in my _document_ hierarchy is "flotsam".

I think part of the problem is that organizational tactics/schema/heuristics aren't global. We need an array of safe, high-quality tools with good system support/interfaces, and the knowledge to reason about how and when to use which. Patterns.

A stack is probably a fine way to think about organizing mail or clothes. It's probably less useful for deciding where furniture or paintings should go. A filesystem that made sense for organizing source code is probably not the best tool for organizing a movie collection or a lifetime of personal documents. Genre apparently seems like a great way to organize most of the world's movie, book, and music store/sections, but I (unless I can get someone to check the store's inventory system) never know whether what I'm looking for is out of stock or just hiding in the taxonomical hinterlands.

Search can help. Tags can help. Hierarchy can help. Metadata can help.

<comment><USER33> I don't see why one couldn't have a tag system whose scope is defined by its location in a hierarchical structure.

<comment><USER33> apparently someone thought that the notion of a tag system with a semantic scope bound to a file tree hierarchy is such a ridiculous--nay, offensive!-- idea that they had to downvote me for it, but couldn't bother to address it on the merits. The problem I see with tag based structures is that tags are global in scope, and so, tags have to mean one thing and one thing only; on the other hand, having one set of tags for a photo directory subtree, and another for my code repository makes a hell of a lot of sense to me.

<comment><USER34> I agree, though I think this kind of hierarchy should be shallow and high level, representing only typical search scope boundaries.

</comment></comment></comment></comment><comment><USER35> Just a braindump of what I don't like about tags instead of directories, no need to repay the advantages (I agree with some of them) :

\- lack of identity. Somewhat watered down in presence of hardlinks/symlinks, but still much closer to identity than a tag cloud.

\- does a file without a tag exist? It is conceptually very clear how removal of the path identity causes the actual bytes to go back into the free storage pool (wiggle a bit for hardlinks, but still pretty clear), it would be quite weird however to have files stick around based solely on secondary tags like "blue".

\- lack of a consistent threshold for tagging: tags are binary, but relevance is not. If some files are tagged close to a full text index while others are tagged following a more minimalistic approach, the combined soup will not be very useful.

\- too powerful for convenience: file creation in a hierarchical filesystem usually happens with a somewhat meaningful default. PWD, an app-wide default or an app-specific last used folder. The default sets some of the information you might put into tags, and this information is easily corrected if it was wrong, with a single operation that might be as easy as dragging to a different folder. "Is the default folder the right one for this file?" is easily discerned and corrected, a default tag cloud however would require a full mental scan to check for applicability to the new file. Every attempt of making those defaults more clever would just force even more scrutiny onto the user.

</comment><comment><USER36> [https://web.archive.org/web/20070927003401/http://www.namesy...](https://web.archive.org/web/20070927003401/http://www.namesys.com/whitepaper.html)

Hans Reiser had some good ideas.

And by the way there is a good analogy with www - originally we had just the addresses (somehow hierarchical), then we had hierarchical catalogue of Yahoo, and then quickly it became too much for that and we now rely on search.

<comment><USER37> There's a difference in searching the Internet and searching your own data. On the Internet, there's much more of everything than you'd ever need, and close to none of it is something you've created, or even seen before. So you use search to get some reasonably relevant results. The search doesn't have to be - _and isn 't_ \- complete nor correct.

On the filesystem, I'd like to know the data browser isn't hiding files from me by reporting only "top 100 relevant results", or not indexing half of it because $reasons, or not showing them because of faulty query. Being able to iterate through all files on your disk in a tree-like fashion seems like a feature.

</comment></comment><comment><USER38> I 100% agree that a tag based file system is better than a hierarchical or folder based system. The problem is that people seem to fall into one of two camps - too confused by how to make tags work, or too enamored with organizing things into folders.

</comment><comment><USER39> A way to see folders & tags is to treat folders as tags but with the property that items may only be of one folder. The same happens in biology where scientists try to classify animals into one folder; a cat is put into the mammals folder (which has multiple hierarchies of subfolders). Instead, this classification system may be much more effective if it is classified as tags instead, where items may belong to multiple folders (or categories). This solves problems like the platypus, which belongs to multiple "biological folders".

It seems that humans have a hard time reasoning with the tag concept. I'm not sure where this comes from; is it decades of working with the folder/subfolder idiom in Windows, where most people are grown up with? Is it the resemblance of the physical world where we also put documents into one folder and one folder only? Or is it our intuition to simplify matters, and therefore seemingly make things simpler to uniquely have items belong to one container? I don't know; most likely, it's all the reasons above plus a few that I didn't mention.

</comment><comment><USER2> Some relevant past discussions on similar articles: [https://news.ycombinator.com/item?id=14537650](https://news.ycombinator.com/item?id=14537650) ; [https://news.ycombinator.com/item?id=15492795](https://news.ycombinator.com/item?id=15492795)

</comment><comment><USER40> I'm going to really enjoy piecing apart the incredible detail put into this article.

[https://tmsu.org/](https://tmsu.org/) This is nice.

</comment><comment><USER41> I wouldn't really call a hierarchical FS a DAG (directed acyclic graph) because of the flaws with links you've already called out. It's not a true DAG.

Are there graph-structure filesystems?

Tagging has always seemed confusing to me because it seems like a degenerate case of a graph if your tags can't contain other tags. A graph that is only two levels deep doesn't have the flexibility of a real DAG. I'm having trouble visualizing the true correspondence between tagging and a graph structure, but I think they're pretty much the same thing _if_ you can tag tags. Does that sound right?

Finding easy fast ways to navigate graphs in various UXs (shell, file explorers, etc) is an interesting challenge. Deletion is tricky.

</comment><comment><USER42> I think this is fundamentally NOT how humans remembers things - I think we are masters in "geospatial" memory compared to abstract unrelated concepts, and "geospatial" memory is probably organised in hierarchies.

Worse, it's next to impossible to efficiently explore a large tag-cloud compared to a hierarchical structure, which means it's much harder to _learn_ about things organised in a tag-cloud compared to hierarchical tree, or a graph that is mostly tree-like.

As an example; it totally breaks the xkcd techsupport cheat sheet - you end up in the "click one at random" branch basically all the time. [https://xkcd.com/627/](https://xkcd.com/627/)

Obviously, tagging things is good too, but file systems and computers should emphasise the tree (a better one than the Windows file strucutre though)- rather than inventing a confusing cloud/fog of unrelated things.

</comment><comment><USER43> The pain with tags is the overhead of generating them and semantic drift. Likely the best solution is simply search (some smart semantic search) with ad hoc tags to help.

</comment><comment><USER44> Just use tagspaces ([https://github.com/tagspaces/tagspaces](https://github.com/tagspaces/tagspaces)).

</comment><comment><USER45> I believe many developers and designers have been annoyed by hierarchies in filesystems.

But I wanted to comment on this: > there is a mismatch between the narrowness of hierarchies and the rich structure of human knowledge

Absolutely true, this is exactly what I think annoys us most than anything, it shows us how limited hierarchies are. But at the same time, I think it's very relevant to keep in mind that our knowledge and the mental relationships we can find between ideas are very hard to make explicit and complete, like you would ideally want in a tag-based filesystem. I feel serious tagging, if manually defined, it's quite expensive if we want it to be really useful (surely, we can also consider complementary automatic tagging, like AI). Hierarchies instead, might not be very expressive, but they are very simple to use in "most" cases. So I would say we're still far from getting the best of both worlds. The problem to "solve" is information organization/structuring, and not even humans handle that ideally (we are more like, faulty, search engines with random inputs, prone to forget XD).

About the other ideas, I think they are all interesting, agree a lot with hashes usage and no-filenames, not so convinced about metadata, but haven't really thought enough about it. I don't think we can talk about the ideas fitting cohesively or not yet (but hey, I don't even think links in HFS are cohesive from any perspective), we would have to see more formal proposals for implementation and interface. This said, I hope we see more work along these lines in the future, it's a very worthy field to explore! Maybe start small, testing some of the ideas, we get a lot of design insight when we are working on the implementation.

</comment><comment><USER46> My pet peeve with tagging systems in general, but especially community-based tagging is _false negatives_. If I search using a tag, there's no guarantee at all that it will display ALL the items that qualify for it.

I'll use my favorite porn site as an example, without going into any specifics and especially linking. I just skimmed the HN Guidelines and I don't think I'm breaking any.

Suppose I try the tag #bigtits. It is highly unlikely that I will get all the pictures with women who have especially large breasts. It's because no one will review all the images and verify if the tag #bigtits applies to them. That would be very time-consuming even for the most motivated individual who uses both hands for typing. So if I were into that particular fetish, I would need to try #bigtits, then #busty, then #nicerack, #slimandbusty, #ygwbt... because each tag has its proponents, and there's definitely overlap between them. You _could_ \- and I've seen non-porn sites doing that - use a program for automatic tagging, but then in my opinion you are defeating the purpose of tagging, which is grouping things by interesting categories. Machine-generated tags tend to be lifeless.

As I've said it is a pet peeve of mine, and I will likely start a project or two to implement my fixes for a web framework or a static blog generator. I mean that I should have confidence that a tag has been considered for all content in the collection. Program-assisted tags can help, such as keeping track of what tags existed at the point when a picture was added.

Then there are almost identical tags. #cat vs #cats, #tortoise vs #turtle, #color vs #colour.

Overall, in practice, I think tagging, as usually implemented, is the most overrated feature of the Web 2.0 era.

</comment><comment><USER47> I purchased the Sony Digital Paper system (DPT-RP1) and it has possibly the most ill conceived file system design possible. All files are stored in a flat directory on the device (eg. one long list).

Users on the Sony community site are frequently looking for updates to the software. I'm curious which file organization solution, hierarchy or tags, would be easier to implement?

</comment><comment><USER48> I'm a big believer in tags. I tried to make a tag-based tool that merely relies on directory names as tags, so ~/t/.tag1/.tag2/tag3/your-file-or- directory and moves your files around so that the tag directories are always organized by tag counts.

I had the idea there would be a series of tmv tcd tls commands that work with the tag directory structure.

[https://github.com/foucist/tagmv](https://github.com/foucist/tagmv)

Warning - The regexp I'm using is likely broken, I suspect directories with .git/ or other dot-directories in them causes issues. It sometimes causes the .git/ innards to be moved out into the project directory or something like that. I never got around to fixing it.

</comment><comment><USER49> OFTN OSWG started work on a tag-based filesystem called TPFS[1] back in 2011. The Haskell source code might be useful to anyone interested in developing platforms that use tag-based filesystems.

[1]: [https://github.com/oftn-oswg/TPFS](https://github.com/oftn-oswg/TPFS)

</comment><comment><USER50> As I recently inherited a huge, well-tagged music collection (tens of terabytes of files) I am very interested in this. Is there something like it, also supporting .cue files and also storing the original filenames and structure? A mediaplayer agnostic way to access this treasure trove would be the best.

<comment><USER26> I highly highly highly highly recommend beets [0].

[0] [http://beets.io/](http://beets.io/)

<comment><USER50> Wow! Thank you.

</comment></comment></comment><comment><USER51> Since 1 June 2012, I've been taking notes in unicode text files, which contain (occasional or adjacent) lines starting with 'nb ' and then a list of tags. I wrote a simple tool ("nb") in Inferno's shell (thanks to Robert J. Ennis for the port to Plan 9's rc), to (1) search for given keywords in per-directory index files pointed to by the global index, (2) index all of the nb lines in files in the current directory, and (3) if necessary, append, to a global index file, a reference to the index file in the current directory.

[https://github.com/catenate/notabene](https://github.com/catenate/notabene)

I've found that I'm comfortable with the eventual consistency this offers, in exchange for fast lookups when I want something (as opposed to indexing first, and/or indexing globally, and so waiting for indexing to get a result). This distributed-file approach also allows me to add tags to a variety of files: local files, or networked file-system files, or sshfs-mounted files, or Dropboxed files, or files under version control, or files with varying text formats; and find tags across all of them and across all the time I've been indexing.

It runs in linear time with respect to the number of tags I've entered, plus the time to read and process the global index, so obviously there are many ways I could improve the time performance (as an easy example, I could permute the index to list all the tags in alphabetical order, and next to each tag list the files that contain that tag).

I also wrote other tools, since the layout is so simple: for example, "nbdoc", to catenate the actual contents of the references returned by the primary tool (nb); and "so" (second-order), to return all the tags which appear in any nb line with the given tag(s).

I've also found that it's not easy for me to remember what tags I might have used in the past, or how I was thinking about something, so I try to use the conjuction of several tags to narrow down search results, rather than try to remember one specific tag (this seems to correspond to the observation that it can be difficult to remember exactly where in a hierarchy you put something).

The modular approach, of per-directory indexes referenced in a global file, also makes it easy for me to combine work-specific notes, with public notes, with private notes, all in the same global index file, at work; but only have the same public and private notes at home.

</comment><comment><USER52> I wonder if Microsoft would be willing to take another shot at WinFS? It would have met most of the requirements. But the project bogged down and never shipped.

[https://en.wikipedia.org/wiki/WinFS](https://en.wikipedia.org/wiki/WinFS)

</comment><comment><USER53> git-annex might have some interesting thoughts about this: [https://git- annex.branchable.com/tips/metadata_driven_views/](https://git- annex.branchable.com/tips/metadata_driven_views/)

</comment><comment><USER54> Have you seen "Collaborative Creation of Communal Hierarchical Taxonomies in Social Tagging Systems" ? It is very relevant and if tags could be accurately machine generated we could derive organizational hierarchies from just tags.

</comment><comment><USER55> > When people realize they need to classify a file in more than one way, they > will start to use shortcuts/links to try to solve the problem. (Windows has > shortcuts, Unix has soft/symbolic links, and Mac has aliases. This is a > ubiquitous feature, but transferring shortcuts across existing platforms is > very hard.) This sounds like a reasonable solution, but will face trouble in > all but the simplest use cases.

They rule out this solution because it's not perfect, but surely if the idea had real merit this would be a serviceable test bed?

I think we have hierarchies because it's human nature to create hierarchies to make sense of the world, we try to force them into place where they don't make sense. We see it in biology, we see it in organisations and we see it in code, I'm sure most of us here have worked with examples of OO hierarchies that made no sense.

<comment><USER2> Hi. Have you tried managing a collection of shortcuts? How do you deal with recategorization, removing files, renaming files, moving files to different storage devices, etc.?

The article has a whole section illustrating why both hardlinks and softlinks won't work in the general case. [https://www.nayuki.io/page/designing-better- file-organizatio...](https://www.nayuki.io/page/designing-better-file- organization-around-tags-not-hierarchies#hard-and-soft-links-as-non-solutions)

<comment><USER55> > The article has a whole section illustrating why both hardlinks and > softlinks won't work in the general case.

Yes, that's the part I quoted.

> Have you tried managing a collection of shortcuts?

No, tagging is the bottleneck for me, which is why I don't think it will ever be useful.

> How do you deal with recategorization, removing files, renaming files, > moving files to different storage devices, etc.?

Aside from moving to different storage device, you can make a minimum viable product with a few shell scripts, one to tag a file, one to untag, one to search by tag(s), one to listen for for events (move, delete). For bonus points you could do some auto tagging from meta data.

A working implementation (even with limitations) would be a lot more convincing than all this theory.

</comment></comment></comment><comment><USER56> I wrote a paper about the same topic while in Uni about 15 years ago, and also developed a proof of concept 'filesystem' with an file explorer that uses tags. Too bad it isn't the standard in any OS yet.

<comment><USER2> Would you care to share the name or a link to your paper?

<comment><USER56> I would need to dig it up from my old harddrive, if it still exists. To be honest not sure it still exists, as I said it was a long time ago.

</comment></comment></comment><comment><USER57> I have thought about this a bit and I think if similarity hashes (probably LSH forest) were used, automatic tagging could occur with a preset of hashes.

</comment><comment><USER58> for simple use cases, folder beats everything else.

when you have a large numbers in deep path then tags should be the way to go, you will need a database to manage it for portability across OSes etc.

with tags we need isolate how-to-store-the-files from how-to-organize-them- for-easy-access, tags can be used to build a virtual folder hierarchy for example.

<comment><USER58> just curious, why the down vote? i wish I can see who down voted, is there a way to check?

<comment><USER59> What would you do with the names?

<comment><USER60> make an enemies of ausjke list probably

<comment><USER58> nope, just ask why? you're what you think I guess, I have not made any lists in my whole life and surprised you immediately guessed that.

<comment><USER59> If they wanted to say why, they would have. You've got no need to go around hassling people for information. You could be down-voted a trillion times and it wouldn't matter for shit in the world, so get over it.

</comment></comment></comment></comment></comment></comment><comment><USER61> ReiserFS flashbacks.

</comment><comment><USER62> You can pry HFS from my dead hands!

seriously, graphs are hard, and the possibility to lose data is serious.

<comment><USER62> just to clarify: not lose data in the hardware sense, but in the cognitive one.

In a tag-based fs I'm sure I will find a way to "lost" files.</comment></comment>