TMSU: a tool born out of frustration with the hierarchical nature of filesystems - <AUTHOR> http://tmsu.org/ <comment><USER4> I'm actually not frustrated with the hierarchical nature of filesystems. I'm most frustrated with the state of filesystem search these days. I don't want to tag and curate my files. I want to search them.

I've strung together something using bleve full-text search and some OCR libs to scratch my particular itch but it still doesn't quite get all there.

<comment><USER5> I agree.

I actually like the hierarchical organisation, and I don't like the 10 year usability trend, particularly driven by Microsofts attempts to patch over their horrid structure with even worse workarounds.

The solution to learning my mother were to find her documents is not hiding the place 15 levels deep and having 10 symlinks to it.

Fast, unobtrusive indexing and a good structure is all that is needed. Humans organise and memorise things in hierarchies, and while categories is another helpful abstraction they tend to be confusing if they do not point to a fixed position in a hierarchy in my opinion.

You can also explore a hierarchy; Not so with a cloud of things connected to a bunch of tags.

<comment><USER6> > Humans organise and memorise things in hierarchies,

Humans also track a lot of things based on spatial memory, which no OS since Mac Classic has even _tried_ to make use of, which is a shame.

<comment><USER7> How did Mac Classic make use of that?

<comment><USER8> In Mac Classic each folder you opened created a new window which remembered all of its settings for size, position on screen, scroll position, icon size/layout, etc. This window is an explicit and exclusive representation of that particular folder and attempting to open the same folder again does not create a new window, it simply shifts the focus to the already-open window. The change in Mac OS X towards a browser for navigating the filesystem hierarchy caused an active debate over spatial[0] vs. navigation (or browser)[1] file managers.

[0] [https://en.wikipedia.org/wiki/Spatial_file_manager](https://en.wikipedia.org/wiki/Spatial_file_manager)

[1] [https://en.wikipedia.org/wiki/File_manager#Navigational_file...](https://en.wikipedia.org/wiki/File_manager#Navigational_file_manager)

<comment><USER6> Ars did an amazing series of articles about the (IMO) missteps made in OS X, and presented a theoretical design that would satisfy both Mac Classic fans and fans of "browser-based" windowing systems.

[http://arstechnica.com/apple/2003/04/finder/2/](http://arstechnica.com/apple/2003/04/finder/2/)

Any software developer interested in designing usable software should read and digest this. Alas, nobody working at Apple did, so I'm no longer an Apple customer.

<comment><USER8> Thank you for this! I vaguely alluded to this (active debate) in my comment but I had forgotten the source. I'm going to read through this now! :)

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> Stuff like this is cool, but it really points out just how poor the filesystem is for organizing certain types of data. The "Why does TMSU not detect file moves and renames?" question in the FAQ really highlights how this isn't helped along by the filesystem at all.

One comment mentioned BFS, which had some really cool stuff. There's an Ars Technica article that touches on some of it[0].

The secret to BFS, in my mind, is that applications use it. The Haiku Mail app, as noted in the article, used the filesystem as its email database by attaching its own attributes to messages. This is also an example used in the "Practical Filesystem Design with the Be Filesystem" book[1].

Unless the metadata becomes a first class citizen in the filesystem, any attempts to layer it on top will have problems. Either applications won't understand it or normal filesystem operations will cause the metadata database to become de-synced with the filesystem data.

[0] [http://arstechnica.com/information-technology/2010/06/the- be...](http://arstechnica.com/information-technology/2010/06/the-beos- filesystem/)

[1] [http://www.letterp.com/~dbg/practical-file-system- design.pdf...](http://www.letterp.com/~dbg/practical-file-system- design.pdf#75)

<comment><USER10> > Unless the metadata becomes a first class citizen in the filesystem, any > attempts to layer it on top will have problems.

Remember when it seemed like Mac OS might give us a modern era of rampant metadata ([http://arstechnica.com/apple/2005/04/macosx-10-4/6](http://arstechnica.com/apple/2005/04/macosx-10-4/6))? Ah, those were the days.

<comment><USER9> Yup.

MacOS does an OK job of helping the user find things with Spotlight, but it's not a full metadata system like BFS had.

Mail.app, for example, keeps each message in a separate file[0] (and probably has a cache or separate database of this to make displaying mailboxes quicker). This makes it easy for Spotlight to index, but all of the stuff that you'd think of as metadata is actually just regular data inside the .emlx file.

If Apple made a huge effort to start treating the metadata (assuming the infrastructure described in the Ars Technica article still exists) as a first class citizen and using it like BeOS did, maybe we can get there. This would be a drastic rethink though. It feels like files, in some ways, are becoming second-class citizens in the Mac world. Photos, for example, are managed in the Photos app - you do not go into the filesystem and organize your photos.

One big problem with filesystem metadata is how do you transfer it? The Ars article showed a sidecar file (._filename) being created when the file was copied to a non-HFS volume. Now the metadata is detached from the file and we're back to the same problem.

[0] [http://mike.laiosa.org/2009/03/01/emlx.html](http://mike.laiosa.org/2009/03/01/emlx.html)

</comment></comment></comment><comment><USER11> A great idea! I think tags are definitely a better way to organize most personal data than trees.

Also I like that they describe what data they actually change on your computer right on the homepage: "TMSU does not alter your files in any way: they remain unchanged on disk, or on the network, wherever you put them. TMSU maintains its own database and you simply gain an additional view, which you can mount, based upon the tags you set up."

Unfortunately building on a foundation of sand (meaning not TMSU's code, but Unix filesystems) has downsides:

[https://github.com/oniony/TMSU/wiki/FAQ#why-does-tmsu-not- de...](https://github.com/oniony/TMSU/wiki/FAQ#why-does-tmsu-not-detect-file- moves-and-renames)

" Why does TMSU not detect file moves and renames?

To detect file moves/renames would require a daemon process watching the file system for changes and support from the file system for these events. As some file systems cannot provide these events (e.g. remote file systems) a universal solution cannot be offered. Such a function may be added later for those file systems that do provide file move/modification events but adding support for this to TMSU is not a priority at this time.

The current solution is to periodically use the repair command which will detect moved/renamed files and also update fingerprints for modified files. (The limitation of this is that files that are both moved/renamed and modified cannot be detected.) "

Ouch.

<comment><USER12> Hmm, seems like they could have gone the other way, throw everything into a DB, and then wrote a fuse plugin to access it all through traditional file system mechanics. That would have allowed for gating direct access such that moves and renames could be dealt with accordingly. Of course, there are other problems with that approach, but probably not as many as you might think (the file system _is_ a database, so you're really just choosing a back-end that is less likely to be directly accessed).

<comment><USER11>
        they could have gone the other way, throw everything into
        a DB, and then wrote a fuse plugin to access it all
        through traditional file system

This is the Camlistore strategy!

        Of course, there are other problems with that approach

Could you elaborate more on these? I've never worked with FUSE.

<comment><USER12> The other problems I was alluding to weren't really with FUSE, but one that does pertain to FUSE is speed, since FUSE imposes overhead through a daemon running in user space, and associated context level switches because of that. From just looking into is again, this may have been mitigated to some larger or smaller degree with some FUSE performance enhancements in 2012.

Specifically, I was referring to the different off the shelf database systems which could be used. Each will have it's own benefits and drawbacks to storing large chunks of data per-record. Benefits might include (relatively) easy sharding or replication. Drawbacks might include not being space efficient for removed files, not being as resilient to corruption due to crashes or corruption affecting more than the files in use, or overly aggressive use of memory to function efficiently.

If a custom database was developed, you could tailor to your exact needs, but then you have much more work to do, and a period of immaturity.

Off the top of my head, if I were designing a general purpose system for tagging files where people were expected to use it as a regular file system and some overhead from FUSE was acceptable, I think I would leverage the file system but in a different way. I would set up a specialized directory for the files themselves, and store then hashed within it, and have a BerkelyDB database relate filename to hash and tags, and use FUSE to do direct file access. But that's my 5 minute assessment, so I reserve the right to change it completely given someone pointing out the obvious problems. :)

</comment></comment></comment></comment><comment><USER13> This idea is rediscovered every few years in a new project. I tried this with StorageBox many years ago and even did some UX research in this context. Turns out many users don't like querying only as it they feel they can not search their data exhaustively, and might "lose" some of their data this way.

Also for not tech savvy users, folders have the nice interaction pattern of question-response via menu selection. They see something, click, see something, click, without realizing that they are navigating a folder hierarchy.

<comment><USER14> Not convinced. Why wouldn't a 'tag browser' work just like that too? Even better, since what I'm clicking on are meaningful tags, instead of 'directory names'

</comment></comment><comment><USER15> I empathize with the OP's view that there's a usability gap with out of the box file systems/namespace (short of bash/sed/awk/perl wrangling, natch), so I applaud this project.

I have a couple of points/concerns after a quick read.

1) So 'tag' is the verb that updates the records, but 'tags' is a read?

Poor taxonomy, IMHO. If I'm using "tmsu" I _know_ I'm working with tags, so I'd think natural switches would be the way to go ( tmsu add, tmsu ls|list, &.) Or at least don't make the "create" verb and "read' verb differ only by the plural 'S.'

2) Is there a way to list all existing tags? —not just the ones already bound to a file but all available in the database (with regex filtering, of course).

That's what I'd need to pick from my 'tag pallette' _before_ actually tagging so I could avoid creating synonyms accidentally that'd later require a merge.

</comment><comment><USER16> The reason hierarchical structures are used in file systems is because they are a pretty intuitive and, most importantly, _generic_ way of classifying and storing information. For the most part, just about any file you have on your computer can be stuffed into some sort of folder structure.

Specialized files like music, movies and images are a solved problem. iTunes and other software do a great job at organizing this information and making it easy to use. There's also software like Quicken for dealing with the other common mountain of data people have.

What _might_ be useful is a piece of software that is capable of extracting and managing metadata automatically. Think of a tool like iTunes that you feed it a collection of files and it uses some form of ML to extract and create a database build logical ontologies for this data. The big problem with this kind of tool is finding a large, complex dataset that an individual has, but that has not been organized by a specialized piece of software. I doubt these exists in numbers significant enough to justify creating a project.

tl;dr: Directory structures are low-effort, generic, and discoverable ways of dealing with files that are not managed by other applications. It will be hard to improve on them without sacrificing one of those three attributes.

<comment><USER14> I don't think its intuitive at all. We don't ask questions like "What parent folder did I put that document it?" We ask "Where is that document I printed yesterday. The one that I got via email."

With liberal use of tags, and the ability to browse them fluidly, we could ask those sort of questions.

<comment><USER16> > My Documents [Sort by Date]

At the top.

The way that one person manages their documents probably isn't going to be the same as another, but generally a person is consistent with all their files.

In this case, it's either a document that you have several types of, in which case you would have an existing folder structure, or it's a one-off that you dump in My Documents along with all your other one-offs. Even if you lose it, file systems all keep date-time information, so you can easily search for the all the files last modified yesterday.

The problem with liberal tagging is that it requires a bunch of up-front effort that you're never going to perform. In this case, if it were an email, you'd just use your email browser (specialized software!) to find the document again based on the things you remember about it (date, source, size, etc).

<comment><USER14> I don't remember it by date or size or maybe even source. That's just parroting what we do now. I remember that I printed it. That could easily be a tag. The tags need not be created by me; the tools could be promiscuously tagging persistent data all the time, with useful clues. I'd learn some clues, learn to use them.

That 'my documents' thing - you could easily create a 'view' on tags that yielded that result. Without relying on Microsoft or whomever to do it for you.

</comment></comment></comment></comment><comment><USER17> How is this different than something like this:

        mkdir -p ~/tags/{music,big-jazz,mp3}
        ln -s /path/to/summer.mp3 ~/tags/music/
        ln -s /path/to/summer.mp3 ~/tags/big-jazz/
        ln -s /path/to/summer.mp3 ~/tags/mp3/

From there, you can use all normal filesystem tools to interact with your 'tags'. You could extend this with a simple script that handles duplicate file names in the same tag by sticking a hash of the file before the extension. Having a separate database for this information seems unnecessary.

<comment><USER18> This supplies you with only one relation between the tags: "And Then". Ex. 'Path' (and then) 'to' (and then) 'summer.mp3'. Or 'tags' (and then) 'big- jazz'.

What you want is to also have AND, OR and NOT available. How would you find 'music' (and) 'mp3' (and not) 'big-jazz'?

<comment><USER19> I'm not saying this would be simple, or as nice as the tool given, but it's not impossible.

First, do a 'ls' directory dump of the 3 folders. Then 'cut' out the softlink destination from each file. This will end up with 3 files with soft links. From there, 'grep' the music and mp3 files together to get a list of soft links that are in both music and mp3. Then you can do an inverse 'grep' to remove softlinks that are in big jazz.

Even if the system creates hard links instead of softlinks, then the same could be done via inode numbers

</comment></comment></comment><comment><USER20> I feel like I must be missing something, but I feel like tags is just a band- aid. If you have enough files, you will end up with too many tags to manage (think of a tag directory filled with tag files, instead of a home directory filled with actual files), so you'll need hierarchical tags. Or kludge it with "tag/subtag" which looks a lot like a directory. GMail added nested tags, which seems to me like an admission that flat tags is not sufficient.

One problem is that regular people don't know how to organize a hierarchy. General -> specific works really well, but that requires the ability to generalize.

The only use I can see for tags is if you want files to be a member of more than one directory. Other than music, everything I have is generally created for a specific purpose, so tags are not particularly helpful.

</comment><comment><USER14> Its been a generation since the hierarchical file system was obsolete. Its lame to have to hang every file on the ceremonial file tree like some Christmas ornament. Hardly any app wants data organized like that.

In fact, nearly every large app does something to avoid it. They create their own representations of a log, or a mail folder, or a document, or an image (and on and on) and manage the details themselves. Because 'file systems' are so lame and underpowered.

This tool begins to help. Creating flexible groupings (tags) resembles a relational database. That's a start. I'd like to replace the OS file system with something like that.

Instead of renaming files when you bring another copy onto your persistent storage, you could just add a version tag to them. Leave the names alone! I can tell my build system (or document store, or mail tool) what version I want to deal with e.g. tag='version' value='2.5'. No collisions any more. No requirement by the 'file system' to mash them into some file tree so they can still be found, but don't collide.

In fact this system can do everything the hierarchical file system can do, and more. Just add the tag 'parent directory name' and voila! You have a file tree (if you want).

</comment><comment><USER21> So tag order is important?

i.e. `music mp3 folk` results in a different virtual file system than `folk music mp3`?

I often think of tags as an unordered set, rather than an ordered list.

<comment><USER22> An ordered list smells a bit like a hierarchy to me...

</comment></comment><comment><USER23> This is actually kind of akin to BeFS. Although BeFS had greater capabilities in some ways, being an actual filesystem. For the uninitiated, BeFS was the native filesystem of BeOS, and allowed for metadata attributes that allowed for querying and indexing capabilities akin to an relational DB. Or at least, that's what Wikipedia says.

</comment><comment><USER24> I just keep all my work-files (documents, downloads etc) in my desktop folder (it's the "top-level" folder in Windows when you Alt-Up, have it symlinked at /desktop/ too).

My default view is a detail view sorted by "date accessed" (descending) which is what I need 99% of the time. Especially handy when uploading random images, quick edits etc from the browser.

btw I highly recommend [https://pathcopycopy.codeplex.com/](https://pathcopycopy.codeplex.com/) for those that use a terminal and win explorer at the same time a lot.

</comment><comment><USER25> Why not use file attributes and provide a nice interface for managing them? (e.g. extend find to search them, etc)? A parallel database is fragile -- it can trivially get out of synch.

<comment><USER26> I thought the same thing, but in practical terms you'd need to maintain a separate db for portability anyways I guess.

</comment></comment><comment><USER27> I seem to recall reading somewhere that one of the reasons Vista was a "bad" os, was that it originally had much higher more optimistic ambitions to build a sql like file system (i suppose similar to this). It however had issues, and a decision to scrap it delayed the vista project, and reduced the scope of "cool" things it was supposed to deliver.

<comment><USER28> The thing you're thinking of was indeed supposed to be in Vista, then post- Vista, and then scrapped all together - WinFS.

[https://en.wikipedia.org/wiki/WinFS](https://en.wikipedia.org/wiki/WinFS)

<comment><USER29> It was promised for Cairo (that become win95), under the name of Object File System.

Different name, exact same promises.

</comment></comment></comment><comment><USER30> UI nitpicking:

These:

$ tmsu tag summer.mp3 music big-jazz mp3

$ tmsu tag --tags "music mp3" foo.mp3 bar.mp3

$ tmsu tag spring.mp3 year=2003

Are confusing. Very non-Unix. They should be:

$ tmsu tag music,mp3,year=2003 summer.mp3

<comment><USER31> I have to ask, how are they non-unix?

<comment><USER30> They're inconsistent. Sometimes the file name comes before something and sometimes after something. It's a lot easier to remember commands if their structure is uniform, and Unix commands usually have the general structure of "program [options] [files]"

</comment></comment></comment><comment><USER32> This is an interesting idea, but it seems like an awful lot of work on my part to go through and organize and tag every file. While some of it could be automated (pulling ID3 tags out of MP3s), a lot of it seems to depend on me figuring out good names for everything I make.

The biggest problem is that I have to figure out which tags are going to be useful to me in the future and where to add them. This is relatively easy for music (but even there can explode in complexity depending on how granular you want to be), but more difficult for things like photos or papers.

IMHO, fully general tagging systems never work because the complexity explodes as the number of potential tags increases. You need to narrow the scope down to a specific domain so your tags can be limited to human scale.

</comment><comment><USER33> Very cool attack on the problem.

I've been working on ideas for managing filesystems and tools for thought for a while off and on, it's starting to coalesce into a set of design ideas as well as prototypes.

What I won't do is set up a filesystem: I don't think that adds value to me; it mostly sounds technically complex and hard to figure out. And, I think that driving the whole business through manual tagging is a lost cause. Manual tagging can be _useful_, but actual attempts to derive semantic knowledge will be _more_ useful. I have many thousands of documents - manual tagging ain't gonna happen.

I need _semantic_ search and _semantic_ cross-referencing; something like a Xanadu or a (much better) wiki/hypertext system.

</comment><comment><USER34> You can do the same thing with git-annex: [https://git- annex.branchable.com/tips/metadata_driven_views/](https://git- annex.branchable.com/tips/metadata_driven_views/)

</comment><comment><USER35> This reminds me of BeOS's BFS, which let you do all this as part of the FS itself :)

</comment><comment><USER36> I can't believe this, but I created something like this in an ad-hoc way. I have a large amount of music, and I needed to be able to transfer them to devices (my phone) based on some sort of tag - I need work-out music, I need driving music, etc. Genres are inappropriate for this. The easiest way to transfer this to the device is have a directory full of links that point to the right files / directories.

So, depending on the need, I will either have two directories (files and tags) or a number of file directories and a tag directory. File directories can have whatever they want, and tag directories have either only tag directories (like workout, driving, etc) or soft links.

Tagging a file / directory is easy - just link to it. Untagging is just as easy. The links don't take up much space, especially next to the music. When I'm transferring the files, I either use a script to make a directory with the links replaced with their file counterparts, or I transfer with something like rsync that can do that itself.

I'm amazed how similar this project is to my own solution. It's nice to have a dedicated script for the whole thing, but the solution itself is very simple, and easy to script with.

This isn't born out of frustration - hierarchical filesystems are perfectly adequate for most tasks. But they have not been trees for a long time - <USER2> have links, which let us make any graph we want out of those trees.

</comment><comment><USER37> This is just PDM.I built a system that does all of this, more, and integrates into our company's existing products for managing manufacturing data. While my version only watches the directories you tell it to, it also detects file movement and changes via a service and through the hashing of files. I'd be more impressed if someone built this into a file system directly. Combine advanced tag based indexing with zfs and you've got something impressive.

</comment><comment><USER38> An old solution to this problem was the Logic File System [https://en.wikipedia.org/wiki/Logic_File_System](https://en.wikipedia.org/wiki/Logic_File_System) (disclaimer: I am one of the author).

</comment><comment><USER39> I looked for a tagging system months ago, did fairly extensive research into existing solutions (rather than writing a FUSE layer myself) and TMSU was the leading result. I installed it and it's all ready to use.

Today, it's still all ready to use. I haven't touched it. I'm actually quite happy with the way my filesystem works, I just had this idea how great it would be to work with tag selections instead.

The only reason I might still use a tagging system is to tag some files I want to back up manually (if at all), like a 50GB disk image or some temporary big download, but in general I create one or two symlinks a year and I'm good. The hierarchy works fine.

<comment><USER40> > in general I create one or two symlinks a year and I'm good. The hierarchy > works fine.

Same here. I have a few things that tags would be nice for, but it's infrequent enough that current filesystems are fine. Couple of examples

* Tagging the source [CD/iTunes/Amazon etc.] of my music - tags would be nice (and possibly doable as IDTags etc.) but "/music/source/artist/album" or "/music/artist/album [source]" works fine

* Multiple paths to the same file - I have various media (movies, music, books, TV shows etc.) related to a single series in one folder. Those should also be in my main music etc. folders. Again tagging with the series name to make a virtual "series" folder would be nice, but symlinks solve that and it happens infrequently enough that it isn't an issue.

Other than those edge cases, I'd say most of my data fits pretty well into a hierarchical structure.

</comment></comment><comment><USER41> Reminds me of ReiserFS's original goals.

[https://reiser4.wiki.kernel.org/index.php/Future_Vision](https://reiser4.wiki.kernel.org/index.php/Future_Vision)

</comment><comment><USER42> Seems like the Tags feature that has been in MacOS X since 10.9

<comment><USER43> Was just going to post this - you can also access them from the CLI with mdfind command, mdfind tag:jazz

<comment><USER44> [https://github.com/jdberry/tag](https://github.com/jdberry/tag)

</comment></comment></comment><comment><USER45> I could see where a tool like this would be a huge help to legal and compliance departments.

There is a frequent need to produce documents for discovery purposes. However, you generally have to review the documents or send the to outside counsel first for review before they are produced. This can take hundreds of man hours and cost thousands of dollars.

Building something on top of TMSU could be a great solution for this task.

</comment><comment><USER46> It would seems to me a better idea to order the VFS around queries - <USER3> command-line command returns a query id, which represents a directory in the VFS (such that the command is a bit like a mkdir for the VFS), the dir might then be '/foo/tmsumountpoint/<queryid>/', and contains symlinks for all files found in the query.

I'm sure FUSE can do this.

</comment><comment><USER47> Wasn't Windows 7 supposed to be built on a revolutionary new filesystem based on a rdbms? I was pretty excited about that because it would have natively enabled a lot of the features listed here. Unfortunately it was one of the features they cut when Win7 went over budget and over deadline, and they never brought it back for subsequent releases.

</comment><comment><USER48> This reminds me of something that would be useful in implementing Desktop Neo [https://news.ycombinator.com/item?id=10932378](https://news.ycombinator.com/item?id=10932378)

</comment><comment><USER49> Spotlight can do the same thing, no ? [https://en.wikipedia.org/wiki/Spotlight_%28software%29](https://en.wikipedia.org/wiki/Spotlight_%28software%29)

</comment><comment><USER50> Hey guys, you should look at diamond.io, we're building something that is a tool to help you solve this problem and the problem of organizing information in general. Check us out!

<comment><USER11> "It's backed by a powerful Artificial Intelligence."

Ahhhhhhh!

You're tackling a laudable goal, but it would help to link to a page with more technical details.

<comment><USER50> Haha, I don't want to go too much off topic of this thread, but essentially we can analyze actual file contents, source and metadata to find patterns between file structures and associate them to user and "common" labels.

On top of that we try to get further accuracy by using user information to help us find the context. We call it a Personal System, a repo of knowledge and learned preferences heavily tailored around each individual user. We're just in beta now but definitely put your email down if you want to try it out eventually!

<comment><USER11> I put my email down to spy on you:)

My personal feeling is that (A) you're totally right that we need better _personal_ organization systems, but (B) the bottom layer (tagging, schemaing, relationships) should not involve fuzzy processes but should be totally understandable by the average user.

I know (B) is a weird opinion though so I look forward to seeing how far you can get with (A) plus machine learning & whatever other tricks you guys plan:)

<comment><USER50> Absolutely, I think (B) is quite a big concern for people. While it's nice to have a fuzzy AI match things _most_ of the time, it's absolutely critical the user can always go in and override and take control of the situation.

So for us, we very clearly distinguish "a user labelled this item" vs "we guessed it was this label".

I'll keep you posted!

</comment></comment></comment></comment></comment><comment><USER51> For some reason, I knew this would be about tagging files before reading the article.

</comment><comment><USER52> > TMSU does not alter your files in any way: they remain unchanged on disk, or > on the network, wherever your put them. TMSU maintains its own database and > you simply gain an additional view, which you can mount where you like, > based upon the tags you set up.

I'm not so sure this is a great design decision. Now your tags are only in your sqlite file, and you'll have to work extra hard to get a copy of the relevant tags when you backup/copy etc.

I think storing tags in extended attributes[x], and possibly a separate utility that maintains and index (hopefully shouldn't be needed just for the tags, but might help with a) exposing file-level tags (like ID3, exif, file- type (magic number) etc), and b) allow for automatic organization based on full text and other content-based indexing.

It appears, on a * nix system, the only major reason to stay away from extended attributes (apart from the limit on size of tag data) is NFS. But samba should (AFAIK) work fine with extended attributes.

As far as I can gather, Gnome Beagle is dead, and Gnome Tracker[t] has taken its place. But it's not crystal clear if Tracker will index tags placed in files' extended attributes or not. If I understand correctly, Tracker's own tagging utility, will only place/edit tags in the Tracker database/index. But the indexers will certainly honour file-level tags for some files.

I don't really use full Desktop environments, but some kind of system with inotify support, and a Xapian or similar back-end (like Tracker), does seem like a good idea. It would certainly be nice to see such a system implemented in Go, but I think an _architecture_ along the lines of Tracker is probably worth keeping: A database daemon, an indexer and a set of query/view tools (I'm not a fan of the centralized tag database, though).

Another alternative to Tracker would be Recoll:

[http://www.lesbonscomptes.com/recoll/index.html](http://www.lesbonscomptes.com/recoll/index.html)

[t] [https://github.com/GNOME/tracker](https://github.com/GNOME/tracker)

[x] [http://www.lesbonscomptes.com/pages/extattrs.html](http://www.lesbonscomptes.com/pages/extattrs.html)

Btw, for editing/automating ID3 tags, I recommend "Ex Falso", the tag-editor for Quod Libet (which is an audio player): [http://quodlibet.readthedocs.io/en/latest/](http://quodlibet.readthedocs.io/en/latest/)

</comment><comment><USER53> This is really complicated

</comment><comment><USER54> This just seems like a different problem.

</comment><comment><USER55> I wish it would support tag hierarchies.

</comment><comment><USER56> Nice logo. D:

</comment><comment><USER57> It seems to me most of this stuff can be done via unix command line if you are so adept... without this app.

<comment><USER58> What existing tools can automatically generate a virtual filesystem based on tag metadata?

<comment><USER57> no not virtual filesystem, i meant querying your files by date or tag... find, grep and something to print the tag would do the job.

<comment><USER59> But where is the metadata? With OP's tool, it is in some db, so you don't have to tamper with your files or their name.

<comment><USER57> well with mp3 files the meta data is in the file, it's called an id3 tag. for other files, depends on the file.

</comment></comment></comment></comment></comment><comment><USER53> Hello</comment>


Infamous 'fake news' writer found dead in his bed - <AUTHOR> https://amp.independent.ie/irish-news/news/infamous-fake-news-writer-found-dead-in-his-bed-36172979.html <comment><USER2> fake news</comment>


When science hits a limit, learn to ask different questions - <AUTHOR> https://sinapticas.com/2020/08/07/when-science-hits-a-limit-learn-to-ask-different-questions/ <comment><USER2> Biology seems easier because we can actually understand questions biologists can't answer, and easily invent our own. Not so, physics.

But that is also exactly why biology is harder.</comment>


LinkedIn down 50% from its first day high - <AUTHOR> http://www.google.com/finance?q=lnkd

<comment><USER2> Mhh. I knew I should have shorted after the initial upo.</comment>


Computer snafu is behind at least 50 'raids' on Brooklyn couple's home - <AUTHOR> http://www.nydailynews.com/news/2010/03/19/2010-03-19_computer_snafu_behind_repeated_raids_on_bklyn_couples_home_the_8year_glitch.html

<comment><USER2> _cops have flagged the Martin's address so no officer will be dispatched to the home without double-checking the address._

Call me cynical, but the first thing I thought when reading this is "so that's how they keep the police from bothering any of the 'connected' people".

<comment><USER3> It might also be using the mechanism they use to stop police bothering subjects of surveillance too soon and ruining investigations.

<comment><USER4> It definitely _isn't_ the system they use to stop harassing innocent people after dozens of inappropriate visits but before the Daily News investigates and reports on it. I'm not sure they have a system for that.

</comment></comment></comment><comment><USER5> Wow. Were none of the same cops ever on the repeat visits? "Hey Joe, isn't this the same elderly couple who had nothing to do with the murder and the coke deal and the prostitution ring? Why the heck are we here again?"

<comment><USER6> "This time it's them."

<comment><USER7> "Time to bag us some cattle rustlers, boys."

</comment></comment></comment><comment><USER6> This is why you should _never_ buy a house located at 123 Fake St.

</comment><comment><USER8> Brilliant.

Enter your address in the system as the default one; complain until the police stop showing up; commit loud crimes.

<comment><USER9> I wonder what it would do for your home's value if you listed "DMZ" as one of the home's features.

</comment></comment><comment><USER10> I'll believe this one. I used friend's addresses in testing a system years ago. It's worth having a "zoo" of addresses because they come in a dazzling variety of formats - Salt Lake City, for example, has 2 Temple Streets that run east-west, one north of the LDS Temple, and one south. So they have a "directional": 123 E Temple St N. Apartment numbers are also a problem

Once you're done developing, the hand off to day-to-day administration sometimes leaves things behind. Test addresses might just stay in the system.

</comment><comment><USER11> Something of a similar nature happened to me at a university a decade or so ago. My e-mail address somehow got stored on a server as the default reply-to address for the e-mail client in all the public labs. I ended up getting on every mailing list in the known universe.

They eventually fixed it, but then there was a crash, and they restored from a backup ....

</comment><comment><USER12> Once at my Mum's (in England), a cop rang the doorbell at 2AM. I thought 'They're coming for me!' but it was just the car's boot lid was wide open and the policeman was concerned!

<comment><USER13> Now I want to know why you thought they were after you. Seems like there's a story in there somewhere.

<comment><USER12> Well I was working on a decompiler at the time (reverse engineering). But I only thought they were 'after me' because they rang at 2AM.

<comment><USER14> Interesting that reverse engineering is still somewhat shady.

</comment></comment></comment></comment><comment><USER15> Two thoughts:

1) It's surprising they didn't decide to sell the house and move.

2) Can you imagine _buying_ property with this particular baggage?

</comment><comment><USER16> Unconscionable.</comment>


Why don't more police cars have video cameras? (2006) - <AUTHOR> http://reason.com/archives/2010/03/25/why-dont-more-cops-go-to-the-v

<comment><USER2> I assume that the watchers don't want to be watched. If you do something bad, and it's his-word-against-yours, the consequences are minimal. If you do something bad, and there's a video of you doing it, that's the end of your career.

I don't buy the "it's too expensive" argument. Considering how much money law enforcement puts into busting pot users, I think they can afford a few cameras if they really wanted to. But cameras are more beneficial to the public than to the police, and the police are writing the checks, so...

<comment><USER3> _But cameras are more beneficial to the public than to the police, and the police are writing the checks, so..._

It's sad that we've gotten away from "To Serve and Protect" as a mindset. We'd be better off if public employees remembered that they are employees of the public. Really, it's the public that is writing those checks through tax dollars, the public just isn't interesting enough in watching the watchers.

</comment></comment><comment><USER4> The $2,500 to $10,000 per car figure doesn't surprise me, but does seem like it could be brought down. A webcam with a suction cup would be better than no video camera, and is <$500. Surely there is an $1000 solution that's acceptable?

My guess is that they're pricing Police Car Camera Systems, which is probably a market with very few, highly priced suppliers.

<comment><USER5> I've seen this phenomenon on display in the fire service. Somehow the exact same axe you can buy for $20 at your local hardware store costs 5-6 times as much...

<comment><USER6> Nobody in local government has an incentive for keeping their costs down. On the contrary, budgets are a sign of your personal authority, and cutting budgets is a sign of neglecting your priorities. ("My opponent slashed the funds available to save your wives and children from grizzly demise in a fire by 84%!" "But it was the same axe!" "SLASHED THE FUNDING!")

</comment></comment></comment><comment><USER5> Cops in my area have cameras on their dash, cameras on their tasers, and audio recording devices on their person. (Taser-cams can make for some pretty hilarious video...)</comment>


Netflix's Reed Hastings Is a Coward - <AUTHOR> https://theoutline.com/post/8277/reed-hastings-netflix-cowards <comment><USER2> Netflix used to be open to accepting very controversial documentaries in the past. Any new developments on such documentaries already released on the platform and they start to sneakily take them down, as if they were never uploaded.

I had high hopes for Netflix to show more interesting content unlike their competitors, but taking down the offending film and then running away for controversy afterwards, removes the context of the full story and is beyond cowardice to the viewers paying for the service.</comment>


Operating Systems vs. Messaging Apps in the Battle for Tech's Next Frontier - <AUTHOR> http://techcrunch.com/2015/08/11/its-operating-systems-vs-messaging-apps-in-the-battle-for-techs-next-frontier <comment><USER2> I'm working on an service that primarily communicates via SMS and works as a bot that you can have a conversation with. SMS is expensive if you want to be able to send and receive a couple hundred SMS per month per user (SMS costs about 0.75 cents a message). This puts a floor on your cost per user at around a couple of dollars and makes it difficult to have a price point of around $5/user/month.

But SMS has lots of awesome benefits. It works for everyone with a phone out of the box without installing anything and it gives you a unique identifier for the user. It would be awesome if iMessage and/or Google Hangouts allowed bots on their platform, but they do not. Telegram looks like the only alternative to SMS for bots.

Telegram even supports custom keyboards for bots so you can have the commands presented directly to users: [https://core.telegram.org/bots#keyboards](https://core.telegram.org/bots#keyboards)</comment>


Announcing Dex, an Open Source OIDC Identity Provider from CoreOS - <AUTHOR> https://coreos.com/blog/announcing-dex/ <comment><USER2> I like how CoreOs folks take each painful thing to do, and come up with a simple to use solution. This should save people (and me) quite some time.

<comment><USER3> Thank you. OIDC and OAUTH 2.0 are non-trivial protocols to get right. But, the opportunity is having a federated set of standards that web infrastructure can use. I think the best parts are: 1) being able to chain identities in a reasonable way 2) not requiring a database hit on every identity assertion and relying on crypto instead.

</comment></comment><comment><USER4> thank you so much coreOS. this might give me the identity solution. will give it a spin this weekend</comment>


Matt Maroon: Advice for Y Combinator Interviewees - <AUTHOR> http://mattmaroon.com/?p=283 <comment><USER2> I liked the advice, but I couldn't help feeling a little peeved by the continuing "yc news readers are a cult" thread.

I mean, come on, there are obviously a lot of people who have big dreams and look at YC as an easier way of getting to them, and yes, deadlines and tests bring out the neurosis and suck-ups in people, but this talk of cult-dom ALSO has a strange narcissistic quality about it which is just as bad.

If I thought the board was too cultish I wouldn't participate. Geesh -- on one side we get the hype of how great YC is, how it's all milk and goodness. On the other side there's a significant sour grapes contingent. For those who have already made it through or have an interview, congrats! For those who have not, it's not the end of the world! If this site wants to be useful to the hacker community, the board has to be, well, useful. I understand it got a little freaky over the past couple of days, but having people tell me I'm part of a cult is not a way to keep me around. I'm here for the other hackers -- YC is just the facilitator for that.

<comment><USER3> Here's another piece of advice (self-serving, I'll be the first to admit) -- work at a YC startup. you can meet all the YC people, including Paul. There are a lot of YC parties and events that you'll be able to go to. Make a good impression and get good recommendations from people already in YC, and you'll be in a great position to get into YC a little while later.

Now the shameless plug -- Scribd is looking for more people. Here's our jobs page:

<http://www.scribd.com/static/jobs>

Check it out.

</comment></comment><comment><USER4> Great advice. What exciting times for the hackers accepted to interview with YC! Best of luck to all. Now listen to Matt and get working on your demo!

</comment><comment><USER5> Maybe this is a retarded question, but what is Matt Maroon's startup? A quick search on google turned up nothing.

<comment><USER6> Not public but you can probably hunt it down if you read the archives of his poker blog which is definitely worth reading if you like poker. The startup has a business model and a very viable one at that.

<comment><USER7> _"... >I got asked whether I was worried about the mafia. Seriously. ..."_

Gambling related?

</comment></comment></comment><comment><USER8> That is excellent, excellent advice. I wish I could have read that last interview round.

<comment><USER6> make it two rounds ago for me

</comment></comment><comment><USER6> >I got asked whether I was worried about the mafia. Seriously.

Seriously, the US Government is much worse. Unless you believe, like I do, that a lot of organized crime in the USA is controlled by the CIA, in which case it is the same thing.

<comment><USER6> You guys didn't see the CIA plane The Web 2.0 caught smuggling drugs about a week ago? Remember that Bin Laden was on our side at one point.

<comment><USER9> Govt and states have persistent interests. They do not have persistent allies.

</comment></comment></comment><comment><USER10> That's all certainly good advice. Thanks for taking the time share it with us.

Speaking of advice though, a little padding on the left margin of your page wouldn't hurt...</comment>


Ask HN: How to do targeted web crawling? - <AUTHOR>

I'm thinking of building a domain-specific search engine. What are some general techniques for doing a targeted web crawl?
First, there's the issue of crawling pages that are relevant to my domain. How do I keep my crawler "on topic"? I know that I can start crawling from relevant seed sites, but how do I prevent my crawler from moving onto the general web once it hits a very general (and high PageRank) site such as Wikipedia?
Second, suppose that I am interested in finding a particular type of file (for example, PDFs). Are there techniques (again, other than seeding) for guiding the crawler toward sites that are likely to have lots of those files?
Thanks for any assistance. <comment><USER2> You could try latent semantic analysis of new pages to determine similarity to other documents in your domain.

I've been doing a lot of web crawling recently, feel free to get in touch if you'd like to discuss further.</comment>


Ask HN: I am spent and exhausted, I don't know what to do - <AUTHOR>

Hello folks of HN
As pretty much the only source of info and help on burnout, I turn to you. It has been a year in this company and I am pretty exhausted. I keep going from one burnout to the other in the name of release cycles. The love for the product is gone. I try talking to folks about a change in terms of jobs and I have had a barrage of rejections. Google in the final round rejected me. Companies looking to hire their first PM dont want to talk beyond the initial phone call. It is not hopelessly depressing yet but is very soon going to be. Couple this with barbaric hours at the startup and I dont even know if there is light at the end of the tunnel. I have done all the sane things like reach out for feedback etc etc but I get the canned "Not a fit" because relevant domain experience/we found someone better.
So, HN, How did you guys/gals handle rejection? Happy to PM my resume and talk to folks if there are any takers.
For clarity, I am not a developer/SDET. I am a PM and based on these rejections, not a good one at that :) <comment><USER2> I've fallen into this trap myself and understand where you are coming from. You get so burnt out on your current job that you start to treat every job interview as a potential relief from this burden in your life. What happens though is that you start to sound desperate when interviewing. No one wants to hire a desperate person. Even if you are the most qualified candidate, if you give off the wrong vibe it is difficult to get hired.

My advice is to stop focusing on the outcome of interviews and just focus on what you can control in the process. Try to do a few things each week that will make you seem like a better candidate than you were last week. Treat it like a game where you are leveling up during each round. Go back and review interview questions that you didn't answer well and rehearse what you would say the next time. Put together some material that you can show to someone interviewing you that helps guide the conversation towards your strong points. Keep an open mind and talk to people in your industry. Making it trough several rounds with Google means you are a strong candidate. If you keep up with the process, you'll start getting job offers.

<comment><AUTHOR> Thanks for the advice.

I have started doing some work with ruby on rails to make sure that I do not lose touch with programming. I have also enrolled in a bunch of coursera classes which have helped out. Thanks for the words of encouragement. It is people like you who make this forum worth coming to time and again.

</comment></comment><comment><USER3> >I am a PM and based on these rejections, not a good one at that

Everyone goes through this. But note! You got to the _final round_ of Google interviews. That doesn't scream "awful PM" to me. How many people make it that far? No, it's not what you want -- but then, you didn't get weeded out in the resume screen.

<comment><AUTHOR> Thanks man! its very encouraging, hope to dig myself out soon

</comment></comment><comment><USER4> I'm not sure how qualified i am to answer this post, when i got approached by google i didn't actually reply because i'm 99% sure i wouldn't even have got to a first phone interview so your way ahead of me in employability already.

But i've had enough canned responses and no responses to job applications or redundancies from startups going tits up, failed promotions or the right thing said at the wrong time and vice versa to suggest the following: a) Never ever take rejection personally, learn what you can from it but definitely don't beat yourself up over it. I know it's easy to say but in the first place it's obviously downright unhelpful, in the second, there's just so many ways from the apparently sensible, to the outright stupid, people come up with to reject you it's unfunny. There's probably a good reason half the time you get no feedback at all. A number of factors are out of your control. A number of them involve prejudice and herdthink. A number of them involve people making mistakes, politics, or having bad days, asking the wrong questions or just not happening to ask the right question that would make you shine over the other person. A lot of them, e.g. they found someone better, don't even involve you not being good enough to do the job anyway or are just luck, taking a path through domain A rather than domain B, not having buzzword X on your CV when your otherwise excellent CV got binned by someone non-technical in HR.

b) At the risk of getting too metaphysical, quite often it's very difficult to know what the job's _actually_ like until you've started it anyway. So the fact you didn't get it might not be as big a deal as you think (putting to one side your current one sounds far from ideal).

Obviously a lot of these are mistakes you wouldn't expect a decent employer, e.g. Google to make, so the fact you got to a final stage interview with them must put you in the top small fraction of the population anyway so don't take that for granted. And on the flip side, given the fact Google have just come to the conclusion that great exam grades don't necessarily mean great people [sure this popped up on HN recently] just shows how random it can be.

c) Obviously farm your cv around as much as possible to friends, other people in the industry on here etc. for feedback. Hone your technique. Don't give up on google, i gather a lot of people fail on first attempt but get jobs after reapplying later on.

d) Applying for jobs, always reminds me of going through school/college/university. Loads of my friends dropped out at various points, got labelled stupid, unemployable etc. etc. but have since gone on to get one (or more) degrees, MBAs, very highly paid and/or very technical jobs or possibly average jobs that they are just really happy in. The one thing that unites them is that they didn't give up even if when things seemed pretty hopeless and then took one (sometimes very small) step at a time.

Good Luck.

As an aside, obviously your current work situation with long hours isn't helping. I'm assume you have to be aware of the general consensus (mythical man month, agile/xp, amongst many) about how counterproductive repeatedly working long hours is. I'm not sure i'd necessarily go as far as quitting if it's possible to salvage the current job with more sensible hours, a break, go part time etc. or even just work contracted or sensible hours and make them fire you. Though often in these cases it can be the person themselves contributing to the pressure, through fear of failure, perfectionism or just taking onboard other people's monkeys (One Minute Manager Meets The Monkey) and they find kicking back and just letting some stuff fail actually works out better in the long run.

</comment><comment><USER5> If you have the financial assets please take some time off. Disconnect from your work and relax.

Rejection happens if you are misunderstood - if you are close to burnout you are not 100% yourself.

Hope this helps you.

<comment><AUTHOR> Thanks for taking the time to reply!

This, more than anything, is what I wish I could do. Unfortunately with a work visa, that is not an option. Closest thing to a vacation might be to find a behemoth of a company and work a fraction of the hours that I am now working. It will probably be immensely unsatisfying, but it will give me a chance to recharge.

<comment><USER6> I think servowire is correct - you need time off. It can get to the point where you simply cannot work.

My advice would be to just quit. Your mental and physical health is more important than anything else. There are always other opportunities out there. You could even work at a non-computer job for a while, and possibly work on your own projects in your spare time (once you get your motivation back).

Having suffered from burnout myself, I think it was actually a useful experience, as it forced me to make changes to my life and I am now living a much more fulfilling lifestyle.

I'm happy to chat more privately.

</comment></comment></comment><comment><USER7> Getting burned out sucks. I'm a PM and I've been there. What helped drag me out was to focus on the process and not the results so much. I spent some time soul searching and landed on what I think is a philosophically sound way to do things, and sticking with it in the ups and downs helped my well being. That being said hitting your numbers / getting results is super important, but you can only impact how you play the game and winning or losing can be (biased) randomness on top of that.

Anyway, happy to look at your resume if you want.

<comment><AUTHOR> Thanks Idoh! This focus on the process seems to be a good way of mending the havoc that comes with reliance on results. It at least makes for a good story and something tangible to take from one job to the other.

</comment></comment><comment><USER8> Keep in mind that the rejections may have nothing to do with your profile or skillset, but simply to how you're presenting yourself during the interviews.

If you're dealing with burnout, low energy, low self-esteem, this will clearly show up during the interview or even a phone screening. You may not even notice it, but it's usually very evident.

The other recommendations here are all valid: invest in yourself. Take some time off. Learn something new. Start a new Coursera (or eDX, or Udacity). Start exercising, if not already. Give yourself a few months to heal, and then start applying again and move on.

Happy to take a look at your resume and give you some honest feedback, if desired.

</comment><comment><USER9> There were plenty of implicit rejections for me in the last two weeks. I decided to vent my frustration by learning how to write Android apps. Ultimately, it didn't do anything for me. But I did learn something new, and wrote something I think is quite cool.

In fact, I just wrote a new blog post talking about how I handled rejection and what I did to make me feel better

Then I rediscovered the joys of running. That made me feel a lot better.

TL;DR - Learn something new, and then do some exercises.

</comment><comment><USER10> For a start, next time, you can avoid startups which go, "You should be prepared to make X a major priority in your life" in their job ad. If you pass by that red flag, next up is the huge blinking red light in the form of the idiot founder who goes, "I work 100 hours a week on my startup". If you pass by even that, you have other issues to deal with.

</comment><comment><USER11> Try to get some rest, eat better, take your vitamins, hydrate. All that stuff makes a difference. At some point, you will solve this. It will go better if you keep yourself together in the mean time.

((hugs)) if you want them.

<comment><AUTHOR> Thanks! Hugs back :)

</comment></comment><comment><USER12> It's just a numebers game brother. Apply to 100 companies and you will have a couple of offers.

<comment><USER13> Yes I agree with this, however my take would be to think of it as a statistic... how many nos you need to get to a yes. Right now you don't know your base 'conversion' stats so you don't know how many nos give you a yes. Of course you feel lost but you don't have a base case to compare to. Until you do, you need to keep plugging away and just keep track of your stats.

Say you find and you need 80 job applications to a job offer, well then that's great because then you need to know how many nos you need to brush off to get to the yes. Say you cut it down to 40 job applications to a job offer... well clearly you are getting better at the process.

I think playing these sorts of mind games with yourself is really helpful for keeping your ego intact and handling the rejection productively.

</comment></comment><comment><USER14> Dude... relax... you're in a bad spot have a beer, eat some pizza and get laid...

You'll find a job. Only about 1 in ten people that want jobs don't find them and only about 1 in 100 end up eating out of a garbage can. You sound pretty smart... get your portfolio together and start going to networking events. You'd be surprised how many people want smart people working for them. You have to go looking thru the haystack... the haystack ain't coming to you.</comment>


The U.S. Congress Discusses the Establishment of the US Embassy in Taiwan - <AUTHOR> https://img3.picload.org/image/dlwwraia/times.jpg
The Taipei Times reported on the discussions at the U.S. Congress on establishing the U.S. Embassy in Taiwan and diplomatic contacts with the Republic of China (Taiwan).
According to the report of such international law firm as Alston & Bird LLP to the Minister of Foreign Affairs (Taiwan), Joseph Wu, this lobbying firm acted as a bridge-builder in re-establishment of diplomatic relations between the US and Taiwanese authorities.
As a result of their joint efforts, President Trump signed the Taiwan Travel Act and Marie Royce, Assistant Secretary of State for Educational and Cultural Affairs, opened a new representative office of the U.S. State Department on the island - the American Institute in Taiwan.
It is also worth mentioning that the U.S. State Department has taken an extremely risky step as it has been quoted in the newspaper: for the first time since the 1995-1996 Taiwan Strait Crisis, it granted visas to Taiwan's President, Chen Chen-jen, as well as, to Taiwan's Foreign Minister, Joseph Wu, and other personnel from this ministry. <comment><USER2> Anyone know why some non-Ask HN's show on the ask page.</comment>


Kasperksy Ad: "Be the MAN, show the ladies your smarts" - <AUTHOR> https://twitter.com/orless/status/741359220763832324 <comment><AUTHOR> Kaspersky Internet Security just popped up this ad on my desktop: "Want to be the MAN? Want to show the ladies your smarts?". A guy with a notebook and a queue of ladies waiting to be brought to the CyberSecurity world.

It is 2016 and we still insist on this "helpless lady" image, unbelievable.</comment>


Funding government without taxes in 4 ways - <AUTHOR> http://ask.fm/objectivist_liberland/answer/127181944997

<comment><USER2> >The government has a unique role in a country because it has a monopoly on three institutions (courts, police, military) that are outside of free market. A citizen can not escape them, short of leaving the country. That is why

Limiting business into the gameplay and excluding it from the rule-making is not realistic.

It's clear that if everyone is business savvy and knows how the play the game rationally, top 50% must pay fee for the bottom 50% that is negotiated like business (negotiation power and utility maximization) and the fee is significantly larger than production cost.

Whereas libertarians see the current economy as unfair. Economists can see that politics seems to constantly adjust the gains just as you expect if everyone is at least semi-rational actor.

Progressive taxation is explained by marginal utility theory and smaller number of wealthy individuals. As long as wealthy need the help from the poor to have system that protects their property they must accept contract that is more than what just enforcing the rules.</comment>


Evilgrade is like Firesheep for Software Updates - <AUTHOR> http://www.infobyte.com.ar/down/isr-evilgrade-Readme.txt <comment><USER2> Firesheep is firesheep because it's easy enough for random college freshmen to use. This is a commandline tool so it is no Firesheep.

</comment><comment><USER3> Are the latest versions of Skype, VMWare, and others listed there at risk of software-update hijacking, with no cryptographic verification of update payloads?

<comment><USER4> VMWare is in the compatibility list, so that's a yes there.

<comment><USER3> I see it on the EvilGrade page; I'm asking here for independent confirmation.

<comment><USER5> Yeah, Virtual Infrastructure and VSphere clients download updates over HTTP from a URL that looks like: [http://www.vmware.com/vmware<%RND1%>.exe](http://www.vmware.com/vmware<%RND1%>.exe)

</comment></comment></comment></comment><comment><USER6> Seems like something metasploit would do. If you're a budding security nut and you haven't tried some 'sploitin, you should definitely give it a go.

<comment><USER5> true that. But AFAIK metasploit doesn't have exactly this kind of functionality. It would be nice if someone would port it though...

</comment></comment><comment><USER5> Here's the list of supported apps:

* Freerip 3.30

* Jet photo 4.7.2

* Teamviewer 5.1.9385

* ISOpen 4.5.0

* Istat

* Gom 18.104.22.16815

* Atube catcher 1.0.300

* Vidbox 7.5

* Ccleaner 2.30.1130

* Fcleaner 126.96.36.1999

* Allmynotes 1.26

* Notepad++ 5.8.2

* Java 1.6.0_22 winxp/win7

* aMSN 0.98.3

* Appleupdate <= 220.127.116.11 ( Safari 5.0.2 7533.18.5, <= Itunes 10.0.1.22, <= Quicktime 7.6.8 1675)

* Mirc 7.14

* Windows update (ie6 lastversion, ie7 7.0.5730.13, ie8 8.0.60001.18702, Microsoft works)

* Dap 18.104.22.168

* Winscp 4.2.9

* AutoIt Script 220.127.116.11

* Clamwin 0.96.0.1

* AppTapp Installer 3.11 (Iphone/Itunes)

* getjar (facebook.com)

* Google Analytics Javascript injection

* Speedbit Optimizer 3.0 / Video Acceleration 220.127.116.11

* Winamp 5.581

* TechTracker (cnet) 1.3.1 (Build 55)

* Nokiasoftware firmware update 2.4.8es * (Windows software)

* Nokia firmware v20.2.011

* BSplayer 2.53.1034

* Apt ( < Ubuntu 10.04 LTS)

* Ubertwitter 4.6 (0.971)

* Blackberry Facebook 22.214.171.124 | Twitter 22.214.171.124

* Cpan 1.9402

* VirtualBox (3.2.8 )

* Express talk

* Filezilla

* Flashget

* Miranda

* Orbit

* Photoscape

* Panda Antirootkit

* Skype.

* Sunbelt

* Superantispyware

* Trillian <= 220.127.116.11

* Adium 1.3.10 (Sparkle Framework)

* VMware

* more...

</comment><comment><USER7> Firesheep only requires that you sniff unencrypted traffic but this requires that you make DNS requests resolve to an address of your choice. The latter is much harder to do. You either need to control the wireless router or break the DNS server some way.

<comment><USER8> True but not that much - it wouldn't be difficult to inject a reply on an unencrypted wireless network, you just have to sniff the trafic and then reply faster than the wireless network and then blast the reply out with a higher signal.

Since everything is cached locally for your, replying faster shouldn't be an issue, and you can sit closer to your intented victim than the wireless router, which should give you a better signal.

</comment></comment><comment><USER9> It says it supports Adium (Sparkle) updates, but Adium definitely uses digital signatures, see /Applications/Adium.app/Contents/Resources/dsa_pub.pem . So...is there something I'm missing? Has anybody tested this?

<comment><USER5> Adium definitely gets its update list over HTTP, so maybe it just prompts the user if the signatures don't match and lets them install anyway?

The relevant module in evilgrade is sparkle.pm if you want to check it out.

</comment></comment><comment><USER10> Can anyone outline how a software package would protect against this?

<comment><USER11> Use SSL for the connection to the update server.

<comment><USER12> A secondary method for authenticating updates would also be wise. When Moxie Marlinspike's null-prefix SSL bug landed, people with vulnerable versions of Firefox were somewhat screwed: Firefox used only SSL to ensure the authenticity and integrity of updates, but SSL was broken, so the update fixing SSL security couldn't be authenticated!

</comment></comment></comment><comment><USER13> I you don't get it I recommend watching the screencast <http://www.infobytesec.com/demo/evilgrade.htm>

</comment><comment><USER13> What's "Internal DNS access"? Host file access?</comment>


Why is nickb's submissions page empty ? - <AUTHOR> http://news.ycombinator.com/submitted?id=nickb http://news.ycombinator.com/leaders ==> http://news.ycombinator.com/user?id=nickb ==> http://news.ycombinator.com/submitted?id=nickb<p>I could find a(another) contact for news.ycombinator.com ... <comment><USER2> This is a known HN bug; any page that is really huge only works about 10% of the time. My threads and saved pages are pretty useless as well.

</comment><comment><USER3> I see submissions on the page.

Might be due to some of the database/scaling issues going on with the code.

</comment><comment><AUTHOR> it also takes a long time to show the page with no submissions...</comment>