Let's start using DuckDuckGo more often - <AUTHOR> Granted that DuckDuckGo.com is quite a childish name, but then so was Google when we first heard about it in early 2000s, wasn't it? I just switched to DDG recently and feel that not only is the interface fast and minimal (like Google used to be in those early days), but even the results seem to be a lot better (at least specific to the programming topic that I search too often). I'm getting a feeling that some day in future, DDG is going to become as big as Google, if not supersede it. But a search engine's results are only as accurate as the number of users who search and contribute to it, so its my appeal to people to give a chance to DDG and start using it more and more instead of Google. <comment><USER2> I use DuckDuck go as my default search on my machines, but I think the fact that Google is not anonymous does give them a big advantage.

An obvious example of this is when I search for "Django" (I am primarily a Django developer). DuckDuckGo will return results about the film as the top hits, whereas Google already knows that I mean Django Web Framework and will return those as the top hits.

I appreciate the fact that my searches are anonymous with DDG, but I doubt that it will be able to be "as good" as Google for that reason.

<comment><USER3> And DuckDuckGo is not really comparable when you don't live in the US. Google is far more superior, even without your info.

<comment><AUTHOR> We have the chicken and egg problem there. Google have that results because it has terabytes of information on everything and that's because millions use it. If more and more people start using DDG, their results will also improve.

<comment><USER4> See, I'm not willing to do that. I'm not willing to put up with inferior results on the promise that maybe, someday in the future, they might get better.

<comment><USER5> I remember how impossible searching was before Google. DDG at its worst is infinitely better than that.

<comment><USER4> It very well might be. But there's also something better. And I'd prefer to use the better option.

</comment></comment></comment></comment></comment></comment><comment><USER6> > I'm getting a feeling that some day in future, DDG is going to become as big > as Google, if not supersede it.

Well, no, they don't even have their own search engine.

> a search engine's results are only as accurate as the number of users who > search and contribute to it

That makes no sense. Anyway, we don't use DDG because its results are shitty.

<comment><USER7> > Well, no, they don't even have their own search engine.

Really? I've never heard this, do you have anymore information about it?

<comment><USER8> It's sort of a meta search engine that utilizes other search indexes. Here's a post from the founder.

[https://duck.co/forum/comment/27893](https://duck.co/forum/comment/27893)

Edit, that's about 5 years old. The current page on Wikipedia provides a more recent explanation.

"DuckDuckGo's results are a compilation of "about 50" sources,[42] including Yahoo! Search BOSS; Wikipedia; Wolfram Alpha; Bing; its own Web crawler, the DuckDuckBot; and others."

[https://en.wikipedia.org/wiki/DuckDuckGo#Overview](https://en.wikipedia.org/wiki/DuckDuckGo#Overview)

<comment><USER9> They actually say hundreds of sources now on [https://duck.co/help/results/sources](https://duck.co/help/results/sources) and specifically call out Bing, Yahoo, and Yandex for search results.

</comment></comment></comment></comment><comment><USER10> Disclaimer: I work at DuckDuckGo so I'm a bit bias. I won't turn this into a sales pitch, but here are a few common misconceptions people have about privacy and search.

1) Many people don't realize that tracking isn't just about having something to hide. But, it can cost you money. From Airline tickets to staplers you pay based on a profile:

[http://www.wsj.com/articles/SB100014241278873237772045781893...](http://www.wsj.com/articles/SB10001424127887323777204578189391813881534)

[http://www.wsj.com/articles/why-you-cant-trust-youre- getting...](http://www.wsj.com/articles/why-you-cant-trust-youre-getting-the- best-deal-online-1414036862)

2) People don't realize what's being tracked. I usually send them to [http://history.google.com/history](http://history.google.com/history) to have a look. Or [http://webkay.robinlinus.com/](http://webkay.robinlinus.com/) to see what their browser can access. That makes a lot of people realize just what is out there.

3) People feel they can't search without personalized searches.

The example is often a matter of disambiguation. For example, if I type "Python" I want code, not snakes. But, really, when is the last time you only typed 'Python' and wanted something generic about Python? You probably wanted a package lookup or the latest news on a release. So if you become more specific there is no issue.

Plus when you get a bit more specific on Python you can trigger things like package lookup:

[https://duckduckgo.com/?q=python+numpy&ia=about](https://duckduckgo.com/?q=python+numpy&ia=about)

Or NumPy Cheatsheet:

[https://duckduckgo.com/?q=numpy+cheat+sheet&ia=cheatsheet](https://duckduckgo.com/?q=numpy+cheat+sheet&ia=cheatsheet)

At the end of the day some people may truly be ok with all the tracking that takes place, and that's ok that's up to them. But, at DuckDuckGo our goal is to educate people on online privacy, and provide a trusted way to access information as best we can. (Not to mention Instant Answers and Bangs which are super addicting)

<comment><USER11> I'm a ddg fan. You guys should have a big, conspicuous WHY? button on the homepage that links to all this. The 'Learn More' link takes me to what you guys don't do, which is good, but it's light on the _why_.

<comment><USER10> Thanks for the great suggestion. We're currently exploring ways to get the 'why' message out there more. I'll certainly bring it up with the team :)

</comment></comment></comment><comment><USER12> DDG has been my default search for the last few years and I only very rarely use it for it's native search results. The bang (!) search syntax is by far the biggest reason I use it. It allows me to search sooooo many websites directly from the address bar. If you want normal Google search results, use !g. If you want Google Images, use !gi. Amazon? !a. Wikipedia? !w. The list goes on and on and on.

Edit: grammar.

<comment><USER13> Completely agree. I also have been using DDG for a few years and got used to the bangs.

The results for a general search usually are good enough, so don't I even bother to use Google directly.

</comment></comment><comment><USER14> DDG is great; what it currently lacks (for me) is not search quality but reach; it has trouble indexing less visited corners of the internet.

Google did not become huge because of the search engine _alone_. AFAICT two things, based on related technologies, made Google oodles of money: AdWords and SERP ads. (AdWords used to be so unobtrusive I never tried to block them.)

I don't know how DDG currently pays its bills. They do feature unobtrusive and clearly marked ads on their SERP, too.

I'm not sure if ads can be reasonable without precise targeting, that is, tracking, tacit privacy invasion of various sorts, etc. Poorly targeted ads are disliked both by users ("dumb!") and advertisers ("poor conversion, money wasted").

The only other option I can see for a private company is to sell a subscription. Pay _n_ USD / mo for no-tracking, no-strings-attached search.

The question is, of course, the value of _n_. It may turn out to be uncomfortably high for many users, just because advertisers value their eyeball rather highly.

You can already opt out of ads on some Google services, e.g. YouTube: try closing a few ads, or visit google.com/contributor when it (re-)opens. You can opt out of personalized ads, too. While many of us still won't trust all these measures, for many these would feel adequate.

I wish DDG all the luck. But being and _staying_ an alternative, privacy- respecting search engine, even a low-profile one, isn't going to be easy.

<comment><USER15> How poorly targeted can an ad be when you've told them _exactly what you 're looking for_? It doesn't get much more targeted than "car dealerships Florida".

<comment><USER4> Well, if you're looking to buy a Honda, and the ad that gets shown is for the Mercedes Benz dealership, that's pretty crappy targeting.

<comment><USER15> It is, but there's no better _potential_ for ad targeting than in a search engine, which is what the GP was lamenting. I read their comment as "how can they target ads well if they don't track me?", to which the answer is "they don't need to track you, you've literally told them what you want".

</comment></comment></comment></comment><comment><USER16> I use searx.me more often than any other search engine because along with giving the result it also display the search engine which it uses to fetch the result. I mean it is very important to know the areas where each of the search engine excel. Privacy is a concern for sure, but DuckDuckGo cannot win me over on the basis of privacy only. DuckDuckGo can be an answer to privacy concerned people, but it cannot beat Google Scholar, YouTube, PubMed, Amazon etc. We must know which search engine is perfect for which kind of keyword.

<comment><USER17> It would be great if they made ! more discoverable. For instance, it could tell you the ! code needed to limit searches to the engine that produced each "niche" blue link and gray box at the top of the screen (the one that inlines amazon product thumbnails/prices, wikipedia article synopses, stack overflow answers.", etc)

<comment><USER16> But the question is, why are we looking for a Swiss Army Knife?

I make all kinds of search queries but I only use privacy centric search engines when required for my particular concerns. In my opinion, the best way to stay anonymous is by disguising yourself as normal. Today's browser are capable of switching search engines with the use of a single letter before the search query to make use of a particular search engine. And one can easily visit Firefox search engine database to include it in their list.

[https://addons.mozilla.org/en- US/firefox/search/?atype=4](https://addons.mozilla.org/en- US/firefox/search/?atype=4)

I haven't use DuckDuckGo's bang features, so I really don't understand what are they trying to achieve other than proxying the search result which I already satisfactorily achieve by having multiple Firefox profile with different proxies.

I think this is never ending quest of integrating new search engines in DuckDuckGo. But it will never match the native use. For example, take Wolfram Alpha or TinEye(Reverse Image Search).

Bottom line is that there are million ways to stay anonymous while using native search engines instead of routing it through third party services.

</comment></comment></comment><comment><USER18> I've used duckduckgo for a few years now, highly recommend it.

For 'easy' searches it's equivalent to Google.

For 'hard' searches it's nearly stricter better than Google, because if DuckDuckGo doesn't find something I also look at the Google results (append !g to the search), and they often come up with very different subsets of the internet.

To me it no longer has anything to do with privacy or not liking Google, it's just that DuckDuckGo has the better product for putting into your search bar.

</comment><comment><USER19> I'd suggest to "Start contributing data to DDG in 2017". Most of their best results are crowdsourced. I've been using it as default for perhaps 2 years now. There's some consistent annoyances, but overall I actually prefer the experience to Google.

It's much nicer about disambiguation than google, and can be _incredibly_ helpful. e.g. Search for Zen - you get a wide selection of possibles in probable order. Sadly there's far too many missing. Crowdsourcing needed.

It's _horrible_ at localisation. eg set Filter by region to UK and search any global multinational. Chances are the UK site is WAY down the list and the .com and US options hard at top. on google UK the local branch is always first.

There's too many instant answers that presume a US only view of the world.

The instant answers when they have adequate data and ! searches are brilliant.

Lyric and video searches are orders of magnitude better than Google.

Maybe 5% of searches go to Google as I'm not quickly finding what I need.

<comment><USER10> We always welcome more contributions. We crossed 1000 Instant Answers this year and they make for some wonderful experiences!

</comment></comment><comment><USER20> For anyone using !g when they don't get the results they need - use !s. It will use Startpage, which fetches results from the Google search engine but without giving any personal information to Google's servers.

</comment><comment><USER21> DDG is really great for easily doing specific types of search, eg. adding "!w" to your query will search for something on Wikipedia. For that feature, it's extremely useful and I don't like to be without it. But its own search results (which I think are actually anonymized Bing search results?) are nowhere as near what google manages by personalizing your search. So I'll often add "!g" to my query to get google results rather than ddg ones.

<comment><USER22> You can add these shortcuts into your browser. On firefox, you can right click on a search box and specify some character(s). I'm sure there's a similar process on chrome too.

<comment><USER23> too much work. changing my default search engine to ddg is faster and easier. i don't even know all of the bang commands! if i followed your advice i would have a significantly inferior experience.

</comment></comment></comment><comment><USER24> Every time I try to use DDG I ended using !g which kills its purpose, I just use Startpage[0]

[0]: [https://www.startpage.com/](https://www.startpage.com/)

<comment><USER25> This site appears to be a front of google and in the end its relying on google to search and get the result, some how I don't see this to be too different from directly using google in a incognito window

<comment><USER26> A incognito window doesn't hide your identity from Google. They still get your IP address and your browsers's fingerprint, which is more than enough to identify you.

</comment></comment></comment><comment><USER27> I took the opportunity of starting at a new job to make the switch of default. I've found it to be generally giving me better tech search results than Google. For the most part, however, I haven't really noticed I'm using it, which I'd consider a good thing. It means it does its job without fuss or bother and gets out of the way.

</comment><comment><USER5> It's my default engine everywhere. Very happy with it generally, and the fact that I can trivially redirect a search to Google makes it a no-brainer.

<comment><USER28> I used DDG for a while as my default engine, but realized that I was using the Google redirects more and more until I finally switched back again. Google's search result were just better for me.

<comment><USER5> I'd estimate I redirect no more than 10% of my searches, and given how much I distrust Google, it's worth it to keep searching elsewhere.

</comment></comment></comment><comment><USER25> I use DDG full time and it has been awesome , I am a devops guy use DDG to trouble shoot my way out of problems I get stuck in and I have personally found this to be way better than google. The Bang search is very cool too , for example I use it to search github for docker orchestration related content directly ! , I also use DDG command line quite a bit and its really awesome , these are things which I feel are much superior compared to google and I hope DDG will become the geeks most preferred search tool in 2017

</comment><comment><USER29> I switched to DDG a few months ago and have found it just as good as Google at least for my usage, at the very least I haven't felt the need to use Google as the results I have been getting have been just fine, and that's without the bangs functionality.

</comment><comment><USER30> I have duckduckgo set on my phone. Recently, I started discovering that when I clicked a google search result link on my phone, google would mask that link and wouldn't actually take me to the webpage. Annoying. DDGo doesn't do this.

</comment><comment><USER31> The "bangs" make all the difference. Just this morning I used !ups <tracking number> to track a package. I use !w (wikipedia), !gm (google maps), !gi (google images), !yt (youtube) and !a (amazon) all the time. Less often I use !so (stack overflow), !gsc (google scholar), and others. Sometimes I even use !g (google) or !b (bing) if the search results are inadequate. But that's very rare for me.

</comment><comment><USER32> Google, Microsoft, Yahoo, they all have something in common: EaaS. Ecosystem as a Service. DDG should start offering email first, then news, stocks, maps, social, and finally ads. You don't survive with search alone, you need to monetize it on one hand and keep your users coming back on the other.

Oh, I wish it was called something simpler like "Ducker" or "Duckit", but whatever, bikeshedding territory.

<comment><USER33> That strategy doesn't seem to have worked out for Yahoo.

Not sure DDG users really need/want another er... EaaS.

</comment></comment><comment><USER34> I switched to DuckDuckGo as my primary SE and will be using Bing as a fallback. I'm slowly moving as far away from Google as I can.

On that note, any good alternatives to G-Suite that aren't necessarily Microsoft O365 (though I'm not against migrating to that either). A straightforward email migration is a big plus, documents not such a big deal.

<comment><USER35> suggest fastmail.com

<comment><USER36> I have a question about fastmail: I tried migrating from gmail but the original date of the emails didn't transfer. Anyone knows how to solve it? I had to go back to gmail since it's a deal breaker and I couldn't find a solution, but I really want to leave gmail (so I can have a direct relationship with the service provider without advertisers involve)

<comment><USER37> How were you trying to migrate? I've been doing a Gmail->FastMail migration and I definitely had no issues with original email dates.

<comment><USER36> It was a few months ago, so I don't remember exactly, but I believe I followed the instructions that fastmail provided. If you say you didn't have this problem then I will try again.

Thanks for the answer

<comment><USER37> In my case, FWIW, I used an IMAP-compatible mail client (Outlook), and dragged my crud from one mailbox to the other. I did have some issues where mail conversations where different messages were under different labels ended up in not copying fully over, but that's partly because I migrated section by section.

Also, of course, note that if you have a mail message under two Gmail labels, and use IMAP to copy your folders, you'll probably end up with two copies of that message.

Migrating out of Gmail, FWIW, is still a pain because they are so nonstandard now.

But I never had an issue with the dates on the emails.

</comment></comment></comment></comment></comment></comment><comment><USER38> They do a nice job of extracting and presenting the top StackOverflow answer:

[https://duckduckgo.com/?q=stl+string+remove+characters&t=h_&...](https://duckduckgo.com/?q=stl+string+remove+characters&t=h_&ia=qa)

</comment><comment><USER39> By far my biggest gripe with Google _and_ DDG is that they have started quite simply ignoring search terms. I search for 'foo bar' (without the quotes), they return results which clearly do not include "bar", neither in the blurb nor on the page itself. At least Google will mostly obey if I quote the search terms, but DDG doesn't even do that.

</comment><comment><USER40> At DuckDuckGo, no cookies are used by default.

>Yet they do store a cookie by default - this cookie is called "user_segment" and is valid for 1 month after it is first set.[1]

They have removed it but this kind of behaviour doesn't exactly raise trust. Also they are based in US so 'privacy' is just PR.

I would recommend to use startpage.

[1][https://archive.is/qntuk](https://archive.is/qntuk) [2][https://8ch.net/tech/ddg.html](https://8ch.net/tech/ddg.html)

</comment><comment><USER41> Well, as Google continues dumbing down their search engine, there's certainly room in the market for a search engine with the same kind of power that Google used to have...

...but that being said, DuckDuckGo ain't it. It's in fact, quite far from it. It's roughly as good as old Yahoo Search (pre-Bing), which also nobody used because Google's is vastly superior.

Give me a search engine like Google circa 2010-11 (back before the menagerie of Bird algorithms began trading "fuzzier" results for raw search power) and I'm good.

</comment><comment><USER42> I've been using duckduckgo as my main search engine for about 2 years but I never recommend it to anybody. It just feels like a homeopathic remedy to me. What's the business model? I don't allow them to show me ads and they don't allow me to pay for the service. I just can't imagine how that can scale.

Search engines are such an important tool that I would be more than willing to pay $10 a month for a good quality one with strong commitment to privacy and maybe additional premium features.

</comment><comment><USER43> I find duckduckgos results quite bad though, i've given it many shots over the past year but often have to keep switching back to Google.

<comment><USER15> I find that the results _feel_ quite bad, but, whenever I _actually_ compared results to Google's, the latter wasn't better at all.

<comment><USER43> I meant I used to find bad results and used google and found the results were more relevant, I tried to feel duckduckgo's results were better.

<comment><USER15> Hmm, odd, that doesn't match my experience at all. I realized how bad Google was when I started using DDG and realized that all the bad results in DDG were bad in Google too.

Maybe it improved since you used it?

</comment></comment></comment></comment><comment><USER44> I don't really get DDG. I guess it's nice that they pattern-match some queries and give you a a query specific UI on top of the results or maybe some results UI that makes a lot of sense for the query -- though Google and Bing do that too, and I don't believe they are hardcoding those rules/patterns like DDG seems to be doing.

I 've given it a few honest tries this year but the results are really not that great, certainly far inferior to Google's or even Bing's, and it 'feels' slow -- but then again, a few dozen ms slower than Google is 'slow' to me (and I am sure the difference is higher than that ).

I suppose the major selling point is that it doesn't track your queries and that's nice and all, but it definitely is not important enough for me to trade that for better results and responsiveness.

I wish them the best though.

</comment><comment><USER45> I think twice a year I get my antigoogle moment and try to use it, it rarelly last more than two weeks when I don't find something or get frustrated because Google always loads faster. It's a shame. As much as I want to love DDG I find pretty hard to use it for a long time.

</comment><comment><USER46> >I'm getting a feeling that some day in future, DDG is going to become as big as Google, if not supersede it

If they become bigger than Google what's to stop them from becoming another Google?

Why punish myself by using an inferior service just to give it the seat at the big 5?

</comment><comment><USER47> If you want a search engine which respect your privacy but with a fancy gui (unkike duck duck go), you should try qwant : [https://www.qwant.com/](https://www.qwant.com/)

French technology which work really well !

</comment><comment><USER48> Already have been!!!

Started using Beaker Browser as my main browser, DDG as my search, and am working on moving the rest of my life off google (read: gmail).

Also have been messing with doing more work on a raspi tablet rigged with a bluetooth keyboard/trackpad combo.

</comment><comment><USER49> I really like that google isn't anonymous and that it pulls everything that I've done in the past to give me better results. I usually have to jump through a lot of hoops to get the same level of results from DDG.

</comment><comment><USER50> > Let's start using DuckDuckGo more often

I am at a loss to understand really why 'us' should start doing this 'more often'. Is there something inherently good about using ddg vs. google? Why should anyone use it more vs. what they have already decided works best for them? This smacks of 'make the world a better place by using ddg' unfortunately as many others have noted it's simply not a better mouse trap. And what does the name have to do with it at all?

</comment><comment><USER51> I use DDG but only because of the bang system. In other words I can supply a !g anywhere in the query and it will use the encrypted google search. This is also useful when you also use other ones like !yt !fl !gh !r and so forth. But to be honest, I hardly ever actually use the ddg search engine.

</comment><comment><USER52> I use DuckDuckGo exclusively on my phone. Half of the time I seem to fallback to Google but since you can use !g to get google results I still use it.

</comment><comment><USER53> I am already using it. and when you want a custom Google search that goes through DDG you do: !g 1 byte to MB or any other search you want

</comment><comment><USER54> I tried to use DuckDuckGo and it was terrible. Couldn't even last a week. The results were dramatically worse.

And putting aside the quality of results, the actual design of the site is not very good. This is probably a preference, but I find it much easier to quickly scan a Google results page (or even Bing), but DuckDuckGo, with the font face and spacing they use, is not as clear.

<comment><USER26> Try startpage.com. They fetch the search results from Google without revealing your identity to Google. My primary search engine since more than a year.

</comment></comment><comment><USER55> Questions for you to ponder:

1) Why do you think their results would get better if more people use their engine?

2) If their results really are better than Google, as you claim, why is their user base so small after all these years?

3) Why should we trust them any more than Google? How do you know they're not actually collecting your data or passing it on to the third party engines they use?

<comment><USER17> 1) More revenue -> more developers -> more deep web onebox results.

2) Result quality is trumped by inertia and ecosystem stickyness (I commented above)

3) They have a smaller team and are focused on one product, which means they can survive without doing this. Also, the risk of detection is high, since it relies on hiding revenue, third parties that keep secrets, and the US never issuing a subpoena for their logs. In contrast, Google would go out of business tomorrow if they stopped collecting and monetizing data.

</comment></comment><comment><USER56> Anyone use DDG's grouping search syntax much?[0] I've always wondered if this is something just for bots or if humans actually write out complex searches like that.

If you use it how are the results?

[0][https://duck.co/help/results/syntax](https://duck.co/help/results/syntax)

<comment><USER57> I often submit equivalent queries when searching using Google. I tend to use OR a lot to specify alternatives to specific keywords (e.g. osx OR "os x" OR macos), though it's a years-old habit often not that important with today's search engines; I sometimes also use it to search sets of domains (e.g. site:w3.org OR site:whatwg.org). In general I feel that when the first few pages of results using a simple query don't get me to what I am looking for, using a more complex expression can unearth hits which I would have otherwise skipped or not reached.

Regarding Duck Duck Go's advanced search syntax, I've use it in a similar fashion, though not as much. Keep in mind that the provided link doesn't seem to reflect the current implementation. For instance, the syntax for grouping using parentheses doesn't seem to be as strict (which is a relief), excluded words are seemingly not limited to being at the end of the search string, and AND has a higher precedence than OR (which is currently a pet peeve of mine because I am used to Google doing it otherwise and I like not having to use parentheses in my usual queries).

</comment></comment><comment><USER58> But DDG still goes to Google (as well as the other providers). And I see it returns less relevant results compared to Google itself. Where it might be shining is in avoiding the search per unit of time throttling done by Google, and less content tailored by user IP which I see an intrusion into ones privacy.

</comment><comment><USER59> DDG is my default search engine because I'm addicted to the !bangs. For example, I use !pf to quickly convert an article into a PDF.

I maintain a blog where I "showcase" the best bangs for the Duck: [http://duckgobang.com/](http://duckgobang.com/)

<comment><USER60> Nice. I like the passion you clearly have for this topic

</comment></comment><comment><USER61> I try to use DDG and set it as my default search. But more often than not I end up doing !g. And if I don't, I feel like I'm missing the answer that will really help me work through my coding issues. After awhile, it just gets to be more annoying than anything, and I go back to Google.

</comment><comment><USER62> I can never remember the (short) url, so I end up Googling it so I don't have to spell it out..

<comment><USER10> ddg.gg or duckduckgo.com

You can also set it as default in any major browser either manually or using one of the extensions.

</comment></comment><comment><USER63> >>> Granted that DuckDuckGo.com is quite a childish name

Call me crazy, but I suspect that's part of the reason DuckDuckGo has had a hard time catching on. A name matters.

If I was DuckDuckGo I would rebrand to something simpler that could be used as a verb. But what do I know?

<comment><USER64> Agreed, it can't be verbified easily.

DDG admins: this is a real problem, you need to get creative on how to fix this.

</comment></comment><comment><USER65> Sure, just make me a chrome add-on that splits the tab in two vertical ones with DDG in the left and Google on the right whenever I search from the location bar which is nearly always so I don't have to search twice when DDG misunderstands what I want.

</comment><comment><USER66> One of many things that disgust me about the latest big revision to Opera (version 41) is that Opera Mobile doesn't include DDG as one of the (7) default search options (though it _does_ include Amazon, eBay, and IMDB). Ugh.

<comment><USER67> Opera got bought by an Asian ads company.

Vivaldi browser seems to be the spiritual successor of Opera by its previous developers.

</comment></comment><comment><USER68> @rms_returns, my only question is why are you asking us to use DDG rather than Google? You seem to be drawing some sort of direct comparison to Google, but you're not stating why we shouldn't use Google or what DDG does better.

</comment><comment><USER69> This is the sort of low-effort post I expect on reddit. And I come here to avoid that.

</comment><comment><USER70> I've tried multiple times to switch completely. Currently I use DDG on my iPod touch (which is what I use for web things on the go) but when I'm on the laptop or desktop I stick with Google as default.

</comment><comment><USER71> I personally found searx to be better than DDG, plus it's self-hosted. (It's a metasearch engine, you don't need an index)

</comment><comment><USER72> > But a search engine's results are only as accurate as the number of users > who search and contribute to it

Are there not privacy concerns about this for DDG?

</comment><comment><USER73> Also use bing more. I know my privacy has gone already - I just want to encourage competition for Google. The results are great 98% of the time.

</comment><comment><USER74> i think they do need to ditch that name though, whilst as you say google wasnt really any less odd in the early days however it did sort of roll off the tongue, where as duckduckgo is somewhat clumsy and unpleasant no matter how many times you say it. i know it shouldnt be important and its not really, its just annoying

</comment><comment><USER75> I've always had problems with their search results and I end up switching back to Google every time.

</comment><comment><USER76> Switched to DDG for most things 3 years ago.

</comment><comment><USER77> I actually tend to use Bing. _hides_

<comment><USER78> >I actually tend to use Bing. _hides_

I would find more appropriate in the context _ducks_ ...

<comment><USER77> Haha absolutely!

</comment></comment></comment><comment><USER79> i got bored from using Google. i like trying new products so i'll go with DDG until i get bored from it.

</comment><comment><USER80> Well ... No.

</comment><comment><USER81> ...

</comment><comment><USER82> Adverts on hackernews now eh?

<comment><USER83> Precisely. This is not what I expect to see on Hacker News. This isn't a discussion of DDG's abilities. It's simply an advertisement.

If this was a link to DDG's contribution page or some kind of information document or review, it would be different. However this just screams "USE THIS SERVICE FOR X REASON!"</comment></comment>


We built a website to provide kids with basic necessities - <AUTHOR> https://storylink.io <comment><AUTHOR> Hi guys, we are a small team from Europe and we built a website where underprivileged families can share their stories and needs. Anyone can directly fund their needs and we order the products from online stores so they get delivered to families' homes. For many of the families this is the only way to get fresh groceries which is usually the biggest challenge. People can also fund their christmas wishes. To prevent scams everything is transparent and families are submitted by our charity partner which is very respected in our country. We would appreciate any feedback. All best, Nasti

</comment><comment><USER2> I think the idea is great because fresh groceries really are the problem since charities usually provide only things such as flour, oil, sugar and stuff. Urban poverty is unfortunately still a big "shame" so the drawings are pretty cool solution for how not to expose families.

</comment><comment><USER3> Hey it looks like some "kids" are in their 30s. eg: "Maja, 34"

What's going on there?

<comment><AUTHOR> It depends on who is telling the story. So in that case this is their mother.</comment></comment>


StackOverflow for your Intranet - <AUTHOR> http://www.coordino.com/

<comment><USER2> What's the difference to other systems like this? For example OS QA [1] seems to be quite popular/established?

Experiences, anyone?

1: <http://www.osqa.net/>

<comment><USER3> OSQA is based on Django, which makes it somewhat harder to deploy. I know because we package it as a BitNami stack <http://bitnami.org/stack/osqa> and also use it for our Q/A site <http://answers.bitnami.org> Without actually having tested Coordino yet, I believe the fact that it is based on PHP should make it easier to adopt and extend. It will be interesting to see if it gets traction, there are plenty of StackOverflow clones that have been abandoned over time. An important factor in choosing OSQA was the fact that even though it was open source, it was backed by a company. A Q/A site, like forum software, is not something that you can move away from easily once you have content in it.

<comment><USER4> Having just installed Coordino in under 5 minutes (from git clone to login) on my Mac workstation at the office, I can attest to the ease of deployment.

We've been looking for an in-office knowledge base solution but we're a bit lazy (more accurately swamped with client work) so this was a no-brainer to try when I saw it was PHP.

<comment><USER5> I find it's installation method, like most PHP programs, to be somewhat off- putting. Yeah, it's great for a development box--place in front of a web server and hit the browser to finish up--but it's a nightmare for automated deployments: source code and configuration are two different assets that _should_ exist in two different places on the filesystem. When they do it's reliable to update one or the other without issue, else it's a bit chancy for my tastes.

</comment></comment></comment></comment><comment><USER6> I remember hearing in one of the first StackOverflow podcast that Jeff wanted to release the software as OS in and indefinite future, guess that Joel did not support that and now they are too successful to consider it again, in any case I can see why it would be disadvantageous after they changed their business model from third party hosted Q&A sites to the Area51 process.

<comment><USER7> Upvote, I posted an almost identical comment to your's a couple hours after your post. That will teach me to read through the comments before commenting:)

</comment></comment><comment><USER8> Was it really necessary to _completely_ rip off Stack Overflow's design?

<comment><USER9> "A FREE PHP StackOverflow Clone" - I think its title shows where it gets its inspiration from :)

<comment><USER10> There's a difference between inspiration and copying. It is also bad marketing if you don't have enough personal image to be identifiable.

</comment></comment></comment><comment><USER11> Hi Folks,

I maintain the Coordino project on GitHub located here: <https://github.com/Datawalke/Coordino> At the moment the project is somewhat stagnent as I work on some major improvements.

If you have any questions please feel free to ask. Thank you.

<comment><USER12> Could you please make it look a bit more unique (i.e. different than stackoverflow and the other stackexchange sites)? Most people won't bother changing the default template, and most people who end up on a hosted version of this software will then think it's run by Stack Exchange Inc. I'd like to avoid that situation as it causes us a bunch of administrative busywork and confusion to the interneters.

</comment></comment><comment><USER13> I think a StackOverflow style app would be powerful, and yet I don't want to add yet another "knowledge base" style application on top of, say, an internal wiki, ticket tracking systems, git commit messages, et cetera.

What do you use to tie these things together, and how do you decide what goes where? The line seems to blur between Q/A-style like Coordino and a wiki, in particular. Does anybody integrate these different knowledge systems well?

</comment><comment><USER14> I'm a tremendous fan and frequent user of various StackExchange sites, and I've been looking for places to use a tool like OSQA or Coordino.

I wonder, though, about the scale required. My sense is that for sites like StackOverflow and Hacker News:

 _Quality = Quantity + (Filtering, Sorting, Ranking)_

The software provides the second term, but I wonder how many responses or frequent posters/commenters one needs to have sufficient quantity. Any data on this?

</comment><comment><USER15> You should checkout Askbot, it's used in many places, easy to install, test cases and documentation.

[1]<http://askbot.org/>

</comment><comment><USER16> Fonts are way too small for the over-40 crowd -- and if it is a product for the Intranet, that's an important demographic.

<comment><USER17> Change them?

</comment></comment><comment><USER7> Interesting. This reminds me of the stackoverflow podcast where Joel and Jeff argued about open sourcing the stackoverflow engine.

Joel was against it because they were building stackexchagne at the time and he didn't want to compete against competitors like this using their own engine.

</comment><comment><USER18> Also;

<http://www.question2answer.org/>

</comment><comment><USER19> Maybe this is not the best place to ask this, but is there any SO clone written in Ruby?

<comment><USER20> Shapado [<http://gitorious.org/shapado>]

<comment><USER21> Here's the Github repo <https://github.com/ricodigo/shapado>

</comment></comment></comment><comment><USER22> I stood up a few OSqa sites recently:

<http://lostquery.com> \- Database FAQ

<http://zealotrush.com> \- Starcraft FAQ (other games too)

</comment><comment><USER23> What's the licence on this? GPL? LGPL? ASL? Can't find any info on their site for this

Edit: looked at the github, seems to be BSD-compatible

<comment><USER11> Torn,

The License is MIT based.

</comment></comment><comment><USER24> Is it written in CakePHP, or is their website just made with that? (Assumption from the favicon)

</comment><comment><USER25> did they steal github's logo?

(edit: nevermind...)

<comment><USER26> It looked like it at first, since I block JavaScript by default. Their menu system covered most of the page except for the github logo next to a "download now" link.</comment></comment>


The Newspeak Programming Language - <AUTHOR> http://www.newspeaklanguage.org/home

<comment><USER2> I have to say that having a language called Newspeak without a ++ operator feels like a huge missed opportunity. Writing ++good in Newspeak would probably be very satisfying.

<comment><AUTHOR> [humorless]

Gilad Bracha describes the reference as to the Newspeak dictionary getting smaller with each edition in _1984_ :

[http://www.se-radio.net/2009/07/episode-140-newspeak-and- plu...](http://www.se-radio.net/2009/07/episode-140-newspeak-and-pluggable- types-with-gilad-bracha/)

The idea of making a language reducible instead of extensible is a rather interesting design motif.

<comment><USER2> I know what I'll be listening to on my walk! Thanks!

</comment></comment></comment><comment><USER3> One of the interesting features is that modules receive parameters instead of importing names. So dependency injection is 'built-in'.

The author speaks about the design focus of newspeak (modularity, security, reflectivity, interoperability) a bit more here[0]

[0]: [http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6898742](http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6898742)

<comment><USER4> Since I've learned LambdaCalc and interpreters I've always been curious why modules were such an oddity, statements in most languages, and even in ML it's still a separate entity (even though research has been made to express it as a single core). Seeing newspeak a while back felt refreshing. I don't know if there are caveat doing things this way though.

</comment></comment><comment><USER5> Unfortunately, there has already been a newspeak programming language:

[http://www.eecs.berkeley.edu/Pubs/TechRpts/1983/5980.html](http://www.eecs.berkeley.edu/Pubs/TechRpts/1983/5980.html)

Yeah, from 1983, but still, I'll bet a search for "newspeak programming language" before they chose their name showed Foderaro's version pretty high up.

And yes, there was a real implementation of it, in Franz Lisp (the Lisp from UCB, not the Lisp from Franz, Inc.).

</comment><comment><USER6> Newspeak's UI paradigm is one of the most interesting things I've seen in years.

The Hopscotch video from 2009 at [http://www.newspeaklanguage.org/podcasts- and-talks](http://www.newspeaklanguage.org/podcasts-and-talks) covers some of it.

<comment><USER7> Like many other UIs influenced by the Xerox PARC environments.

</comment></comment><comment><USER8> Might be worth mentioning that the project has been abandoned for a few years (it ran out of funding) and that Gilad now works on Dart at Google.

<comment><AUTHOR> I suppose for some definition of _abandoned_ : The current download is December 1, 2014.

[http://www.newspeaklanguage.org/downloads](http://www.newspeaklanguage.org/downloads)

As I write this, several parts of the repository have commits less than 24 hours old:

[https://bitbucket.org/newspeaklanguage](https://bitbucket.org/newspeaklanguage)

</comment></comment><comment><USER9> Gilad Bracha & Peter Ahe are also working on Dart btw ([https://www.blossom.co/blog/who-is-behind- dart](https://www.blossom.co/blog/who-is-behind-dart)).

</comment><comment><USER10> Mildly disappointed that this wasn't a 1984-themed joke language.

> "war" == "peace"

true

> 2 + 2

5</comment>


Ask HN: Tax lawyer referal for founder stock questions - <AUTHOR> Can anyone recommend a personal tax lawyer that is versed in the complex avoidance schemes common among those who make it in the valley?
Context: I am a serial entrepreneur embarking on my next startup, and this time I intend to capitalize the company by buying my restricted shares at a nominal fee with funds from my Roth IRA. I believe this to be legal and not fall under the self-dealing / disqualified person rules because all of the founders, including me, are minority shareholders, we are not related, and my Roth IRA custodian is buying the shares directly from the company.
If Forbes reporting is to be believed, Max Levchin did this trick to shelter founding shares of Slide and Yelp, and Peter Thiel did similar things with Paypal and Facebook. However the lawyer at my IRA custodian (who is used to setting these things up for holding real estate and gold bullion, not founding shares of a private company) doesn't want to touch the deal even though he can't, when pressed, point to a reason why this wouldn't be allowed.
So I ask this community, who is in a unique position to worry about and take these sorts of things seriously, is there any tax lawyer in silicon valley who is likely to know the rules regarding self dealing and IRA accounts and be willing to make an opinion on this? Or generally a tax lawyer who knows how startups work and is willing to investigate?
EDIT: Repost because apparently accounts created through Tor have their submissions hidden. <comment><USER2> I'm very interested to see how this works out.

</comment><comment><USER3> Robert L Summers

<comment><AUTHOR> Which Robert Summers? Do you have contact information? A google search shows a few obituaries...

<comment><USER3> [http://www.taxprophet.com/](http://www.taxprophet.com/)</comment></comment></comment>


Is Tomcat an application server? - <AUTHOR> http://www.javaworld.com/javaworld/jw-01-2008/jw-01-tomcat6.html <comment><USER2> This is what's wrong with Java:

J2SE, J2EE, EJB, JDNI, JMS, JTA, WAR, JAR, ERP, DAO, EIS, JSP, JDBC, XML, HTML, EAR, JAAS, JAF, JPA, JMX, JAXP, JAX-RPC, JAXB, SAAJ, JCA, SOA

Every single one of those acronyms (and probably more) were used in that article.

And, if you read this [http://discuss.joelonsoftware.com/default.asp?joel.3.219431....](http://discuss.joelonsoftware.com/default.asp?joel.3.219431.12) and thought he was exaggerating, think again. This is what I had to do to parse an XML file in Java the other day:

        DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
        DocumentBuilder builder = factory.newDocumentBuilder();
        Document doc = builder.parse(new File("something.xml"));
        Element root = doc.getDocumentElement();

_Sigh_

</comment><comment><AUTHOR> Bleh, this article made me want to hurl. While Java EE developers are debating the definition of 'application server', developers using other platforms are building applications (and 'enterprise'-worthy applications at that)!

</comment><comment><USER3> Yes, the author spent way too much time before getting to his relatively trivial point:

 _Where Tomcat and other Web servers fall short is in the area of features such as distributed transactions, EJBs, and JMS. Applications requiring support for these components are usually more at home in with a Java EE application server such as JBoss, Geronimo, WebLogic, WebSphere, or Glassfish. Many Java EE application servers actually use Tomcat as their Web container._</comment>


Ask HN: Are documentaries becoming more destructive than informative? - <AUTHOR> I won't name any but with regards to lets say the past 20 years or so (perhaps since 2002, wink wink), and by mere virtue of the genre, it seems like the public considers them as proven fact, especially if you push the emotional narrative to dark places that people could never typically envision and ever so slightly twist "facts" around though creative editing. <comment><USER2> Interestingly I just watched this video. [https://youtu.be/BgQ79evjylc?list=LLbN1lC_4raev2geuc6RhTpQ](https://youtu.be/BgQ79evjylc?list=LLbN1lC_4raev2geuc6RhTpQ)

And I was surprised about how they manipulate the information, well not the information but their knowledge of how you will react and thus push an emotion that does not match the information.

</comment><comment><USER3> Can you give examples? I'm a huge doc nerd and in my opinion we're in golden age of documentaries right now in terms of film-making. Documentaries have always been a way to express opinions though (personal or political), nothing surprising here. Basically anything made my humans will be stained with their worldview.

Some people believe Google or news outlets to be objective depictions of reality, I think, we just tend to oversimplify and not look too deeply into things that are not our primary concern.

<comment><AUTHOR> Bowling for Columbine, Blackfish, most MJ documentaries.</comment></comment>


Ask HN: Does anyone else not like LinkedIn's redesign? - <AUTHOR>

I'm no designer but to me I think it's a step backwards. <comment><USER2> Really, really cluttered, but probably really, really functional. It's probably (and hopefully) the result of mining through a _lot_ of analytics data.</comment>


The North Pole is an insane 20C warmer than normal as winter descends - <AUTHOR> http://www.smh.com.au/environment/weather/the-north-pole-is-an-insane-20c-warmer-than-normal-as-winter-descends-20161117-gss3bg.html <comment><USER2> While I'm certainly no denier of the scientific part of the global warming phenomenon, I tend to have healthy scepticism of these headlines which seem to solicit an alarmist reaction. Short-term weather can have high volatility and deviance from the norm, and that's nothing new.

<comment><USER3> We are aware of the difference between weather and climate. Most of us are also aware that these significantly elevated temperatures have persisted for a while now rather than being outlying data points, and are heartily sick of attempts to derail the debate with facile arguments like the one advanced here.

Of course, that might not be your motivation in posting it, but since you saw fit to speculate about the motivation behind the headline I feel equally free to speculate about the motivations behind your 'skepticism.'

<comment><USER2> The title is:

>The North Pole is an insane 20C warmer than normal

To be fair, in that context it is really not facile to point out the difference b/w weather and climate. If anything the title is facile. It says nothing of this difference, yet exclaims, "insane."

While you speculate about me derailing "the debate," (what exactly is "the debate?" Is there only one? regarding the science, the politics, what am I exactly derailing?) You're not clear about how I would be trying to derail the debate. I did assert the scientific consensus regarding global warming in my first sentence, so I don't know what you're getting at.

My point is that the alarmism around global warming is what's up for debate: I think we have all met people who think the world is soon coming to an end. Yet, despite the hysteria, we aren't doing enough in our public policy to combat global warming. Have you seen two irrational minds discuss global warming: one convinced the world is coming to an end ("go travel now before all the world's coastal cities are underwater in 5 years") fighting against the other "that's just lieberals making stuff up." It's not longer about the science, that case has been closed, it's about the politics, and how is it constructive for either side to radicalize their followers?</comment></comment></comment>


Public Domain Day: Works Enter Public Domain for the First Time in 21 Years - <AUTHOR> http://www.openculture.com/2019/01/public-domain-day-is-coming.html <comment><USER2> Or perhaps more significantly: for only the second time in 41 years.

For an entire generation, works simply didn't enter the public domain:

        1921 works became public domain in 1977.
        Copyright Act extends the term by 20 years in 1978.
        1922 works became public domain in 1998.
        Sonny Bono Act extends the term by 20 years in 1998.
        1923 works became public domain in 2019.

Were it not for these two extensions, we would today see works from 1963 entering the public domain.

Instead, we aren't even up to the first 'talkie' motion pictures.

<comment><USER3> Is this true? The few references I find indicate 1921 works became public domain only in 1997. Perhaps it was 1920 works which became public domain in 1977?

</comment></comment><comment><USER4> A rare victory for public interest over rentier interests. I so hope it becomes a trend.

</comment><comment><USER5> [https://news.ycombinator.com/item?id=18801531](https://news.ycombinator.com/item?id=18801531)

</comment><comment><USER6> It is crazy to think that if you live to be 95+ you'll get to see something enter the public domain in your lifetime.</comment>


What Is Zoned Storage and the Zoned Storage Initiative? - <AUTHOR> https://blog.westerndigital.com/what-is-zoned-storage-initiative/ <comment><USER3> For anyone wondering "why", this linked article in the source gives some background, [https://blog.westerndigital.com/storage-architectures- zettab...](https://blog.westerndigital.com/storage-architectures-zettabyte- age/)

So instead of SMR harddrives and SSDs doing extra work to hide their deficiencies, they're pushing this up into the filesystem. In return you get slightly more storage and slightly faster IO. Perhaps not useful on a desktop, but very useful when dealing with thousands of storage devices in a data centre.

In terms of how it would be exposed to users, this feels very well fitted with object storage. Unlike file-based storage, each object can only be read or replaced. So any partial write relies on the program to read the whole object, then write the updated object.

<comment><USER4> It also fits well log-structured and copy-on-write filesystems.

<comment><USER5> Yes! That was my first thought. The way SMR and SSDs work actually aligns pretty well with log structured filesystems.

In this case, the firmware translation layer is only getting in the way.

</comment></comment></comment><comment><USER6> " In SMR, unlike conventional recording, tracks are written in an overlapping manner. ... once the tracks are overlapped, they cannot be written independently. there are disadvantages for device-side localized management. ... Managing the complexity on the host side is almost a requirement"

also links: [https://www.zonedstorage.io/](https://www.zonedstorage.io/) for further information.

Sounds good to me; exposure of the actual mechanisms instead of outdated abstractions makes sense. I'd even argue for open access to motor controllers and raw signal buffers but apparently I'm insane.

<comment><USER7> Host managed also means messing with the file system, and the patch set for btrfs is not small. Plus you have to disable a bunch of features (like preallocation since you can't move the write pointer backwards) which is going to surprise people in unfun ways. These drives are useless for general purpose. If you are going to use them as expensive tape then by all means, but otherwise I have serious doubts about their usefulnes.

Edit: I'm talking specifically about the SMR side. The general zoned stuff is interesting, but when you start putting restrictions on how you can write to certain zones you wind up with a lot of weirdness that application developers are going to be surprised by.

<comment><USER8> Drive-managed SMR looks OK to me for general use, although I don't have hands- on experience with it. They also have a non-shingled region that can accept random I/O at higher speeds, and the drive deals with moving data to a linear- only region once it fills up, or at garbage collection time.

It's very much like the QLC drives where a portion is treated as SLC/MLC.

<comment><USER9> It's like the QLC drives with SLC/MLC cache (e.g., Intel 660p) except the performance cliff is far, far worse (and obviously the "fast" mode is far worse as well).

</comment></comment></comment></comment><comment><USER10> Candidly, the manufacturers haven't shown enough of a price savings to make these drives worthwhile. As others have mentioned, there's a fairly significant cost in development to using these drives efficiently. The only way that makes sense is if there's a significant discount over non-SMR drives. Nice blog post WD, but you're going to have to drop the price to about half of what it currently is for these to make even a little bit of sense

<comment><USER11> Half of the world bits from HDDs are estimated to be on SMR in 2023 - <USER2> gains are significant when deploying at scale.

For SSDs, it gets even more fun as zones aligns with the characteristics of the media and you get this effect of significant increase in capacity (20% with a 28% OP drive), order of magnitude reduction in dram, and eliminates device side garbage collection on the drive (commonly between >1-5), which improves the QoS considerably.

Additionally, one can now run the drives at 100% capacity utilization - conventional drives becomes slower due to increasing device write amplification.

</comment></comment><comment><USER9> Zoned SMR storage has almost no use case outside of tape replacement.

The same drives configured in 100% random I/O mode lose a handful percent capacity and the tradeoff for SMR zones is awful: rewrite your entire filesystem stack (thousands of labor hours) _and_ expect awful performance. For most businesses this is not an obvious win.

Meanwhile, big cheap dense SSDs get bigger, cheaper, and denser every day. You don't have to rototill your filesystem to use them effectively. So I don't see a lot of sense in investing in zoned filesystem support when archive tier bottom-dollar storage can just use the unzoned, marginally smaller spinning rust today, and switch to the unzoned, super lower write endurance flash media tomorrow.

<comment><USER12> SMR does have a use case outside of tape replacement. It has higher density. Take a look at DropBox MagicPocket - they designed their entire storage system around SMR technology

<comment><USER9> It has _marginally_ higher density; I didn't forget this benefit. The tradeoff is something like 5% more space vs much worse performance and many man-years rewriting filesystems.

</comment></comment></comment><comment><USER13> I want write once storage. For photo backup. No virus or user error can ever delete my photos.

<comment><USER9> There's always optical media or SD cards with physical write-protect toggles. (If you leave the optical media in the writer, though, conceivably an absurdly specific virus could cause the device to burn out existing contents.)

<comment><USER14> The read/write switch on SD cards is 100% up to the reader device to honor or ignore (seriously)

</comment></comment></comment><comment><USER15> What's the difference between this and RAID?

<comment><USER15> Like RAID-0 specially

</comment></comment><comment><USER16> On my Firefox Android this link goes into what seems infinite redirects. Um, thanks?

<comment><USER17> Works fine on Firefox for android for me

I did just experience Firefox opening infinite tabs when I opened a PDF last week though, but I tracked that down to a problem I introduced, where I told the OS to open PDFs in Firefox, and Firefox was still set to defer to the OS for PDF handling. Could there be something similar going on here?

</comment></comment><comment><USER18> I seriously question the value of having this bloat up the kernel and general purpose filesystems. Tapes work just fine with userspace support and there isn't any reason for not having custom FUSE's for this as well.

Put another way, everyone suffers with code bloat and extra complexity for the tiny percentage of users (which are large, and can manage their own systems) this supports.

Edit:

The SMR drives should just support the SCSI streaming command set and be done with it. That way nothing really needs to change anywhere but in the drive firmware.</comment>


The music classifying nightmare - <AUTHOR> http://blog.pkh.me/p/15-the-music-classifying-nightmare.html <comment><USER5> You're making the problem a lot harder than it needs, or ought, to be. The most telling example, IMO, is the Aphex Twin one...

        > Oh, and his real name is Richard David James. What are you
        > supposed to use for the file system directories and files
        > name? His name? The most common nickname? Both? One file
        > system solution is to have symbolic links (do you link
        > Richard David James to Aphex Twin, or vice versa?). For
        > tags, if you don't want to lose information, this is
        > another story...

You should—obviously—not attempt to "link" AFX to Aphex Twin to GAK to Blue Calx to etc. etc. The artist behind all of the monikers has made a deliberate decision to release work under different names. Organize accordingly.

Many of your nightmare scenarios appear to be a result of the same kind of over-thinking, or invention of nonsensical requirements. How are you supposed to deal with Japanese artist names? It literally doesn't matter—pick a scheme you can understand, and be consistent. How are you supposed to deal with multiple artists? List them, separated by commas, in the artist field. If they appear on an album released by a different group or person, use the "album artist" ID3 field. And since (if?) you use the ID3 fields to store your metadata, and presumably navigate your collection through an interface over that metadata, all of your questions regarding how to store files physically on disk are totally irrelevant, as long as you pick some scheme which doesn't generate conflicts. The default iTunes structure (Artist/Album/01 - <USER2> Title.mp3) seems to work fine, for example.

<comment><AUTHOR> Well, it's the same person behind. Even if the artist tried to give a different "personality" to his music over the time, following all his creations is actually a good listening experience. The "hey I just want to listen to the musical evolution of this guy" isn't a rare feeling IMO. Grabbing all his nicknames is kind of a problem.

Of course, you have a point, I'm making the problem harder than it needs, and you can just don't care about most of these issues. Hey, that's actually to the conclusion I reached. But still, I believe the current solutions are not optimal if you want to match N songs, store all the related "context" information, or just keep a consistent way of storing them.

<comment><USER6> But that's not a problem, either. In my music collection I have, e.g. tags ARTIST=AFX and FILED_UNDER=Aphex Twin. Then when you make a playlist with your favorite music player, you can just sort things by FILED_UNDER, if you prefer. What's so hard about that?

<comment><AUTHOR> Interesting, so you have a way of making the relationship at the end. Now I could start nitpicking about how you decide to make the link to Aphex Twin and not his name (or another nick), and how would you make that decision for any similar cases.

Note that I don't consider this issue the main problem, it's just one I hit a few times, and I wasn't able to select a correct solution.

<comment><USER7> Discogs tracks Alternative Nicks that artists use, and can help you link songs together. Here is Aphex Twin example:

<http://www.discogs.com/artist/Aphex+Twin>

Also, discogs will track which albums were released under different artist tags. Foobar+discogs tagger combination lets you pick, if you want to use most common general name (so all tracks are Aphex twin), or the alternative name that songs were released under (aka: AFX songs stay as AFX).

\---

This is same issue as a book ''Running Man'' (yes, followed by a movie with Arnold). Was it written by Richard Bachman, or Steven King? How does your public library list book?

<comment><USER8> Same for MusicBrainz: 'aliases' <http://musicbrainz.org/doc/Artist#Alias>

</comment></comment></comment></comment></comment></comment><comment><USER9> Have you tried Quodlibet? <https://code.google.com/p/quodlibet/>

It comes with a Musicbrainz plugin which allows you to select an album in your library and search for it on musicbrainz (e.g. by artist name and year or number of tracks-- so you really dont need to have much info) and then you can select a match (there are usually many releases of a given album and it will actually differentiate between them) and then tag your album with the musicbrainz tags.

In fact you don't even need to use quodlibet to get this feature. It has a separate tagging component called "ex falso" which you can run standalone and then use the player of your choice.

But I would strongly recommend Quodlibet for its organizational capabilities as well. For example it uses special internal tags (not id3 but stored in a separate db) that allow you to associate "people" and "performers" to a track so that the track will appear when you search for any of those people.

Also there are sort tags that allow you to customize where stuff shows up, e.g. I can have a track with Artist tag "London Symphony Orchestra", composer tag "Ludwig von Beethoven" (the proper ID3 tags) BUT I give it the artistsort tag "Beethoven" so it shows up under B. Perfect!

See <https://code.google.com/p/quodlibet/wiki/AudioTags> or more info.

Lastly it has regex search! And you can make "saved searhes" e.g. playlists. And it's lightweight and has an uncluttered (but highly customizable) interface. And it's very easy to write plugins for it.

<comment><AUTHOR> Nice, I'm sure a lot of people would be interested in trying this.

But talking to myself, I'm disgusted with all of this, so I'm just maintaining my mess in its current state for now.

Still, I like to see such solution, and I'd be really interested in a counter article to what I wrote dealing with each issue. Even if at the end, I will likely not use the given solution.

About the regex search, I'm not sure that's really the solution to the "textual problem". As mentioned in the article, the music content retrieval system is in my opinion the future. Echonest and similar services are trying to achieve something like this. Looking for one artist isn't really what you actually want most of the time. It's likely you are looking for good music, and just want to listen to things who sound "like this".

<comment><USER9> Sorry-- I was unclear. The regex is not for organization. It's just an easy and flexible way to browse subsets of your library, like smart playlists in iTunes, but more powerful.

<comment><AUTHOR> I was actually talking about browsing. The musical content analysis has the goal to provide new ways of representing your music, and browse it.

</comment></comment></comment></comment><comment><USER10> I only go so far as sanitizing and standardizing my music collection through Tag&Rename (and I haven't found a good OSX equivalent to this yet sadly). It gets the data from Amazon in 98% of cases, adds the album art (which I like having on my player), etc. Then I store the files in:

Artist/Album/Track# - Song name [- Artist name]

The last is only there for soundtracks and other "Various Artists" type collections.

This is Good Enough [tm] for me. I can sync this across hard drives (backup), minimize duplicates (although I end up with these through compilations of various sorts), etc.

Unfortunately the ID3 tag system is All Wrong [tm] for this in many ways stated (in this post and elsewhere). For example:

\- Albums don't really have an artist; songs do;

\- Programs for automating this that get info from Amazon and elsewhere tend to use what year the particular CD was released rather than when it was _originally_ released, which is far more useful and relevant (eg if you want the Beatles White album you don't care the CD was released in 1998, it should come up under 1968;

\- Albums don't really have years either. Or at least they have publication years. The songs have years. Normal studio albums have a common year. Compilations and soundtracks do not;

\- Genres are coarse-grained, arbitrary and (IMHO) mostly useless;

\- What I like is greatly influenced by the circumstances around the song, sometimes more than the song itself. I might like a song because it reminds me of a particular person, place or event. Or even _mood_. Sometimes its the lyrics. Sometimes its the sound. No recommendation engine is going to capture this sort of angle.

This goes beyond music: people just aren't interested in classifying, well, pretty much anything. Playlists seem to be about as far as most people are willing to go. Playlists are a fairly convenient way of coming up with s event-specific music eg for working out, for relaxing, for dancing, for a party, etc.

Efforts at far strong and more accurate metadata, classification and organization speak more about one's festidious--even anal-retentiveness--more than any real need or better outcome (IMHO). It's just rabbit-holing really.

<comment><USER11> I'm not sure how much of this is conjecture. My thinking differs from yours with respect to the following -

"Albums don't really have years either. Or at least they have publication years. The songs have years. Normal studio albums have a common year. Compilations and soundtracks do not."

An album that is released by a record label does have a year, as upon release it becomes a publication. By this train of thinking i would also say a soundtrack does have a year, too, as it is published as a collection timely to the context of its release. The same can be said of compilations, as they are a contemporary release.

<comment><USER12> Yes, but you're usually more interested in when the song came out than when the album came out.

If you sort your playlist by date so you can pick songs from the 90's, you probably wanted to hear something like Nirvana instead of The Best of The Who: Volume 3.

<comment><USER7> I agree that year is screwed up, but some taggers (like the picard-lastfm tagger) reads also decade feel tags. So music that sounds like 90s is tagged as 90s music, even if it isn't recorded in 90s.

For example, ""La Roux -- In It For The Kill"" will get the song tagged as sounding 80s like:

<http://www.last.fm/music/La+Roux/_/In+for+the+Kill/+tags>

</comment></comment></comment></comment><comment><USER13> Somewhat related but as I've been building a rails project I've been meaning to open source the song parser I've been building alongside it. It scans an mp3 and pulls out the artists along with the type of role they played on the song. Here's a quick gist I pulled from my model:

<https://gist.github.com/3680949>

Some examples:

Drake - The Motto (Jon Bellion Cover)

=> [["Jon Bellion", :cover], ["Drake", :original]]

David Byrne and Brian Eno - Strange Overtones

=> [["David Byrne", :original], ["Brian Eno", :original]]

Cheri Coke, MELO-X - <USER3>

=> [["Cheri Coke", :original], ["MELO-X", :original]]

Avicii - Street Dancer (Whelan & Discala Remix)

=> [["Whelan", :remixer], ["Discala", :remixer], ["Avicii", :original]]

RAC - Hollywood featuring Penguin Prison (The Magician Remix)

=> [["Penguin Prison", :featured], ["The Magician", :remixer], ["RAC", :original]]

And a ridiculous example:

Eight, Nine & Ten (Eleven cover - Song name feat. One, Two & Three (Prod. by Four) (Five & Six remix) (Seven cover)

=> [["One", :featured], ["Three", :featured], ["Two", :featured], ["Four", :producer], ["Five", :remixer], ["Six", :remixer], ["Seven", :cover], ["Eight", :original], ["Nine", :original], ["Ten", :original], ["Eleven", :cover]]

If there's any interest, I'd love to turn it into a proper github repo and accept some pull requests.. it's far from perfect (both code-wise and generally) but works well for most cases.

<comment><USER14> This looks awesome. You should consider sticking it somewhere and giving it a name, so that I might find it 6 months from now when I'm motivated to fix my shitty Python script [1].

Also, a side question: is it normal/considered a best practice in ruby to monkey-patch built-in libraries like that? I know ruby _has_ open classes, but I'm just curious how using them in this way is regarded.

[1]: <http://news.ycombinator.com/item?id=4494437>

<comment><USER13> I put the link to the gist with the code for now, when I get a couple hours free sometime (soon I promise) I will do just that and put it on my github, so feel free to watch my account there.

As for patching string.rb I don't know 100% but I believe thats the point of being full OO is that I can make a patch like that where it makes sense to do it. I was using those functions in multiple places, and given they are meant for strings it made sense. I can see the potential pitfalls when it comes to sharing things like this, but again, Ruby makes it easy for anyone to make the same patch for a reason.

</comment></comment></comment><comment><USER15> Use hierarchical directories. They work. I sort my mp3's into _two_ distinct music collections:

      /Albums/
      /Singles/

Then I sort by genre:

      /Albums/%GENRE
      /Singles/%GENRE

In Albums, I then sort by artist, then by album:

      /Albums/%GENRE/%ARTIST/%ALBUM/# - %SONG.mp3

Whereas in Singles, I organize by:

      /Singles/%GENRE/%ARTIST - %SONG.mp3

This system scales very well. I have a 100GB collection and can nail down any song or album in my collection in a few clicks.

<comment><USER7> I can see few issues with this that wouldn't work with me:

1) How do you deal with songs that have multiple genres?

It is a standard part of the TCON (content type / genre tag) to separate multiple values through either coma, slash.

For example this allows you to select songs that are tagged as both 'Ambient' and 'Rock'.

2) how do you deal with album that has songs of varying styles?

For example: "Prodigy - Music for Jilted Generation" has a dance track like "Voodoo People", and a chillout track "3 Kilos"

<http://www.last.fm/music/The+Prodigy/_/Voodoo+People/+tags>

[http://www.last.fm/music/The+Prodigy/_/The+Narcotic+Suite:+3...](http://www.last.fm/music/The+Prodigy/_/The+Narcotic+Suite:+3+Kilos/+tags)

3) Compilations

Let's say we have a compilation made by a group called 'Air'. This compilation is called 'Deck Safari'. Do you use Album Artist instead of artist if it is a compilations (aka: if TCMP flag exists, use TPE2 instead of TPE1?)

Checkout the album:

<http://www.discogs.com/AIR-Deck-Safari-Part-1/release/215068>

4) I don't understand how you handle singles.

Let's say you have a single ""Wamdue Project – King Of My Castle"" released in 1999 by Urban (563 891-2). Your formatting doesn't store album at all?

[http://www.discogs.com/Wamdue-Project-King-Of-My- Castle/rele...](http://www.discogs.com/Wamdue-Project-King-Of-My- Castle/release/44719)

And then, how do you combine it with the fact that a year later was a different mix released by Avex Asia Ltd. (AVTCDS-235) with different set of songs?

[http://www.discogs.com/Wamdue-Project-King-Of-My- Castle/rele...](http://www.discogs.com/Wamdue-Project-King-Of-My- Castle/release/1352484)

To me it looks like your singles folder is intended for random songs, not to handle actual CDS/CDM style singles.

<comment><USER15> 1) How do you deal with songs that have multiple genres?

In "Singles" I sometimes copy individual MP3's into both Rock and Metal. It comes up less often than you'd think.

2) how do you deal with album that has songs of varying styles?

I just pick the one genre that makes the most sense. As long as I know, or can quickly find which genre an artist is in, that's what matters. How long does it really take to check if I put an AC/DC album in Rock, or Metal?

3) Compilations

I either place it in /Albums/Soundtracks (if it's a soundtrack), or make a "Various Artists" directory in /Albums/Electronic or whichever genre makes sense. Almost all of my compilations are dance or electronic music.

4) I don't understand how you handle singles.... To me it looks like your singles folder is intended for random songs, not to handle actual CDS/CDM style singles

I actually removed most of the EP's and CDS rips from my MP3 collection because I found I never listened to them. I would take the individual MP3's that I really liked and placed those into my /Singles/ directories. Otherwise I could treat them just like I do my Albums.

<comment><USER7> I guess I enjoy putting more tags into collection. I try to fit few from last.fm , and few that came with the tagging. So a single track can have multiple tags (I think I average 10 tags per track). It is normal to have things like like 'Symphonic' and 'Rock' and '80s' and 'Krautrock' as separate tags on one song. Or in case of some electronica (like ones found at <http://ektoplasm.com> ): 'IDM', 'downtempo', 'goa', 'psy-dub'.

For compilations I try to keep track of who made compilation, especially if it is a mix by a DJ. I do have many that are just VAs too.

In the end, I have three types of music plays:

\- everything by an artist -- this is why I keep CDS / CDM / EP to have remixes, and to track them with covers, etc

\- specific album -- this works great for movie scores, or albums that have songs tailored to go one to another (Jean Michelle Jarre work from 70s and 80s is awesome in this)

\- party mode -- something like pandora -- last.fm is queried about tracks similar to current one (and past one), and sets up playlist of similar songs to play. See <http://forum.xbmc.org/showthread.php?tid=83915>

</comment></comment></comment></comment><comment><USER16> What I lack (and I doubt I'm the only one) is well-thought-out tag system and more advanced players.

All textual entries (title, album, author, ...) should be stored in original language using original alphabet. Player could transliterate them if user doesn't know the alphabet (e.g. doing romanization of hiragana, katakana and kanji using Hepburn system for japanese music). Such entry should be able to store also a translated text, usually at least English one for non-English stuff.

That would solve also another problem the blog post author mentions, first name and last name ordering issue. Quoting Wikipedia:

"In Hungary, along with China, Korea, Japan and in many other East Asian countries, the family name is placed before a person's given name."

Thus in original language the order would be original, but in English one, Western-style, i.e. placing last name after the first name (and of course transliterated already).

It would be up to a user to choose what player should show her or him: original names, transliterated names, translated names.

But AFAIK ID3v2 and Vorbis don't support such stuff (well, you can try going with custom keys, but non-standard means mess) and I haven't heard about player that would do any transliteration either.

\---

As for filenames I think that the best is Latin alphabet, with the most-widely used romanizations of non-Latin alphabets and simplifications of extended Latin alphabets (like removing diactric marks etc.). Clean visible ASCII!

I know that Asian people would mostly disagree on such file naming rule. :)

</comment><comment><USER17> doesn't the musicbrainz schema cover most/some of this? <http://musicbrainz.org/doc/MusicBrainz_Database/Schema>

also, given the ubiquity of UTF-8, why the need for ASCII?

<comment><AUTHOR> It would be interesting to make an analysis on how MusicBrainz deals with all these problems, I admit I didn't look much into this. But AFAICT it wouldn't really solve the file system problem, except if you decide to name your files with a hash. Also, you might still want at some point to keep extra information MusicBrainz wouldn't handle, even if you have a MusicBrainz ID stored in the file to identify the music.

About the ASCII, my point was just all about the fact that you can't actually keep only the international name and you need to store the name in different language version. Obviously, I don't have any problem with using UTF-8.

PS: note that UTF-8 won't be able to represent properly mathematical formula... :)

<comment><USER17> yeah, sorry, was in a rush to go eat. didn't really mean that your worries would all go away, only that musicbrainz might be a good place to look for more info.

</comment></comment></comment><comment><USER18> The single most annoying thing I encountered was the inability of ID3 to handle multiple albums. Every artist sooner or later releases the exact same piece of music on another album. WHY DIDN'T THEY THINK OF THAT?

<comment><USER19> That's irrelevant, it's either ripped from one or the other, and that's the information that should be there. Lookinng up on which albums a song has appeared should be a database / wikipedia lookup. Think about it, is it relevant on which christmas compilations "so this is christmas" has appeared?

<comment><USER20> It's not irrelevant. If the exact same recording of a song appears on two albums, an ideal categorization system would allow it to appear as such without storing multiple (redundant) copies of the song. An easy way to think of it is a normalized relational database. If the song has an id of 5 and is on two albums with id 2 and 3, then there would be an AlbumSong join table with schema (album_id, song_id) and two rows: (2, 5) and (3, 5). It should be irrelevant which album you ripped or downloaded first.

<comment><USER5>
        >  If the exact same recording of a song appears on two
        >  albums, an ideal categorization system would allow it to
        >  appear as such without storing multiple (redundant) copies
        >  of the song.

I don't agree. The song was released twice; if you have both albums, you ought to have two copies of it.

<comment><USER20> Yeah, it's not going to be a big storage hit in reality (and even if it were, you could just rely on file system level compression/deduplication). I, like the author, just enjoy musing about the "perfect" way to organize a music library.

<comment><USER7> Some artists re-record or alter their song on secondary release. Or if it is live performance.

Deduplication should be done by algorithm that detects that it is same binary copy -- not through tags.

</comment></comment></comment></comment></comment></comment><comment><USER21> There seems to be confusion between the filesystem and the metadata.

The filesystem is for storage... it's only important to be able to group tracks together in small batches (releases - albums/singles/EPs) to be able to manage the files

The metadata is for searching, grouping and locating in your music player.

With that in mind, a lot of the problems he's cited vanish.

I have ~84,000 tracks from over 6,000 albums. The result of a 5 year ripping spree after a decade worked in the music industry.

Every track, with no exception, has been scanned by MusicBrainz Picard, had it's PUID generated and meta data normalised.

I've allowed my definition of genre be influenced by general opinion... I simply learned how the mass tag things.

The result is that I can find everything in my interface (Squeezebox) within seconds.

The file storage I only need to care about for management of the files, and copying to my portable player or deleting old releases (which I do rarely, but it does happen).

The file storage starts with file type:

MP3|FLAC|FLAC_9624

Below those are folders for high level type of content:

Artists|Classical|Spoken Word|Compilations|Soundtracks

Within Artists I used

Artist/Release Name[ - Catalogue Number]/[Volume/][Media/]Track Number - <USER4> Name

I only fill in catalogue number if I have two versions of the same titled release... i.e. Quadrophenia by The Who I have a couple of versions of.

I only fill out Volume if this is a multi-volume release.

I only fill out Media if this is a release that spans multiple CDs or DVDs.

So the short version of that might be:

FLAC/Artists/The Who/A Quick One/02 - Boris the Spider.flac

And a long version might look like:

FLAC/Artists/The Who/Quadrophenia - Polydor 2777840/CD2/01 - 5.15.flac

I have no problem at all storing and finding tracks, and I've no problem at all searching for tracks.

One of the good things about Squeezebox is that when you search it searches all metadata and the full file path. So a search for "quadro poly" would turn up the Polydor version of Quadrophenia.

<comment><USER22> What is Squeezebox exactly? I found many programs named like that and the one I think is you are talking about seems to have been discontinued now. Do you have a link or something?

<comment><USER23> I think he is referring to the Logitech Squeezebox:

[http://en.wikipedia.org/wiki/Squeezebox_(network_music_playe...](http://en.wikipedia.org/wiki/Squeezebox_\(network_music_player\))

<comment><USER21> That's the one.

I use Squeezebox server on a QNAP NAS storage thing.

Then I use a Squeezebox Transporter for the superlative DAC that it has.

Squeezebox server allows me to listen remotely too, so for more than a decade I've had the ability to listen to my home music collection from wherever I am in the world (assuming I have an internet connection). I use DNSMadeEasy for their dyndns stuff.

</comment></comment></comment></comment><comment><USER24> Having worked with it for a few years, modelling music metadata is indeed an absolute nightmare.

There are some efforts to standardise this stuff though, see <http://www.ddex.net> which a lot of the digital supply chain is starting to adopt. It's something of a set of scary great kitchen-sink XML schemas (schemata?) but might be of interest to those who get massively nerdy about this sort of thing.

</comment><comment><USER25> This sounds like exactly the kind of problem that relational databases were designed to solve. You can organize it with an entity-relationship model fairly easily. Once you have stored your information in a database, the filename doesn't really matter as long as each mp3, ogg, etc. gets a unique name; you can look up the file by querying the database for files that have the properties you care about.

<comment><USER7> This isn't something that relational database fixes. It is about schema and data tagging. It can all still sit in ID3v2 tags, as long as people use all tags properly.

The problem is that most tags are badly written, and most software that displays songs only displays bare minimum. Typically tags I miss seeing: Style / Moods / Genres / Composers / Performers. Even Album Artist or Compilation tags are bad. Same with some misspelled tags, or inconsistent (is it 'D&B' or 'DnB' or 'Drum-n-Base', etc).

<comment><USER26> I think Figs is onto something though: the OP is clearly worrying about how to stuff all this information into the filesystem, and the plain fact is that while all of the information needs to be accessible beyond the database, either in the hierarchy or in the tag embedded in the file itself, a lot of that information is not structured in a manner conducive to access via those two rather limited methods.

For the sake of argument, let's be platonic idealists for a second and say that a song is a concept and the files are simply recordings of performances—they're not "songs" at all, per se. Say I play "Black Betty" by Lead Belly. I get curious about the song, so I want to hear other versions of it. There's no way to express this data in ID3 or through a naming convention—it's just a reference to a concept that has no existence in a pile of MP3s—and it's hard to imagine a linking regime that would be maintainable. To do this, you need a place to put metadata that isn't attached to the files themselves, which can point to them. Note that this kind of scenario is not outside the OP's scope: they say in the article they are interested in providing all music to everyone, so these kinds of relationships would need to be documented and searchable.

I personally think a relational database is a fine tool for this problem, but lately I've been looking at RDF for things and I think RDF might be an even better choice, since it's really designed for metadata description, ontologies (like music categorization) and graphs. I think it would turn out to be a ton of work, but I think it would satisfy many of the requirements.

</comment></comment></comment><comment><USER27> Oh god, I tried renaming files on my own (and I'm kind of a beginner programmer). Then I found out about unicode problems and then came 2 problems.

problem #1 was apparently I set the encoding wrong while renaming the ID3 for the music files so the foreign languages turned into question marks. I thought scratch that, time to use someone else's tool. foobar2000 solved everything I had a problem with except problem #2.

for problem #2, I have no idea if it'll ever be solved. The question marks or blank boxes ending up on my computer deal with the encoding for the OS itself (or something). And even with multi-language support, windows doesn't let me fix that. There's obviously something wrong when my blank box issue disappears when I restart my computer from time to time...

<comment><USER14> This is the shitty python script I use to organize music, which seems to work pretty well except in one case[1]: <https://gist.github.com/3681165>.

For the love of god, try it on a small sample of your music or it will wreak all kinds of havoc and you will yell and curse my name.

What it does: takes a directory of folders with MP3s, and makes them into the a directory of folders like yourdir/[ARTIST]/[ALBUM]/[TRACK No.] - [Track name].mp3. More importantly though, I _believe_ it handles unicode correctly, and it's just over 100 lines of code, so you could adapt it to whatever system you prefer.

Find the python library called Mutagen that deals with parsing ID3 tags and put it in the same directory as this horrible script.

[1]: The one case it handles spectacularly poorly is when you have an album with multiple artists. This could be a soundtrack, a compilation, or even that thing that happens especially often in rap where the artist is like "[Some dude] ft. [Some other dude]". Honestly, I just handle those cases manually, but it might not be that hard to make the script do it right.

By the way, I've really only run this script on Windows. There are undoubtedly some tweaks that should be made to make it work correctly on Unix-y systems.

<comment><USER28> That's nice, thanks for sharing. I used exfalso, for which mutagen was written, to build a similar directory structure for my music. Mine was "Audio/[ALBUM_ARTIST]else[ARTIST]/[ALBUM]/[TRACK 00]. - [TITLE].mp3" The one thing it did that your script doesn't is it will enter TPE2 if present and backtrack to TPE1. And similar considerations can be entered for adding a disc prefix for multi disc albums. (Since that anachronism is still relevant.) The only problem was I needed to go back through and delete a bunch of empty directories.

</comment></comment></comment><comment><USER29> Rob Pike said it best when he quoted his friend:

       My late friend Alain Fournier once told
       me that he considered the lowest form of
       academic work to be taxonomy.
       And you know what? Type hierarchies are
       just taxonomy. You need to decide what
       piece goes in what box, every type's parent,
       whether A inherits from B or B from A.
       Is a sortable array an array that sorts or a sorter
       represented by an array?
       If you believe that types address
       all design issues you must make that decision.

Reference: [http://commandcenter.blogspot.com/2012/06/less-is- exponentia...](http://commandcenter.blogspot.com/2012/06/less-is- exponentially-more.html)

</comment><comment><USER30> It looks like the author has come to an understanding with his/her disorganized music collection, but beets makes it pretty easy to bring sanity back to a messy music library: <http://beets.radbox.org>

<comment><USER31> I second beets, it handles big libraries better than any tool I've come across. My once messy library is now pretty much perfectly tagged (thanks to MusicBrainz) and organised.

</comment></comment><comment><USER32> I feel your pain. I have also stopped caring about music classifying, for much the same reasons you listed. Nowadays I use only Spotify, and I trust in their tagging abilities, and in Last.FM's auto-correcting ability.

My main concern when I used foobar used to be how to handle multiple artists on the same album, which always resulted in the album being split up when displayed in a list. I was unbelievably happy when I discovered the "Album artist"-tag, which unites all songs in an album under the same banner, while still preserving (and scrobbling) the original artist name.

<comment><USER33> Last.fm and Spotify generally give me terrible recommendations and no discovery at all. I think this is because they use machine learning on the person <-> track graph to determine which tracks are related, so they're very good at finding you DragonForce or IOSYS, but very bad at finding you Hammers of Misfortune, Gallowbraid, Umbrella, or m's. Of course, if you listen to music that Last.fm thinks is adjacent to IOSYS or DragonForce, you've probably already heard of IOSYS or DragonForce.

Pandora doesn't have this issue because they use human analysts to determine the properties of the tracks and give you similar tracks rather than tracks that are listened to by similar people. Unfortunately their music library is pretty limited. Google Music's automatic playlist feature does something similar without the human analysts, but this only works if you already have all the music you want and it fits onto your Google Music account. They don't seem to provide an option to pay for extra space.

</comment></comment><comment><USER34> I don't think there is a perfect way to organize a music collection in a hierarchical manner, so I don't even bother. Good tag metadata and foobar's search does all the work for me.

To me, the purpose of a filesystem is not to implement fine-grained categorization, but to provide basic grouping of related files so that I can easily operate on them all at once. To this end, my music collection mostly consists of one folder per album in a root music directory. Folders are usually named "Artist/Group Name - Album Title". That naming scheme doesn't always fit (albums featuring various artists, soundtracks in which I'm more likely to care about the title of the work rather than the artist that composed it, etc), but I don't try to separate soundtracks from regular albums or anything like that, I just throw them in the same root directory. With this scheme, it's easy to delete/share/transcode an album when it's contained within a single directory, convenient for people I share with, and I don't waste any time obsessing over something that I rarely need to see.

Some people have advocated a more database or metadata-oriented approach where you strip all metadata from the filename and folder hierarchy and stuff all your files in one directory. It's an interesting idea, for sure, more closely resembling the way web services like Youtube store their content. It makes one begin to imagine a desktop operating system that featured a metadata database as the primary filesystem organization scheme in place of the traditional hierarchical filesystem.

With our currently available tools, however, having some kind of useful metadata in the filename and/or filesystem hierarchy, even if it is redundant, is incredibly useful when performing manual file manipulation, especially the aforementioned sharing of files. You'd need ubiquitous categorization metadata in files (that is, not just ID3 and company for music files) and ubiquitous support for parsing this metadata in everyday applications (that is to say, when beginning a download of a song or a document, your web browser would show you the relevant metadata and hide the filename, if it exists. when opening a file, one would have to be greeted with a search box instead of a traditional hierarchy dialog) before we could ever entirely transition from having meaningful filenames to having meaningless hashes, timestamps, or garbage as the primary identifiers of files.

</comment><comment><USER35> Some of these problems (like creators with the same name) have been solved by librarians years ago.

<comment><AUTHOR> You mean by having a guy to ask where to find something because you're lost in the store and can't find shit?

<comment><USER35> No, there is a whole field that deals with issues like this. It's called library science. Specifically, dealing with this problem is called [authority control](<http://en.wikipedia.org/wiki/Authority_control>).

Some of the other problems are also solved by cataloging experts.

</comment></comment></comment><comment><USER36> I have two classifications for my digital collection: Old Music and New Music. Old Music is everything recorded before I was born. New Music is everything recorded after I was born. It cuts down on putting too much time into it all.

</comment><comment><USER37> Another annoyance the author didn't mention is that unzippers frequently mangle the filename and/or metadata if it uses Unicode. Archive Manager is the worst at this.</comment>