DuckDuckGo: illusion of privacy? - <AUTHOR> http://etherrag.blogspot.jp/2013/07/duck-duck-go-illusion-of-privacy.html <comment><USER2> _If you are specifically targeted in an investigation [...]_

The purpose of services like DDG is to reduce the amount of casually collected, personally-identifiable private data you might be strewing about - data that might potentially be recovered and might potentially be used against you or used in ways that you don't like.

If you are specifically targeted by an investigation, a law-enforcement agency like the FBI, armed with probable cause and warrants can tap your phone, search your house, track your location, log your keystrokes, etc, etc. DDG can't help you there, you'd also be vulnerable even when using a service provider which really doesn't have access to your data, like tarsnap. DDG is not going to magically protect you from targeted (and perfectly legal, civilian, non-NSA-related) surveillance if you happen to have the bad luck of being a subject of such an investigation. It's a silly standard to hold any service provider to.

<comment> brown9-2 Were these features of DDG ever it's _purpose_ or merely an incidental feature?

<comment><USER3> It was supposed to be Google except with privacy.

</comment></comment></comment><comment><USER4> 1\. Since the U.S. government has given itself both the power to compel U.S. corporations to spy for them and the power to prevent them from revealing this, we can't take the claims of any U.S. corporation at face-value.

2\. DuckDuckGo is a U.S. based company.

<comment><USER5> Problem is, even if DDG would be hosted in, say, Europe, the NSA would still be able to snoop on the traffic. As has been recently revealed, the German BND may siphon off up to 20% of the traffic at DE-CIX (Internet exchange in Frankfurt) and on average siphons off 5% [1]. The BND closely cooperates with the NSA. As we've seen in the last couple of weeks, Europe is basically a US colony and Germany in particular is not a true sovereign state. [2] What to do? Host in Russia, Latin America?

(Disclosure: I am German and DDG is my default search engine.)

[1] [http://h-online.com/-1909989](http://h-online.com/-1909989)

[2] [http://sz.de/1.1717216](http://sz.de/1.1717216)

<comment><USER6> DDG searches are delivered over HTTPS. It doesn't matter if the NSA can siphon off DDG traffic.

<comment><USER7> It does matter and the article has reasoning about why it does. "The NSA could get the DuckDuckGo master cert in one of three ways:

1\. Be given the cert

2\. Physical access to servers or load-balancers

3\. Remote access to servers or load-balancers"

<comment><USER6> If they can get direct access to the DDG servers, then it doesn't matter if they can siphon off traffic at the ISP level. They can just access the data.

<comment><USER7> But wouldn't that require constant access to the server, whereas the key they could steal once with short access to server and use until it expires without the victim noticing?

</comment></comment></comment></comment></comment></comment><comment><USER8> I like the blog post, but I think that it is somewhat unfair against DDG since the argument works against any internet company. The argument rests essentially on two points:

1\. Client/Server architecture has a single point of failure, namely the server. ( Or the network equipment directly upstream of the server.) So that whatever nasty surprise is embedded directly at the server, or in the jurisdiction the server is in, affects whoever is using the server.

2\. We do not have a treat model for the NSA, they are somewhere between a usual state level attacker and Cthulhu. We do not really know, what the NSA can or can not do, can they crack the discrete log or factor large numbers? Or do they 'just' have a assorted 0day collection? Is it realistic that they can coerce anyone into revealing public keys? And if they can actually break TLS, can they also break all TLS or does this require some not insubstantial effort on their part?

So both of these are real problems and the combination is potentially undermining the trust into the entire internet. But it is not really about DDG.

<comment><USER9> "... it is somewhat unfair against DDG since the argument works against any internet company"

What other search engines are being suggested for use as a safer alternative to the major search engines? Did any of those companies respond by affirming their ability to protect your privacy in a way that is not technically possible?

The author's point is that you can't dodge the NSA's scrutiny, and if you think you can then you are either lying or uneducated about the NSA's capabilities.

UPDATE: What I'm trying to say is that the article doesn't really work when directed at Google or Yahoo because we already know that our privacy is compromised there.

<comment><USER10> What about startpage? I know they're still dependent on google, but in terms of privacy they seem to be at the same level as DDG. At least they're smaller than DDG, so it's less likely that the NSA will bother with them

<comment><USER9> We need a definitive answer on whether PFS is truly perfect. If ssl->vpn->https->anon-dns->tor->tls2->no-logging-search is safe then our searches for weird porn might be private.

</comment></comment></comment></comment><comment><USER11> _" Option 2 Many smaller internet companies, including DuckDuckGo, do not operate their own data-center, but instead are "hosted" in another provider's datacenter. In DuckDuckGo's case, they are hosted by Verizon Internet Services. We've all learned about the cozy relationship between the NSA and Verizon, it is quite imaginable that Verizon would simply give them access to a DuckDuckGo server, or the load-balancer which is likely owned and operated by Verizon and upon which the SSL decryption key is installed. They don't need continuous access, 30 seconds is all that would be necessary to copy the cert."_

And Gabriel's response to that: _" There are many additional legal and technical inaccuracies in this article and I will not address all of them in this comment. All our front-end servers are hosted on Amazon not Verizon, for example."_

Might as well %s/Verizon/Amazon/g…

I also found what Gabriel said here to be interesting:

 _" We have not received any request like this, and do not expect to. We have spoken with many lawyers particularly skilled and experienced in this part of US and international law. If we were to receive such a request we believe as do these others it would be highly unconstitutional on many independent grounds, and there is plenty of legal precedent there. With CALEA in particular, search engines are exempt."_

So apparently speaking to a couple of lawyers who are probably not upon the FISA court (who apparently pretty much just stamp what has been decided) now have a say in whether such actions can be taken by the NSA and whether they are unconstitutional are not?

Don't get me wrong, I've been using DuckDuckGo for a couple of years now, but that's laughable.

<comment><USER12> Probably the lawyers are experts in the FISA court.

<comment><USER11> Contact with lawyers with access to closed hearings and classified proceedings?

I wonder if Google talked with those lawyers too…

</comment></comment></comment><comment><USER13> Response from DuckDuckGo CEO from the article comments:

"Hi, this is Gabriel Weinberg, CEO and founder of DuckDuckGo. I do not believe we can be compelled to store or siphon off user data to the NSA or anyone else. All the existing US laws are about turning over existing business records and not about compelling you change your business practices. In our case such an order would further force us to lie to consumers, which would put us in trouble with the FTC and irreparably hurt our business.

We have not received any request like this, and do not expect to. We have spoken with many lawyers particularly skilled and experienced in this part of US and international law. If we were to receive such a request we believe as do these others it would be highly unconstitutional on many independent grounds, and there is plenty of legal precedent there. With CALEA in particular, search engines are exempt.

There are many additional legal and technical inaccuracies in this article and I will not address all of them in this comment. All our front-end servers are hosted on Amazon not Verizon, for example."

</comment><comment><USER14> The beauty of the NSL system and the NSA acting outside the constitution is that no matter what anyone says there is no way to prove any statement made about receiving or not an NSL. In fact it wouldn't surprise me that if the NSA wanted the data they could compel someone lower in the company and the CEO might never know. How do you prove you are NSA-free if you are the CEO of an American based company? Really the only thing statement that people would believe today is actually showing your NSL publicly and telling the NSA to stuff it. If you haven't been targeted (or someone in your organization was) there is no way to prove it.

</comment><comment><USER15> The key thing about Google et al is that they maintain user accounts and try to get you to stay logged in which means (a) they maintain huge data stores that are (b) tracked by user. While DDG can be required to turn over its records, could theoretically hand its unencrypted traffic over to the NSA, and in any event the NSA could simply pluck the packets off the air en route, it would then have to figure out which packet was from whom and join the dots itself. This is essentially no different from what I assume the NSA can do with any damn website, foreign or domestic, it likes.

As a further wrinkle, if you are logged into Google then it can watch your web surfing activity onto any website with embedded google code (analytics, adwords) which is pretty much most websites.

All of this comes down to Google is an advertising company. If DDG becomes an advertising company, watch out.

<comment><USER16> How is DDG not an advertising company already? One of the top hit on every search I do on DDG is an ad (actually in my testing I found if I do an informational search like "Barack Obama" I get an ad on DDG and none on Google). If advertising itself is the problem, DDG has already doomed itself.

<comment><USER15> You're right of course about DDG selling advertising — I assume at much lower cost than Google given their relative inability to gather intelligence about users. The big question is whether they have a non-evil business model on the horizon (sponsored links aren't bad of themselves, but they are the entrance to the rabbit hole Google has long since gone down and expanded into a giant underground complex).

</comment></comment></comment><comment><USER17> DDG have a tor hidden service for search [1] and so you can search annonamusly through that, even if someone has access to the ssl privet key.

[1] [http://www.gabrielweinberg.com/blog/2010/08/duckduckgo- now-o...](http://www.gabrielweinberg.com/blog/2010/08/duckduckgo-now-operates- a-tor-exit-enclave.html)

<comment><USER18> This almost makes no sense. Hidden Services exist to hide the location/identity of the server. You gain the same anonymity by visiting ddg.com via Tor. It's cool that they're running a relay though.

<comment><USER19> I think that connections to hidden services on the Tor network are encrypted end-to-end.

<comment><USER20> They are not encrypted at all. Tor provides anonymity (nobody can say where the captured traffic comes from), not encryption (nobody can read my trafic). You should use independent encryption inside Tor.

I2P is similar to Tor, but encrypted AFAIK.

<comment><USER21> Tor absolutely does provide encryption, except for the last link (unless it's a hidden service) just like the parent poster said. More information is available here:

[https://ssd.eff.org/tech/tor](https://ssd.eff.org/tech/tor)

[https://www.torproject.org/docs/hidden- services.html.en](https://www.torproject.org/docs/hidden-services.html.en)

EDIT: To be clear, I am saying that tor DOES provide end-to-end encryption, but only when you're using a tor hidden service -- which is also what the parent poster said.

<comment><USER20> The parent said _end-to-end_ encryption, which Tor does not do. Yes, information through a tor circuit is encrypted. No, it's not enough if you need end-to-end encyption, the participants must arrange independent of tor for that.

<comment><USER22> But with a hidden service, the last unencrypted bit is all within the same server. It's a very different scenario from accessing the general internet: There's no exit server to worry about, except for the service you're connecting to.

</comment></comment></comment></comment></comment></comment></comment><comment><USER23> > "DuckDuckGo can easily be compelled either under the Communications > Assistance for Law Enforcement Act (CALEA), standard court orders, or by > secret orders from the Foreign Intelligence Surveillance Court (FISA) to > provide tap-on-demand"

Can they actually do that? I mean it's one thing to just "hand over the data" you already have about the user, and maybe even compel the company to decrypt it (although I still think that's BS [1] and companies should fight against it), but can they actually _force_ a company to _spy for them_ , and change their service in such a way that makes it possible? Does FISA and the Patriot Acts actually imply that? Or does he mean it might be yet another one of their "interpretations" of the laws?

Either way, if that's possible, just start using StartPage.com. They're based in Norway.

[1] [http://paranoia.dubfire.net/2010/09/calea-and- encryption.htm...](http://paranoia.dubfire.net/2010/09/calea-and- encryption.html)

<comment><USER24> I think that a common misconception of the current discussions is that people still expect NSA to obey some kind of law.

Never been to the US, but I grew up in a country trying to recover from a dictatorship, so I never believe that secret services have any law to obey.

<comment><USER23> > I think that a common misconception of the current discussions is that > people still expect NSA to obey some kind of law.

I realize that, and I know many things they are doing are extra- constitutional, and even extra-legal. But that was precisely my point. How can the companies just stand to being bullied by law enforcement like that just because "they say so", _especially_ when they think that what they're asking them is at least unconstitutional.

And I know there's a difference between how easy this is in theory, and how easy it is in practice, but come on - what are they going to do? Put Ballmer or Larry Page in prison for not obeying them? The companies would have the best 100 lawyers in the world fighting for them.

It just pisses me off that some companies seem to cooperate _way_ too easily with the mass spying, either because they find no problem with it themselves, or they're just too scared to try anything, even as big corporations.

<comment><USER25> _"...what are they going to do? Put Ballmer or Larry Page in prison for not obeying them?"_

Sure, why not? It may sound ludicrous but do you really think it's beyond the realm of possibility? You should look up stories about the CEO of Qwest and what he claims happened to him when he said no to the NSA (in 2007) [1].

[1] [http://www.businessinsider.com/the-story-of-joseph- nacchio-a...](http://www.businessinsider.com/the-story-of-joseph-nacchio-and- the-nsa-2013-6)

</comment></comment></comment></comment><comment><AUTHOR> If what's written in this article is true, that basically means that even hosting a server in the US is basically a breach in privacy.

Do you think this is reliable information?

<comment><USER26> What makes you think that hosting a server in one of the world's oldest democracies, the country among the first to have a written constitution and a strong bill of rights, the land of the free, would not be a breach of privacy?

Oh wait…

<comment><USER27> One of the world's oldest democracies? You mean Switzerland? Well, hosting a server _there_ might be a good idea.

<comment><USER26> 'one' is still an indefinite, rather than definite, article, and the USA are certainly one of the older, still-existing democracies (even though, of course, there are even older ones).

</comment></comment></comment></comment><comment><USER28> Before drawing any conclusions from this, I recommend reading Gabriel Weinberg's response in the comment section. I felt considerably better after reading it.

<comment><USER29> I felt considerably worse, because he takes a very naive view of the legal realm.

Let me expand a bit: I don't think Gabriel is lying/dumb/whatever. However, the statement given essentially comes out to "If we had to, we would fight the good fight and we very strongly believe we would win". I'm all in favor of fighting as hard as you can against broad/illegal/user harmful orders/etc.

But at some point, you _will_ lose, even with the law and moral justice on your side. This is a certainty. Google has lost. Yahoo has lost. Twitter has lost. Microsoft has lost. Contrary to the belief that they are cooperative, they don't want _anything_ to do with anything, and fought with more resources and energy than DDG probably can muster (again, no offense to Gabriel).

Let's ignore for a second whether they have any data to give or not. The point is that at some time in the future you will not just lose temporarily, you will lose in a way that you have to make a choice because yourself/your business/your livelihood and your users privacy.

Believing otherwise makes you naive in my eyes.

<comment><USER30> > Google has lost. Yahoo has lost. Twitter has lost. Microsoft has lost.

Lost in what sense? The question here is whether service providers can be compelled to send data that was never stored on disk to law enforcement. I've seen the news that Skype calls are streamed (in accordance with their privacy policy[1]), and I guess this can be viewed as Microsoft losing, but what about the others?

1\. "Skype, Skype's local partner, or the operator or company facilitating your communication may provide personal data, communications content and/or traffic data to an appropriate judicial, law enforcement or government authority lawfully requesting such information. Skype will provide reasonable assistance and information to fulfill this request and you hereby consent to such disclosure."

</comment></comment></comment><comment><USER31> MegaUpload, LastPass and others are provably not able to access your information. Storing the encryption key yourself is the way to go.

But then the govt can capture you and make you give up the key. A whistleblower can threaten to have more incriminating evidence disseminated encrypted somewhere, and if he doesnt check in every 30 days it gets released ... but then the government can just torture him until it makes him give up the key he uses to check in every 30 days. It wouls take a really stubborn guy to persist and let the information be released. Since you dont have any information like that anyway, just assume that if you can access your own data the govt can compel you to do it for them.

<comment><USER32> _MegaUpload, LastPass and others are provably not able to access your information_

Where can I find the proof you're talking about?

<comment><USER33> [https://www.grc.com/sn/sn-256.htm](https://www.grc.com/sn/sn-256.htm)

<comment><USER34> Gibson is about as credible as I am when it comes to security - ie, not very.

<comment><USER32> Credibility doesn't matter, as I'm not asking for a confirmation from an expert: EGreg claims there is a security proof of some kind.

</comment></comment></comment></comment></comment><comment><USER35> Hmm, CALEA is really not the right law to be referencing. CALEA generally applies to wiretaps and specifically derives from telephony surveillance and is more relevant—and worrying—to a Twilio or SendHub, rather then DDG.

It's more likely to be a portion of the PATRIOT Act (Sec. 215 and possibly 217, h/t to Marcy Wheeler for the education here: [http://www.aclu.org/free- speech-national-security-technology...](http://www.aclu.org/free-speech- national-security-technology-and-liberty/reform-patriot-act-section-215) and [http://cyber.law.harvard.edu/privacy/Introduction%20to%20Mod...](http://cyber.law.harvard.edu/privacy/Introduction%20to%20Module%20V.htm)) or the specific update to it (Protect America Act of 2007, FISA Amendments Acts of 2008 and most recently 2012) to bring the warrantless wiretapping scandal back into "compliance," and seemingly updating PATRIOT for the current round of surveillance, which was likely reauthorized Dec '12.

Now, the FBI recently floated a trial balloon of what we're calling CALEA II, but that's focused more on compelling the providers of in-browser chat products to create backdoors for surveillance: [https://www.cdt.org/files/pdfs/CALEAII- techreport.pdf](https://www.cdt.org/files/pdfs/CALEAII-techreport.pdf) It's not current law yet, and we're fighting to prevent the proposal from becoming law.

It's a point of precision that doesn't detract from the author's main point.

Just as an update, the legal debate is continuing over both the NSLs themselves and their related gag orders: [http://www.networkworld.com/community/blog/fbis-national- sec...](http://www.networkworld.com/community/blog/fbis-national-security- letter-gag-orders-violate-1st-amendment-ruled-unconstitutional) I'm not sure how or where this case escalated to, but the last time a court declared the gag order to be unconstitutional, it took an act of Congress to reauthorize it, which will be a difficult sell right now.

For a final note, here's a counter by the DoJ about how I'm wrong, for whatever that's worth: [http://www.justice.gov/archive/ll/subs/add_myths.htm](http://www.justice.gov/archive/ll/subs/add_myths.htm)

And for full disclosure: I consult with Center for Democracy and Technology (CDT) on reforming Electronic Communications Privacy Act (ECPA) of 1986, which is a similar but not directly related issue.

</comment><comment><USER36> You don't even need to ask ddg for a private key, go straight to certification authority.

<comment><USER37> I see this comment in so many stories lately.

Certification authorities do not, in general, have the private keys corresponding to the public keys they sign. Some CAs will generate a key for you and claim not to keep the private half once you download it, but any security-conscious site will opt to upload a signing request for their own public key instead.

So, that rules out the possibility of passive spying on HTTPS traffic.

As for active spying, a CA could certainly produce a certificate for a MITM attack, but many modern browsers or browser extensions will rapidly detect that, so doing it on a large scale will fail and be detected. The same goes for most security exploits: a large scale systematic exploit will not pass silently.

MITM or exploits on a small, targeted scale have some chance of working, depending on the target, but if a government-scale entity targets you _personally_ , you're pretty much screwed anyway. HTTPS still effectively protects against a large, systematic, undetected dragnet of traffic.

<comment><USER38> Yeah, but your "I know how SSL works" story is probably an NSA plant so that the NSA can read my secretz. NSA NSA blah blah NSA

NSA

<comment> grey-area You're really not helping improve the conversation by posting sarcastic responses like this.

</comment></comment></comment></comment><comment><USER39> Thus article states that the NSA will get the information anyway. Even if this is true it may be a good thing to choose a search engine that makes a point of at least not tracking you itself. Sadly Google has much better search results...

</comment><comment><USER40> The article is completely missing the point. Of course the NSA can get information from DDG, the point is that there is not much information to be earned there in the first place, and the searches are not associated with your google account, let's say.

</comment><comment><USER41> Lots of comments here. Not sure if it is already covered or not. So I will be brief.

DDG, hushmail, etc. Doesn't really matter does it if the NSA gets you at your internet connection and reads what you are doing from your service providers trunk?

You can DuckDuck and Go and hush your email. If they are grabbing it at the point of your modem and your internet provider, the illusion is you are secure but really you are not.

<comment><USER27> Data for such services leave your computer encrypted with SSL.

Bugging your computer (and/or phone) is probably the best way to track your communications clandestinely.

</comment></comment><comment><USER42> >> Can they refuse to collaborate with the NSA if approached? What I would like to know if they can really refuse when big corps such as Google, Microsoft and others can't?

>> Gabriel Weinberg comment: We have not received any request like this, and do not expect to. But if they receive such request can they just really say NO while other big companies can't?

</comment><comment><USER43> EDITED: Thanks guys, it seems like I managed to paste over most of my post with the clipboard filled with the last one. Thanks for being so nice pointing it out. Sucks, the on topic one i destroyed was leaps above the banal content that replaced it. Let something here to avoid you guys being orphaned. Sorry for reducing the signal to noise ratio!

<comment><USER44> You posted this same comment here: [https://news.ycombinator.com/item?id=6040733](https://news.ycombinator.com/item?id=6040733)

But added a section on DDG to the bottom...

</comment></comment><comment><USER45> Also consider the holland based search engine www.startpage.com which has al kinds of certificates.

</comment><comment><USER46> They also really hate it when people point out their privacy flaws. See [http://www.alexanderhanff.com/duckduckgone](http://www.alexanderhanff.com/duckduckgone)

<comment><USER47> Except those aren't privacy flaws. He's wrong about DDG setting cookies by default, the easter egg issue, affiliate revenue, and US law. He got so much wrong that I can't take the post seriously.

<comment><USER48> He's so pretentious it makes me chuckle.

</comment></comment></comment><comment><USER49> the main argument of this page is "nsa can hack everything cuz google got hacked by the chinese"

yeah thanks for the tip, that's so insightful.

</comment><comment><USER50> This "blog" has one post and was specifically started to bash DDG.

I don't use the duck (yet?) but to me it looks like an attempt at black PR.

Even if the duck doesn't give full anonymity, I would still prefer it to Google - the new leaked slides revealed that NSA has direct access to Google, whereas with ddg, they'd have to snoop upstream.

<comment><USER51> I am the author. I _did_ start that blog just for that post. I have never blogged before in any substantial way, but the naivete of comments from the CEO of DDG and the fact that users reading articles like in The Guardian might be misled into believe they are protected from NSA monitoring by DDG, motivated me to blog. No black PR. I don't work for any other search engine company, I am an American working for a small software company in Tokyo, Japan.

<comment><USER52> > I am the author. I did start that blog just for that post.

Can you prove that you are not an agent of the NSA trying to dissuade people from using services like DDG? I mean, you are using BlogSpot of all services, so you are directly contributing to the tracking of individuals (heck, you are encouraging it), so forgive me if your motives are suspect.

Frankly, that you cannot prove otherwise, I think you should say as much: that you should not be trusted.

<comment><USER53> Can you prove that you are not an agent of the NSA trying to dissuade people from using services like Google? I mean, you're using HN of all services, so you're directly contributing to the Silicon Valley culture that arms the NSA with tracking tools (heck, you're encouraging it!), so forgive me if your motives are suspect.

Frankly, that you cannot prove otherwise, I think you should say as much: that you should not be trusted. I say good day, sir.

<comment><USER52> Yeah that was the message. Not really sure if you got it, or missing the point though.

</comment></comment></comment></comment></comment><comment><USER54> Shhh, don't ruin the ongoing circle jerk.

</comment><comment><USER55> are we still all just forgetting the fact that DDG is just an aggregator of other search engines. Specifically Bing.

</comment><comment><USER56> How about the article writer test this before writing? Do DDG-searches about:

      - A violent political/religious subgroup.
      - Politician XXXX's opinions about said group
      - The future travel to your locality of XXXX
      - How to build bombs with YYYY.

Just make certain to wear a bullet proof jacket while reading... :-)

But seriously:

Sure, the NSA probably index searches on DDG. Of course. It is fully possible DDG isn't aware of it, too. It is too obvious to be at the top of HN.

I might have followed the subject too shallowly, but it really surprises me that I haven't seen the responsible politicians talk about the real problem here.

You can make a good argument that pervasive monitoring is a good thing in the short perspective (which the supporters do), but over a longer time?

After a long time of military or economic problems you get paranoid tendencies (see McCarthy, Putin, 9/11). I have no idea how likely a 1984 scenario (or a present day Russia!) is -- and neither do anyone else.

The people responsible should have answers to these questions _before_ making the decisions. And now they should discuss that. (Or doesn't a risk exist if it can happen after they are out of office? Or do they really think there will never be any political/economic bad times again?)

<comment><USER57>
       Sure, the NSA probably index searches on DDG. Of course.

Well the searches are encrypted so the nsa should have access to the servers themselves right? I dont now if DDG provides forward secrecy though..

<comment><USER56> We are talking about the same (or a sister) organization that created the viruses in Iran and the Middle East. Civilian search companies aren't as high on the agenda as rouge states' nuclear weapon programs, but still.

</comment></comment></comment><comment><USER58> _DuckDuckGo: illusion of privacy?_

His point is that if NSA wants they can do this and that. I suppose they can also take you to a Romanian black ops site and beat you bloody till you cough up your passwords. The idea is for average Joes and Janes to have a little bit more privacy. When you search on Google, they try everything possible to have you search while logged on and save your search history. All this is linked to:

your real name, location (Android)

maybe credit cards (Google Play, Adwords)

emails sent

videos watched

books read

sites visited (Google Analytics, sites visited that serve Adsense, Google hosted jQuery)

your private docs (G Drive, Documents)....

This is of course to make more money off you, they can charge more money to advertisers, and all this info is ready to be siphoned by Booz Allen and Hamilton employees and to be added to your file.

What do they get from DDG? Relatively speaking, nothing. The idea is to split your activities to make it harder for them, which is great for an average user. By using everything Google, it's even better than sending them a memo detailing everything you did each night.

If you're a target, long live TOR, which is not really usable on Google.

</comment><comment><USER38> I'm surprised by how many world class super spy/hacker types there are on HN...who knew that, besides just generally being snarky about shit that you half understand, that you had so many secrets to protect?

<comment><USER59> I'm surprised you don't recognize the principle of the matter. There was an article posted a while back about "in a world of perfect surveillance, legal change would be impossible". Much of social progress came about because people could communicate in secret.</comment></comment>


Calculate how much your site is worth  - <AUTHOR> http://www.cwire.org/website-value-calculator/

<comment><USER2> According to the tool on the site:

Google: $4552,092,873

Myspace: $302,159,931

facebook: $221,692,985

ycombinator: $88,266

My own site (which admittedly is just a static page) is valued at $58. Got some work to do there...

</comment><comment><USER3> I put in a site that has been making $1200+/month for the last two years and it tells me that the site can make $20/month. Fail.

<comment><USER4> Just out of curiosity, can you disclose which site is that?

</comment></comment><comment><USER5> Bogus.

For a recursive twist I typed in cwire.org.

""" Your site is valued at: $4,906

Your site could sell text link advertisements at a rate of approximately $0/Link/Month.

If you sell eight links, you could earn approximately $0/Month selling text links on your site.

Your site could earn approximately $135/Month through affiliate marketing".

Your site could earn approximately $0/Month through independent advertising arrangements specific to your site's niche. """

</comment><comment><AUTHOR> This tool is definitely crude (a newly created site is worth at least $50) but gives a good perspective on the factors one might use to interpret a site's value.

</comment><comment><USER6> Paulgraham.com - $29,889

I certainly think that the information there is a bit more valuable, though I'm also pretty sure that it doesn't make the $829/month that it could.

</comment><comment><USER7> My user page on reddit is worth $66. There's a lot of people who hate me and click on my profile to down-vote past comments. Attention advertisers!

</comment><comment><USER8> sweet $359, any buyers? Now it's time to start filling out my executive summary.

</comment><comment><USER5> recursive twist: cwire.org valued at $4,906 - which "could earn approximately $135/Month through affiliate marketing"!

</comment><comment> daniel-cussen Google = 580,000,000. Fail.

</comment><comment><USER9> "Your site could sell text link advertisements at a rate of approximately $1,087,500/Link/Month.

If you sell eight links, you could earn approximately $8/Month selling text links on your site."

Right...</comment>


The Pirate Bay Moves to .SE Domain To Prevent Domain Seizure - <AUTHOR> http://torrentfreak.com/the-pirate-bay-moves-to-se-domain-prevent-domain-seizure-120201/ <comment><USER2> It appears that thepiratebay.org points to a server that then handles redirection. Question: Why hasn't the .org URL already been seized at the DNS level? DiG output below:

      ; <<>> DiG 9.3.2 <<>> @localhost thepiratebay.org A
      ; (2 servers found)
      ;; global options:  printcmd
      ;; Got answer:
      ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5372
      ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

      ;; QUESTION SECTION:
     ;thepiratebay.org.		IN	A

      ;; ANSWER SECTION:
     thepiratebay.org.	3587	IN	A	22.214.171.124

      ;; Query time: 0 msec
     ;; SERVER: 127.0.0.1#53(127.0.0.1)
     ;; WHEN: Wed Feb  1 15:37:02 2012
     ;; MSG SIZE  rcvd: 50</code>

<comment><USER3> Probably because it's a lot harder to seize a site in another country which isn't technically breaking any laws.. It's different from say, megaupload in that no infringing content resides on their servers.

It's only going to get harder once TBP stops hosting even torrent files and moves to magnet links entirely. Once that happens, the site basically goes viral, since any random can just spider the whole site for magnet links and host a mirror.

<comment><USER4> And Google's Cache will contain the full information of the site itself. Think about that.

<comment><USER5> No doubt a court order will soon be sent to Google so that they can no longer index or cache TPB.

<comment><USER6> They're already getting the government to demand essentially that:

[https://torrentfreak.com/copyright-industry-calls-for- broad-...](https://torrentfreak.com/copyright-industry-calls-for-broad-search- engine-censorship-120127/)

</comment></comment></comment></comment></comment><comment><USER7> I wonder, why they didn't move to a .is domain, since Iceland claims to be a safe data haven.

<comment><USER8> That law is pretty new and quite untested.

<comment><USER9> Link please? I wasn't aware Iceland has passed anything that might put it in the position of a data haven, but then I don't follow Icelandic :-)

<comment><USER7> [http://en.rsf.org/islande-nouvel- article-18-06-2010,37771.ht...](http://en.rsf.org/islande-nouvel- article-18-06-2010,37771.html)

</comment></comment></comment></comment><comment><USER10> Supposing one law or another was passed giving the US govt the power to block offending domains, if push came to shove couldn't the US government ultimately block any domain anywhere via control over the .NET TLD of ROOT-SERVERS.NET and GTLD-SERVERS.NET? e.g. punish/block any nameserver allowing thepiratebay.se to resolve to it's real IP

<comment><USER11> This isn't about blocking domains, this is about the DHS or FBI seizing the domain, which they can only do with TLDs whose registrar is located in the US (.com, .net, .org, .us, amongst others)

<comment><USER12> It's not the registrar but the registry (Verisign) they go to. Lots of people switched from GoDaddy to Gandi last month, a French registrar, but that won't help against this .. problem.

<comment><USER11> Sorry, my bad. I'm not that familiar with the specific terminology, so I tend to confuse some things. A better formulation would have been "TLDs which are based in the US".

</comment></comment></comment></comment><comment><USER13> All of the mentioned services only work if the government does not fiddle with your ISP to block certain IPs/HOSTs/DNS Servers etc. Since our way into the net is controlled by local companies who have to comply to local laws, theres always a way to prevent people from using certain services.

</comment><comment><USER14> There is something called "darknet" which apparently runs as part of the freenet. I have heard about and read a few articles. This may be the way forward.

</comment><comment><USER15> Well well, there seems to be an issue with their SSL certificate. It still says .org even if you try to access .se domain. - fail.

<comment><USER16> SSL certificates take time to authorize/generate, depending on the issuing CA.

</comment></comment><comment><USER17> Anyone else noticing non-response from the site? I'm supposing it's temporary, as usual...

</comment><comment><USER18> Is there any way to make a website that has no central location, that lives on hard drives and internet connections of citizens and self replicates and adapts when the government wants it to just disappear?

A free internet is going to need a "deploy website to the hive" tool. Something that would force the FBI to have to raid computers in 50 different nations.

I'd like to see a message: "this site has been DNS blocked in your region, please download this program to seed this website for others over an encrypted subnet".

<comment><USER19> I believe the Freedombox project would help in situations like this, though wouldn't solve it entirely, and that situations like this would help gain support for such projects.

From <http://freedomboxfoundation.org> and <http://freedomboxfoundation.org/learn>

What is FreedomBox?

        Email and telecommunications that protects privacy and resists eavesdropping

        A publishing platform that resists oppression and censorship.

        An organizing tool for democratic activists in hostile regimes.

        An emergency communication network in times of crisis.

If you live near New York City, there will be a hack-fest February 18, 19, 20.

<comment><USER18> Wow. Freedombox is exactly what im looking for. You could put a secondary dns system on it which can't be fiddled with by ISP's and oppressive governments. My freedom box could reach out to nearby freedom boxes to find the quickest route in, around and through oppressive firewalls through encrypted tunnels and what not. Where do I buy?

<comment><USER20> My understanding is that they are non-existent, and people are still arguing over the design and implementation details.

But if you've got the skills, you can help. You know what it's supposed to do. Chances are, whoever stops bickering and builds one first wins. I'm going to see if I can get my local hackerspace to help me experiment and build at least some sort of prototype.

</comment></comment></comment></comment><comment><USER21> Permit me to address a few things in a way that is contrary, maybe, to the common HN reader's sentiments. I'd like a serious discussion of said matters rather than knee-jerk reactions.

1) " _It's not stealing, because you only take a copy_ ". We're playing with words here. First, the point is not the common definition of the word, it's what a court deems "stealing". Second, even in everyday life, we say things like "he stole the exam answers" or "He stole the recipe" (when in fact he only _read_ them).

2) "The losses are imaginary, because not everybody would have bought the stuff he pirated". Sure, the estimations of the losses are imaginary. All estimations are imaginary --by the very definition. You can argue about the accuracy of the estimations, but not of the need of an estimation (even if you put that estimated loss to zero).

Also, the fact that "not everybody would have bought the stuff he pirated" bypasses two things:

a) some people WOULD have bought the stuff, because they want it, if it wasn't available for them in pirated form. Those are actual money lost (I have downloaded for free lots of things that I would have bought if they weren't available because I just had to have them. Surely, other people have too. How many, is the topic of the aforementioned estimation).

b) those that would never in any case had paid for the item, do not represent "actual lost money". But they have violated the wish of the content creator/distributor regarding the (paid) use of his product. This can also be punishable by law, and fined, and in many legal systems, it is.

This "respect of the will" of the copyright owner, is like GPL etc works. If you want GPL respected, you also want copyright respected, even if the owner is not using a permissive license. Because the court doesn't upheld GPL for it's permissiveness, but because of it being a copyright license.

3) "The creator gets nothing anyway, it's all the big media". This doesn't matter. For one, people still pirate things that are sold by their creators directly. People have pirated even the Radiohead "pay what you want" album.

Second, if the creator sold the rights to some "big media" company, that's his choice (or mistake) to make. That doesn't mean you have the right to grab a copy of his work as you see fit.

Third, for a lot of the content the creator _is_ the big media company. People pirate like crazy BS teen-pop idols and hollywood blockbuster movies, which represent like 1% creative talent and 99% big media bucks for effects, marketing and production.

4) "Their business model is broken, they should find something else to make money" - Actually their business model is not broken. They make and sell something people want. The only broken part is that people copy it for free and give it around illegally. If I get me some GPL code, and make a closed source black box extended program out of it without giving back my source, just because I can, would people say that the "GPL model is broken"?

5) "People only pirate because the prices are absurdly high / the buying process is inconvenient" - Well, companies can set their prices to whatever they want for their own stuff, and they can make the buying process as convoluted as they want. That's not an argument in favor or copyright infringement, as much as high prices for Ferraris is not an argument in favor of stealing them, and the difficulty of attaining a university degree is not an argument for printing a fake in our printer.

6) "They can make it up on concerts" - Directors and shrink-wrapped software programmers don't give many concerts.

And even for musicians, a lot of musicians don't like giving concerts. Not everyone is like your local bar band or Rolling Stones. Especially in modern kinds of music, from electronica to ambient, there are lots of people that don't like playing concerts that much or feel that that is not their artistic medium (even the Beatles stopped playing concerts after '65). They still deserve some money for the use of their work.

Even for artists that _DO_ play concerts, a gig used to be mostly a loss leader for album sales. The only people making big money from concerts are mainly established artists with a managerial team, everything organized, big following and sold out venues and bands with a largish cult following (think Phish, Pearl Jam, etc). That doesn't bode well for artists that still used to make a decent living selling around 50.000-100.000 albums a year. Some artists also have actual integrity and don't want to be seen dead selling "merchandise". Not everything is "rock'n'roll".

7) "What's the proof of piracy's harm? Record company profits are at a high this year" - How is this contradictory? One can have record profits and still be ripped off. The sets of "people buying record company stuff" (profits) VS the number of "people that WOULD have bought record company stuff but they download it illegally" (actual lost money due to piracy) are not the same set.

Those are some arguments.

My personal opinion?

1) Software patents must go. Especially trivial. A patent period of 5 years of actual use of the patent in production seems ok for everything else.

2) Copyright must be restricted. 20-30 years at max.

3) Illegal downloading should remain illegal. Not everything should be available for free against the owner's wish. "Metallica" releases and "The managing secrets of Attila the Hun" books are not an undeniable right for everybody, there are just stuff for sale. There are lots of copyleft software, music and books, one can use.

<comment><USER22> If you want to understand, you have to start by doing one thing: stop taking copyright for granted, and imagine it as _not_ a fixture of reality, _not_ like a basic law of physics -- because it isn't.

Copyright is a law we have made, and those who reject it see it as bad and unjustified. From a principled position it is basically immoral, and from a pragmatic position it is unproven by evidence.

From the anti position, all the notions of 'stealing', 'losses', 'will of the creator' no longer exist -- they are entirely dependent on accepting the copyright concept. If there is no copyright, there is no stealing, losses, or primacy of creator's will.

If a law is not justified and well-founded we really ought not to have it. Is that not important?

<comment><USER23> That's a fair point. I suppose it relates to why people follow laws in general: is it because all laws we've adopted are good? Is it because laws, by virtue of being a law, are good? Is it because we are afraid of being punished for violating the law? These are major legal and political science debates (e.g. H.L.A Hart, Ronald Dworkin, etc.).

Anyhow, point being, although the law may be unjust, punishment for disobeying the law is not either unexpected or unfair. So while we can bemoan that people are punished for violating copyright law, that doesn't mean that they weren't justly punished by a government enforcing the law.

<comment><USER24> 1) _although the law may be unjust_

2) _that doesn't mean that they weren't justly punished_

Didn't you equivocate on "justice" there?

If "justly" only means "legally", the second quote is right. There's a reason why the Department of Law-interpretation-and-enforcement is called "Department of Justice" instead. Lawyers love to think that they're administering justice and not simply a piece of legislation. But we should not fall for this linguistic trick and equivocate justice with what is legal.

Political philosophers often use the word "justice" or "fairness" to refer to some moral ideal that is above existing laws, and which we can appeal to in evaluating existing and proposed laws. There are disagreements about the content of this ideal, of course, but none of that makes the concept of justice less authoritative than positive law. On this definition, the first quote would stand, but the second quote wouldn't make much sense.

There's also a sizable literature on civil disobedience, whereby citizens are morally permitted -- or even required -- to disregard laws that they believe are grossly unfair or unjust (provided that certain other conditions are met). IIRC Dworkin wrote an article or two on this topic, and so did Rawls.

<comment><USER23> Sure--I don't disagree with you. I'm not suggesting that people shouldn't engage in civil disobedience, in fact, I personally think that people should engage in civil disobedience as often as they feel morally compelled to.

In my last statement (and setting aside the word "just"), I was putting forward the legal positivist (i.e. HLA Hart) viewpoint that laws are valid by virtue of being law (without getting into what "law" is and so on). Accepting that copyright law is therefore a valid law on this level, I don't find its enforcement invalid. This is separate from whether I find the law sensible, or morally reprehensible, or whatever--and, depending on my views on that issue, I might want to disobey it and be justified in doing so. But, even if I think the law is completely insane, that's not saying that I would think of it as some artificial concept that I might completely ignore and then be shocked at being prosecuting for violating it, as a I read a previous post to suggest. By rebelling against it I am tacitly acknowledging that the law is what it is, and I should be prepared to accept the punishments.

Apologies if I am being opaque, it's obviously a minor point that has no real bearing on how people actually act.

<comment><USER24> Oh, I see. I agree. I wouldn't be surprised that a government tried to enforce positive law, the same way I wouldn't be surprised if the Mafia trashed your store because you failed to pay for protection. Just because the powers that be are wrong doesn't mean that their threats are any less real. To disagree would be naive.

It's also great to meet someone who knows that legal positivism is not incompatible with the existence of valid but unjust laws. Seen too many first- time philosophy of law students who don't seem to understand this!</comment></comment></comment></comment></comment></comment>


For Science: Does ZFS deduplication work on intros of TV shows? - <AUTHOR> http://manuelgrabowski.de/2014/09/29/zfs-deduplication-tvshow-intros/ <comment><USER4> I challenge you to record 30 seconds of a fixed test pattern from TV on two separate occasions and have it encode to identical bits. I don't think it can be done.

Many media formats (including all digital TV formats) encode a rolling hardware timestamp of 33 bits (or more) which won't be the same for two separate segments by random chance. Audio, video, subtitles and other metadata will be ordered differently in the stream because they all comes from sources that have their own separate clocks. Synchronization between clocks at different stages in the media pipeline will cause different frames in the sequence to be dropped, padded, made into keyframes, etc, which then affects every bit in subsequent frames. TV stations use time-based watermarks. TV stations use digital compositing software that may retain bits from previous frames on an ongoing basis. Many studio media pipelines still involve analog steps.

The list of complications goes on.

Unless your pipeline is lossless, uses only digital sources and uses totally synchronized clocks for all stages, you're about as likely to get an MD5 hash collision by accident as you are to get any two non-trivial sequences of compressed video to be bit-identical.

<comment> a-dub Could make for an interesting codec idea though! A codebook based compressor that works on corpora of audio/video seems doable. More interesting would be a codec that aims not only to exploit models of human perception to throw away bits, but also exploits those models to produce "canonical" compressed forms that would be suitable for byte/block level deduplication. Of course, interesting things are hard, and errors would be hilarious. (See: Xerox and their copiers that change letters and numbers in documents...)

<comment><USER5> Also, I don't know if this is already implemented in some codec, but it would be interesting to re-use key frames from previous shots. For instance when you have a dialog and the camera switches from one person to another, a conventional codec probably creates a new i-frame for each shot whereas an intelligent codec would recognize that this is the continuation of the previous shot.

</comment></comment></comment><comment><USER6> The reason I assumed deduplication wouldn't be effective on TV shows is because the intro isn't always at the same point as most shows have a prelude these days. Since the preludes are of different lengths, it means that certain things like key frames (if used on that particular codec) would appear on different frames of the intro sequence. Which makes me wonder how effective deduping would be on older TV shows where the intro is at the start of the programme.

As for compression being largely ineffective, the shows are already compressed anyway. In fact file system compression seems to be less relevant these days as most modern file formats have compression built in (even Office documents are just ZIP files). But as the author said, the overhead for compression isn't damaging for the performance* of ZFS - unlike with deduplication

* ZFS compression is particularly performant† when using the newer lz4 algorithm available in OpenZFS - which I'd highly recommend people using if they're not already.

† I know "performant" isn't technically a word. But it should be.

<comment><USER7> If people use and understand the word "performant", it's a word. That's how language works.

<comment><USER6> In the evolutionary sense languages, I agree. But there's also a lot of people who strictly disagree with slang, "verbing" and other colloquialisms - <USER2> believing language should be applied strictly (people otherwise known as "language Nazis").

My "performant" footnote was just an attempt to pre-empt such attacks; though ironically including the footnote has now sparked the tangent itself.

<comment><USER7> It's a fun thing to point out to those people how many of the words they're using to lambast you have actually changed meaning or are new words. They get -very- flustered and sulky.

</comment></comment></comment></comment><comment><USER8> For shows with identical intros, mkv format supports linking to a separate file to store it. The separate file just has to be in the same directory.

<comment><USER9> A feature which is frequently used by anime fansubbing groups, and almost never elsewhere.

Those groups often seem strangely bleeding-edge. They always adopt new features years before western groups will... I wonder why.

<comment><USER10> I'm guessing it's because they have to do much more work: western rip group just rip and upload, fansubs have to rip, translate (or find a translation), integrate subbing, recompress and upload. And they'll be going through each episode multiple times to QC timings, subs readability and check that subs doesn't prevent seeing anything of interest, at least for quality groups.

As a result they're always on the hunt for options which could help them do their stuff and:

* they'll find out about other possibly interesting features at the same time as they're poring over release notes

* because they might have to re-export their rips multiple times during QC, they can toy around extensively with features & settings (especially on early exports when they'll most likely have typos & al and will have to redo the export anyway)

tl;dr: they have plenty of opportunities to find and try out bleeding-edge features.

<comment><USER11> I only wish "western" releases were already packed with subtitles. Especially now that opensubtitles tries to sell ad space directly in the subtitles.

</comment></comment></comment></comment><comment><USER12> Since videos from the iTunes Store are encrypted, it seems like the individual files should look like random noise to the filesystem anyway. Unless there is something I don't understand happening.

<comment><USER13> There are ways to remove the DRM. In case you don't want to rely on Apple keeping their activation servers online forever.

<comment><USER14> I don't think OP did it in the post, so I'm also inclined to believe that he did not see deduplication in part because of that.

<comment><AUTHOR> dewey is correct, DRM/encryption is not the issue.

</comment></comment></comment></comment><comment><USER15> If you were going to do this for science, a good start would be to analyze the intro of shows in question, using rudimentary tools such as md5sum on a per- block basis.

You would then quickly discover that the chances of huge gains would be small, without ever even looking into the ZFS aspects.

<comment><USER16> ZFS supports byte-level deduplication where it can compute anchor points so that it works on unaligned data. However, quite a few a few assumptions about the entropy of the data set could have been inferred from the "iTunes TV Show" file choice, which clearly means the files would be compressed and encrypted.

<comment><USER17> Is that new? Zfs has supported block deduplication for a few years but I'm unaware if it supporting byte level deduplication.

<comment><USER16> [https://blogs.oracle.com/bonwick/entry/zfs_dedup](https://blogs.oracle.com/bonwick/entry/zfs_dedup)

<comment><USER18> Not new.. dated 2009.

Oracle seems to be doing alot of the lately where they are posting really old collaterial

</comment></comment></comment></comment></comment><comment><USER16> Yes, ZFS deduplication is not very useful when trying to deduplicate encrypted and compressed video files. Encryption is supposed to make the contents of the file look random. Compression is itself a form of in-file deduplication. Where ZFS deduplication will be extremely useful: on a SAN in a video editing studio where originals are kept in raw format. Even then, file-level and block-level dedups will probably be the least effective means of deduplication. What we need is byte-level deduplication which computes anchor points from which data ought to be deduplicated. Even then, it sounds like the dedup feature would be scripted so that it be a part of the tooling, and would be updated with the team's projects. I would go as far as to say the author chose a deliberately eye-catching headline and coupled up a few Linux commands with some "hilarious" GIFs to address the wannabe geeks out there while displaying a staggering pile of shamelessness to even the remotely competent of his blog's visitors.

<comment><AUTHOR> The files are not encrypted. I mention the aspect of doing it with files that are already compressed and that I never expected it to work for that very reason. I just wanted to to for once actually do it and I don't see why you need to get so mad about that.

<comment><USER16> Unless you stripped the DRM from the files, yes, they are encrypted. At least according to the Wikipedia page on FairPlay [[http://en.wikipedia.org/wiki/FairPlay](http://en.wikipedia.org/wiki/FairPlay)], all the iTunes files are encrypted.

Also, wouldn't proving it doesn't work by setting up a deliberately flawed scenario amount to a bit of blogsturbation? Should I write an article about "finding out whether my computer will turn on even if it's unplugged"? Answer: no. But read my article about me TRYING it just to be sure.

I think you should write a follow-up article where you splice together raw video files and include a similar segment in all the different files. Then, put that through the dedup test. I would actually be interested to see the ability of ZFS implementations to find worthwhile anchor points within files and do smart byte-level deduplication.

</comment></comment></comment><comment><USER19> There's basically no way to make this work because, as has been mentioned, in compressed formats there's every chance that title sequences will occur at different times in the show, so will occur at different points in the group- of-pictures sequences in compressed media.

And especially for rips from an already-compressed signal source (like TV), the odds are well against a sequence, although identical visually, encoding into exactly the same sequence of bits digitally, as there are so many factors that can vary the datastream you receive, even before it's ripped and re- encoded.

To stand any chance of this working, you'd really need to be storing and comparing uncompressed frames direct from the master, before ANY kind of variable encoding or compression. But if you have enough storage to be working with files on that scale, deduplication is probably not your biggest concern. :)

</comment><comment><USER20> Reminds me of the DVD sets of Sailor Moon I made years ago where I spliced the same video sequences in for the intro and outro. I never got so far as deduplicating the henshi sequences because there often are different voice- overs during them.

</comment><comment><USER21> I am curious and did not read the docs: Does ZFS deduplication work on parts of files? I would have expected that filesystem deduplication just deduplicate complete files (for conveniance). Deduplication of file parts could of course be good for some types of files, with a fixed structure and with no "noise" \-- but with videos, there is no chance I guess without manually linking parts.

<comment><AUTHOR> Yep, it does. [http://arstechnica.com/information- technology/2014/01/bitrot...](http://arstechnica.com/information- technology/2014/01/bitrot-and-atomic-cows-inside-next-gen-filesystems/) has a pretty nice overview.

</comment></comment><comment><USER22> A few bits of change in uncompressed video can change the following stream in the compressed video completely. Deduplication isn't intelligent enough to understand "video" and compress for that -- that's what video codecs are for. So it's not surprising that deduplication would not prove useful for compressed video. You could see the advantage in uncompressed streams.

</comment><comment><USER23> A codec that could search for reference frames from large collections of media might be pretty interesting for some applications, especially ones with large storage demands and repetitive material. The indexing system would be an interesting problem since it would have to be very low overhead. Wonder if any systems like this exist?

<comment><USER24> The player would have to be able to reference a huge number of sparsely- positioned reference frames.

The seeking might make it impractical.

</comment></comment><comment><USER25> I've always dreamed that subbers would just provide one version of the credits in their season packs and provide a bash/mencoder/whatever script to glue the bits together after download. We're wasting so much bandwidth on opening and closing credits that people end up skipping anyway, it's crazy...

</comment><comment><USER26> TV shows may have the same intro sequence, but the overlaid credits are always a bit different.

Hence any compression applied will always produce different results between episodes of a show. This would make any de-duplication extremely difficult.

<comment><USER6> I think it's a bit of a stretch to say the opening credits are " _always a bit different_ ". I can think of quite a few shows display the episode directors and any guest stars after the intro (ie during the first scene of the main episode). I presume they do this to save editing time and cost changing the intro - though there's also the convenient side effect of still having the credited names appear on screen even when viewers fast-forward through the intro.

</comment></comment><comment><USER27> I think this was a pretty dumb experiment, and the outcome was to be expected.

There will always be some sort of noise, pixels aligned differently etc., in a production like a tv series, and expecting the encoded output to be identical/matching on a blocklevel is pretty naive to say the least.

I did some experimenting using ZFS deduplication on MPEG2 files, where I encoded hundreds of MPEG2 dvd-sized videos, where 90% of the material was identical (the last 10% was affected by applying different watermarking techniques to the footage), and got some decent deduplication ratio (x1.2:1 or so).. But ZFS deduplication is expensive in memory/SSD, and it was definitely not worth it.

<comment><USER28> It may be a "dumb experiment" if you already understand how the underlying systems (both the file system, and video encoding) work - but the author did not, and now having done the experiment, they understand more about them.

</comment></comment><comment><USER29> Perhaps I'm just in a bad mood today, but really, why must technical articles be festooned with meme GIFs featuring abusive language and children injuring themselves? What does it add? If your content is so boring to you, or to your prospective audience, that you think it needs livening up with that sort of garbage, maybe just don't write it in the first place.

<comment><AUTHOR> I tend to agree with you, but the article wasn't supposed to be very technical in the first place, and: both those GIFs are from Dexter, the show of which I used a season in my tests. It's not completely unrelated.

<comment><USER29> I didn't realise they were from Dexter, though I stand by what I said. Clearly the masses on HN agree with your choices, though, since I'm getting down- voted!

I'm writing some technical documentation for a project right now - <USER3> I'll put in a few tasteful images from Baywatch to make sure the project managers who have to read it will make it to the end.</comment></comment></comment>


Uber becomes legal in NSW, Australia – taxi owners to be compensated - <AUTHOR> http://www.dailytelegraph.com.au/news/nsw/uber-legal-in-nsw-taxi-owners-to-be-compensated/story-fni0cx12-1227656507684?nk=35cb421f9f76b5c0063a30276fd580f9-1450392057 <comment><USER2> The Tele is a tabloid but appears to be the best source for this article.

September: [http://www.abc.net.au/news/2015-09-28/uber-drivers-face- susp...](http://www.abc.net.au/news/2015-09-28/uber-drivers-face- suspension/6808582)

Here's what's probably the press release that was used for this article: [http://www.rms.nsw.gov.au/about/news- events/news/ministerial...](http://www.rms.nsw.gov.au/about/news- events/news/ministerial/2015/151217-ride-sharing.html)

</comment><comment><USER3> This is awesome news!

As a NSW taxpayer, I'm not too sure I agree with the government doling out our taxpayers to compensate a dying/legacy industry, or tacking on $1 per Uber ride to pay them out but I suppose the taxi lobby is strong, and the government needs to pander to them.

I'm curious what licensing is required, will be good to hear more details.

<comment><USER4> The government has sold taxi licensed/medallions for the right to provide this service. If this were real estate eminent domain would apply and the owners would be compensated. It seems consistent to compensate existing license holders.

<comment><USER3> Well, not really.

It's not like the government took away the rights of taxi drivers to drive people around. It's just that there's a new entrant now, and they actually need to compete.

In fact, taxi medallion prices went down post-2011 (before Uber) because the government released more, because consumers (i.e. everyday people) complained about crazy prices. You don't see the government compensating people there.

It would be like if the government handed out licenses to build 1000 apartments.

Then, they realised, jeez, house prices are crazy, we better issue more licenses. Or we better re-zone to allow super-high density apartments, or allow people to sub-let apartment (cause in this make-believe world, that wasn't possibly before).

Would the original 1000 apartment owners be clamoring for government handouts?

<comment><USER5> But the whole selling point of a taxi medallion is that the government will limit supply. If a developer was sold the rights to build 1000 apartments under an agreement that the government would limit future development in the region and they later opened the market, the developer they made that promise to would be right to expect some compensation.

The problem isn't the compensation. The problem is that the government promised artificial scarcity to the taxi companies in the first place.

<comment><USER6> Does the legislation specifically state artificial scarcity will be enforced, or just that supply will be regulated by the govt?

</comment></comment></comment></comment></comment><comment><USER7> As an American resident of NSW, I had no idea Uber wasn't legal, I just knew they couldn't pick up from the airport. I've been using the service regularly since moving here. Perhaps this helps explain why locals I've met seem to favor taxis.

<comment><USER8> I would have to differ on this. I only ride uber in Sydney since I got back from o/s in Jan-2015. I would say, once the locals have ridden uber (i have introduced family and friends on the service), no one that i know prefers taxis.

</comment></comment><comment><USER9> Actually, this article raises a point I'd not considered: Why doesn't Uber just start buying medallions, especially in large / angry markets? In NYC, medallions are down to $600,000: [http://www.bloomberg.com/features/2015-taxi- medallion-king](http://www.bloomberg.com/features/2015-taxi-medallion-king). Presumably Uber wouldn't have to buy all the medallions out there, but if it bought enough it could circumvent much of the political opposition from current holders.

<comment><USER10> Because buying them would push up the price, especially once the sellers realized the motive. If they could really buy them all for $600,000 / each they probably would.

Nevertheless, having the government compulsorily aquire and compensate previous owners in an orderly fashion, where they all get the same compensation instead of being able to push up the price by holding out is a wonderful solution to this kind of problem.

It would be great to see this happen more in the US, taxi medallions and rent regulation are obvious areas where just compensating the current beneficiaries as a one-off, then opening up the market would do everyone a world of good.

</comment></comment><comment><USER11> > _The perpetual owners of taxi plates, about 5800 people, will receive a > $20,000 payment for the plate. Multiple plate owners will receive a maximum > of $40,000._

> _Both taxi and Uber drivers will also have to pay the government a $1 levy > per trip for a maximum of five years to fund the compensation package._

So, "here's $20k, now start paying us back"?

I wonder how long it takes to accrue 20k rides in a taxi.

<comment><USER12> For an average of 7 rides/day ~7 years, 20 rides/day ~3 years. Assuming the car is working daily.

<comment><USER13> Having a taxi collect a $1 just to redistribute it back to taxi drivers seems a little pointless. The real impact will be in the levy money collected by Uber drivers, so the ratio of Uber drivers to taxi medallions is very relevant.

</comment></comment></comment><comment><USER8> Can someone explain what I'm missing? 20,000 in perpetuity at the approx 2.5 rate in oz is PV of 800,000? But the plates hit there highs of 475,000 in 2011?

</comment><comment><USER14> Why aren't we addressing the safety aspects of Uber rather than making it legal? Can we not ensure that the cars are professionally insured, professionally driven and kept up to standard and safe to be in? That should be our number one priority before allowing the company to continue operating.

Some random examples:

[http://www.cnet.com/au/news/sydney-ridesharing-driver- charge...](http://www.cnet.com/au/news/sydney-ridesharing-driver-charged-with- sexual-assault/)

[http://www.heraldsun.com.au/news/law-order/uber-driver- arres...](http://www.heraldsun.com.au/news/law-order/uber-driver-arrested- over-alleged-sex-assault-in-melbourne/news- story/200c9101a4f31ec7218507d7bd6b6d4a?=)

[http://metro.co.uk/2015/09/08/uber-driver-dragged- passenger-...](http://metro.co.uk/2015/09/08/uber-driver-dragged-passenger- out-of-car-and-ran-over-her-5381679/)

<comment><USER3> Sorry, but these examples are pretty laughable. Do you live in Sydney, or even Australia?

As a Uber customer in Australia, safety is not something I need to worry about getting into an Uber. I can see the photo of the driver before I get in, I can see his rating, and I know the entire ride including GPS location etc. will be stored in the cloud.

Your examples only prove my point.

That's a lot more accountability that can be had with taxis.

Now taxis on the other hand....I have had some great drivers.

However, more often many of them are rude, and quite honestly, some of the most a*sehole drivers on the road I've seen. Even as a customer in a hurry, I don't want him or her driving like it's Grand Prix Turismo, or being a jerk to other people on the road.

With Uber, I just rate him or her 1 star, and say aggressive driving. I have only had to do that once ever with Uber (and I have taken around 150 Uber trips, across four countries) - and within minutes, somebody from Uber emailed me back, to follow up with the complaint.

Have you ever tried complaining about a Sydney taxi driver? Or heck, talking to their lost property department? Good luck with that, mate....

<comment><USER14> None of those examples strike me as funny; I am somewhat concerned that you can laugh in the face of human suffering.

I am also not defending taxi drivers. The other day I had to take a taxi for a $9 ride and the guy was less than graceful about the fact that I wasn't going very far from his taxi line.

But the solution is not knee-jerk deregulation - although I agree that a shakeup is necessary and warranted.

<comment><USER3> Nice attempt at a strawman fallacy there, ank_the_elder....

Nobody ever said "human suffering was laughable" \- you just made that up. It's the equivalent of uttering "think of the children!", when it has nothing to do with the topic at hand.

I said that your examples were laughable - and you have said nothing to make me change my view.

I made numerous points - in Uber, you see a photograph of your driver before they arrive. In Uber, you see their rating (and can rate them afterwards). In Uber, the drivers are constantly tracked by GPS, which is stored in the cloud. You addressed _none_ of these points in your reply.

Please list to me your reasons how a taxi driver is somehow _more_ accountable than an Uber driver?

I have had numerous bad experiences with taxi drivers, both as a passenger, and as a fellow driver or cyclist on the road. I can assure you that attempting to contact the taxi companies (at least in Sydney) is usually an exercise in futility. It's only when you involve the police, that things get a bit more traction, and the taxi companies are suddenly ever so co-operative - fortunately I have only had to do that once.

In contrast, Uber customer support is fantastic. Even better, they have GPS records of everything - so if you say, the driver never showed, it is very easy to prove or disprove. Try doing that with a cab. (I've never actually had that issue with Uber, to be honest, of a driver not showing - only with taxis.)

Or one time, I left some headphones in an Uber. Immediately contacted the driver via the app, who then drove back to me for _free_ , to give them to me. Try doing that in a cab.</comment></comment></comment></comment>


Pyze – Growth Intelligence for Every App - <AUTHOR> http://pyze.com/ <comment><AUTHOR> Hey guys, super excited to finally post this as we just launched. We provide a BI suite for mobile analytics that is really easy to use, would love any feedback on the product or anything else.</comment>


Ask HN: Linux on a Macbook Pro. Viable or Get an XPS 13? - <AUTHOR> I have a 2015 Macbook Pro (15 inch, Core 17, 16gb, ssd, retina display).
I'm not happy with OS X 10.13. Is Ubuntu or Fedora a viable option? Things I care about: trackpad, wi-fi, sleep?
Would an XPS 13 be a better laptop option?
Edit: clarify retina display <comment><USER2> Typing this from Fedora running on retina MBP. Everything other than the camera works out of the box and has been a pleasant experience for the limited amount of time i have had this setup. Linux install is running from an external SSD connected through USB3 so OS X is untouched. I don't notice any performance issues with this setup, none of my workloads are disk heavy. External monitors requiring different scaling than the desktop might need some setup.

EDIT: No experience with touchbar equipped models

<comment><USER3> Do you get F keys on the touchbar (if it's a touchbar model)?

</comment></comment><comment><USER4> I'm using Ubuntu 17.10 on a 2015 Retina MBP 15. All three of those things work for me.

If your hard drive is big enough, consider dual-booting. It's easier if you use refind as the boot manager.

You can even triple boot (MacOS, Ubuntu, windows).

EDIT: You didn't mention whether your macbook has a retina screen or not. If not, then I'd prefer an XPS 13. If retina, then I'd stick with the Mac.

If you don't need to dual-boot, then installing Ubuntu is really easy.

<comment><AUTHOR> Great. Can you share any interesting apps that you use? I mostly use GitHub from a command-line, Markdown, normal daily tasks, C++, etc.

<comment><USER5> there's some good recommendations in the reddit thread I bookmarked: [https://www.reddit.com/r/linux/comments/6di5s4/what_are_your...](https://www.reddit.com/r/linux/comments/6di5s4/what_are_your_must_have_programs/)

I also have some of my favorites listed in my HN profile

</comment></comment></comment><comment><USER5> I have a mac pro (trash can) and two MBPs (one 13" with retina display). I exclusively run linux distros on all of them and it works great. Just have a $10 linux compatible usb wifi dongle on hand for setup - once you're running you can install the appropriate nonfree broadcom wifi driver.

<comment><AUTHOR> I have the a trash can too. I was considering what to do about that. Can you share what linux distros you are running and an example linux compatible usb wifi dongle?

<comment><USER5> I'm running Debian stretch on the mac pro.

The following usb wifi dongle seems to have decent linux compatibility but I haven't tried it [https://www.amazon.com/Panda-300Mbps-Wireless-USB- Adapter/dp...](https://www.amazon.com/Panda-300Mbps-Wireless-USB- Adapter/dp/B00EQT0YK2)

There's a lot of tutorials on installing linux distros on macs since there's a few special steps you'll probably have to take. I googled the following tutorial (haven't used it) but it should get you started: [https://linuxnewbieguide.org/how-to-install-linux-on-a- macin...](https://linuxnewbieguide.org/how-to-install-linux-on-a-macintosh- computer/)

</comment></comment></comment><comment><USER6> Curious if you would consider using a VM? I use fusion to run a Windows and a couple of Linux VMs with 0 issues save for hindered battery life.

<comment><AUTHOR> Currently VMWare Fusion doesn't run for me on anything 10.13.1+. I have talked to Vmware. They blame the bugs on Apple. All my virtual machines over the years have become unreliable in some way or another at some point for a variety of reasons.

OS X just doesn't work as well as it used to. I cant turn off things. Bugs. Etc. I feel like more control and flexibility would help me.

</comment></comment><comment><USER7> I recommend dual booting for safer firmware security updates, but it totally works.</comment>


An 'impossible' card puzzle - <AUTHOR> http://mindyourdecisions.com/blog/2010/09/22/a-fun-card-puzzle/

<comment><USER2> I really like these kinds of puzzles. I'm not sure if they have a name, but I like to think of them as "invariant puzzles". The key to understanding them is finding an invariant - something that doesn't change when you manipulate the objects in the puzzle.

You can tell it's an invariant puzzle because they seem to be missing one vital piece of information. This is because one of the factors (in this case, whether a card is face up or face down) has been canceled out due to the invariant.

So what's the invariant? Well, let's start with what we're trying to find. We want the number of face up cards in the new deck (let's call it Deck 2) to equal the number of face up cards in the old deck (Deck 1), so Up1 = Up2. We know we can flip cards, so that's equivalent to Up1 = Down2.

But we're still missing a bit of information. How can we add more face down cards to the second deck when we don't know which way they're facing? Here comes the invariant... it doesn't matter! We can make Up1 and Down2 closer to equal by increasing Down2 (by moving a Down card) _or_ decreasing Up1 (by moving an Up card). Essentially, the value of (Down2 - Up1) decreases by 1 when we move a card no matter what.

So we want to keep moving cards until (Down2 - Up1) = 0. (Down2 - Up1) starts at 10 in this case, so we do it 10 times. We flip Deck 2; Up becomes Down, Down becomes Up, and we're left with Up1 = Up2. QED!

Another cool invariant puzzle is the wine mixing problem (also known as the water/wine problem, the red/white marble problem): [http://www.smart- kit.com/s330/red-wine-and-white-wine-brain-...](http://www.smart- kit.com/s330/red-wine-and-white-wine-brain-teaser/)

(Oh, and incidentally there are some problems that seem to be missing an important piece of information, but aren't invariant puzzles, like xkcd's Blue Eyes (<http://xkcd.com/blue_eyes.html>). Puzzles that include language like "everyone behaves rationally" are almost always recursion problems.)

</comment><comment><USER3> People are way too quick to call logic puzzles "impossible".

Most logic puzzles (maybe all?) have a particular way of thinking about them that will lead you easily to the answer. If the puzzle seems hard, that's probably a sign you need to change how you're thinking about it.

Edit: Looking at it again, there's something else: Every logic puzzle is going to try to slip some important fact by you with vague language. In this case, the hint gives it away: the piles can be different sizes. In general, if I tell you to divide this pack of cards into two decks, I mean they should be equal, and you understand that. The puzzle is using that deliberately to trick you.

It's easy to get around, though; you just have to define the puzzle logically. A "deck", in logical terms, is an unordered set of any number of cards. Once you realize that, the answer seems stupidly obvious.

<comment><USER4> Well... it depends how do you look at the rules. Saying "You have been asked to reorganize this pack of cards into two decks" is a bit misleading, as "reorganize" typically means moving things around without changing their properties. Sometimes you just assume rules that seem logical to imply.

To one person it might be obvious you can turn the cards upside down, to another it's obvious you can't. Then again... you could just get all cards into a deck, split in half and put each half on a side, so that no cards are facing up, or destroy the cards, or bring your own new deck and hide the cards you find, or take the cards out of the room to reorganize, or turn on the light, or bring a torchlight, or ...

<comment><USER3> That's true. In the context of a logic puzzle, we're supposed to assume that a card is a logical entity— having two states, face up and face down, belonging exclusively to one deck, etc.

If the puzzle relies on doing things such as tearing a card or turning on the lights that don't really make logical sense, you would usually call it a lateral thinking puzzle instead.

Obviously some people take a lateral approach to logic puzzles like this and declare it "solved", which is a perfect segue into a lengthy discussion on what the criteria for solving such a thing actually are, and who judges that.

</comment></comment></comment><comment><USER5> Typically a good strategy is to solve a simplified version and generalize.

In this case, assume only 1 card is face up. Take a single card from the deck:

       - if it was face up & you flip it, both have zero
       - if it was face down & you flip, both have one

</comment><comment><USER6> Go through the deck card by card and rip each card in half, stacking one half in one pile and the other half in the second pile. Done :)

<comment><USER7> This is the anarchist solution ;-)

<comment><USER8> Nah, that would be to set the deck on fire.

<comment><USER9> Maybe it's the communist solution? Everyone gets a piece of everything.

<comment><USER10> The existentialist's solution: throw all the cards in the trash and make two piles of zero.

<comment><USER11> The theoretical physisist's solution: chuck the cards into a sealed box with a cat and accept that you have both solved and failed the puzzle.

<comment><USER12> The Darwinist solution: the cards will eventually evolve into something that will sort itself

The creationists solution: Let God sort it out

<comment><USER9> Maybe that's already happened: we're sorting them. Are we sure we're not sorting them _for_ them?

<comment><USER13> The metaphysical solution: can a card be face up if one can see it?

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER14> Face up cards in 42 card deck = N

Face up cards in 10 card deck = 10-N

Face up cards in 10 card deck after flip = 10 - (10-N) = N

N == N, so they are equal

</comment><comment><USER15> I had to see this myself: <http://gist.github.com/592622>

Ruby proved it right, but I still don't get it. :-(

<comment><USER16> To prove the solution works:

Take ten chosen cards from the deck. Call this set of cards C. Number them c1 through c10. Similarly, number the face-up cards f1 through f10, such that if a face-up card was taken, it has the same number in both orderings.

Note that, at the end of arranging the decks, if a card is face-down, it doesn't matter where it is. The only thing we care about is the locations of the face-up cards. So we can ignore all the cards that aren't in the C or F sets, as they're all face-down in the original deck.

Now, for each card cx in the C deck, flip it over. There are two cases: either cx == fx (i.e., the card was originally face-up), or cx != fx.

If cx == fx, you've flipped it face down, and it doesn't contribute to the count of face-up cards for either deck.

If cx != fx, you've flipped it face-up, and fx is in the original deck. So we can increase the count of face-up cards in both decks by one.

So, for each card we've chosen, either that card contributes nothing to the count of face-up cards in either deck, or it contributes one to the count of both decks.

<comment><USER15> Thank you. Telling me to ignore the face-down cards let me wrap my brain around this one.

</comment></comment></comment><comment><USER17> This puzzle seemed very simple. "Impossible" puzzles should be a little tougher.

</comment><comment><USER11> This is pretty old, I've been told it many times in the past... albeit usually as "there are X coins in a dark room".

<comment><USER18> I think a variation of this was in the one of M. Gartner's books. It was about red and black cards, but the idea was the same, iirc.

</comment></comment><comment><USER19> How about turning the light on?

</comment><comment><USER20> Fascinating puzzle, although I agree impossible was a bit strong. I never would have concluded on my own this to be the solution.

</comment><comment><USER21> What about shuffling the cards REALLY well and then picking 26 in random.

<comment><USER22> Wrong

</comment></comment><comment><USER23> I solved this in like a minute. I rule!</comment>


Searching for Good Neural Networks with Neuroevolution - <AUTHOR> https://eng.uber.com/deep-neuroevolution/ <comment><USER2> I wrote about this in my book 'C++ Power Paradigms' that I wrote about 25 years ago. I devoted a very long chapter to an implementation I called Vari- Gene where I started by using a small number of bits to represent weight parameters and slowly increased the number of bits per weight. At the time, I had lunch with John Koza and we discussed my idea. He said that it was a net idea but that it wouldn't scale. He was correct, I never was able to train a large recurrent net. BTW, I didn't see any source code and data mentioned in the first linked article. Any links?</comment>


Mongo: Mr Right... or Mr Right Now? - <AUTHOR> http://blog.scripted.com/dev/why-mongo/

<comment><USER3> I had Mongo lose data.

The box it was hosted on was shut down (possibly uncleanly), and when it came back up, mongod refused to start. I ran mongod --repair and ended up with a running system, but the data was mangled. There were parts of records inside other records and a large amount of data that I just couldn't find. Fortunately, most of it was backed up elsewhere.

I've been told that maybe some other version is more reliable. I've been told I need to configure it properly by hand before I use it. I don't care. This was a trivial single-machine installation and there is _one_ thing any sane database absolutely _must_ do: avoid completely losing all the data that has been stored in it.

<comment><USER4> What version was this? 2.0 (which has been out almost a year) has journaling enabled by default which almost always provides complete single server durability for these situations.

Any database can suffer corruption if you shut down the server uncleanly - <USER2> depends on the state of the filesystem when it comes back up. That's why you have replication to protect against failure of a single node.

Having run Mongo in production for over 3 years, I've never lost any data despite numerous failure scenarios (including unclean shutdown).

<comment><USER3> I believe it was 1.8. I know it wasn't a high-reliability configuration, and it _could_ be that an unclean shutdown was to blame. I do not claim to know objectively how reliable MongoDB is.

I am, however _very_ hesitant to rely on it for anything now, much as I would be hesitant to touch a machine that previously gave me a severe electric shock even though its creator and other users assure me it's safe now.

<comment><USER5> I can tell you now that it's _extremely_ reliable - we store a lot of data in our mongodb - several million documents a day. As mentioned, it would be because journalling wasn't enabled which would mean possible corruption on non-clean shutdown. That's history.

<comment><USER3> That makes me willing to consider subjecting a test environment to abuse to see if it loses data so as to consider it for production use, but only if I had a compelling use case for it.

</comment></comment></comment></comment></comment><comment><USER6> I don't understand how writing migrations takes "weeks (if not months)". I can see "hours (if not days)". But, you still have a schema and migrations, they just exist in the application layer instead without a clear upgrade path.

<comment><AUTHOR> I think you're underestimating just how many times I rethought my schema and refactored my models :)

</comment></comment><comment><AUTHOR> Anyone else have a good story about Mongo as a speccing tool (as opposed to a long term solution)?

<comment><USER7> I did need a schemaless store for a project I was prototyping, but then I didn't see why I had to install MongoDB just to get a simple schemaless store. So, I wrote Goatfish:

<https://github.com/stochastic-technologies/goatfish/>

<comment><USER8> Wait, you found it too difficult and time-consuming to _install_ software, so instead you _wrote_ software?

<comment><USER9> I think he mains maintaining a mongodb installation, which is not trivial.

Same reason people use SQLite instead of MySQL or Postgres. Don't want to have to deal with keeping a database server up.

<comment><USER7> Exactly. This way, I get my schemaless datastore without leaving the stdlib, _and_ it doesn't eat my data at unsuspecting times.</comment></comment></comment></comment></comment>


Divshot 1.0: Visual Front-End Development for Bootstrap and Beyond - <AUTHOR> http://blog.divshot.com/post/55791825061/divshot-1-0-visual-front-end-development-for-bootstrap

<comment><USER2> 80,000 beta testers?! Wow, great work guys!

</comment><comment><USER3> I dig the concept. I would surely give it a try since I am playing around with foundation framework.

One thing though, the video on the landing page is really too fast to actually see anything. Is it just me ?

<comment><USER4> Playback speed seems to be defaulted to something other than 1x; check out the bottom-right of the video for the > (versus the >>)

</comment></comment><comment><USER5> Here are the steps I did.

1\. Signed up for an account.

2\. Created a project called "test" using default settings.

3\. Created a blank file called "index.html".

4\. Your icon keeps flashing in the main area, and Firefox seems to be waiting for something to finish. As far as I can tell, this never ends.

The result being that I cannot actually create the contents of index.html. My Divshot username is same as my HN username.

</comment><comment><USER6> Love this product and the story!

Another company created out of Startup Weekend, they went on to Launchpad LA and most recently raised $1.1m in March: [http://techcrunch.com/2013/03/18/web-app-interface- builder-d...](http://techcrunch.com/2013/03/18/web-app-interface-builder- divshot-raises-1-1-million-seed-round/)

</comment><comment><USER7> The interface looks so nice and clean and crisp. Nice work, guys!

</comment><comment><USER8> I've been with Divshot for at least three months now, and it's by far the best development tool I've used. Good luck with your 1.0!

</comment><comment><USER9> If you don't mind my asking, what are the competitive advantages of Divshot over, say, JetStrap?

<comment><AUTHOR> I would say some of the biggest differences are our multi-framework architecture and our focus on being a viable full-fledged development platform.

Divshot supports Bootstrap, Foundation, and Ratchet (at present) and our new project structure works just like a folder on your hard drive. Because Divshot allows you to edit the entire HTML document you can actually build entire applications in it using a third party service like Firebase or Parse.

There are lots of smaller differences, but our focus on being a full development platform as opposed to a mockup/prototyping tool is probably one of the biggest.

<comment><USER9> Thank you, as an occasional user of Jetstrap, I was wondering what the benefits might be. I'll have to take a deeper look now.

</comment></comment></comment><comment><USER10> Absolutely fantastic product, well done guys

</comment><comment><USER11> I could use this for Rails projects, yes?

<comment><AUTHOR> Divshot works great as a prototyping tool for Rails projects (and if you're using something like Angular for templating, you can do much much more with it).

We built [http://revision.io/](http://revision.io/) for the Rails Rumble last year (2nd place, woo!) and used Divshot to prototype all of the interfaces. It's a huge time-saver, but I'm a bit biased :)</comment></comment>


What colour are your bits? - <AUTHOR> http://ansuz.sooke.bc.ca/entry/23 <comment><AUTHOR> An old one, but after reading the thread on the reddit warrant canary it seems relevant.</comment>