Writing a Very Fast Hash Table with Tiny Memory Footprints - <AUTHOR> http://www.idryman.org/blog/2017/05/03/writing-a-damn-fast-hash-table-with-tiny-memory-footprints/ <comment><USER3> Not to disgrace the author (it's a good article), but I've spent a lot of time studying hashtable performance recently and I groan whenever I see benchmarks against libraries like std::unordered_map or libcuckoo. No shit your hash table performs better; it's like claiming you're good at running because you can beat a crippled child in a race.

For a test to be meaningful it needs to include comparisons to linear probing (both regular, robin-hood variants, and SSE variants), and other low-cost hash tables like coalesced hashing or separate chaining with a memory pool.

Speaking of robin hood hashing, pretty much all of the existing articles about it are missing some crucial facts about it. Don't take what blog authors say for granted.

<comment><USER4> > pretty much all of the existing articles about it are missing some crucial > facts about it.

Would you mind expanding on those facts?

<comment><USER3> 1) The linearly-probed robin hood variant is just a sorted array. That's it. It's just a sorted array. I haven't seen an article explain it so succinctly though.

2) The linearly-probed robin hood variant is only faster than regular linear probing for searches that fail. Successful searches have the same average probe count. Too many articles bullshit about how robin hood reduces variance (which is irrelevant 99% of the time) and then go on to ignore the fact that they cannot beat the performance of linear probing when it comes to successful searches. One guy I saw spent hours trying to optimize his hash table for the K-nucleotide benchmark without realizing that a trivial 5-minute implementation of linear probing would beat it because searches in that benchmark never fail.

3) There's a simple 40 year old bitwise trick by Knuth which allows one to combine the probe distance and the hash into a single value. This saves up to 4 bytes per table entry, but I haven't seen anyone use it.

There's also a few articles that get deletion straight-up wrong, but meh. That doesn't bother me too much.

Also, for double hashing I'm pretty sure a SSE/AVX variant is optimal. I haven't tried implementing one though.

<comment><USER5> 1) I don't see why a linear probing robin hood is a sorted array? Can you explain more?

Robin hood hashing doesn't limit which probing scheme you use. I end up with quadratic probing which gives me both good cache locality and good probe distributions. The probing schemes I tried was omitted in this post because it would bring too much noise. But I can give you some quick summary here:

1\. linear probing: probing distribution has high medium and high variance. Performance is not that great either because of the high probing numbers.

2\. quadratic probing: probing distribution is not the best, but the medium sicks to 2 probes. Since the first two probes are very close in quadratic probing, its overall performance wins.

3\. When probing, hash the key with the probe added to the seed. This gives very good hash distribution, but hash on each probe is very slow. Also you cannot do deletion using this scheme.

4\. rotate(key, probe). This is somewhat like rehash, but way faster. The probe distribution is also very good, but items goes too far away so we lost the cache locality.

5\. Combination of different schemes listed above. Still, the quadratic probing gives me best performance.

I also tried to use gcc/clang vector extension to speed up probing, but it actually slows down for 30%! I guess I have to hand tune the SSE intrinsics and measure it carefully with IACA to get the optimal performance.

Deletion itself is quite complicated and deserves its own post.

<comment><USER6> It's a sorted array because your sorting by 2 keys (A,B). A is the hash, normally a u32 or u64. B is the probe count, also normally the same sized b/c system ints are easy.

Strictly speaking the Robin Hood "sort" isn't purely lexiconally ordered. A larger A value maybe replaced a smaller A, with a much larger B.

But this relation nonetheless is just a weird solution to build a cmp function one. One that arguably doesn't work in a strict sense. But one that _nearly_ works.

<comment><USER7> So if A is the hashed-to index into your table, then B is a function of A. When you are doing your linear probe and you see a value along with its probe count, it either

1\. Has a higher probe count, this means it's hash is further away than yours => less than yours

2\. Has a lower probe count than yours. This means its hash is closer than yours => greater than yours

3\. Is equal in probe count to yours. That means it's in the same bucket => equal to yours.

</comment></comment></comment></comment></comment></comment><comment><USER8> One nit, as one of the authors of libcuckoo: Of the tested tables, libcuckoo is the only one that's a concurrent hash table. I'm glad that without modification, our single-threaded performance is still doing well, but this isn't the primary use case of libcuckoo.

If you want to compare it against a thread-unsafe table, it's a bit more honest to use the locked_table class, which grants ownership to all locks in the table in advance. It's a fair bit faster.

It doesn't change the memory use - I've created a stripped-down version in the past that elides all the locks, which is better for memory efficiency - <USER2> that's not really our goal.

For more on using the locked_table interface, see: [https://github.com/efficient/libcuckoo/blob/master/tests/uni...](https://github.com/efficient/libcuckoo/blob/master/tests/unit- tests/test_locked_table.cc)

(Also, for hashing strings, please make sure you've got libcuckoo configured to use cityhash for string hashing, as you had "opic", to make the comparison equivalent.)

In fairness to the OP, however, it's worth noting that this is a pretty new addition to libcuckoo (February), and we haven't documented or pushed it hard.

<comment><USER5> Thanks for pointing out the thread-unsafe alternative. I'll update the benchmark with it and double I configured it with cityhash. (will update in a day or two).

Speaking of cityhash, have you tried farmhash as well? I'm not sure what I did wrong, but farmhash's performance wasn't good as cityhash in my hash table. Did you experience the same problem?

<comment><USER8> Thanks! Appreciate the responsiveness. :)

We haven't - I had some other issues with Farm. I tried for a while to see if I could get FarmHash included as the standard hash for TensorFlow instead of the murmur-like homebrewed one it uses, but the cross-platform build issues became too complicated to justify it. We ended up sticking with the murmur- like one for "fast but unsafe", and providing SIP via the highwayhash package as the "strong hash".

</comment></comment></comment><comment><USER9> If you're interested in _truly_ space-efficient hash tables—as in "at every point in time, at least 97% of the space of the table is used for elements"—check out this paper by a colleague of mine: [https://arxiv.org/pdf/1705.00997.pdf](https://arxiv.org/pdf/1705.00997.pdf) (the relevant plots are on pages 10 and 11). He's going to publish the code soon-ish as well.

<comment><USER5> In my experiments using linear probing in robin hood doesn't give good results as well. I hope the author has tried robin hood with quadratic probing. But it's a good reference. I didn't dive deep in the cuckoo route. This paper gives me more confidence to try it out.

<comment><USER8> If you're curious about blending some of the "don't increase by a power of two" memory efficiency gains, I implemented this in a cuckoo hash table for TensorFlow, also using Lemire's trick: [https://github.com/tensorflow/tensorflow/blob/master/tensorf...](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/util/presized_cuckoo_map.h)

I never back-ported it to libcuckoo, because it adds a fair bit of complexity (and requires an extra bit), but if there's demand for non-2^k tables, I'd love to know more, either externally or internally. (dga@, for various values of cs.cmu.edu and google)

<comment><USER5> Lemire's fast range works perfectly with cuckoo, as it doesn't require probing (?). The fast mod and scale trick was invented to address this issue.

It's actually pretty funny. I didn't know Lemire's trick until I start to write this article. But I my first try on fixed point arithmetic was exactly the same as his work. Then figured out the lower bits would get omitted with fast range (I named it scaling). Finally I came up with this fast mod and scale idea.

I don't know for other peoples need on non power of 2 tables. My target is to build large scale hash tables and other data structures. The memory and data compactness is quite critical for this goal.

</comment></comment></comment></comment><comment><USER10> I see often discussions about hash maps on HN. However they mostly focus on performance (memory and speed) of one huge hash map with millions of records. In my numerical calculations I often need lots of small hash maps (say up to 100 elements). Currently I use unordered_maps from C++ standard library. Does anyone know what is recommended hash map implementation for my scenario?

<comment><USER11> For very small number of elements, an array or a list can be more efficient than hash maps. My rule of thumb is to avoid hash maps if you know you'll never have more than 100 entries, and then try to find the bottleneck later on. Using hash maps for everything isn't always the best solution.

<comment><USER12> Using a sequence instead of a map is fine if you know it will _never_ be more than some small number of entries, but so often we are wrong about how many entries will be in a table that the N-squared use of a sequence as a map kills performance unexpectedly.

<comment><USER13> But that's what profiling is for.

<comment><USER14> Profiling won't tell you what the chances are that the collection will someday need to contain more than N elements.

<comment><USER13> You don't replace every map in your code with a distributed Key-Value store, in case it might need to store terabytes in the future. Most of the time you have a pretty good idea whether a map has to store a hundred, or a hundred thousand elements.

</comment></comment></comment></comment></comment></comment><comment><USER15> I only skimmed over, but about memory overhead:

 _> The input is 8 M key-value pairs; size of each key is 6 bytes and size of each value is 8 bytes. The lower bound memory usage is (6+8)⋅2^23= 117MB_

In many case, hash table implementation won't (can't) assume fixed size of keys and values, and use pointers. On 64-bit architectures, this can mean there's an uncompressible overhead of 8 * 2 = 16 bytes (one pointer for each key and value) for each item.

In fact, a quick look at the OP's benchmark code shows he's using std::string as keys and uint64 as values. With e.g. std::unordered_map, while pointers won't be used for the values, there obviously will be pointers for the key.

It's actually worse than pointers, since the std::string memory layout is a size_t and a char* pointer. And it looks like his hash table essentially uses the equivalent of char[6] as keys. So a fairer memory overhead comparison should use char[6] as keys, instead of std::string...

<comment><USER16> Small string optimization should kick in here, so you're probably not looking at a size_t + pointer + a object with the 6 bytes of payload data. Instead it'd be a single object of about 32 bytes (depending on the implementation).

But yes, this means that the author's own hash table implementation is getting a roughly 4-5x density benefit from using a different string representation. That totally invalidates the memory usage testing, and probably also has a major effect on the speed tests.

<comment><USER15> Thank you for making me look at the implementation of std::string in GCC's libstdc++... The "small string optimization" indeed makes every std::string at least 1 size_t + 1 pointer + 16 bytes large (for a total of 32 bytes on 64-bits platforms).

Strings smaller than 16 bytes are stored in the 16 bytes inline buffer, which means strings larger than that actually take 32 + their length + 1 bytes in memory on 64-bits platforms.

I guess things are not packed further to allow to just directly use the pointer instead of having to do some computation for each access.

<comment><USER16> > I guess things are not packed further to allow to just directly use the > pointer instead of having to do some computation for each access.

There are C++ standard libraries that pack things further. I think clang has a 24 byte std::string, which can store strings of up to 22 bytes inline.

</comment></comment></comment></comment><comment><USER17> This note about his other project (that the robin hood hash table depends on) is quite curious:

> This robin hood hashing is implemented using my project Object Persistence > In C (OPIC). OPIC is a new general serialization framework I just released. > Any in-memory object created with OPIC can be serialized without knowing how > it was structured. Deserializing objects from OPIC only requires one mmap > syscall. That's say, this robin hood implementation can work not only in a > living process, the data it stored can be used as a key-value store after > the process exits.

> Right now, the throughput of OPIC robin hood hash map on small keys (6bytes) > is 9M (1048576/0.115454). This is way better than most NoSQL key-value > stores. The difference might come from write ahead logs or some other IO? > _I'm not sure why the performance gain is so huge. My next stop is to > benchmark against other embedded key-value store like rocksdb, leveldb and > so forth._

<comment><USER5> Thanks for raising this up. I spent way way more time on OPIC (almost a year) than robin hood hash map (2 weeks to complete the POC). Glad to see people noticing this project :)

The other comment pointed out MDBM, which I didn't know about. From their performance number I think this may show that why OPIC robin hood is quite optimal. [https://yahooeng.tumblr.com/post/104861108931/mdbm-high- spee...](https://yahooeng.tumblr.com/post/104861108931/mdbm-high-speed- database)

MDMB gives users raw access to the mmaped data and pointers. And from its benchmarks it results 10x faster than rocksdb and leveldb. The design of OPIC has even less overhead (may not be a good thing) than MDBM, and it also works on a mmaped file (or anonymous swap). There's no lock, transaction, or WAL in OPIC. OPIC just brings you the raw performance a hash table can gives you.

</comment></comment><comment><USER18> Look up MDBM. I spent a lot of time with a logic analyzer watching the cache misses go across the bus. I'd be pretty surprised if someone has done better.

I can find the code and repost it.

<comment><USER5> Thanks for the reference. I'm collecting a list of embedded key-value store to benchmark against. I'll tryout with this one first!

<comment><USER19> Fun note: Because of this line in the article

> Because MDBM gives you raw pointers into the DB's data, you have to be very > careful about making sure you don't have array over-runs, invalid pointer > access, or the like.

Someone wrote a Rust binding [http://erickt.github.io/blog/2014/12/13/rust- and-mdbm/](http://erickt.github.io/blog/2014/12/13/rust-and-mdbm/)

<comment><USER5> Wow that's a very interesting post! Maybe I should try to bring a rust binding to my project as well..

</comment></comment></comment></comment><comment><USER20> How does it compare to the "Fastest Hashtable" by Malte Skarupke? ( [https://probablydance.com/2017/02/26/i-wrote-the-fastest- has...](https://probablydance.com/2017/02/26/i-wrote-the-fastest-hashtable/) )</comment>


Guess the most expensive domain name - <AUTHOR> https://domain-pricing.glitch.me/ <comment><USER2> Since it's a Glitch app, you can also just look at the source to see which domains are cheapest. Good if you can parse json with your eyes. [https://glitch.com/edit/#!/domain- pricing?path=domains.js:1:...](https://glitch.com/edit/#!/domain- pricing?path=domains.js:1:0)

</comment><comment><USER3> There are more terrible domain names on sale for hundreds of millions of dollars: [https://dofo.com/search/?extension=com&sort=price_d](https://dofo.com/search/?extension=com&sort=price_d)

</comment><comment><USER4> Many of the people sitting on good domains have no desire/incentive to sell these days. Everyone wants to use it as leverage to "build a brand" and get equity in your company while maintaining ownership of the domain.

Rent seeking, but we fetishize .coms and will continue to into the future, so unlikely for anything to change.

</comment><comment><USER5> these are pointless to guess. anyone can put up a domain name for large amounts of money and never get a buyer. it should be based on actual sold prices.

</comment><comment><USER6> [https://www.godaddy.com/garage/the-top-20-most-expensive- dom...](https://www.godaddy.com/garage/the-top-20-most-expensive-domain- names/)</comment>


You Can't Copyright Porn, Harassed BitTorrent Defendant Insists - <AUTHOR> http://torrentfreak.com/you-cant-copyright-porn-bittorrent-defendant-insists-120206/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+Torrentfreak+%28Torrentfreak%29 <comment><USER2> This is actually an incredibly interesting (and cunning) assertion. I hope she takes it all the way, and doesn't accept a settlement from the defendants. Having some case law on this matter would be, at the very least, entertaining. It seems the judge would have to rule either that porn is "useful art", or that it cannot be copyrighted. One little copyright troll may just have inadvertently kicked the proverbial hornets' nest.

<comment><USER3> What's interesting is that while some porn does have artistic elements, a lot of newer porn does not: it is literally just people fucking. In the case of the latter, I think the producers would be hard pressed to argue that anything creative or expressive is taking place.

<comment><USER2> So a big-budget knockoff of a Hollywood movie (a la Pirates XXX) would be protected as art, but backroom casting couch wouldn't? Similarly, Star Wars Kid wouldn't be covered, but Tosh's show on it would be?

I'm simply fascinated by the possible ramifications of a court deciding this issue.

<comment><USER4> Those things are not obscene.

What he's saying it that to rise above obscene, porn would/could add extra artistic stuff to it.

</comment></comment></comment></comment><comment><USER5> tl;dr; Argument of the defendant is based on:

"Article 1, Section 8, Clause 8 of the United States Constitution, known as the Copyright Clause, empowers the United States Congress: 'To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries," the lawsuit details, adding:

"Early Circuit law in California held that obscene works did not promote the progress of science and the useful arts, and thus cannot be protected by copyright."

<comment><USER6> Sure, except anything that passes the Miller Test is, by definition, not obscene.

<comment><USER7> But the point of the Miller Test is to define what obscenity is. If porn isn't obscene, then what is?

I'd say porn pretty clearly fails the Miller Test, with the possibility for argument of works containing elements relating to the prurient interest being granted clemency under the third criterion, that is: someone may argue that a work containing a subsection that may generally be considered "obscene" should not be considered "obscene" on the whole because of its other literary, artistic, political, or scientific qualities.

So this case means that they have to prove that their specific pornography does not meet the 3rd criterion of the Miller Test to classify it as non- obscene.

<comment><USER8> Child porn, cruelty and (possibly) some kinds of criminal assault, _ie_ material that cannot be produced without unlawfully injuring the subject.

<comment><USER9> Your comment merely serves to affirm falsehoods which we, as a society, have come to accept without question. There is a good amount of "child porn" which does not injure the subject.

<http://news.cnet.com/2100-1030_3-6139524.html> (Federal case may redefine child porn)(2006-NOV-30)

[http://blog.al.com/spotnews/2011/02/tuscaloosa_county_photog...](http://blog.al.com/spotnews/2011/02/tuscaloosa_county_photographer.html) (Tuscaloosa County photographer sentenced to federal prison in child modeling porn case)(2011-FEB-09)

Both articles are about Jeff Pierson, and his case is just one of several examples that I can give regarding the misuse of child porn laws. What about bestiality? Is it cruel and/or illegal because animals can't give consent? We eat animals much more often than we fuck them, yet we don't care that they can't consent to being killed. What about those "bum fights" videos, where homeless guys are paid to fight each other for our entertainment? Is that "cruel", or "criminal assault", or "unlawful injury"?

I agree that children should be protected, and that animals shouldn't be hurt unnecessarily, but the crux of the matter is that there is a big difference between the act, and the recording of the act. The former is illegal, whereas the latter is just information, which in itself cannot harm, and which should thus not be illegal. The bottom line is that we've been made to believe that there is no difference. There used to be the Iron Curtain, whose purpose was to control people through political ideology. Then we got the Moral Curtain, behind which we still live, and which consists of child porn, bestiality, and all things "obscene". Now we are busy constructing the Copyright Curtain.

<comment><USER10> I think it's about time to start Shit HN Says.

Did you really just say that the distribution and ownership of child porn should be legal? I think you just did.

I refuse to argue vile shit like that but I have to tell you that it is indeed vile shit.

<comment><USER11> I was about to read his comment with interest to see what claims could be made... then I realised who wrote it.

I could look up a few of his past comments on an older account of his, but it's surely easier just to link to his personal site and let your assumptions take over from there.

<http://thaddeusquay.com/>

Literally the only person I have ever found the need to speak negatively off on HN, as opposed to either agreeing or disagreeing with just their opinions.

<comment><USER9> Please continue. Your comment appears unfinished.

<comment><USER11> Hmm?

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> This is pretty interesting to me as I used to work at an adult media company a couple of years ago.

Amazingly, it was one of the best jobs I've ever had. Although the CEO started getting obsessed about 'protecting their copyright'.

It had gotten to a point where they hired a full-time lawyer that spent all his time essentially black-mailing customers much like the lady in this article. They would ask for a settlement upfront of several thousand dollars. Most people paid because they didn't want their name associated with that type of lawsuit and were afraid of the backlash they would receive if their friends or family found out.

This, among other things led to me leaving that company.

</comment><comment><USER13> The Internet needs more of this, that people don't just defend against the copyright lobby, but strike deep into their home territory. Put the copyright lobby on the defensive.

Idea for a startup: a litigation funder against copyright trolls. A copyright troll might say: we can make this case go away if you give us 'y', at which point a litigation funder steps in and says we will make it go away for 'z', where z<y. Basically the business case revolves around diverting settlement funds away from the copyright trolls, and giving the victims some benefit in the form of a discount. The trolls would have to fight back by bringing stronger cases or by competing on price and reducing the settlements demanded, in an effort to make the litigation funder uneconomic. It would be even more profitable if lawmakers could be lobbied to force losing copyright trolls to cover the cost of both sides of the litigation.

A sideline for the litigation funder could be funding the aggressive defence of copyrights on free software in return for a cut of the damages. Part of the trick would be to chose targets that free software authors might be pleased to take down: such as aggressive movie studios.

<comment><USER14> Are you saying the litigation funder simply agrees to fight the case and pay all damages in the loss but if they win they get some % of the original settlement?

That could be interesting, especially if they got very good at defending patent law suits. Most likely after a few cases the patent trolls would simply settle for a lower amount and the litigation funder would pocket the difference.

I know the term startup has different definitions, but to me this is more of a law firm then a startup since it would be very difficult to scale.

</comment></comment><comment><USER15> That'd be a terrifying precedent to set. "I don't see artistic value in your art, so I can mass produce copies of it."

<comment><USER2> The converse, however, would be to declare porn to be useful art, and thus deserving of the constitutional protections granted thereto. I have my popcorn at the ready.

<comment><USER16> Declaring pornography to be unprotected by copyright on account of "it is not useful" is an extremely scary proposition to me. The apathetic responses in this discussion are also pretty scary. Try to understand the potential implications here.

 __People enjoy it __\- if that is not enough to declare that it is "useful", then what is? How can you define any other forms of entertainment to be "useful"? How far-fetched would it be to eventually declare video games to be not "useful"? Or movies?

<comment><USER17> Seeing how copyright is being used to undermine people's constitutional rights, I think copyright should be abolished for absolutely all works, not just for porn.

The concept of copyright is broken. We need something new and that doesn't go against the human nature or against people's constitutional rights. We need something that doesn't restrict sharing or freedom of speech.

         How far-fetched would it be to eventually declare video
         games to be not "useful"? Or movies?

Not as far-fetched as SOPA.

<comment><USER18> Are you saying that copyright should be abolished in conjunction with replacing it with a new system? Or that abolishing it altogether would be acceptable? I'm not sure I can imagine that transition.

Copyright (and the patent system) are fundamentally broken in the US, but I don't agree that perpetual copyright terms and SOPA follow naturally from any legal copyright system. I'm a believer in limited copyright actually encouraging innovation, and that includes with restrictions on sharing and freedom of speech (eg I publish a book, I don't believe it should be a form of protected speech to then make a copy of that book and distribute it. On the other hand, I also don't believe that should be a felony or that you owe damages of more money than you might make in your lifetime).

Freedom of speech is already restricted in many ways, and not just in ways that a cynic would point out. None of us lives alone on this planet, which implies that we will have to make concessions in systems already without an optimal solution for all parties.

</comment></comment></comment></comment></comment><comment><USER19> From a legal layman's perspective, this should be wrong. For simplicity, let's consider a work of porn that has absolutely no "artistic value" whatsoever (judging this will not be objective, but let us assume we can do it). I unequivocally believe that _you should be able to copyright this work_ , because it's your creation.

The main point here is not whether a work has artistic value but if it has _any_ value at all (if not, nobody cares anyway). You should not be able to say: "You work is not artistic so I'm gonna copy it and torrent it".

<comment><USER20> copyright is not a "creation" right. It is a use(performance) and distribution right.

It didn't use to be before Sonny Bono Act IRRC (and shouldn't) be automatic. You should have to apply for this monopoly right.

<comment><USER19> Fully agree that it shouldn't be automatic, that would be too stifling. But you _could_ be able to do it, no creation should be un-copyrightable.

<comment><USER21> Even if it does not promote the progress of science or the useful arts?

On what basis then should it be copyrightable?

</comment></comment></comment></comment><comment><USER22> I'm not sure if she's saying "you can't copyright porn" exactly. She's saying that particular movie they accused her of downloading is obscene, and you can't copyright obscene materials.

porn != obscene, they have different definitions under law.

</comment><comment><USER23> This almost looks like a win-win situation. If they win people will be granted the right to freely share porn - which might have a downside[1]. If they lose that would certify porn to _promote the progress of science and the useful arts_ which is both fun and liberating for consumers :)

[1]less porn productions?

<comment><USER24> There would definitely be a downside commercially if porn were to be declared uncopyrightable (not that i think it has any chance of happening). Sure there might be amateur free porn that would still be made, but there wouldn't be many production houses that would stay in business if they no longer received profit from the sale of their work.

Many smaller production houses run on extremely tight margins, and after making sure cameramen, models, and everybody else gets paid, there is not much capacity to lose money on distribution and still remain profitable.

You would no doubt seeing at least smaller porn production houses closing. I would hope at least the people who at least believe in the utility of porn enough to distribute it would see this as a bad thing. Anti-pornography crusader types would likely see this as a victory.

<comment><USER25> I'm neither type of person: However I wouldn't boohoo this weakening of copyright.

</comment></comment></comment><comment><USER26> "to be smut, it must be utterly without redeeming social importance" -- tom lehrer, "smut"

</comment><comment><USER3> I don't know if this will fly but it is an interesting argument to make. I will be watching this one.

</comment><comment><USER27> Although the Constitution says the stipulation of copyright is 'to promote the Progress of Science and useful Arts,' it does not say that everything copyrighted must further that aim. Seems to me in general it just suggests that by giving people control of their intellectual creations, science and art will benefit.</comment>


Judge orders Wyoming to return $92k seized by officers - <AUTHOR> http://fox6now.com/2017/12/01/judge-orders-wyoming-to-return-92000-seized-by-officers/ <comment><USER2> Where are the criminal charges against the officers? When the worst that can happen as the result of a theft is there's an outside chance that you might have to give the money back and there'll be no negative repercussions why wouldn't you keep on stealing?

<comment><USER3> Don't hold your breath, they were doing their job.

<comment><USER4> If this is their job then the entire institution of policing is irredeemably flawed and we ought to scrap it.

If these officers were acting outside of the law then they should be fired and charged with a crime.

<comment><USER5> > If this is their job then the entire institution of policing is irredeemably > flawed and we ought to scrap it.

Our main problem as a society is that half the country believes the above, and the other half believes that the half that isn't them are all lawless savages who need order imposed by a heavy hand.

This can't end well.

<comment><USER6> > Our main problem as a society is that half the country believes the above, > and the other half believes that the half that isn't them are all lawless > savages who need order imposed by a heavy hand.

I have just completed an 18-month tour of the country by land (thisisthebus.com). I've met people from every corner of the land and talked politics, technology, religion, and all sorts of other things with literally hundreds, probably a couple thousand, at urban watering holes, suburban campfires, and rural churches.

Please point me to the place where I might find the latter half of the population you've described.

From the Everglades to the Oregon Coast, from the desert to the Hudson Valley, I've searched long and hard and haven't found them.

Everybody in this country wants peace. And a chance for their families to live in harmony in their communities. Nobody - (excuse my language) fucking nobody - wants two million people in prison. Nobody.

<comment><USER7> With all due respect and not to have a pissing contest, but I've been traveling, first in a motorhome and now with an old Avion travel trailer, for about 7 of the past 9 years (sort of hard to pin down exactly what "traveling" means in this context since sometimes I park in one place for a few months, but it includes multiple drives from coast to coast and from Alaska to southern Mexico), and I've met quite a few law-and-order Republicans who absolutely believe that the vast majority of people in prison deserve to be there, that if people just do what police say there won't be any more police brutality, and that maybe we should spend more money on law enforcement and give police more power. It's certainly not half, but it's a sizable portion of the US population, and clearly a sizeable portion of the voting population, since our government has been dramatically reshaped in the past year by them.

It's sometimes surprising, too. I've met sweet old ladies that as long as topics were of the travel and weather variety seemed perfectly normal. The moment politics came up, they'd show a mean side that would really throw me for a loop. Racism with a shocking lack of self-awareness about their racism is one of the biggest surprises.

That said, rather than arguing that you're wrong about the people you've met, I suspect you've experienced bias in the people you meet in traveling the way you do. I've noticed there is a bias in the people I interact with most, for sure; lots of hippies, train kids, as well as people who love the outdoors (in a hiking and biking in national and state parks sort of way, rather than a hunting and fishing sort of way), etc. frequent some of my favorite places to go. But I still meet enough middle-class white folks who have alarming (to me) ideas about police and race and justice in America.

If nobody wanted 2.2 million people in prison, we wouldn't have 2.2 million people in prison. _Somebody_ sure as hell wants it because they vote over and over for the people that perpetuate it. We can't pretend no one is responsible. There are people who made decisions that led us inexorably to this outcome, and people making decisions daily to maintain and even expand the prison industrial complex.

<comment><USER4> As an aside, the train kids are my favorite. They can be a real handful (to put it mildly) but damn if that isn't an interesting culture.

<comment><USER7> They are certainly interesting and most have nothing but kind intentions. They lead interesting but dangerous lives. I like chatting with them in measured doses but wouldn't want to hang out with them for long. There's a lot of fighting, a lot of yelling, a lot of drugs of questionable formulation, a lot of untreated mental illness, etc. But, they often know a lot about the areas where they roam and have fun stories.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> Every time I hear about asset forfeiture I can't believe this is legal. It just blows my mind.

<comment><USER9> Just another reminder that there is no rule of law in the USA; its judicial system is an arm of oppression, not service.

There's nothing we as individuals can _do_ about it besides acknowledging the illegitimacy of laws and abiding by a basic empathetic moral code, but don't kid yourself about how that will be treated.

If you feel like a law is wrong, look at why it was passed. There might be reasoning you aren't thinking of, like environmental damages. If you still feel like it's wrong, look at how likely the action it forbids is to hurt someone else by intent or accident. If you feel like that's extremely unlikely, and don't think you'll get caught, break that law!

Part of civil disobedience is acknowledging that you may get caught and punished. But in doing so, you are riding on the fact that the punishment _is_ arbitrary and offensive, which would rally people in support of you if it is carried out. At least, that's the idea. The real problem comes when your government is more than happy to simply arrest/beat/shoot people until they quiet down.

<comment><USER10> Fortunately you're wrong. In reality, the US is one of the 20 least corrupt nations on the planet, on par with Ireland and Japan and less corrupt than France. It has very strong property protection laws, which is both why the judge returned the money (naturally that would happen in a nation with no rule of law huh), and why the Obama Administration took important steps to neuter civil asset forfeiture abuse. Can more be done to fully end the abuse? Of course.

[https://www.transparency.org/news/feature/corruption_percept...](https://www.transparency.org/news/feature/corruption_perceptions_index_2016)

The US also has rare universal Congressional support for ending civil asset forfeiture abuses. Jeff Sessions undid some of the fixes the Obama Administration made; it turns out that might have been ideal, as Congress is now going to put it down for good by law.

Sept 2017:

"In Surprise Vote, House Passes Amendment to Restrict Asset Forfeiture"

"The amendment passed with a voice vote, meaning it had overwhelming support."

[https://theintercept.com/2017/09/12/in-surprise-vote- house-p...](https://theintercept.com/2017/09/12/in-surprise-vote-house-passes- amendment-to-restrict-asset-forfeiture/)

Nov 2017:

"A bipartisan group of senators wants to defund Attorney General Jeff Sessions' expansion of the Justice Department's civil asset forfeiture program, following similar efforts by libertarian-leaning and progressive members of the House earlier this year."

[http://reason.com/blog/2017/11/08/senators-press-for- defundi...](http://reason.com/blog/2017/11/08/senators-press-for-defunding-of- jeff-ses)

<comment><USER11> > least corrupt nations on the planet

Only because you don't define lobbying as corruption. In most (any?) other nation, individuals and corporations paying legislators to vote a certain way fits the definition of bribery and corruption perfectly.

<comment><USER12> This combined with civil forfeiture gives me a really hard time classifying it as anything else than corrupt.

<comment><USER13> Let's not forget about bill riders: [https://en.wikipedia.org/wiki/Rider_(legislation)](https://en.wikipedia.org/wiki/Rider_\(legislation\))

</comment></comment></comment></comment></comment></comment><comment><USER14> At the very least, we should try to control the vocabulary here. Instead of using _their_ terms (like "seize" or "asset forfeiture"), we should call this theft, robbery, and so forth, to reflect what's happening. Those would make for very different headlines.

<comment><USER15> > _Parhamovich told The Associated Press that he was traveling to several > performances in Western states and decided to bring his "life savings" > because maintenance staff often came into his rented apartment in Madison, > Wisconsin. The 50-year-old hid the money inside a speaker he was bringing > along on the trip. [...]

Parhamovich said the officers implied that carrying that much cash was illegal. He lied and said it was a friend's._

He brought his "life savings" with him on a road trip, hidden in a speaker, and then lied to the police about it. It doesn't take a cynic to see how ridiculously suspicious that is. There's plenty of material to start a civil forfeiture case, where the government has to ultimately prove by a preponderance of the evidence (ie "more likely than not") that the money was the proceeds of illegal activity, the same standard as lawsuits between private citizens.

This is not a theft or robbery. It's the police failing to convince a court in the end.

Why seize things upfront in cases like this? If they let the money go, the government would likely never see it again. The goal is to stop organized crime from successfully moving money around and laundering it. It was only a few decades ago that organized crime operated with near impunity in this country. Laws such as RICO and those strengthening civil asset forfeiture are no small part of our success in turning the tide against criminal organizations.

It's easy to forget how we got here, but if we let uninformed outrage rule the day, then we risk returning to the time when these criminal enterprises spread fear and violence without consequence.

It is always worth investigating reform, but you should think twice about demonizing those who are carrying out the instructions we gave them.

</comment></comment><comment></comment><comment></comment><comment></comment><comment>==

Edit: HN is rate-limiting me, so I'm responding to erpellan's very good point here:

You're right that it's not sufficient reason for seizure by itself, it's actually been litigated to that effect before iirc. But combined with other factors, like lying to the police (a crime itself in many jurisdictions as Michael Flynn has just learned), it can help establish the burden of proof necessary for seizure pending litigation.

(Further, he consented to the seizure in this case.)

<comment><USER16> Civil asset forfeiture is never justified. If you don't have a case, you don't have a case.

'Acting suspicious" warrants asking questions, NOT taking things when those questions don't turn up anything. The idea that it stops organized crime is also rediculous. No crime Lord was ever brought down by civil asset forfeiture. They work that statistic into their prices like any other for- profit Enterprise would and go on with business as usual.

I'd also clear up the "given orders" bit. You cannot really make a good case for police departments ordering their officers to simply take any and all money they see. That is left to the officers discretion. In this case, the officer decided to abuse his power over the general population. To take your orders idea to the end; Let's assume that the police department as a whole requires such theft. We aren't in a country without freedom of the press. Any one of those officers could (and indeed had a duty to) record and publish the illegal order. If such an order exists, then EVERY officer is willingly complicit.

Finally, let's say that we do need limited ability to lock up people's things. That should most definitely involve a case against the individual rather than against the possession itself. It should involve the rights to an expidited trial. Most importantly, it should involve a large burden of proof like any criminal case rather than simple majority.

<comment><USER15> Lying to police is more than acting suspicious.

I'm not claiming that police have a moral carte blanche for overzealous seizures. I'm saying that when they do their duty in good faith and then lose in court, we shouldn't jump to calling them robbers.

To your last point, the standard in civil court is preponderance/more-likely- than-not/"majority". Civil asset forfeiture is not a criminal conviction, carries no risk of prison time, and involves solely the transfer of property from one (who is not entitled to it) to another (who is at least more entitled to it). If I sue you for everything you're worth, or claim rightful ownership of your land in an action directly on the property, I only need to prove my case by a preponderance. Why should it be any different here?

A lot is made over the fact that the civil action is against the property instead of the person it's being seized from. This isn't as relevant as it sounds and it has nothing to do with the standard of proof being lower: the standard in all civil cases is preponderance of the evidence.

The suit is against the property because it's an _in rem_ lawsuit where the only thing being litigated is the disposition of the property. They are not trying to prove that the person it was seized from did anything illegal, just that the asset itself is the proceeds of a crime, whoever may have committed that crime.

</comment></comment></comment><comment><USER17> These stories make my blood boil, as they should all other people. The worst part of it is that people like to think that most police officers are good people just doing their jobs. However, their job is NOT supposed to involve stealing people's life savings. I can't see how any cop doing civil forfeitures, except in cases with extreme proof, (known drug dealer, drugs in the car, etc.) has any redeeming qualities as a human being. The officers involved in this and thousands of other cases are worse than the criminals they are paid to pursue - they are nothing more than armed members of a gang that conducts highway robberies.

<comment><USER5> I find a lot of it is selection bias -- because the police generally interact with criminals, they come to view the whole population as criminal. Do it long enough, you start to get jaded and ask yourself "why do I follow the law when all these people break it with few consequences?"

Also see the effects of the "thin blue line" eroding institutional morality... it's easy to get to a place where we are -- even white people in America no longer trust the police.

</comment></comment><comment><USER18> Having money is not suspicious. I'm sick of agencies acting as self- propagating thieves. The whole drug war needs to end.

<comment><USER19> Well, having tens of thousands of dollars hidden in a speaker is suspicious. But it seems that there is a verifiable story to go along with it

<comment><USER20> Even without a story to go along with it, engaging in "suspicious" activity is no cause for having your property stripped from you without due process.

</comment></comment></comment><comment><USER21> One more reason we need cryptocurrency. Asset forfeiture is incentive for the police to steal. Given the wrong incentives, even to the best of people, you will get the wrong outcome.

<comment><USER12> You're getting downvoted but you have a good point. You can seize cryptocurrencies as well but it will be harder to force someone to give up their password/pin. If they even manage to find your wallet which is much easier to hide than a stack of 92k cash.

<comment><USER22> If they manage to 'seize' your encrypted wallet, you can restore your backup on another device and move your crypto elsewhere.

<comment><USER23> The FBI transfers seized cryptocurrency to its own wallet to prevent this: [https://www.wired.com/2013/12/fbi_wallet/](https://www.wired.com/2013/12/fbi_wallet/)

<comment><USER12> They would need to break your encryption first.

</comment></comment></comment></comment></comment><comment><USER24> This is a prime example of why you never answer questions like this from police. Give them nothing and they have nothing to twist against you. In a rigged system, the best option is to not play whenever possible.

<comment><USER25> He didn't answer their questions though. He did not give them permission to search the car. He refused, so they brought in a dog, (supposedly) made it act like it found drugs, and used that as probable cause to conduct a search.

<comment><USER26> But he did sign a form 'releasing' the money to them.

<comment><USER27> Anything signed would have been arguably under duress. If someone is actively threatening you and coercing you to sign something the signature is irrelevant.

</comment></comment></comment></comment><comment><USER28> Any word on damages?

</comment><comment><USER29> Why is this here? On HackerNews? Since we are on the topic why are there more and more political stories on hackernews?

</comment><comment><USER30> Is there a lower bound on how much cash is "suspicious" under these laws? I feel like it's only a matter of time until some enterprising department sets up along one of the routes leading into Las Vegas and makes a killing.

"Who carries around a thousand dollars in their wallet? Obvious drug money!"

<comment><USER25> > Is there a lower bound on how much cash is "suspicious" under these laws?

$100. And even then it depends on your skin color and where this is being called into question.

It's more plausible to have $10000 in your pocket in Vegas than $1000 in the South Bronx.

</comment></comment><comment><USER31> I must be missing something -- what did TechDirt add to the original article?

<comment><USER32> Ok, we've changed the url from [https://www.techdirt.com/articles/20171202/15391338726/judge...](https://www.techdirt.com/articles/20171202/15391338726/judge- hands-back-92000-taken-musician-cops-failing-to-buckle-his-seatbelt.shtml) to the news article it points to.

<comment><AUTHOR> That's kinda dumb: the techdirt article added quite a bit to the original story

</comment></comment></comment><comment><USER33> Starting to think those 50 pesos the Mexican police took from me were civilly seized as part of an investigation I'm unaware of.

I must have like a dozen civil asset forfeiture cases open in Mexico.

</comment><comment><USER34> This is the reason why I sympathize with drug dealers more than law enforcement officers.

</comment><comment><USER35> [deleted]

<comment><USER24> Civil Asset Forfeiture has nothing to do with Trump. This outright theft of money and other assets during traffic stops has been going on for years. It was originally enacted to help deal with gangs and drug dealers who tend not to use banks for obvious reasons. This would allow law enforcement to seize the cash, drugs, cars, etc. to help suppress their illegal operations by confiscating their capital and equipment/vehicles.

It has since been perverted into the kind of travesty this article details. Personally, I don't even support Civil Asset Forfeiture on any level because it seems to violate every notion of property rights as well as some basic human rights that we supposedly believe here in America.

EDIT: my reference to Trump was in regards to OP's reference, which has now been edited out.

</comment></comment><comment><USER36> While I agree that this seizure of money wasn't right, and shouldn't have happened, one does have to wonder about the guy...

...has he never heard of safe deposit boxes?

It sounds like he works strictly on cash. Usually, to get a safe deposit box, you have to have an account with the bank you are getting one at (and there might be other restrictions as well). If he didn't have a checking account and couldn't meet any other restrictions, then he couldn't get the box.

But once you had the box, you could put the money in it, and not really worry about it. I just don't know why he didn't do this. It certainly couldn't have been lack of funds. He had more than enough to open an account, and pay for one year on a box big enough to hold the rest of the money.

So if he didn't do this - why? One good reason I can think of might be - operating with cash only - not paying income taxes. Sadly, his efforts at getting his cash back will likely cause him more problems with the IRS down the road.

He really didn't think this whole thing through, and it ultimately caught up with him. Of course, the fact that he wasn't wearing his seatbelt, against a law which makes it a crime of some manner in just about every state (save one), and it's also a federal law - all says something about his being ill prepared.

<comment><USER25> FWIW, cash is not FDIC insured while in a safe deposit box. And it doesn't solve his "problem" of how to safely move it around without getting it seized.</comment></comment>


Evaluating the Unsupervised Learning of Disentangled Representations - <AUTHOR> https://arxiv.org/abs/1811.12359 <comment><AUTHOR> Link to the Google AI blog post: [https://ai.googleblog.com/2019/04/evaluating-unsupervised- le...](https://ai.googleblog.com/2019/04/evaluating-unsupervised-learning- of.html)</comment>


Wolfram Alpha Pro Launched - <AUTHOR> http://www.wolframalpha.com/pro/

<comment><USER2> The discussion is here: <http://news.ycombinator.org/item?id=3569746>

</comment><comment><USER3> just out of curiosity...who would pay for this? i am not trying to be cynical or anything. I am just genuinely wondering who the prospects are.</comment>


Snap CEO sets goal of profitability in 2019 - <AUTHOR> https://cheddar.com/videos/snap-ceo-evan-spiegel-company-memo-on-2019-strategic-goals-and-profitability <comment><AUTHOR> A few key highlights:

2019: Invest in the root if you want the fruit.

2019 Strategic Priorities Deliver our Core Product Value Widen our Moat Operate as Owners Invest in the Future Be a Force for Good

2019 OKRs Make Snapchat the Fastest Way to Communicate Find Best Friends for all Snapchatters Achieve Full Year Profitability Lead the Way in Augmented Reality Spread Positivity</comment>


IOS6 html hardware acceleration changes and how to fix them - <AUTHOR> http://indiegamr.com/ios6-html-hardware-acceleration-changes-and-how-to-fix-them/

<comment><USER2> Can someone please explain what tradeoff is being made when Apple decides to hardware accelerate something in their browser? Is it that hardware accelerated elements are more fluid/responsive but require more battery consumption to perform?

I'm used to the PC world where battery/heat are hardly a problem and software rendering is usually a bad thing when hardware rendering is available.

<comment><AUTHOR> Yes, battery is consumed faster when using the GPU - I cannot tell you by how much, but obviously it is enough on iOS devices for Apple to change its feature sets there.

<comment><USER3> Can you give a citation or backing research? In my experience, if you've got two similar operations, the one on the GPU will consume less power (assuming you don't need to transfer substantially more data over the bus).

<comment><AUTHOR> i'm not an hardware expert, so I cannot tell you excactly how it works, but the point is: The device will calculate the same result in shorter time -> from what I know: If you want to calculate faster, you need more energy Also I don't think the devices are built to supply the CPU with less power while using the GPU.

Again: these are mostly assumptions, I'm not a hardware architect!

</comment></comment></comment></comment><comment><USER4> Having to hint hardware acceleration is bad. The browser shouldn't need to be told. :/

<comment><AUTHOR> Not all sites/apps need hardware acceleration and those who do consume a lot more battery. So there is kind of a reason for this - however the way apple implemented this is not from the best quality I think.

The bottom line is, that Apple forces developers to take care of this instead of having everything automatically hardware accalerated and therefore shortening the battery life of iOS devices(by a lot I'm guessing).

<comment><USER4> >and therefore shortening the battery life of iOS devices

Does rendering a few more polygons actually kill battery life that badly? I believe much of the OS is already hardware accelerated anyway.

<comment><AUTHOR> GPU usage does cost battery life, I cannot say how much - but on the other hand I cannot think of any other reason why they wouldn't use hardware acceleration on websites automatically (or allway)

<comment><USER4> I'm mainly wondering why developers need to hint hardware acceleration. Surely Safari can figure out when it is needed.

<comment><USER5> What I can't figure is why 2D transitions needs to be hardware accelerated to be smooth in mobile Safari.

Neither why their mighty Nitro JavaScript Engine can't handle smoothly a setTimeOut based fade in/Out.

Google Chrome on 8 years old hardware (less powerful than actual iPhone hardware) renders the moving web better.

Safari is a poor mobile browser.

<comment><USER6> [http://developer.apple.com/library/ios/#releasenotes/General...](http://developer.apple.com/library/ios/#releasenotes/General/RN- iOSSDK-6_0/_index.html#//apple_ref/doc/uid/TP40012166-CH1-SW19) (referenced from the article) states

 _"WebKit on iOS now supports the requestAnimationFrame and cancelAnimationFrame methods in JavaScript, as described here:<http://www.w3.org/TR/animation-timing/*>

That link in turns states:

_"script-based animations are most often performed by scheduling a callback using setTimeout or setInterval and making changes to the DOM to effect the animation in that callback.

A disadvantage of this approach is that the author of the animation script has no idea what the ideal frequency for updating their animation is. Instead, the easiest way forward for the author is to simply call setTimeout with a very small value, which in practice will be clamped to some minimum time like 10ms anyway. It likely won't be the case that 100 updates per second are required for the animation, especially if the page is in a background tab or the browser window is minimized."*

So, it looks like setTimeOut is discouraged in favour of this (draft) standard. Given that, I would understand that Apple would optimize setTimeOut for minimal power use, rather than maximum update frequency (not that I have the faintest idea about how one should go about that)

Also: have you tried your claim about Chrome's performance in a similar amount of memory as what iOS has to deal with?

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> If they disabled the hardware acceleration because the GPU requires more battery I wonder if they have factored in that the rendering needs more time to complete => users will have to stay on the website longer => they have to keep their screen turned on longer which costs also a lot of power...

</comment><comment><USER8> The conspiracy-minded might suspect that the real reason for this might have been to keep web view based apps inconvenient to develop, and poorly performing, to nudge developers towards native ObjC-based apps. Which are coincidentally harder to port to other platforms.

<comment><USER9> It would, however, indeed be paranoid and crazy.

The WebKit developers spend all day trying to drive the web forward. That's what they do. They're not pulling their punches. Try hanging out in #webkit on irc for a bit and see if you think they want the web to lose to native.

</comment></comment><comment><USER10> Easy solution, just implement the application in native code.

No need for workarounds and hints in an application whose main function is to display _documents_.

<comment><USER11> Here's how you make a tasty burger: make a tasty chicken sandwich instead. :P

The issue affects websites as well as hybrid apps. Given that this change is poorly documented by Apple, and it directly affects a JS plugin I'm working on, I'm quite grateful for the OP sharing their findings.

<comment><USER10> hybrid apps are not native.

<comment><USER11> I know that. But your comment was like answering a question about optimizing a Python snippet with a recommendation to use C++. If you don't like hybrid apps (or Python, or whatever), that's fine, but it doesn't mean that bringing it up aids the discussion. Trust the developer to understand the tradeoffs of their tools and stack.

<comment><USER10> My point is that native applications are the only way to take proper advantage of the hardware.

You cannot try to bend something thought to display documents into some kind of application, and they complain about lack of hardware acceleration.

<comment><USER11> Why _not_ have a document with hardware accelerated effects? Anyway, we're talking about a feature which already existed and worked quite well, which Apple felt the need to discourage the use of, probably for battery reasons. Which is fine, but it's annoying that they didn't document it in more detail.

It's ultimately the developer's responsibility to make good choices on behalf of users, whether on native, hybrid, or the mobile web.</comment></comment></comment></comment></comment></comment>


Apple's Silence in China Sets a Dangerous Precedent - <AUTHOR> https://www.nytimes.com/2017/07/31/technology/apple-vpn-china-dangerous-precedent.html <comment><USER2> Apple stood for privacy in the US where privacy is part of law. It didn't stand by privacy in China where privacy in some forms is prohibited as part of law. Asserting the US law in China would discount China's sovereignty, implicitly declaring "your principles are inferior to ours".

A global company needs to be sensitive to these political/cultural differences and accommodate them. Even in the US where "privacy" is part of law, the legal system recently ruled that Facebook can continue to track users online even when they're logged out. So pretending that the privacy issue is one of "morality" is just borrowing the emotional power of the "morality" label for something that is not a universal law on the planet ... not even within aspects of the US. It is not different from a middle-eastern global company offering christmas discounts for its products in western countries, or Amazon not selling banned books in the countries they operate, or selling books on witchcraft and sorcery as though the books were of the same standing as, say, Principia Mathemtica.

Permitted? Do. Not permitted? Don't. ... unless the goal of your company is to champion a particular political ideology. So, frankly, this NYTimes article is, from a global perspective, "politically incorrect".

<comment><USER3> This amount of cultural relativism is silly and frankly just plain ignorant. China doesn't restrict free speech for cultural reasons. The CCP restricts free speech because it fears losing its ruling status.

<comment><USER4> While you may be right about the CCP's motives, China has its own long history of political thought, including the correct role of the State and public dissent. Guess what? For the vast majority of Chinese history, rulers viewed public dissent as tantamount to open rebellion and reacted accordingly.

Political systems are in part products of their culture. Free speech is not part of the cultural or political history of China, as it is in the West.

<comment><USER5> This is called culturalism. Saying X is not for race Y is racism. Saying that X is not for culture Y is culturalism. There is really not that much difference here. Because I am Chinese/Saudi/Russian etc. in no way implies that I, or my culture, does not value such a basic right as free speech

<comment><USER6> I don't understand the point you're trying to make. Is it culturalism to say that Western cultures value free speech? If you just wanted to say that parent was wrong that Chinese don't value free speech, why the diatribe about culturalism and racism?

You also haven't really put forward any argument that the Chinese value free speech - you can make a base argument that it's valued in some Western countries because they've codified it in their laws - most notably the FA in the US(it's questionable how much this "basic right" is respected in reality). Calling it a "basic right" is culturalism by your own definition, as you apply your own cultural values(whatever they may be) over the whole of humanity. What makes you think humanity as a whole values this "basic right"? Millions of people live in countries where free speech is not as important as religion, for example, and blasphemy laws exist.

<comment><USER7> You seem to omit that's possible that we have universal pricinciples that's superior to whatever culture. You can attain these principles via philosophy. Any individuals can rediscover the values of free speech, freedom of government arbitrary, freedoms of thoughts and actions by just reflexion itself.

<comment><USER8> One cannot disassociate philosophical principles with the culture and history that brought about it.

By claiming there's some Universal Principal that's superior is basically saying that you're own beliefs and culture (presumably the one and the same as the proported Universal Principle) is superior.

This isn't mathematics where there's some objective truth.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> As much as I dislike Apple, they made the right choice here. While it may be easy to praise Apple's stance against FBI backdooring the iPhone's secure boot and condemn their compliance with removing VPN apps from the Chinese app store, the latter policy is completely unlike the former for several good reasons. With the Chinese market, you need to choose your battles.

Removing VPN apps from the Appstore does not compromise the privacy of any Apple users, and does not prevent existing users of these apps from continuing to use them on their devices. I would imagine Cook fought pretty hard to protect the privacy of Apple users to get this concession from the Chinese censors. For comparison, on Android, China is forcing users in Muslim regions to install apps that scan for "terrorist content" and track user activity.

[https://thinkprogress.org/china-muslim-surveillance- policy-c...](https://thinkprogress.org/china-muslim-surveillance- policy-c0ee4721882b)

<comment><USER10> > Removing VPN apps from the Appstore does not compromise the privacy of any > Apple users

That's such BS. VPN apps are banned _because_ they give users privacy. So it's logical that by removing them it hurts people's privacy in China.

> and does not prevent existing users of these apps from continuing to use > them on their devices.

And that's a copout and you know it. As soon as people reset their phones or buy new ones they won't be able to "continue using them". Taking that as a "win" is absurd.

</comment></comment><comment><USER11> This is only possible because Apple has a "walled garden" app store and doesn't let people install stuff from outside it. The solution is simple: Publish an official jailbreak allowing expert iPhone users to root their phones and use third-party app stores.

Then Apple's not violating Chinese law by directly providing VPN's, instead they're removing themselves from a situation where they're in a position to be capable of enforcing Chinese law. The user is free to install whatever they want on a device they've bought.

<comment><USER12> Is this actually necessary? You can install a VPN profile on your iPhone by manually typing in the settings, without going through the App Store.

<comment><USER13> If that VPN profile is limited to well-known VPN protocols, there is no possible combination of settings that gets you past the Great Firewall, a VPN app needs to add their own or it won't work.

</comment></comment></comment><comment><USER14> I don't think Apple has any other choice, except to open up iOS system so that it doesn't have the responsibility to censor apps. It's interesting to see how Apple's centralized control turns out to be a convenient tool for authoritarian governments.

<comment><USER15> This is the point John Gruber made:

 _To me, the more interesting question isn't whether Apple should be selling its products in China, but rather whether Apple should continue to make the App Store the only way to install apps on iOS devices. A full-on "install whatever you want" policy isn't going to happen, but something like Gatekeeper on MacOS could.

Keep iOS App Store-only by default. Add a preference in Settings to allow apps to be downloaded from "identified developers" (those with an Apple developer certificate) in addition to the App Store. In that scenario, the App Store is no longer a single choke point for all native apps on the device.

The App Store was envisioned as a means for Apple to maintain strict control over the software running on iOS devices. But in a totalitarian state like China (or perhaps Russia, next), it becomes a source of control for the totalitarian regime.

I don't expect Apple to do this. They'd rather deal with the negative consequences of the App Store as a choke point than give up the benefits (including the profits) of maintaining complete control over all software on the platform.1 But if you're angry about Apple's role in this VPN crackdown in China, I suggest you direct your anger at the App Store as the single source for third-party software._

<comment><USER16> As someone who just recently started using an iOS device and finds the lack of sideloading mildly annoying, can anyone with some expertise in iOS security chime in with whether or not allowing sideloading of apps on iOS represents any serious security implications for iPhones? From my admittedly amateurish understanding, iOS sandboxing is significantly more robust than Android, is this just a way for Apple to maintain total control of, and profit off of, apps, or does this policy have a basis in security as well?

<comment><USER17> The decision to prevent sideloading dates back to the release of the App Store. Upon release of the App Store, Apple charged a 30% royalty on every app sold. Developers were outraged by these fees, and fought against them. If Apple allowed side loading, then developers would sell their apps on their own websites and then tell users to sideload the apps. Apple had to prevent side loading to ensure success of their App Store. At the time, people with a developer account could always sideload apps, however they were much more difficult to obtain back then (had to send in verification).

Security was not the primary consideration. iOS was a rushed project and it took them many revisions to implement security features. In iOS 1.0, all apps ran as root. iOS didn't have ASLR until 4.3. This was fine since mobile malware barely existed back then.

I would argue Apple's sideloading motivations are primarily monetary based and for "protecting the user experience" (typical Apple PR talk). Why though, does android allow sideloading? I'd say it is because Android was ~2 years behind iOS and they needed to catch up. Therefore they made it very easy to build apps for their platform and let any developer install/test apps without an Android Market account. Also Android chose to be open source and there is little stopping someone from building their own version of Android without sideloading restrictions.

</comment></comment></comment></comment><comment><USER18> I'll repost my comment from the other thread that got flagged for being a dupe:

They absolutely stake out moral positions on issues like this all the time. The difference is they only do it in the West where rule of law prevails and they can actually exert influence.

They've done it with various anti-LGBT efforts that have taken place around the US, they've done it with the fight with the FBI and they're doing it right now in Australia [1], among others.

What they don't do is shoot themselves in the head in some misguided attempt at ideological purity.

Also this seems absurd to me:

>The company has not fully tested its political and economic leverage in China. It hasn't tested the public's immense love of its products. It hasn't publicly threatened any long-term consequences — like looking to other parts of the world to manufacture its products.

If China's authoritarian regime is objectionable then do what countries have always done in history: isolate them economically, diplomatically or politically. Try to foment and support an insurgency or revolution or just invade them yourselves. I'm not sure anyone is willing to risk nuclear war to bring democracy to China but those are your options. Apple isn't a vessel for anyone's foreign policy objectives and it isn't a substitute for the state.

[1] [https://www.macrumors.com/2017/07/20/apple-talks- australian-...](https://www.macrumors.com/2017/07/20/apple-talks-australian- government-encryption/)

<comment><USER19> Apple has most certainly taken a moral stance in the past as you point out. The galling part of the article to me was "The company has not fully tested its political and economic leverage in China." It is not Apple's _job_ to take a political stance. They can if they want, as evidenced by the examples you laid out, but there is no requirement. If the NYT wants to talk politics and precedence, perhaps they should direct their criticisms toward the U. S. government, et. al., who collectively think that China's membership in the WTO, and therefore I assume free trade with China, is just a-okay.

Apple is playing by the rules, and this time decided that this is not the hill they want to die on. That's their prerogative. Don't like the rules? Then go complain to those that make the rules to begin with, not those that play by those rules.

<comment><USER20> it's not their job to do it.

It's nobody's job to do it.

They still potentially can fix the problem. Wouldn't it be great if they used their influence to help fix things?

I agree that there's a "choose your battles" aspect to this. Considering what Apple did with the San Bernandino case, this seems (ideologically) right, but the potential losses are pretty high.

<comment><USER21> Google and Facebook took this stance and look where it got us? China just made their own bloody search engine and social network.

We need to stop looking to corporations as doing the jobs our government should be doing.

<comment><USER22> Let us keep passing the buck... We need to stop looking to governments as doing the jobs United Nations should be doing.

"Act as if what you do makes a difference. It does."

<comment><USER23> That's a good point but China is 1 of 5 nations in the UN with veto power.

</comment></comment></comment></comment></comment></comment><comment><USER24> Apple's silence reflects the cynicism of consumers that buy Apple products.

Apple is mainly selling a brand and not a product. Apple is archetypal emotional brand. It's intimate with its customers and it is loved.

As most global companies, Apple as a company has no morals. It takes moral stance when it's builds up the brand without hurting the sales. Only way to make Apple to sacrifice something is to threaten the brand and make Apple users feel dirty.

More stories like this. People must stop loving and identifying with brands. All brands.

</comment><comment><USER25> Is this a news piece or an opinion piece? The article is terrible. Apple has two choices: comply with the laws of China when operating in China or not operate in China. They can refuse to comply on principle, get kicked out of China, and get sued by their shareholders for breach of fiduciary duty. They can use all available means (none) to fight it. Or they can obey the laws of the country they are operating in.

I bet the author would take a dim view of a Chinese company refusing to obey an American law in America.

<comment><USER26> It's not just about Apple. Apple is still headquartered in US. Thus, it's also a moral quandary for US, because American lawmakers can make it a more stark choice for Apple: they can pass a law saying that no American company can aid and abet human rights violations. Then the choice for Apple would be to operate in China and comply with laws of China, or operate in US and comply with laws of US.

Is that too extreme? In this case, perhaps (or perhaps not!), but where is the boundary? We all remember how IBM helped implement Holocaust. I don't think anyone would argue that if someone wanted to do something like that today (say, use machine learning to identify closet atheists and gays in Saudi Arabia?), we'd just say "Oh well, they're just operating under these laws, and who are we to judge? It's all culturally relative anyway."

So, at some point we draw a line and say, "no, you can't do this, not if you want to be present in our society". Where is that line? And why shouldn't it lie at authoritarian government censoring political speech?

<comment><USER27> > We all remember how IBM helped implement Holocaust

Do you have more details of this? I've never heard about it before.

<comment><USER28> [https://en.wikipedia.org/wiki/IBM_and_the_Holocaust](https://en.wikipedia.org/wiki/IBM_and_the_Holocaust)

<comment><USER27> Thanks! Added to my reading list

</comment></comment></comment></comment></comment><comment><USER29> I won't call obeying local laws or following host's rules sets precedent. MNCs have been following the local rules since day one. What do you expect?

Would you stop speaking local language because it helps the rulers communicate? Would you stop using the bank notes because it's issued by a dictatorship? You didn't because it's essential to run your business within the country.

The US is your own country, you have obligation to fix it, that's why you stand up against FBI. But not for other countries, they have to fix their own country by themselves.

</comment><comment><USER30> It makes no sense for Apple to do anything but comply. Its only real options are to manipulate corrupt government officials (which is against US law) or leave and:

\- There are plenty of competitors in China and its people will continue to be well served by them should Apple choose to leave.

\- The people of China aren't going to benefit at all from any Apple departure.

\- Apple is a publicly traded company; its duty is to its shareholders. 20% of Apple's Q2 17 revenue came from China: [https://www.apple.com/newsroom/pdfs/Q2FY17DataSummary.pdf](https://www.apple.com/newsroom/pdfs/Q2FY17DataSummary.pdf)

So leaving is also not an option, an attempt to do so would be pointless and crater the stock so bad investors would revolt.

The article suggests a "public fight" but that seems incredibly naïve to me. China just doesn't work that way. The fight was lost when the GFW went up.

</comment><comment><USER31> Worth pointing out that this happened on the same day that TestFlight increases the number of beta testers from 2k to 10k. If someone open sources an iOS VPN app, you've now got 10k people that can sideload an app via the efforts of a single person with an apple developer account.

Obviously this is a loophole and not meant as a mainstream solution, but past solutions to getting around the great firewall (including VPN itself) have also been only for the technically informed and connected - often foreigners working in China. Basically it lets the technical elite have a way around the firewall to keep business running.

Personally I think it's unlikely these two bits of news are connected (upping testflight is likely about iOS 11), but still it may have a positive effect until it, too, is clamped down on.

<comment><USER25> I think anyone can download Xcode and compile to their device, so there's nothing stopping an open-source VPN app.

<comment><USER31> True, and really there's always some kind of super-high friction loophole in computing. For these types of loopholes to have an impact though a) people have to know about them, b) people have to be able to do them, and c) they have to be easy to do.

TestFlight is as easy as getting an email, downloading the testflight app, and downloading the app you need. Its a trivial amount of steps, which makes it a powerful loophole.

</comment></comment></comment><comment><USER32> When China next tells Apple to backdoor iMessage encryption, the question is not whether Apple will cooperate, but how the public will ever find out that they had.

</comment><comment><USER33> Wow, this is quite a ridiculous article.

>When Apple took a public stand for its users' liberty and privacy, the American government blinked.

Nope. They got access via other means, and didn't need Apple's help.

>While American tech companies frequently criticize decisions by American officials, they appear loath to do so in China.

Do you also expect a Bulgarian company to comment on Sri Lankan politics just because they're doing business there? Its not wrong to feel for citizens of another country, but that is a personal matter. I don't see why a corporation should be meddling in the internal politics of another country.

To the author: Have the US tech companies stopped dealing with the NSA and the CIA? Has the CIA given up their backdoor access to Google/FB/MS/Apple/Oracle/etc?

</comment><comment><USER34> What dangerous precedent? That Apple is a company that is governed by the rule of law?

I find it fundamentally strange that we expect our corporate overlords to do anything other than maximize their profits. I'd certainly like them to do more than that, but I don't have the expectation that they will. And expecting them to break the law, or leave the Chinese market, is simply just crazy.

It sucks that China treats its citizens in this manner, but I plan on abiding by Chinese law the next time I'm in China or I wouldn't go. Why should Apple be any different, and why is this somehow their fault?

<comment><USER35> >I find it fundamentally strange that we expect our corporate overlords to do anything other than maximize their profits.

There is difference between what we can _realistically_ expect people to do (often, not much, and even after accounting for that, still disappoint) and what they _should_ do and we should expect them do (do what is right).

To continue with the illustrative examples from fiction. Nobody _expects_ Theoden to do much anything useful after listening to all that poison of pouring out of mouth of the Wormtongue. But is it what he _should_ be doing?

</comment></comment><comment><USER36> Does anyone here think China is more free or informed because Google held a hard line and got banned?

</comment><comment><USER37> Easy for China to threaten seizure of Apple's manufacturing capabilities. Shareholders won't like that. When something doesn't make sense, follow the money.

<comment><USER38> China would probably never go that far. At best, they would just cut off the retail market in China and leave it completely to the 黄牛 yellow cow scalpers like it was 10+ years ago.

</comment></comment><comment><USER39> VPN apps are completely different than what happened with the FBI unlocking request last year.

</comment><comment><USER40> Good advice I got once: not every surface is a slippery slope.

</comment><comment><USER41> The day after Apple stands up to the Chinese government, Foxconn will be producing phones that no longer have the Apple logo, but will otherwise be Apple iPhones.

There is nothing Apple can do. In the name of profits, they have put their entire business in the hands of the Chinese government.

<comment><USER42> >The day after Apple stands up to the Chinese government, Foxconn will be producing phones that no longer have the Apple logo, but will otherwise be Apple iPhones.

That's just silly. Foxconn would do nothing of the sort.

<comment><USER41> I don't know if it is just silly. Their IP is in the wind.

The third shift is a real phenomenon for brands manufacturing in China even under good circumstances. I am simply extrapolating from that reality.

</comment></comment></comment><comment><USER43> Privacy concerns aside, without running VPN half of the "allowed" (i.e. not explicitly banned) Western Internet is completely unusable thanks to Google CDN, Google Fonts, Facebook Retargeting, Twitter panels etc. any of which stall the loading forever and the content never shows in Safari / Mobile Chrome. Many hybrid apps are unusable as well. Thankfully they didn't remove manual VPN configuration. Just made it all even more inconvenient.

</comment><comment><USER44> Who do people think Apple is? A saint? a 501(c)? an NGO? No, they are a multi- national profit maximizing corporation that sees not capitalizing on the growing middle class of China as an existential threat. So while the restrictions that the People's Republic of China (sic) imposes on it's citizens are reprehensible it is a place where Apple must abide by the rules if it wants to sell phones and other hardware and services.

</comment><comment><USER45> Absurd article. Why is this special to Apple? Why can US govt not take a stance and ban all chinese articles from their country?

<comment><USER46> That'd be very bad for business. The government is particularly hypocritical in this regard. Our BFFs in the Middle East _execute gays_, oppress women, and finance terrorism, and we pretend not to notice.

</comment></comment><comment><USER47> I don't know why people that love Apple find it so hard to admit that ultimately Apple first and foremost care about money. (And that's ok!)

That's _all_ that this issue is about at the end of the day.

</comment><comment><USER48> The latest data have Apple nearly dropping out of the Top 5 handset vendor. May be it should be the Top 4 vendor to complain first ?

</comment><comment><USER49> Interesting contrast to Google deciding to leave China. Apple has also chosen to use a China gov controlled cloud provider.

</comment><comment><USER50> Why does a western moral standard give Apple the right to break Chinese law?

Where do you draw that line? What US laws are ok to break? Should Apple export other US laws like labor standards? If so, Alabama or California law?

Like it or not, globalism means that we're participants in a huge global community. Part of being in a community is respecting your neighbors, even when you disagree with them.

<comment><USER35> As many others commented elsewhere, moral relativism is not very robust ethical system in a global community.

If you ever have kids, why would you teach them some morals over others? Would you teach that some moral standard (pick any! say, "it's not generally okay to steal others' stuff") would just _stop_ by the virtue that they just happen move to different jurisdiction where the government is not interested in e.g. property rights? Or it is okay that just the natives of Thiefmark have their stuff stolen by others? Can you go and take their stuff with you to Hobbiton, given they live in Thiefmark and King Thiofden is not going to punish you for that? What if the raiders of Rohan move to your neighborhood?

This is a different issue whether it makes sense or is productive in the long term to fight against the Chinese law in some particular way. But sensible ethical systems are universal.

<comment><USER50> I don't disagree, but I won't walk into the home of an acquaintance and tell him why he's wrong either.

There's obviously a line there. But a computer company isn't going to win against a police state.

</comment></comment></comment><comment><USER51> The fanboyism here is staggering.

</comment><comment><USER52> Money talks, human rights walks.

<comment><USER53> Without a minimum threshold of money and power you can't maintain human rights, let alone turn them into international norms.

<comment><USER52> Isn't it the opposite case here? Lot's of power and money involved, while human rights are violated.

</comment></comment></comment><comment><USER54> Lets assume that censorship for any given tech is allready existing and tight.

That means censorship only will struggle when new tech arrives and deploys to fast. This means, most vital for the fight against censorship is actually the development of new tech that is censorship resistant. This means tech that is not of the walled garden variety- tech that allows users to chose other severs, circumvent the defaults, allows users to run a local service instead of a centralized server. Tech that allows to draw copys of softwar without central registration. Apple by beeing apple does harm to free speach right there. The VPN removal is just the cherry on top.

<comment><USER55> > Lets assume that censorship for any given tech is allready existing and > tight.

That's an interesting assumption, though one could argue otherwise. How did you come up with this reasoning?

</comment></comment><comment><USER56> I would hardly consider this "setting a precedent", given the fact that this is standard practice for the Chinese government and any company who works there.</comment>


Teen Vogue's Yanked Facebook Sponsored Content on Election Integrity - <AUTHOR> https://web.archive.org/web/20200108170304/https://www.teenvogue.com/story/how-facebook-is-helping-ensure-the-integrity-of-the-2020-election <comment><USER2> The OP archive link is broken for me "Page doesn't exist"

<comment><USER3> I'm sure that this is just archive.org playing nice.

Edit: I'm assuming that there was actually content at that URL when it was submitted. But maybe I'm wrong. Perhaps jashkenas could clarify that.

<comment><AUTHOR> It was cached correctly when I posted it, but it's certainly gone now. Sorry that I can't update the URL...

<comment><USER3> Thanks.

That's what I meant about archive.org playing nice. If sites ask them to update captures, they do. I'm not criticizing them for that. It's sort of a "right to be forgotten" thing.

But it means that they're not a reliable source for anything too controversial.

<comment><USER4> That's not what happened here. You're making huge leaps of logic, and while we might do a takedown, we will not "modify" a capture to satisfy a third party.

<comment><USER5> Seems to be working now, can you provide any insight into what folks were seeing earlier?

<comment><USER3> Please see my reply to textfiles.

</comment></comment></comment></comment></comment></comment></comment><comment><USER6> Original discussion here [https://news.ycombinator.com/item?id=21994173](https://news.ycombinator.com/item?id=21994173)

<comment><USER7> Ok, we'll move the comments thither. Thanks!</comment></comment>


Ask HN: How do you focus on working - <AUTHOR> I have a hard time focusing on my tasks, whether it's because of YouTube , Reddit or HN, I always find myself doing something that completely break my work mindset.<p>Do you have any tips, philosophy or tricks? <comment><USER2> It depends on the context, however, start with the mindset:

Done is better than perfect. To finish first, first, you have to finish. Motivation follows action – not vice versa. You work or you pause. You cannot do both. If you do not work, you stare at a blanket NOT Reddit, hn. Schedule time for Reddit, HN, etc. where you MUST consume this stuff (believe me, after being forced to do it, you will get over this addiction thing very fast!)

With this internalized, I set boundaries, usually time constraints, and define outcomes so that there is something tangible after the final whistle rings.

I set a timer, usually at one minute, and let it repeat. This helps me to get into the groove and stay in the present moment. Soon this will annoy you, so I increase the period gradually from 1 to 3 to 5 and 10 or 15 minutes.

I use Noise Cancelling Headsets, however lately I use deliberate distraction, which seems counterintuitive, but highly advisable: techno music as well as Medical Doctors.

Why? Content should not excite you too much, so that you deliberately listen to it, but are annoying so that you zone out.

This lets me focus. Sports psychologists recommend similar tactics. Very interesting!

Train yourself, repeat it until it is a second habit. Rinse and repeat. Use an arsenal of tricks to get you going.

Good luck!

</comment><comment><USER3> adderall</comment>


Contrast looking for Exceptional Ruby hacker (remote) - <AUTHOR> http://www.contrast.ie/blog/exceptional-engineer-required/

<comment><USER2> Contrast.ie is an amazing group of dudes, this is a serious opportunity for someone.

<comment><USER3> Exceptional has been a pretty useful tool for us.

</comment></comment><comment> oleg-d their main page doesn't look too usable to me..

<comment><AUTHOR> "But who is wurs shod, than the shoemakers wyfe, With shops full of newe shapen shoes all hir lyfe?"

[http://www.answers.com/topic/the-shoemaker-s-son-always- goes...](http://www.answers.com/topic/the-shoemaker-s-son-always-goes- barefoot)

:-) Thanks for your feedback!</comment></comment>