Smaller and faster data compression with Zstandard - <AUTHOR> https://code.facebook.com/posts/1658392934479273/smaller-and-faster-data-compression-with-zstandard/ <comment><USER3> I have been waiting for this to hit 1.0 and more importantly get popular so that I can use it everywhere. I am really a fan of Yann Collet's work. These are extremely impressive work specially when you consider that lz4 seems to be better than snappy (by google) and zstandard from LZFSE (from apple). I think he is the first one to write a practical fast arithmetic coder using ANS. And look at how his huffman implementation blazes past zlib huffman though compresses less than FSE [0]. I also like reading his blog posts. While a lot of them goes over my head I can generally make a sense of what he is trying and why something's working despite the complexity.

[0] [https://github.com/Cyan4973/FiniteStateEntropy](https://github.com/Cyan4973/FiniteStateEntropy)

<comment><USER4> > I think he is the first one to write a practical fast arithmetic coder using > ANS.

I don't think he is the first; although RAD game tools has been cagey about the details, many strongly suspect their recently announced Kraken, etc. use ANS in some form and some of their previous products; see the discussion here:

[https://news.ycombinator.com/item?id=11583898](https://news.ycombinator.com/item?id=11583898)

...and here:

[http://encode.ru/threads/2492-Kraken- compressor](http://encode.ru/threads/2492-Kraken-compressor)

[http://cbloomrants.blogspot.ca/2016_05_01_archive.html](http://cbloomrants.blogspot.ca/2016_05_01_archive.html)

<comment><USER3> Yeah read about Kraken a while ago and found the encode.ru thread you pointed. Its unfortunate that we can't compare that side by side. Those guys seem another of those compression geniuses. Overall the world of compression is moving very fast and well these days.

But even with that Yann might be the first. Yan's work on FSE is old [0] and if I'm reading this correctly his work on FSE is a mix of his own work and Jarek Duda. But it seems one of his failed attempt challenged Jarek to invent another version of ANS called rANS [1] and as you can see in the encode.ru comments from the RAD game tools that they seem to be using rANS at least for Kraken. Regardless these are impressive works and these people are bouncing ideas of each others, being challenged and inspired which is a very good thing for the rest of us :).

[0] [https://fastcompression.blogspot.com/2013/12/finite-state- en...](https://fastcompression.blogspot.com/2013/12/finite-state-entropy-new- breed-of.html) [1] [http://encode.ru/threads/1821-Asymetric-Numeral- System?p=360...](http://encode.ru/threads/1821-Asymetric-Numeral- System?p=36050&viewfull=1#post36050) [2] [https://github.com/rygorous/ryg_rans](https://github.com/rygorous/ryg_rans)

<comment><USER5> Before Yann, this ANS variant was implemented by Andrew Polar in 2008: [http://www.ezcodesample.com/abs/abs_article.html](http://www.ezcodesample.com/abs/abs_article.html) Here is a list of implementations: [http://encode.ru/threads/2078-List-of- Asymmetric-Numeral-Sys...](http://encode.ru/threads/2078-List-of-Asymmetric- Numeral-Systems-implementations)

</comment></comment></comment></comment><comment><USER6> There is just so much awesome stuff in this article. Finite State Entropy and Asymmetric Numeral System are completely new concepts to me (I've got 7 open tabs just from references FB supplied in the article), as is repcode modeling. I love that they've already built in granular control over the compression tradeoffs you can make, and I can't wait to look into Huff0. If anyone outside of Facebook has started playing with it or is planning to put it into production right away I'd love to hear about it.

<comment><USER7> Indeed ANS is difficult: it is the biggest innovation in compression in the last 20 years. Its author has some nice but dense slides about it.

[https://dl.dropboxusercontent.com/u/12405967/ANSsem.pdf](https://dl.dropboxusercontent.com/u/12405967/ANSsem.pdf)

Not sure exactly when repcodes were invented. Igor Pavlov has already used them in 7zip.

<comment><USER8> The best TL;DR of ANS is something like this (without being too wrong). It's still too long:

Huffman requires at least one bit to represent any symbol, because it finds unique prefix codes for every symbol by varying the leading bits.

Arithmetic coding encodes symbols as fractional numbers of bits, by using binary fractions. It divides up the range to make this work. In the end, you get one fraction per "message" that can be decoded back into the message (IE a fraction like 0.53817213781271231 ....)

range coding is similar, it just uses integers instead of floating point. You get one number that can be decoded back into the message (IE a number like 12312381219129123123).

Note that in both of these, you have not changed the number system at all. The number of even numbers and odd numbers still has the same density.

Another way to look at the above is based on what a bit selects.

In huffman, a single bit is generally not enough to select something, the prefixes are long. So you need to walk a bunch of bits to figure out what you've got.

In arithmetic or range coding, a single bit selects a very large range. They change the proportions of those ranges, but at some point, something has to describe that range. This is because it's a range. Knowing you have gotten to the 8 in 0.538 doesn't tell you anything on it's own, you need to know "the subrange for symbol a is 0.530 ... 0.539", so it's a. So you have to transmit that range.

ANS is a different trick. Instead of encoding things using the existing number system, it changes the number system. That is, it redefines the number system so that our even and odd numbers are still uniformly distributed but have different densities. If you do this in the right way, you end up with a number system that lets you only require one number to determine the state, instead of two (like in a range).

<comment><USER9> With regular arithmetic coding and Huffman coding you don't need to send over a dictionary. Instead, you can have an adaptive model that learns to compress the data as it goes (e.g. keeps running track of symbol frequencies), and it will still be reversible.

I thought this wasn't possible with ANS. Or has this changed?

<comment><USER5> It depends if you have static or adaptive coder.

Static is much cheaper, uses the same probabilities for the entire data block (e.g. 30 kB), probabilities are stored in the header - practically all Huffman and tANS compressors (however, there are considered exceptions: [https://en.wikipedia.org/wiki/Adaptive_Huffman_coding](https://en.wikipedia.org/wiki/Adaptive_Huffman_coding) ).

Adaptive can start with e.g. uniform probability (no need to store in header) and learns on the way - it is more costly but gives better compression, used with arithmetic coding or rANS. See [https://fgiesen.wordpress.com/2015/05/26/models-for- adaptive...](https://fgiesen.wordpress.com/2015/05/26/models-for-adaptive- arithmetic-coding/) [https://fgiesen.wordpress.com/2015/12/21/rans-in- practice/](https://fgiesen.wordpress.com/2015/12/21/rans-in-practice/)

<comment><USER9> I know that's what adaptive coding means, I thought that was impossible with rANS. All the descriptions of rANS I could find (back when I looked at it) described it terms of a static model, and I ran into problems trying to generalize it to an adaptive one.

Thanks for the resources.

</comment></comment></comment></comment></comment></comment><comment><USER10> The plot of compression ratio against speed for the various compression levels is pretty helpful for understanding its performance: [https://scontent.fsnc1-3.fna.fbcdn.net/t39.2365-6/14146892_9...](https://scontent.fsnc1-3.fna.fbcdn.net/t39.2365-6/14146892_944159239044397_638267599_n.jpg)

"The x-axis is a decreasing logarithmic scale in megabytes per second; the y-axis is the compression ratio achieved."

I'd love to see a version of this chart that also included Brotli. (And I'm somewhat surprised Brotli isn't mentioned at all.)

(Disclaimer: I work at Google, which made Brotli)

<comment><USER11> I thought that brotli was tuned for typical web workloads, that it contained a dictionary tuned for web workloads. Our internal testing shows that it performs very poorly for binary 3D vector data.

So a test between zstd and brotli would show brotli in a poor light if it used a mixed corpus, but a test between zstd and brotli on a web corpus would give an advantage to brotli...

<comment><USER12> >Our internal testing shows that it performs very poorly for binary 3D vector data.

Uh ? What format of data was this ?

I did a pretty large test (2gb+) on OBJ/STL 3d data and brotli compressed within ~5% margin of lzma, and this holds true on other binary data I've compared.

It also compressed better than zstd (as in compression ratio) on the same data at their highest respective compression settings:

bro -quality 11 -window 24

zstd --ultra -22

So I find it baffling that you find brotli to be poor for any binary data, could you share the data in question ?

<comment><USER13> What is the compression time for brotli to achieve compression ratios comparable to LZMA on these types of data sets: [https://github.com/google/brotli/issues/165](https://github.com/google/brotli/issues/165)

I understand that brotli is incredibly slow as compared to LZMA when using these high ratio settings (q11, w24), so slow as to be impractical in production even in a write-once, read-many scenario if you have any non- trivial amount of data being regularly produced. I do not want to have a farm of machines just to handle the brotli compression load of our data sets just because it is 10x slower than LZMA.

<comment><USER12> >What is the compression time for brotli to achieve compression ratios comparable to LZMA...

Compression time is indeed the achilles heel of Brotli, which is why it's something I would only use for compress once (and preferably decompress very often) scenarios.

Compared to lzma at it's best compression setting for this particular data (-mx9 -m0=LZMA:d512m:fb273:lc8), brotli took 6 minutes and 4 seconds to compress, while the same data took 1 minute and 47 seconds for lzma.

On the other hand, brotli decompressed the same data in 0.6 seconds, while it took lzma 2.2 seconds.

</comment></comment></comment></comment></comment><comment><USER14> Note: this is from the same guy who created the popular lz4 compressor, Yann Collet: [http://cyan4973.github.io/lz4/](http://cyan4973.github.io/lz4/)

[https://twitter.com/Cyan4973](https://twitter.com/Cyan4973)

</comment><comment><USER15> Yann will be giving a talk on Zstandard at today's @Scale 2016 conference, and the video will be posted. He can answer the most technical questions about Zstandard, but I may be able to answer some as well; we both work on compression at Facebook.

<comment><USER3> I am really looking forward to this. I usually like to read more than vidoes but for complicated topic with a good presenter it can actually be a comprehensive starting point. Would the video also be posted today or we will have to wait?

One thing I haven't figured out from either today's post or Yann's blog is whether Zstandard is switching between huff0 and FSE depending on compression level or is it somehow using both together? Also the post says its both OoO friendly and multi-core friendly but the speed benchmarks are those in a single core context or multi-core? Does only the format/algorithm multi-core friendly or the standard cli can run multi-threaded.

<comment><USER15> I'm not sure when videos will be posted -- hopefully soon.

All benchmarks today are single threaded. The algorithm itself is single threaded, but can be parallelized across cores. We will soon release a pzstd command line utility to demonstrate this, similar to pigz, which accelerates both compression and decompression.

Zstandard uses both huff0 and FSE together when it compresses -- it doesn't switch between them based on the input.

</comment></comment></comment><comment><USER14> The modern trend of compressors is to use more memory to achieve speed. This is good if you're using big-iron cloud computers...

 _" Zstandard has no inherent limit and can address terabytes of memory (although it rarely does). For example, the lower of the 22 levels use 1 MB or less. For compatibility with a broad range of receiving systems, where memory may be limited, it is recommended to limit memory usage to 8 MB. This is a tuning recommendation, though, not a compression format limitation."_

 _8MB_ for the smallest preset? Back in the mid-2000s, I was attending a Jabber/XMPP discussion, about the viability of using libz for compressing the stream. It turned out that even just a _32kb_ window is _huge_ when your connection server is handling thousands of connections at a time, and they were investigating the effect of using a modified libz with an even smaller window (it was hard-coded, back then).

I know Moore's law is in ZStandard's favor w.r.t. memory usage (what's 8MB when your server's got 64GB or more?), but I think it's useful to note that this is squarely aimed at web traffic backed by beefy servers.

<comment><USER16> I don't think 8MB is the smallest preset, the text you quoted says that the lower levels use "1 MB or less".

The concern I have is that this makes it sound like the compressor can choose how much memory the decompressor will need to use. Does this mean that zstd can't be used in a potentially adversarial environment? (Eg. is there a denial-of-service vector here by forcing the server to use large amounts of memory to decompress my requests?)

<comment><USER17> The decompressor can receive a parameter to refuse compressed frames requiring more than a selected amount of memory.

</comment></comment></comment><comment><USER18> I'm a complete dunce when it comes to compression and how it fits in the industry, so help me out here. Say that everyone accepts that Zstandard is amazing and we should start using it. What would the adoption process look like? I understand individual programs could implement it since they would handle both compression and decompression, but what about the web?

Would HTTP servers first have to add support, then browser vendors would follow?

<comment><USER19> >what about the web

The browser sends the server a request header indicating which compression methods it understands. Current Firefox for example sends

        Accept-Encoding: gzip, deflate, br

meaning the server is free to send a response compressed with either gzip, deflate or brotli. Or the server can choose to send the data uncompressed.

This means the adoption path for the web would be an implemtation in at least one major browser, which advertises this capability with the Accept-Encoding header. Then any server can start using Zstandard for clients accepting it.

<comment><USER18> Nice, so really browsers and servers can implement it independently. Seems elegant, I like that browsers can advertise multiple compression formats. Thanks for the info!

<comment><USER20> It's also possible to implement a decompressor in javascript to support browsers which don't do it natively. The performance would likely suck but if you're truly bandwidth constrained and don't mind users having a bit of a lag, it's an option...

<comment><USER21> Assuming the size of the decompressor isn't larger than the savings you gained from using this compression algo over another...

<comment><USER22> Well the decompressor would be cached by the browser so it would pay off for repeat visitors. And if it gets used enough, the user will already have downloaded the decompressor from another site.

<comment><USER23> And easily shared across sites if they all use a popular CDN'd version.

</comment></comment></comment></comment></comment></comment></comment><comment><USER24> Some more benchmarks on this[0] page

Also, I actually discovered something _very_ interesting (to me at least). At the bottom of the link mentioned below, the link attached says [https://github.com/Cyan4973/zstd](https://github.com/Cyan4973/zstd) but then redirects to [https://github.com/facebook/zstd](https://github.com/facebook/zstd) . Anyone know why?

[0]: [http://facebook.github.io/zstd/](http://facebook.github.io/zstd/)

EDIT: After a little bit of sleuthing, it looks like the author of zstd (github.com/Cyan4973) is now contributing[1] to github.com/facebook/zstd

And the page layout for lz4[2] looks the same as zstd[0]

Anyone know if Yann Collet works for/with facebook on things other than zstd?

EDIT 2: In the time it took me to google a couple things, looks like the children comments have already answered my questions.

Also, previous discussions on zstd (not that its completely relevant) -[https://news.ycombinator.com/item?id=8941955](https://news.ycombinator.com/item?id=8941955) [https://www.reddit.com/r/programming/comments/2tibrh/zstd_a_...](https://www.reddit.com/r/programming/comments/2tibrh/zstd_a_new_compression_algorithm/)

[1]:[https://github.com/facebook/zstd/pull/312](https://github.com/facebook/zstd/pull/312) [2]: [http://cyan4973.github.io/lz4/](http://cyan4973.github.io/lz4/)

<comment><USER25> > the link attached says > [https://github.com/Cyan4973/zstd](https://github.com/Cyan4973/zstd) but > then redirects to > [https://github.com/facebook/zstd](https://github.com/facebook/zstd) . > Anyone know why?

I believe this is just GitHub's standard behavior when a repository is moved to another namespace. We recently renamed [https://github.com/D-Programming- Language](https://github.com/D-Programming-Language) to [https://github.com/dlang](https://github.com/dlang) , and e.g. [https://github.com/D-Programming- Language/dmd](https://github.com/D-Programming-Language/dmd) redirects in the same way.

<comment><USER24> Oh, D language is dlang now? Looks like you guys did a redesign of the website and stuff too! I hear good things about D, keep up the great work!

<comment><USER25> Thanks! To clarify, we didn't rename the language, just the GitHub organization. However, dlang.org is what we've been using as the canonical address for the website, and "dlang" is a much more searchable term than just "D". (Now, if only Google would stop auto-correcting "dlang" to "golang"...)

</comment></comment></comment></comment><comment><USER26> The goals sound similar to Apple's LZFSE (see [https://github.com/lzfse/lzfse](https://github.com/lzfse/lzfse) for more). Any comparison out there?

<comment><USER27> That was my first thought, too. I installed and ran both against a tar'd set of PDF files totaling 435MB in size. My timings:

        lzfse  45 MB/s encode, 229 MB/s decode, 1.12 comp ratio
        zstd  181 MB/s encode, 713 MB/s decode, 1.13 comp ratio

The numbers are so dramatically different that I ran several different tests, but those results showed the same rough results. I used default command-line options for both tools, and both created very similar compression ratios.

Note that LZFSE has a somewhat different goal, however: it's designed to be the most power-efficient compression algorithm out there, in other words on mobile devices LZFSE optimizes for bytes-per-watt rather than bytes-per- second. Zstandard, on the other hand, runs multiple pipelines and such--it's banking on having a server-class processor to run on.

Edit: hardware is a 2013 MacBook Pro, pretty fast flash storage, and 2 cores/4 threads. I warmed cache before each run and sent output to /dev/null, so the numbers above are best-case.

<comment><USER28> I use "fastest to complete == least power usage" as a rule of thumb because of "race to sleep". I suppose that might be thrown off by power usage characteristics varying based on number of cores working? How does one even begin to write code that prioritizes power-efficiency over performance?

<comment><USER29> For short runs, any fixed-cost warmup/cooldown periods might dominate - <USER2> latency/throughput tradeoff kind of affair. (I am just shooting my mouth off and have no idea whether that's the case here.)

As for how to do it, I'm not sure... but if I were a valued customer of various CPU suppliers, and were famous for the depth of my pockets, I'm sure I'd be able to find somebody to explain it to me ;)

</comment></comment></comment></comment><comment><USER30> From the bits of testing I've done today, it's phenomenally fast on x86. Much better than gzip (and pigz for that matter) in every metric I think I generally care about: CPU Usage, Compression Speed, Decompression Speed, Compression Ratio.

On other architecture the picture gets a bit murky, it seems to get handily beaten by pigz through what at first blush I'd guess is just sheer parallelism. It's got solid performance, and without a shadow of doubt faster than vanilla gzip. If/as/when I get time, it'll be interesting to dig into why performance is worse there.

<comment><USER30> Dug in a bit further. On the non-x86 architecture I use, it looks like it's really just straight core performance that explains it. pigz's only advantage there really seems to be the brute force parallelism.

In particular note the huge difference in branches between gzip and zstd on decompress:

8959780663 branches # 143.024 M/sec

2969481781 branches # 64.454 M/sec

and on misses:

542158823 branch-misses # 6.05% of all branches

89060880 branch-misses # 3.00% of all branches

<comment><USER17> There is now a pzstd implementation, if you wish to compare it to pigz

<comment><USER30> Had a shot. It's slightly buggy, but holy crap is it fast.

I'm not a C programmer, understanding what happened is a bit beyond me but: 1) to compile on linux it needs the -pthread flag passed to it, Makefile is missing that (compiles fine on OS X) 2) decompression over stdin appears to be effectively impossible, still demands in input file. Compression over stdin works fine.

</comment></comment></comment></comment><comment><USER31> This is an awesome blog post that is very well written, but the lack of incompressible performance analysis prevents It from providing a complete overview of zstd.

Incompressible performance measurements are important for interactive/realtime workloads and the numbers are extremely interesting because they can differ dramatically from the average case measurements. LZ4 for instance has been measured at doing 10GB/sec on incompressible data on a single core of a modern Intel Xeon processor. At the other end of the spectrum is the worst case scenario for incompressible data where performance slows to a crawl. I do not recall any examples in this area, but the point is that it is possible for algorithms to have great average case performance and terrible worst case performance. Quick sort is probably the most famous example of that concept.

I have no reason to suspect that zstd has bad incompressible performance, but the omission of incompressible performance numbers is unfortunate.

<comment><USER17> zstd goes at > 1 GB/s on uncompressible data. It has some fast heuristics for such cases too.

</comment></comment><comment><USER32> A recent compression discussion I saw involved how do compressors fare on uncompressible input? For example, suppose you wanted to add compression to all your outbound network traffic. What would happen if there was mixed compressible traffic along with the uncomressible kind? A common case would be sending HTML along with JPEG.

Good compressors can't squeeze any more out of a JPEG, but they can back off fast and go faster. Snappy was designed to do this, and even implementations of gzip do it too. It greatly reduces the fear of CPU overhead to always on compression. I wonder how Zstd handles such cases?

*Ignoring security altogether

<comment><USER33> Dropbox say they can _losslessly_ compress JPEG files (~20% saving):

[https://blogs.dropbox.com/tech/2016/07/lepton-image- compress...](https://blogs.dropbox.com/tech/2016/07/lepton-image-compression- saving-22-losslessly-from-images-at-15mbs/)

<comment><USER34> Known techniques. JPEG uses Huffman coding for entropy coding, you can replace that with arithmetic coding. This requires knowing the format details, though.

</comment></comment></comment><comment><USER35> >> "It is written in highly portable C, making it suitable for practically every platform used today"

I love C, it is not the enemy everyone makes it out to be.

It's already in debian: [https://packages.debian.org/stretch/zstd](https://packages.debian.org/stretch/zstd) and judging by the small requirements,it is portable indeed.

</comment><comment><USER36> Quick benchmark on a 194MiB SQL dump:

        gzip -9: 27.574s, 48MiB output
        zstd -9: 14.182s, 41MiB output

Thanks, I'll gladly use zstd as a drop-in replacement for my daily backups. :)

</comment><comment><USER37> Isn't it a bit presumptuous to call your own thing "standard"?

<comment><USER38> Like these guys: [https://github.com/feross/standard](https://github.com/feross/standard), which is permissive to the point it's completely pointless. You can follow it to the letter and still end up with vomit code.

</comment></comment><comment><USER39> Really nice work compared to what I consider to be the quite bad Brotli -- an incredibly slow compression standard that only ended up in browsers because it was created by Google.

<comment><USER40> except for the PATENTS file =/

<comment><USER41> Curious what your issue is with it -- it basically says "if you dont sue us, we wont sue you". Thats about as good as I can expect from a large tech company these days with regards to patents.

<comment><USER34> Compare it to the Opus patent license (also including a retaliation clause), which includes grants from Broadcom, Mozilla and Microsoft.

Using zstd gives Facebook a free license on ALL your patents.

Using Opus gives Facebook only a license on patents that apply to Opus.

So no, large tech companies can and have given MUCH better grants for compression tech, than Facebook is doing.

<comment><USER3> I don't understand these things well. But if true this would be really bad. LZ4 is everywhere because it was completely free and I think most of Zstandard's real work was pre-facebook by just Yann alone. Now to have its hand tied because of his job at facebook is the worst thing possible.

I just read the Opus patent summary. It seems like if zstd followed the same license using it wouldn't give facebook any license but if I sue facebook I loose the licese to use zstd. Am I correct in that.

<comment><USER34> _Now to have its hand tied because of his job at facebook is the worst thing possible._

If you accept the job you give up your work. It's not like there's no compensation.

 _Am I correct in that._

Yes, the retaliation (and hence, implicit license to Facebook) is only for the technology itself.

</comment></comment></comment></comment></comment></comment><comment><USER42> I think for typical JS/CSS/HTML sizes, and decompression times, probably maximum compression ratio, followed by decompression speed is what I'd look for. I don't care too much about compression speed, in the sense that if I have to spend 1 minute compressing JS to crunch it by 10%, but I serve that file a million times, then as long as decompression doesn't negate the gain in network time saved, it's a win.

I guess the other factor for mobile is, besides memory and decompression speed, how do various compression schemes fare battery wise?

<comment><USER43> Regarding decompression times, I think it's much more important to save transferred network data than to prioritize quicker decompression.

HTTP request times have a long tail; they can get really slow for the many, many people limited to slow connections. Decompression times are going to be much more consistent. Our aim here should be to improve the 10% slowest requests, and you do that by optimizing the actual transfer.

</comment></comment><comment><USER44> For this to become anything like a standard, Facebook would have to remove its patent poison pill.

</comment><comment><USER45> If anyone wants to try it on windows there is a 7-zip install with support for ZSTD

[https://mcmilk.de/projects/7-Zip-zstd/](https://mcmilk.de/projects/7-Zip- zstd/)

</comment><comment><USER46> If facebook hopes the new compression algorithm to be a standard, why doesn't it publish an IETF RFC draft? Will it follow OpenDNS way of dnscrypt by open- sourcing the reference implementation without publishing any IETF RFC draft?

</comment><comment><USER47> The following link points to a fairly good benchmark / tool that showcases the tradeoffs in real life: since (de)compression takes time, what is the fastest way to transmit data at a given transfer speed?

[https://quixdb.github.io/squash- benchmark/unstable/#transfer...](https://quixdb.github.io/squash- benchmark/unstable/#transfer-plus-processing)

Spoilers: zstd wins at ethernet and wifi (and is among the best in 4G), lz4 wins at hard drive encryption… both were designed by the same author.

<comment><USER48> Charles Bloom's Oodle new codecs handily beat them across the scale:

[http://cbloomrants.blogspot.com](http://cbloomrants.blogspot.com) [http://www.cbloom.com/oodle_arm_report/](http://www.cbloom.com/oodle_arm_report/) [http://www.cbloom.com/rants.html](http://www.cbloom.com/rants.html)

Caveat: proprietary commercial product.

</comment></comment><comment><USER49> I was looking for a windows version of zstd. On the github page, I could only get the version 0.81 version of the windows tool. Can someone release the 1.0 version of zstd for windows ?

</comment><comment><USER50> How difficult is this new standard going to be to implement in another language? It seems highly sophisticated -- which is great, of course -- but the cost of that is relying on giants like Facebook to maintain their One True Implementation. For software this is (usually) fine; for a nee standard, it's a problem.

<comment><USER15> The format itself is documented ([https://github.com/facebook/zstd/blob/master/zstd_compressio...](https://github.com/facebook/zstd/blob/master/zstd_compression_format.md)) with the intention of other implementations and language bindings being readily available. We also have a zlib-compatible API for easier porting to applications already using Zlib. Our hope is that Zstandard is both easy to use and easy to contribute to.

</comment></comment><comment><USER51> That's truly beautiful. Thanks, Facebook! I particularly love that you can pre-compute and reuse dictionaries, say if you're regularly compressing similar JSON objects.

</comment><comment><USER52> Zstandard is both a command line tool (zstd) and a library.

</comment><comment><USER53> I'd love HTTP content encoding to support this and see a comparison to Brotli. Looks like it might be yet another good alternative to gzip.

</comment><comment><USER54> Is this being pushed as being standard as part of the HTTP spec, seeing that it comes from Facebook?

<comment><USER39> Just about anything is better than brotli.

</comment></comment><comment><USER55> turbohf claims to be 4x faster than zlib's huffman coding and 2x faster than FSE and is a generic cpu implementation. Even if claims are only partially true and turbohf is a clean dropin replacement for zlib and licensing were friendly the appeal of zstd drops substantially in my book.

[http://encode.ru/threads/2276-TurboHF-1GB-s-Huffman- Coding-R...](http://encode.ru/threads/2276-TurboHF-1GB-s-Huffman-Coding- Reincarnation)

[https://sites.google.com/site/powturbo/entropy- coder](https://sites.google.com/site/powturbo/entropy-coder)

</comment><comment><USER38> Unless they integrate it into software like web servers and web browsers it will be hard to see it really flourish as a "standard".

But at least within the perimeter of your own systems you can totally profit from this technology now.

</comment><comment><USER56> Beautiful.

</comment><comment><USER57> Tables should be in HTML.

</comment><comment><USER58> Looks very interesting, however I'm not impressed by the name. "Zstandard"??? With ".zstd" as the extension? I don't like it.

They should have named it _letter_ -zip, along the lines of gzip, bzip, and xzip, with the extension _letter_ z. "fz" would have been a good one since they work at Facebook.

<comment><USER59> [https://en.wiktionary.org/wiki/bikeshedding](https://en.wiktionary.org/wiki/bikeshedding)

<comment><USER58> This isn't bikeshedding. Bikeshedding is about quibbling over unimportant details. Names are critically and absolutely important. Lots of great things have been hobbled or ruined by poorly-chosen names. A terrible name can cause something worthy to be ignored in favor of something inferior but with a better name. And you don't need to even be competent in the inner workings of a project to criticize its name or suggest better ones. The people who name cars aren't the same people who design the engine-control algorithms for them.

If you disagree, what do you think of naming your kid "Adolph Hitler [lastname]"? Most people agree that a name like that will cause great harm to a child growing up because of the constant ridicule and ostracization he'd inevitably face. That's an extreme example, but names _are_ important.

Note that I don't think this project's name is horrible, but I don't think it's very good either, and could be a lot better.

<comment><USER60> [https://en.wikipedia.org/wiki/Godwin%27s_law](https://en.wikipedia.org/wiki/Godwin%27s_law) ? :-D

</comment></comment></comment></comment><comment><USER61> How does this compete with PiedPiper?

<comment><USER62> This is general purpose whereas Pied Piper, Hooli et al. focussed on video.

</comment></comment><comment><USER63> Awesome

</comment><comment><USER64> Best middle-out in the game.

</comment><comment><USER65> Whats their weissman score?

<comment><USER66> I bet it doesn't even touch pied piper's.

</comment></comment><comment><USER67> Probably nictpicking but "Smaller data compression" makes no sense really

</comment><comment><USER68> Should we start with the pied piper jokes now or later?

<comment><USER62> Not until I know the Weissman Score of Zstandard.

</comment></comment><comment><USER69> Should've named it "Pied Piper"

</comment><comment><USER69> They should've named it "Pied Piper".</comment>


Why some European countries reject refugees, and others love them - <AUTHOR> https://www.washingtonpost.com/news/worldviews/wp/2015/09/08/this-map-helps-explain-why-some-european-countries-reject-refugees-and-others-love-them/?postshare=1821441801185179 <comment><USER2> What is missing, as a context from the article, is that the majority of Eastern Europe spend half a millennium under the rule of the Ottoman Empire as a second class citizens. Islam is not welcomed on these lands.</comment>


Using Multi-Byte Characters to Nullify SQL Injection Sanitizing - <AUTHOR> http://howto.hackallthethings.com/2016/06/using-multi-byte-characters-to-nullify.html <comment><USER2> In order for this to take effect, you need: 1\. To "sanitize" SQL injection by quoting parameters and building SQL strings manually 2\. Your quoting function needs to be in a language that iterates over multibyte chars properly (i.e., you're not running naively on binary strings) 3\. The output multibyte charset must be one of the afflicted charsets (mostly CJK charsets) 4\. Your DBMS must be incapable of properly handling the relevant charset, and is instead checking it in a different charset.

Point #4 makes it really hard to envision this ever being a problem, since if you're using a CJK charset on your platform already, you're quite likely to notice very quickly that something is horribly, horribly wrong.

<comment><USER3> I was thinking the same thing, and does people really still build SQL-strings programmatically in the server application?

Seems like this bad practice should be dead by now, not only does it open up for injection attacks but it also prevents the database from optimizing the query by precompiling, data aggregation and building smarter execution plans.

<comment><USER4> It happens a lot in junk WordPress plugins. Even with built-in support for doing it better [1], far too many just use straight concatenation.

[1]: [https://developer.wordpress.org/reference/classes/wpdb/prepa...](https://developer.wordpress.org/reference/classes/wpdb/prepare/)

</comment></comment></comment><comment><USER5> As always just use parameterized queries. This is the best and only defense you need against sql injection.

<comment><USER6> This is absolutely good advice, but people giving it rarely acknowledge that it isn't always possible.

A particular irritant is that most (all?) db APIs make it impossible to use parametrized queries with IN clauses. I find myself having to implement string escaping functions sooner or later in every project, because of stuff like that.

<comment><USER7> At least in Postgres, you can use ANY and pass an array of values. From what I've read, the generated query plan is equivalent.

<comment><USER8> > From what I've read, the generated query plan is equivalent.

I believe at one point IN (values) was just sugar for = ANY(ARRAY(values)), don't know if that's still the case.

Still requires that your DB adapter supports array parameters though.

<comment><USER9> Alternatively, you can use a temp table and a join. Not ideal, but still safe from injection and should have roughly the same overhead in the database.

</comment></comment></comment></comment></comment><comment><USER10> "Web applications sanitize the apostrophe (') character in strings coming from user input being passed to SQL statements using an escape (\\) character."

Please, please don't say this. In the SQL standard, backslash is _NOT_ the escape character for a string literal.

PostgreSQL, starting in v8.2, began transitioning from C-style escapes (using backslash) to SQL standard escapes (where a single quote is escaped with another single quote). Standard behavior is the default, but can be controlled with the variable standard_conforming_strings.

But you shouldn't have to know that anyway. Use out-of-band parameters that are passed in the wire protocol separately from the string. Web frameworks should already ensure this, and if they don't, they are likely broken.

If you are writing a web framework and you need to use escaping for some reason: first, make sure you can't use parameters on the wire instead; then read the product-specific literal parsing rules very carefully, considering things like multibyte characters.

</comment><comment><USER11> In-band: bad. Out-of-band: good.

The same goes for JavaScript, escape sequences, SQL string concatenation and so on. It always _seems_ like a great idea but it is just about impossible to get it airtight.

Better do your signaling, commands and meta-data through another channel.

</comment><comment><USER12> This is a pretty well-known problem, but mostly irrelevant if you use UTF-8.

<comment><USER13> Or, you know, just doing the right thing(tm) and using parameterized queries

<comment><USER14> Yes. If you are building SQL by concatenating user inputs (escaped or not) you are doing it wrong.

<comment><USER15> IMO building SQL by concatenating anything _feels wrong_.

I still do it, and i haven't used an ORM yet that is actually useful, but it still feels wrong.

</comment></comment></comment></comment><comment><USER16> I wonder, does that 0x5c byte has to be the last one in the character or can it also be the first, or the second one, followed by the hex value of ;, the end of instruction in mysql?

</comment><comment><USER17> String literals in SQL were a mistake.</comment>


Google shows off Project Glass at I / O with live skydiving and bike jumps - <AUTHOR> http://www.theverge.com/2012/6/27/3121164/project-glass-demo-io

<comment><AUTHOR> Best. Product. Demo. Ever.</comment>


Why dissonant music strikes the wrong chord in the brain - <AUTHOR> http://www.nature.com/news/why-dissonant-music-strikes-the-wrong-chord-in-the-brain-1.11791

<comment><USER3> I think the hypothesis described here is pretty much refuted by the work of William Sethares (<http://sethares.engr.wisc.edu/>), who finds that severely non-harmonic intervals can sound just fine when played with (synthetic) instruments whose overtones are suitably non-harmonic, and that using such instruments can make nice harmonic intervals sound just as nasty as severely non-harmonic ones do on conventional instruments with harmonic spectra.

There's a slightly convincing example at <http://sethares.engr.wisc.edu/mp3s/challoct.mp3> \-- what you hear there is two successive sequences of (note A, note B, chord A+B) where B is an octave above A in the first case and about 1.07 octaves -- an octave and 0.8 of a semitone -- in the second. The instrument has a weird spectrum that makes the second A+B sound better than the first -- but it's weird enough that, to me at least, even the individual notes sound really odd.

A better example: [http://eceserv0.ece.wisc.edu/%7Esethares/mp3s/tenfingersX.mp...](http://eceserv0.ece.wisc.edu/%7Esethares/mp3s/tenfingersX.mp3) is a piece in "10-tone equal temperament", in which notoriously almost all chords sound dreadful with normal instruments. <http://sethares.engr.wisc.edu/mp3s/Ten_Fingers.mp3> is the exact same notes, but played on an instrument with a spectrum designed not to sound horrible in 10-tet.

<comment><USER4> Norman D. Cook has some very nice papers about harmony and tri-chords that make the case that intervals are not what matters when it comes to characterizing more complex harmonies:

<http://www.res.kutc.kansai-u.ac.jp/~cook/>

</comment></comment><comment><USER5> I'm surprised at the lack of understanding of the music phenomenon surfacing in this article. Is this the current state of cognitive psychology?

Complex mathematical relations in harmonics induce more tension than simple ones. We've known this at least since Pythagoras. Schoenberg knew this perfectly well -- he published the bible for the theory of tonal harmony -- and never argued against it, as is claimed in this article. What he did claim was that our ears gradually grow used to the common dissonances which start to lose the tension potential and to be heard as "colors" [EDIT: hence one of his first atonal works, "Farben"]. This is what he calls the "emancipation of dissonance". Basically, we have to find the "edge" in more and more complex mathematical relations.

[EDIT: further, he thought this race for complexity was doomed and exhausted and we should let fully emancipated dissonance live a life of its own, not trying to build up and release tension; this opened a wealth of musical possibilities while alienating a good part of the audience]

Pure major chords sound more "pleasant" in a pepsi-challenge type of study. So does pure sugar in a taste study. That doesn't mean the perfect meal is a pile of sugar cubes.

<comment><USER6> Just wanted to add: Schoenberg's historical view of western music was one of a sort of slow "climb" up the overtone series, from early music utilizing almost exclusively perfect harmonies, to modern (in the west) compositions that use quarter tones, etc.

> Pure major chords sound more "pleasant" in a pepsi-challenge type of study. > So does pure sugar in a taste study. That doesn't mean the perfect meal is a > pile of sugar cubes.

Love this.

<comment><USER7> One example of this "climb" is Wagner, who was found to be excessively dissonant by many contemporary listeners. Nowadays, I think even listeners with very conservative tastes in music don't consider Wagner all that dissonant.

</comment></comment></comment><comment><USER8> Hmph. Read through to the end, and even the paper's co-author agrees nothing was really proven, and learning is quite likely still a major factor in what we prefer to hear.

There's also no discussion of cultures whose traditional music _focuses_ on creating audible beating, like gamelan. There's a ton of audible beating, it's completely intentional and part of the sound, and even if you've grown up listening to music with a lower normal level of dissonance the sound can grow on you.

It's also worth mentioning that highly consonant music is pretty lifeless -- even in the world of western music, the drive and power of a piece comes in tension and release.

<comment><AUTHOR> He doesn't say nothing was really proven - he just says that they didn't establish whether this dislike of dissonance was innate or not. That wasn't what the experiment was about. It was to find what caused the dislike - <USER2> they ruled out 'beating' and propose it's harmonicity. It's also not beating per se which people find unpleasant but rapid beating -roughness. Does Gamelan try to produce 'roughness'-ie rapid beating - or just beating - which I could imagine would be quite cool.

<comment><USER8> Fair enough, and I shouldn't just knock it without even reading carefully. It's frustrating sometimes to see discussions like these that seem to completely ignore the very wide variety of sounds that people find pleasant. I've perhaps read too many claims that "music is a universal language" when it far from that.

You should search out some recordings; I played in a gamelan ensemble for a while and quite enjoyed it. The sound is cool, and the interlocking rhythms do neat things in your head. :)

The beats -- I'm not sure if they're rapid enough to qualify as "roughness" or not; I've never encountered that term before, but they're fairly rapid and give a shimmering effect.

</comment></comment></comment><comment><USER9> _Then the researchers tested how both groups felt about beating. They found that the amusics could hear it and disliked it about as much as the control group._

followed by:

 _Co-author Josh McDermott at New York University reported previously that harmonicity seems more important than beating for dissonance aversion in normal hearers. In the new paper he and his colleagues argue that the lack of sensitivity both to harmonicity and dissonance in amusics now adds to that case._

Am I thick? These paragraphs seem to directly contradict each other (1st says amusics are sensitive to dissonance, 2nd says they are not).

<comment><USER7> beating != dissonance.

</comment></comment><comment><USER10> > "Rock bands often deliberately introduce roughness and dissonance into their > sounds, much to the delight of their audiences", she says.

From reading this, you might think that this trick was invented in the mid-20th century.

<comment><USER7> Besides, rock tends to be harmonically quite conservative, compared to classical music and jazz.

The reason something like the intro to Jimi Hendrix' "Purple Haze" stands out is that tritones are very much the exception rather the rule in rock.

</comment></comment><comment><USER11> This is a bit disappointing. If I understand correctly they only gathered some evidence that it is not due to beating (interference.) So we still don't even know whether it's prewired or learned. The fact that you can learn to like dissonance tells us little - you can also learn to like some kinds of bitter taste and dislike for bitter is prewired.

<comment><AUTHOR> That's underselling it a bit, I think? They've ruled out the dominant hypothesis with their experiment (beating/roughness) and propose another - harmony of overtones. Quite clever to do the experiment with people with amusia. Wonder if this has anything to say about the old CD versus vinyl debate? Or was that nonsense anyway?

<comment><USER12> The beating/roughness couldn't have explained the dissonance of the tritone (C to F# for example).

The harmony of overtones doesn't come as a surprise for people familiar with sound synthesis, but I suppose that it had never been rigorously tested...

</comment></comment></comment><comment><USER13> This short video is a fascinating explanation of how sound interacts with our ears, and why chords sound good. <https://www.youtube.com/watch?v=i_0DXxNeaQ0></comment>


San Francisco sues DoorDash over driver classification - <AUTHOR> https://www.axios.com/san-francisco-sue-doordash-driver-classification-ee8968f9-0542-4522-8bcd-948a4fdc623c.html <comment><USER2> Because, of course, people looking for a flexible gig and wanting to work a few hours a week while they have other jobs can't possibly exist.</comment>


Dragons, Memory and Navigating the Globe Using Only Your Wits - <AUTHOR> http://nautil.us/blog/dragons-memory--navigating-the-globe-using-only-your-wits

<comment><USER2> This article would be a lot more interesting if it gave even a tiny hint of how to do these things, rather than just suggest that they are possible.

I'm going to be spending the better part of the next two years on a boat and look forward to learning this type of stuff.

<comment><USER3> You might enjoy reading [http://arachnoid.com/sailbook/](http://arachnoid.com/sailbook/)</comment></comment>


Ask HN: People doing open-source development full-time - <AUTHOR>

Is it possible to make a living doing open-source development full-time? How do these great people make money for living? <comment><USER2> Some examples:

\- Hires: You start an open-source project, it gets some nice traction, a company start using your code in production, but they want to make sure you spend your time working on the project. They hire you to work full time on it. So you're technically employed by the company, but all the community is benefiting from your work.

\- Consulting: you sell consulting services for your project, or an hosted implementation (think MongoDB or Wordpress, for example)

\- Sponsorship: you get sponsored either by a company that has an interest in the project (but you're not an employee of the company), or by universities / research grants / etc... I think Nokia sponsors some KDE developers, for example. The European Union also spends some money sponsoring open-source projects.

\- Donations (from individuals or corporations). Either in money, or in hardware / software / services / etc... Usually not a good way to make a living, as a developer, though.

One thing is for sure: it is definetely possible to make money with open source. But more likely as a project (ie: you found a company that develops a piece of technology which is open-source), than as a developer (ie: you're not very likely to make a living of being a regular contributor to open source projects)

<comment><USER3> _You start an open-source project, it gets some nice traction, a company start using your code in production, but they want to make sure you spend your time working on the project. They hire you to work full time on it._

It must be difficult in this situation to keep the needs of other users in mind, if one user in particular is paying you to maintain the project. How does this usually pan out?

</comment></comment><comment><USER4> Salvatore Sanfilippo, creator of Redis, has been hired by VMWare to work on Redis full-time. The development pace surged as a result. Joyent did the same with Ryan Dahl, creator of Node.js

It's possible, but keep in mind that those two are great developers, not everybody can do what they did.

</comment><comment><USER5> Rich Hickey of Clojure ([http://groups.google.com/group/clojure/browse_thread/thread/...](http://groups.google.com/group/clojure/browse_thread/thread/cc77df25e98ce46b?pli=1))

Matt Mackall of Mercurial, before it was acquired ([http://selenic.com/pipermail/mercurial/2010-April/031543.htm...](http://selenic.com/pipermail/mercurial/2010-April/031543.html))

<comment><USER6> Thanks for the link to Rich Hickey's post. How software development should be funded is something I've been thinking about for a while now, and I hadn't come across that link yet. He gives a good summary of the existing situation. (For anyone interested, it's also reprinted on the Clojure website [1].)

I tend to agree with the points that Bradley Kuhn makes in one of his blog posts [2], but I'm still trying to figure out how direct payment for time spent would work for non-obvious scenarios, like end-user desktop or mobile apps.

[1]: <http://clojure.org/funding> [2]: [http://www.ebb.org/bkuhn/blog/2010/07/07/producing- nothing.h...](http://www.ebb.org/bkuhn/blog/2010/07/07/producing-nothing.html)

</comment></comment><comment><USER7> I know Microsoft has a team that does OSS work on .NET MVC and related projects. Not sure if they do that full time, but I'm sure if you ask Scott Hanselman or Phil Haack they can tell you.

In a lot of cases companies will buy the talent behind an open source project to get the support they need. Sounds funny, but when you consider a piece of software mission critical to your business its a lot safer to be able to pay the guy or gal and get their immediate attention. :)

I think Apple did that with the CUPS creator, and Rapid7 did something similar with w3af. Sometimes its called "partnering" or "sponsorship" other times it is an all out acquisition. It depends.

</comment><comment><USER8> I am full-time R&D engineer at <http://nuxeo.com> working on integrating semantic analysis technologies into our open-source document management platform and applications.

All my code is released under LGPL or ASL2 on either the company mercurial repo <http://hg.nuxeo.org> or on apache projects we contribute to (Apache Chemistry and soon Apache Stanbol).

The company sells support subscriptions, consulting, training and custom dev.

</comment><comment><USER9> I wrote an article last year about the "how to" of making a living writing open source: [http://www.itworld.com/open-source/80180/building-your- caree...](http://www.itworld.com/open-source/80180/building-your-career-open- source) ...or, actually, a series of articles. Maybe it'll help. (Because it's not just "who's doing it" under consideration, but "how.")

<comment><USER10> thank you so much. I'm trying to explore the possibility of addressing an industry-wide problem through starting an open-source foundation.

Do you know anyone who has gone through the effort of convincing the boss not just to leverage open-source, but start such a thing from scratch?

</comment></comment><comment><USER11> donations. from inidividual users and from corporations. its not unusual for corporations to donate money to oss software that play a big role in their business/infrastructure.

also some companys make the step to employ these people full time and sometimes even to work on the software full time.

</comment><comment><USER12> There is a good breakdown of employers for the Linux kernel code here: <http://lwn.net/Articles/373405/>

</comment><comment><USER13> I was a software engineer for a company doing open-source backup software for two years. Almost all of the code I worked on was released under the GPL (there were a few proprietary components I worked on).

It's entirely possible to find a salaried position where you develop open source software professionally. However, your employers will most likely not consider that a factor when assigning work.

</comment><comment><USER14> It's very simple, just work for a big company that does open-source: RedHat, Novell, Canonical immediately spring to mind.

<comment><USER15> don't forget Mozilla or Automattic :P

<comment><USER16> We (Automattic) love hiring open source contributors and creators, they always go to the top of the queue.

</comment></comment></comment><comment><USER17> Aside from donations, some get hired directly by corporations to develop the projects further. Obviously, there is some pressure to move the project in a certain direction when that happens.

</comment><comment><USER18> Does grant-funded research count? Until last year I was working on <http://esphealth.org> full-time, and it's all GPL.

</comment><comment><USER19> I believe that the Banshee media player is maintained and developed largely by Gabriel Burt and Aaron Bockover, who are paid by Novell.

I'm betting that the of OSS development that is done by people who are paid for it, is seem as a marketing effort. Any large corporation have more than enough money for a few developers in their marketing budget.

<comment><USER20> speaking about Novell, Isn't Miguel De Icaza a Novell employee ?

</comment></comment><comment><USER21> Engine yard pays Charles Nutter and Tom Enebo work full-time on JRuby as well as Evan Phoenix and Brian Ford on Rubinius.

</comment><comment><USER22> My employer (a math textbook company) uses previously-developed GPL-licensed software to do online homework. We host it and provide support, further development, and preset problems and courses that correspond to our books. I get paid for doing server maintenance and hacking on the code.

</comment><comment><USER23> I used to work on an Apache project full-time at Yahoo!

</comment><comment><USER24> 37signals. Sorta.

</comment><comment><USER25> SourceFire (<http://sourcefire.com>) makes Snort and ClamAV, both of which are open source, but they also have content revenue models, commercial products and a service branch.</comment>


GNAT Community 2020 is here - <AUTHOR> https://blog.adacore.com/gnat-community-2020-is-here <comment><USER2> Does Ada have dynamic strings, or do I still have to specify the number of characters...etc.

<comment><AUTHOR> Since 1983.</comment></comment>


Digital Comic Publisher Quits, Customers Lose All Their "Purchased" Titles - <AUTHOR> http://www.jmanga.com/urgent-message <comment><AUTHOR> Yet another case that demonstrates that as long as DRM is involved (in this case, being restricted to online reading of titles only), you're never actually _buying_ anything, merely renting them indefinitely. <http://xkcd.com/488/> keeps being sadly relevant.

It's also such a damn shame that genuine digital purchasing (as in, something that involves DRM-free downloads) is such a rarity outside the music industry. I just can't understand why every other industry (especially the video industry) thinks that offering DRM-free downloads would somehow kill them or whatever when digital music sales have only got better since the wide-scale ditching of DRM, _especially_ since huge video files are much harder to share casually than small music files. There is value in the catalog subscription model that the likes of Netflix offers, but it should really have actual buying options available next to it (ideally, a service could combine both models - pay a lump sum for catalog streaming rights, then actually _buy_ individual titles, and as a bonus, not only do you get DRM-free downloads, but you could also offer extras, formats that aren't possible with streaming, unlimited streaming rights for actually purchased titles since they have the available infrastructure for it, and so on).

<comment><USER2> I've been able to buy a lot of e-books without DRM (mostly tech books). But still, it is rather ridiculous, I can't help but think that the people of, say, 2050 will look back at the way we have gone about "managing" content in this era and think we were all mad.

<comment><USER3> Maybe. Or perhaps people just give up and the corps get their way and all content is rented; and can only be accessed by devices with DNA biometrics.

There will be niche boutique artists creating dead tree books of obscure short stories and bad poetry, for phenomenal pricing.

<comment><USER2> Based solely on what seems to be a technological advantage to those who desire content to be un-DRMed, I'd say that scenario is very unlikely. At least not without an extensive world-wide police state enforcing it.

<comment><USER3> For many years when I bought a console game I was able to sell that game when I'd finished with it. Japanese[1] companies were especially unhappy with the second-hand market, and took vigorous efforts to stop it. US companies are also keen to prevent this re-sale of games.

Now many games are unable to be sold on.

When I buy a DVD there is a clear benefit to me in being able to play that DVD in my player, no matter where I buy it. Yet DVDs are region locked. VHS tapes had Macrovision anti-copy signals; DVDs had region protections and CSS; BluRay has region protections and several layers of DRM; restrictions continue to get tighter, not more relaxed.

Laws are also getting stricter. The 1996 WIPO treaty (<https://en.wikipedia.org/wiki/WIPO_Copyright_Treaty>) led to the introduction of the DMCA and similar laws in other regions. (Circumventing technical protections is illegal under the DMCA but it's also illegal throughout Europe, with various difference in each country).

I agree with "unlikely", but I'm not so sure about very unlikely.

[1] Japanese companies have been doing this longest, since SNES times. I think, but I could be wrong, that they managed to make second hand sales of games illegal. Maybe they just wanted to do that?

</comment></comment></comment></comment></comment><comment><USER4> Companies should have the term "buying" forbidden if DRM is involved.

(This page [http://www.jmanga.com/hoshi-no-samidare-the-lucifer-and- bisc...](http://www.jmanga.com/hoshi-no-samidare-the-lucifer-and-biscuit- hammer/5) is filled with "buy" this and that)

<comment><USER5> There's actually a "We The People" petition on this: "Force companies to change language from "buying" to "licensing" when dealing with DRM-restricted goods." [https://petitions.whitehouse.gov/petition/force-companies- ch...](https://petitions.whitehouse.gov/petition/force-companies-change- language-buying-licensing-when-dealing-drm-restricted-goods/myq4rZjX)

Before you derisively dismiss this, you might want to take a look at their response to the cellphone unlocking petition: [https://petitions.whitehouse.gov/petition/make-unlocking- cel...](https://petitions.whitehouse.gov/petition/make-unlocking-cell-phones- legal/1g9KhZG7)

<comment><USER6> A petition - especially to the executive branch rather than the legislative - is hardly the way to approach this.

How about class-action lawsuits for fraudulent misrepresentations for businesses that use the word "buy" in deliberately deceptive ways?

<comment><USER7> > _How about class-action lawsuits for fraudulent misrepresentations for > businesses that use the word "buy" in deliberately deceptive ways?_

And then one or two judges declare it "OK", and that's it.

<comment><USER6> Why do you think that would happen? Given the complexes of incentives attached to the judiciary and the legislature, respectively, it seems more likely that new statutes rather than case law would attempt to legitimize this practice, which probably already runs afoul of existing case law.

And if it was likely to happen, wouldn't it happen anyway, even if there were new legislation involved?

</comment></comment></comment></comment></comment><comment><USER8> The punchline:

>> Is there a way to download the manga I have purchased? It is not possible to download manga from My Page. All digital manga content will no longer be viewable after May 30th 2013 at 11:59pm (US Pacific Time)

</comment><comment><USER9> Just a quick plug for <http://fixthedmca.org>

If you care about issues relating to DRM, please spare a few minutes and take action by tweeting at your Congressional representatives. Not only should owner's rights be protected, but people should have the right to circumvent DRM systems when there's no copyright infringement.

</comment><comment><USER10> For what it's worth, it looks like the preview books simply have every byte XOR'd with 0x42, and you can see the image requests coming in on the Network tab of the Chrome developer tools while you're in the Flash-based reader. (I can't say for sure if the previously-for-sale books have the same scheme though.)

I doubt that this will reach the eyes of most of the users being screwed by the DRM though...

</comment><comment><USER11> It will probably be unpopular to say so: But learn how to strip DRM. If you paid for it, you should _own_ it.

<comment><USER12> It wouldn't help in this situation, since you can't even download the stuff you bought[1]. Your "purchase" only got you on-line viewing.

[1] <https://news.ycombinator.com/item?id=5377617>

<comment><USER13> What about a caching proxy? You _can't_ view it on your screen unless the browser downloads it anyway.

<comment><USER12> It could be something like a streaming Flash video, which the browser never has a complete copy of.

<comment><USER14> Streamloaded flash video is stupid simple to grab.

On windows, navigate to:

Users\USER\AppData\Local\temp\acro_rd_dir

And create a >hard< symbolic link for the .tmp file you will find there, renaming it to .flv . I am sure, on a mac, the location name is similar.

Close the browser window, the tmp file will be deleted and the flv will no longer be locked for access, and is now yours.

[http://www.howtogeek.com/howto/16226/complete-guide-to- symbo...](http://www.howtogeek.com/howto/16226/complete-guide-to-symbolic- links-symlinks-on-windows-or-linux/)

</comment></comment></comment></comment></comment><comment><USER15> So let's change the title: instead of "Digital Comic Publisher", put "Steam", "Sony" or EA. DRM is the 21th century worst trap of all and we must stop it from spreading.

</comment><comment><USER16> Ah... JManga... I didn't think this place was the best example of electronic publishing to begin with. Their page structure was very misleading, too. They had some "informative" page, that is unavailable to "buy" and such.

The idea of the service it was actually interesting, and they perhaps had a better chance if it was all-you-can-eat subscription model as opposed to retailing.

A bit of shame especially after coming into the market with big fanfare of how all Japanese publishers joined the force making this possible... (and, it is plausible whole business model was dictated by influences of those publishers.)

</comment><comment><USER17> Another reason to backup all your purchases from the "cloud" services, scraping off any DRM from them first.

</comment><comment><USER18> The problem is that there are no consumer protections built into DRM law. People should never be locked out of their content.

In my opinon, JManga should be forced to continue to support their authentication systems or be forced to permanently unlock the content.

<comment><USER19> The problem is, how do you define DRM vs just a regular online service?

If I launch a web-app for example I don't necessarily want to have to support it forever.

<comment><USER20> This is, at its core, the problem with webapps.

<comment><USER7> Really? How is it any different from:

a) regular proprietary apps.

b) regular open source apps that they don't have a community to continue them after the original coders lose interest, and you don't know enough programming to support them nor have the money to pay someone to work on them?

<comment><USER21> b) is like saying 'how is this any different from a car if you drive it until it runs out of gas, and then push it into the ocean, and then wait until it rusts from being under the water, and then you forget where it is?'

Yes, of course it is possible to deny yourself the ability to view content, use software, or access goods that you have purchased. That is beside the point, which is that webapps are inherently transient - unless you have access to the ability to configure and run your own server to host the application, the ability to use it can be revoked at any time, either through malice, negligence, or the regular operation of business. There are numerous examples of all three categories. This is, in fact, worse than proprietary software. At least with proprietary software (DRM/authentication notwithstanding), you have an executable, and it works on the machine you have right now. VM software like VMWare allows you to preserve a particular software configuration indefinitely if you wish, and access that software as long as you like.

</comment></comment></comment></comment></comment><comment><USER22> What has me puzzled is why they didn't try and sell their usebase, I'd be willing to make a modest offer. But now that they've gone off really pissed off their customers the value of that asset just dropped like a rock.

</comment><comment><USER3> How does the DMCA handle cases like this?

Are you allowed to rip the protection to keep access to something you've "bought"? Or is that circumvention and thus a criminal offence?

<comment><USER23> It is circumvention and it is illegal. There are only 6 allowed exemptions to the rule as of the last rulemaking, and making backup copies is not one of them, no matter the situation. The only exemption that exists for DRM- protected ebooks is to make stripping that DRM legal when no version of the ebook compatible with screen readers or read-aloud functions exists, so that the visually impaired can access the text.

<comment><USER24> So, why buy any media then? I'd rather just pirate it and be on my way. Xkcd/488 makes a great point: buying and then freeing from a dying system is just as illegal as copyright violation.

So, media types that use any DRM: why should I buy your product?

<comment><USER23> I doubt any media companies are much interested in swaying self-professed pirates that demand rights not offered to their side. You're coming from this imagined position that you deserve license to others' work on your terms at your price. The companies offering media with DRM for sale on the market have no such delusion that they have an innate right your money; they're OK with you not taking their offer. That's fine, since the value proposition is suitable to millions of others who will make the purchase on the terms offered -- watching a movie today is still worth some number of dollars even if you might not be able to dust it off 20 years from now to watch again. They're not OK with you infringing their rights to their property instead of declining the offer. But really, your moral and ethical compass won't be turned by logic or debate.

<comment><USER24> In terms of a capitalist debate, one side charges money for substandard product. The other side charges less, while providing a much better product. It just turns out, with copyright applied, the better side is illegal.

But being illegal hasn't stopped pirates yet, has it?

One major side effect isn't them trying to sway us; instead they are forced to release movies in the range of weeks, and not the traditional years. So yes, pirates are a competitive pressure downward.

Don't get me wrong: I think people should be paid for creative works. I like creative stuffs. I just disagree with putting locks on our culture, that are physically or legally hard to remove.

</comment></comment></comment></comment></comment><comment><USER25> Friend of mine runs something similar. <http://www.thefabler.com/>

<comment><USER24> If your friend uses strong DRM and/or uses cloud based "purchasing" to keep users from possessing the bought files, I hope he fails.

The last thing I want is more DRM peddlers catering to people who aren't in the know about every tech nuance out there. So yes, it can he unduly harsh. But in our legal environment (with little recourse), drm providers almost always deserve it.

An exception to that is companies that sell "drm" servers to authenticate that only X licenses of software are used at one time. License compliance is another drain on innovation, but can be effective in keeping legit in compliance.

</comment></comment><comment><USER19> The purchasers are issued a refund in the form of a gift card. So, if you purchased it and DRM stripped it then you get to keep the comic and get the refund!

Edit: Looks like you don't

The point I am more worried about here is that we are actually reliant to an extent on piracy and DRM stripping in order to preserve abandoned works.

<comment><USER26> The refund is only for the points you haven't spent yet. If you 'bought' something you will not get the money back and you will never get to see it again.

<comment><USER27> Fraud. Isn't that what chargebacks are for?

<comment><USER28> Yeah, that's what I've been thinking. But they've been in business since sometime in 2011, and chargebacks only seem like forever, aren't they limited to 6 months or so?</comment></comment></comment></comment>


Leaked Photos: Tesla Model S Sedan - <AUTHOR> http://blog.wired.com/cars/2009/03/leaked-photo-te.html <comment><USER2> Given how heavy and otherwise problematic batteries are, I've always wondered why not use water.

You would split it into H and O using a capacitor, most likely at home, but possible also at gas stations.

Then when it's turned into H2O again, don't dump the water, store it in a tank, a closed system in other words.

The only tricky part here is that H is very explosive, but gasoline ain't exactly gravy either.

<comment><USER3> Liquid gasoline is actually fairly stable; gasoline gas on the other hand, combusts fairly violently.

<comment><USER2> Good point, that's why I'm thinking of a VERY robust tank, sealed to the outside world, except for the electric connections, which contains the H and H20.

I'm assuming it's not very hard to make a tank that can survive almost any collision.

It would be a lot more expensive then a gas tank but I'm guessing less costly then all those lithium batteries?

</comment></comment></comment><comment><USER4> That looks exactly like the Jaguar XF. I wonder if Jaguar will sue Tesla

<comment><USER5> or maybe they licensed the body model</comment></comment>


Ask HN: What about rechargeable pricing plans? - <AUTHOR>

I live in europe and I can't use Stripe for recurring payments. I didn't want to go with other services because they seem too expensive and quite complicated to setup.
So I thought to myself, why not implement a simple rechargeable system, where the user simply pays through a PayPal Pay Now button? 1 month, 3 months, 6 months and 12 months plans. After the period is over, the users are reminded to re-charge in order to continue using my service.
From a user perspective this has several positive outcomes:
1) The user isn't locked into any month-to-month thing.
2) They don't need to store their credit card info anywhere - Stripe and other 3rd party services are PCI compliant, but users feel safer knowing that their CC info is only used for that specific transaction and not being stored somewhere.
3) The user is in control. Just like my pre-paid cellphone, I charge it whenever I'm out of credit - it's just simpler than having something that bills me monthly. Just the feeling that a service is connected to my Credit Card makes me feel a little unsafe.
Why aren't services being built this way? Am I missing something? <comment><USER2> The primary reason that a company wants to get a hold of your payment details (or at least a way of charging to them; Stripe etc) is to streamline the charging process when the customer is to be charged again. Plus it has the added benefit (for the compnay) of having the customer feel like it's a sunk cost once it's been charged (Netflix monthly access, Amazon Prime yearly shipping, etc). Think about everything that you get billed for regularly that you wouldn't necessarily pay for if it were left up to you.

In the case of your prepaid cell phone the cell company knows that you: 1) Want to be able to make phone calls 2) Want to be able to receive phone calls 3) Want to keep your phone number

I'd wager that you're much more likely to make sure you're prepaid phone is paid then your Netflix account each month.

All that aside I think it's a very pleasant model from the perspective of the customer (I personally avoid auto billing whenever possible and even resort to single use credit card numbers for anything new or remotely shady).

If you have an engaging enough product I see no issue starting off with this model though I'd recommend eventually migrating to a model where you implement your own recurring payments.</comment>


Startup Offense and Defense in the Recession - <AUTHOR> http://blog.eladgil.com/2020/05/startup-offense-and-defense-in-recession.html <comment><USER2> Pragmatically, it's defense, and then offense. Operationally, hunkering down needs to be rigorous. For me, the most interesting bits are at the tail, on offense. Locate potential markets as going up or down and ephemerally due to COVID or longer term sustainably. That's a 2x2. Net, the author recommends preparing to hunker 2 to 3 years...

<comment><USER3> The hard part is when there is a offense opportunity that it at odds with the most prudent defense. It is different for every situation and is a high stakes game.

</comment></comment><comment><USER4> That makes sense, and I feel like copying the first part that is a general prediction of what is to come in the economy and send it to friends and family who suffer from too much optimism on the economy recovering quickly. I am retired (ha!) but even so I have had his book High Growth Handbook in my book buying wish list for along while.

Sorry for being off topic and sorry for the following rant, but I think that the whole world is starting a permanent process of change/upheaval and not all results will be bad. I expect cruft like stupid consumer buying of needless shit to slow down, people and institutions will increasingly cherish strength and long term viability. In the recent sickness of neoliberalism (basically more rights for corporations than people) my hope is that there will be more investments in people and training in ways that enhance technological progress. For years the world will be harsh for the 'bottom 50%' but we can hope for some sort of transformation.

<comment><USER5> _> For years the world will be harsh for the 'bottom 50%' but we can hope for some sort of transformation._

All I want for Christmas [in 2020] is for that "bottom 50%" to not pick up pitchforks. Never forget serious tail risks. Will we still be using the same system of currency 5 years from now? Will entire stock markets become nationalized? Will we get true dictators in some "medium quality" countries or perhaps even in some "high quality" countries? Will China survive the massive lack of demand without political or military turmoil?

<comment><USER4> good questions! I don't know the answers either.</comment></comment></comment>


Food delivery robots target college towns - <AUTHOR> https://www.axios.com/food-delivery-robots-target-college-towns-e9884f70-466d-490d-9a3d-d30f23866ed3.html <comment><USER2> These were always a bit odd to me, and by that I mean like they're solving a problem that's not really there. They had a similar startup during my time at Berkeley, and I always preferred the nice break going to get food provided.</comment>


How to write great Javascript Unit Tests - <AUTHOR> http://pivots.pivotallabs.com/users/nick/blog/articles/455-better-javascript-testing-through-screwunit

<comment><USER2> Isn't is this a useful piece of information. :bmark</comment>


In defense of Tor routers - <AUTHOR> http://arstechnica.com/security/2015/04/op-ed-in-defense-of-tor-routers <comment><USER2> I, for one, have completely resigned myself to the idea that my employer, government and neighbors know all of the most shameful elements of my Internet usage. I haven't yet seen an approach to the privacy problem that delivers acceptably anonymous results for acceptably low effort, while simultaneously leaving my connection speed and latency relatively intact.

My experience with Tor was limited and was several years ago, but it felt like I'd returned to a 28k modem...except the entire Internet had begun to serve content on the implicit assumption that I at least had a few megabits of pipe. Do these routers actually offer a solid cornerstone of the kind of solution I've been waiting for?

<comment><USER3> It's really not that slow any more, it's actually quite usable for regular browsing and if you catch a slow route a reconnect usually fixes that.

</comment></comment><comment><USER4> I'm a babe in the woods, but I had no idea that I was broadcasting my list of installed system fonts and browser plugins.

But it seems to me that this kind of fingerprinting would be pretty easy to defeat if people new it was happening and wanted to defeat it. Something like panoticlick could give advic on fonts and plugins to add to lose yourself in the herd.

<comment><USER5> That's the point of Chameleon[1]. I've got it installed, and the issue is a lot of websites assume "oh, they don't have $plugin (e.g., Flash) installed", and don't fallback to something reasonable. So there are edge cases where you'd have to disable it. Otherwise, it's pretty effective.

[1] [https://github.com/ghostwords/chameleon](https://github.com/ghostwords/chameleon)

</comment></comment><comment><USER6> Both the original and this retort forget to consider that different users have different needs. Security is never going to be a 'one size fits all' kind of arrangement. Different users will find differing levels of utility in Tor routers.

</comment><comment><USER7> > we've been vocal about the need for people to use privacy add-ons with their > web browsers

This contradicts what I have read on the Tor Project site:

"Site-specific or filter-based addons such as AdBlock Plus, Request Policy, Ghostery, Priv3, and Sharemenot are to be avoided."[1]

[1] [https://www.torproject.org/projects/torbrowser/design/#philo...](https://www.torproject.org/projects/torbrowser/design/#philosophy)

<comment><USER8> Filter-based addons leak at a minimum 1 bit of identifying information -- that the addon is installed.

With something like RequestPolicy the specific policy in place likely uniquely identifies the user.

Maintaining anonymity with a modern web browser is virtually impossible.

Maintaining pseudonymity substantially less so, but you're probably identified as "user running tails version x"

<comment><USER9> Why/how do browsers allow sites to query the list of installed addons? That seems like it should very much be opt-in behaviour, that an addon would need to explicitly inject functionality to be discovered.

<comment><USER10> It's not that the list of addons is known, but rather infered from the number and types of blocked elements.

</comment></comment></comment></comment><comment><USER11> Perhaps I'm missing something, but isn't the primary argument against Tor routers simply that it's a really bad idea to send _all_ your traffic through Tor? You certainly don't want to be sending logins, banking data, etc., through some unknown exit node. I mean, presumably you can configure the router to only use the network for some traffic, but then what do you gain over using the Tor Browser bundle?

<comment><USER12> If banking isn't using https, get a new bank.</comment></comment>


Some Tools for Go Lang That You Might Not Know Yet - <AUTHOR> https://blog.gopheracademy.com/advent-2016/some-tools-for-go-that-you-might-not-know-yet/ <comment><USER2> I've used go-bindata[1] for embedding things like shell scripts and data files into my applications, it's a fantastic tool!

[1] [https://github.com/jteeuwen/go-bindata](https://github.com/jteeuwen/go- bindata)

<comment><USER3> I use go-resource[1], it works nicely for embedding websites as well as scripts and other resources.

[1] [https://github.com/omeid/go-resources](https://github.com/omeid/go- resources)

</comment></comment><comment><USER4> The tool I have found very useful is goimports [https://godoc.org/golang.org/x/tools/cmd/goimports](https://godoc.org/golang.org/x/tools/cmd/goimports) really great for automatically adding and removing imports.

</comment><comment><USER5> Really? Interfaces everywhere you can possibly imagine? Is that where we're heading in Go land?

<comment><USER6> > Really? Interfaces everywhere you can possibly imagine? Is that where we're > heading in Go land?

That _should_ be where you are heading, because contracts should be separated from implementation, things should not be tightly coupled, etc. This is good programming practice. How can you be opposed to accepting interfaces instead of implementations, especially in a standard library or the standard tooling for the language?

<comment><USER5> [https://github.com/simeji/jid/blob/97c7f902bc210d7b0ebd3eadd...](https://github.com/simeji/jid/blob/97c7f902bc210d7b0ebd3eaddabb8ecc62056bbc/query.go#L8-L30)

<comment><USER7> That's not an interface everywhere you can imagine, that's a huge, bulky interface that should probably be many smaller interfaces. No one is arguing for 16-method interfaces when all you want to do is call s.Alert().</comment></comment></comment></comment>


One thing I actually like better about DOS shells than bash - <AUTHOR> http://www.reddit.com/r/commandline/comments/kmb9z/ok_sacrilege_of_sacrileges_one_thing_i_actually/ <comment><AUTHOR> Add -- set show-all-if-ambiguous on -- to your .inputrc

Also don't forget -- set completion-ignore-case on

</comment><comment><USER2> zsh.</comment>


Helen Keller and the Glove That Couldn't Hear - <AUTHOR> http://www.theatlantic.com/technology/archive/2014/09/helen-keller-and-the-hearing-glove/380336/?single_page=true

<comment><USER2> The Apple Watch promises haptic feedback to use in, for example, turn-by-turn navigation. I doubt that anything that crude could come close to achieving what the "hearing glove" inventors are/were after, but I'd be interested to see whether, upon putting such capabilities on the wrists of so many people, and opening up access to them for countless app developers, whether we might be able to come up with some extremely interesting cybernetic user experiences, both for hearing or sight-disabled people and for those will their full senses who could benefit from the various kinds of augmented reality that's on offer.

I recall reading about a guy who wore a device that gave him an indication of his North south east west orientation through a vibration, and how he used that to gain a instinctive sense of his place, perhaps similar to how pigeons experience the world. I think that's only scratching the surface of cybernetic potential that we could be unlocking by making even simple haptic feedback devices like the Apple watch widely available.

</comment><comment><USER3> Well that's an unsatisfying ending. No mention of trying to use larger areas, or more sensitive areas like the tongue, just "isn't sensitive enough".</comment>


Show HN: Parametrek product search - <AUTHOR> http://parametrek.com

<comment><AUTHOR> This has been my side project for the past month-ish. The concept was spawned by a friend's observation that most online stores looked the same, and that perhaps they were climbing towards a mere local optima. I hacked together a prototype in a weekend.

It languished for some time. Then Hmexx came along with his HN Proposition. I threw the basic idea at him and it was turned down. While the pitch was not enough to convince him, it was enough to convince me. Thanks, Hmexx. Parametrek would not have shipped without your encouragement.

</comment><comment><USER2> it seems you as maintainer need to insert every product and that won't fly with myriads of products in existence.

also see established geizhals.at</comment>


Turst me, I know what I'm doing... - <AUTHOR> http://torvalds-family.blogspot.com/2010/02/turst-me-i-know-what-im-doing.html <comment><USER2> The article itself is only so interesting, but the ensuing discussion might be of value to people who like wired home networks.</comment>


How friendly is your AI? It depends on the rewards - <AUTHOR> http://robohub.org/how-friendly-is-your-ai-it-depends-on-the-rewards/ <comment><AUTHOR> New research from the Computational Neuroscience Lab at the University of Tartu looks at what happens when multiple AI agents are competing or collaborating in the same environment. The research (built on some of the pioneering Deep Reinforcement Learning work done by Deepmind) shows that AI agents will tend to try to maximize their rewards. Leaves interesting questions about the role of researchers in defining these rewards clearly and ethically.</comment>


Is Apple planning a secret product launch this thursday? - <AUTHOR> http://www.itproportal.com/2011/05/17/apple-plans-secret-product-launch-ten-year-anniversary-retail/

<comment><USER2> Short answer: maybe!

Long answer: who cares? Either they will or they won't. If they do, either you'll buy one or you won't. If they don't, then they'll come out with a new product some other time.

<comment><AUTHOR> considering who it is, it would be silly not to care. How many people here on HN earn money in markets that wouldn't exist if Apple didn't create them.

<comment><USER2> By all means, feel free to care when there's an actual product.

But two days of speculating on whether or not there's going to be an actual product seems like a waste of time. It's just the noise the internet makes.

</comment></comment></comment><comment><AUTHOR> Note that I'm asking the question. I personally think it's highly unlikely, but I wondered wether anyone had any further insight...?</comment>


New here.  How do you collapse comments in Hacker news threads? - <AUTHOR> How do you collapse comments on Hacker News? I read a comment which I want to get past, but if it has 3 pages of responses, I have to scroll through them all.  I often just get to frustrated and leave the thread after reading the top 2 or 3 comments if the discussions is very active.  Is there a way to collaps comments? <comment><USER2> The site is fairly minimal, and doesn't support comment collapsing. If you are using Chrome then you could install the Hacker News Collapsible Comments extension [1].

[1] [https://chrome.google.com/webstore/detail/hacker-news- collap...](https://chrome.google.com/webstore/detail/hacker-news- collapsible-c/hockhafcdegocajmjhafgjncjpodihkd?utm_source=chrome-app-launcher- info-dialog)</comment>


UK unveils new Tempest fighter jet - <AUTHOR> https://www.bbc.co.uk/news/business-44848294 <comment><USER2> I'm not an aerospace nerd. I think I was once, in my teenage years, but that was a long time ago and I don't know what I'm talking about. Here is my completely unqualified opinion.

It seems like manned fighter bomber aircraft are fast becoming an outdated concept. Drones can be lighter, turn harder and crucially not endanger the life of the operator. Why is the MOD committing £2bn to building new ones? Fighter aircraft historically take _decades_ to be developed. The current typhoon mentioned in the article began life in 1983, before I had my teenage plane fascination and subscription to takeoff magazine, and as far as I know the UK is still waiting on delivery of some of the initial orders, 35 years later. We can assume that this tempest isn't going to be delivered until at least 2040 at which point drones will be phenomenally more capable than they are now. A fighter built to be flown by a human but also function as a drone _as an afterthought_ will have next to no hope against whatever unmanned tech exists then. Unmanned planes, built from scratch to be pilotless can be much much lighter because they need no human support systems (air, controls, ejector seat) They can also iterate a lot faster because they don't need to meet safety standards. This thing feels like a dinosaur already, by _todays_ tech.

Pouring £2bn into a fighter in 2018 seems like a completely political move, a Brexit bluff and mere posturing from Theresa May.

But maybe I'm wrong. Somebody explain why I'm wrong?

<comment><USER3> My experience in the field is about 15 years old at this point, but here is my take on it. Your supposition is premised on one of two things: (1) drone AI that can out-think a human; or (2) perfect communications between drones and an operator on the ground.

These are unsolved, and very difficult problems. The state-of-the-art right now is that we have trouble getting back the video feeds from our reconnaissance drones due to limitations in the downlink system. Drones also create new attack vectors--you might be able to ground hundreds of drones by taking out a handful of satellites or communications hubs.

It is possible that these obstacles can be overcome. But waiting on technology to be where you need it to be can be a very iffy thing. (Cue: "we were supposed to by on Mars by 2025."). I certainly wouldn't gamble the national defense on it.

<comment><USER4> It seems like what you need is a manned fighter that stays in the air some distance behind a fleet of semi-autonomous drones doing the actual fighting/killing. The pilot acts as an officer, selecting targets and providing general orders, while his or her plane provides the short-range encrypted comms which are hard to intercept or interrupt.

<comment><USER5> Did you mean: air to air missiles

EDIT: active or semi-active guided

<comment><USER6> Probably more like flying missile-pods?

<comment><USER5> When does the recursion stop?

</comment></comment></comment></comment></comment></comment><comment><USER7> Apart from BAE flogging their kit, I don't see what the point of this exercise is. There is zero chance of this being built by the UK, which can barely afford the nuke subs it has which are its main military threat.

<comment><USER8> The big spend in the UK defence budget is Eurofighter/Typhoon @ £23bn. Astute comes in at £1bn per pop, £7bn total program cost (I believe), future SSBN will be similar.

Turkey and India are trying to develop fighters as well, the UK has the tech to build these, there are markets.

<comment><USER9> It's pretty tough when you've got Russia and China out there competing with you and you're running from behind.

The Tempest looked interesting, until I saw the hoping to fly by 2035 part, which really means 2040 or later.

It's really a technology make-work project to keep their skills up and retain defense capability, which isn't a terrible idea so long as the cost is tightly contained.

<comment><USER10> Russia announced the abandonment of the su-57 yesterday. [https://www.news.com.au/technology/innovation/military/russi...](https://www.news.com.au/technology/innovation/military/russias- stealthkiller-fighter-project-has-been-abandoned/news- story/03e50ea76d6a5dfb5aaac69b9c75fbd6)

</comment></comment></comment></comment><comment><USER11> I've been a fan of the original Tempest since reading Clostermann's book as a kid.

One of the finest late War piston engined planes, in my non-expert opinion. Too few (and too late) produced to have any major impact on the course of the war, though.

<comment><USER12> That book was awesome! It

</comment></comment><comment><AUTHOR> Better info: [https://ukdefencejournal.org.uk/uk-unveils-new-next- generati...](https://ukdefencejournal.org.uk/uk-unveils-new-next-generation- fighter-jet-called-tempest/)

</comment><comment><USER13> Man these 5th gen fighters are unattractive. Even the F-22 being the best looking of the bunch looks blah from several angles.

<comment><USER8> I saw the F35's in the air over east Anglia this week - they looked pretty cool in the flesh.

<comment><USER14> Here I thought the F-35 was mostly made of vapour.

</comment></comment></comment><comment><USER15> "The hope is to see it flying by 2035." Sigh.

<comment><USER16> I'd say this is more possible than the "Ban on fossil fuels by 2040" deadline

<comment><USER17> Realistic or not, the timeline comes pretty straight from the plan of retiring Typhoon in 2040. So of course the target date for the replacement would be 2035, regardless of what it is.

That said, I wouldn't be surprised at all if they end up extending Typhoons service life by a decade or so.

</comment></comment></comment><comment><USER18> Their hangers must be called teapots. Or would it be kettle?</comment>


Women's Terrible Pockets - <AUTHOR> https://www.outsideonline.com/2400562/how-womens-clothes-pockets-changed-over-time <comment><USER2> It's not only women's pockets, in recent years the average pair of male pants has had the depth of its pockets decrease to the point where the pockets are becoming useless.

I've lost mobile phones after I've sat down and they've slid out, especially in theaters etc. where the seats tilt backwards. When sitting in cars I regularly have my keys and coins end up on the vehicle's floor after they've slid out. It's gotten to the point where I'm a bit paranoid about checking the seat and floor after I get out of the vehicle.

Surely, I'm not the only male who experiences this problem.

At every opportunity I whinge about it and no one (male or female) seems the slightest concerned other than me. I cannot understand why males everywhere aren't up in arms over it—after all, losing things like an expensive mobile from one's pocket is no small deal for the average male apparel wearer—just because some damn greedy manufacturer wants to save a few inches of cloth.

I've some old army BDs (dungarees) designed decades ago which have side pockets that are 33 cm (13") deep and they're wonderful—nothing ever accidentally escapes them. They are even tailored with a curved contour at the bottom so that when one sits down the stuff in one's pockets sinks down along the side of one's thighs rather than tending to pop out of the pocket. Once, not that long ago most pants had pockets this deep. (I'd buy more of these ex- disposals BD pants but I cannot get this type anywhere these days.)

Same goes for the flimsy, stupidly-small pockets on shirts not to mention the overly tiny buttons that are used to button up shirts, then there's the overly short zippers on the fly of male pants—which nowadays open to a length of at least two inches shorter than they did some decades ago.

Why on earth aren't more males complaining about their stupidly-impractical clothes?</comment>