Introducing the 'mozjpeg' Project - <AUTHOR> https://blog.mozilla.org/research/2014/03/05/introducing-the-mozjpeg-project/ <comment><USER2> Bravo. I love JPEG. Amazing that it's been 23 years since its release and it remains as useful as ever.

I remember what it was like to watch a 320*200 JPEG image slowly build up on a 386SX PC with a VGA card. Today, a HD frame compressed with JPEG can be decoded in milliseconds. This highlights the secret to JPEG's success: it was designed with enough foresight and a sufficiently well-bounded scope that it keeps hitting a sweet spot between computing power and bandwidth.

Did you know that most browsers support JPEG video streaming using a plain old <img> tag? It works also on iOS and Android, but not IE unfortunately.

It's triggered by the "multipart/x-mixed-replace" content type header [0]. The HTTP server leaves the connection open after sending the first image, and then simply writes new images as they come in like it were a multipart file download. A compliant browser will update the image element's contents in place.

[0] [http://en.wikipedia.org/wiki/MIME#Mixed- Replace](http://en.wikipedia.org/wiki/MIME#Mixed-Replace)

<comment><USER3> Unfortunately chrome recently removed support for "multipart/x-mixed-replace". It's too bad too. It was a simple way to implement a webcam.

[https://code.google.com/p/chromium/issues/detail?id=249132](https://code.google.com/p/chromium/issues/detail?id=249132) [http://blog.chromium.org/2013/07/chrome-29-beta-web-audio- an...](http://blog.chromium.org/2013/07/chrome-29-beta-web-audio-and-webrtc- in.html)

<comment><USER4> According to those links, they still support it for images, but removed support for other types of resources.

It's too bad IE never supported multipart/x-mixed-replace or we might have seen more live updating websites earlier. Now that we have WebSockets and well understood long polling approaches it doesn't matter any more, since x-mixed- replace would keep the download spinner spinning forever and the newer approaches don't have that problem.

<comment><USER5> I used multipart replace in firefox for streaming data for years with no download spinner. I couldn't use it in chrome because chrome always seemed to have some weird bug where 'frames' (of data in my case) were delayed.

I was disappointed when they were unceremoniously ripped out, but yes, websockets are better.

<comment><USER4> Ok, I believe you, it's been over 10 years since I tried it :-) Maybe I was thinking of the forever iframe, where a regular application/javascript document had content added to it incrementally over time.

<comment><USER6> It is actually pretty bizarre. If you point Chrome at a .jpeg that does multipart/replace it will stall, giving you roughly 0.2 fps. Point it at a. .html that contains an <img> tag and it works fine. Found this out while hacking on Hawkeye: [https://igorpartola.com/projects/hawkeye/](https://igorpartola.com/projects/hawkeye/)

</comment></comment></comment></comment></comment></comment><comment><USER7> This is very promising. Images by far dominate a web page, both in number of requests and total number of bytes sent [1]. Optimizing image size by even 5-10% can have a real effect on bandwidth consumption and page load times.

JPEG optimization using open source tools is an area that really needs focus.

There are a number of lossless JPEG optimization tools, but most are focused on stripping non-graphical data out of the file, or converting the image to a progressive JPEG (since progressive JPEG's have rearrange pixel data you can sometimes get better compression since there may be more redundancy in the rearranged data). Short of exceptional cases where you can remove massive amount of metadata (Adobe products regular stick embedded thumbnails and the entire "undo" history for an image) lossless optimization usually only reduces file size by 5-15%.

Lossy JPEG optimization has much more potential. Unfortunately, beyond proprietary encoders, the most common lossy JPEG optimization exclusively is to reduce the JPEG quality. This always felt like killing flies with a tank, so advances in this area would be awesome.

I've written extensively about Lossy optimization for JPEGs and PNG, and spoke about it at the Velocity conference. A post and my slides are available[2].

[1] - [http://httparchive.org/trends.php](http://httparchive.org/trends.php)

[2] - [http://zoompf.com/blog/2013/05/achieving-better-image- optimi...](http://zoompf.com/blog/2013/05/achieving-better-image-optimization- with-lossy-techniques)

</comment><comment><USER8> JPEG has shown amazingly good staying power. I would have assumed "JPEG is woefully old and easy to beat" but Charles Bloom did a good series of blog posts looking at it, and my (non-expert and probably hopelessly naive) takeaway is that JPEG still holds its own for a 20+ year old format.

[http://cbloomrants.blogspot.com/2012/04/04-09-12-old- image-c...](http://cbloomrants.blogspot.com/2012/04/04-09-12-old-image- comparison-post.html)

<comment><USER9> In my opinion, the biggest drawback of JPEG is that its window is non- overlapping. Ring-artifacts are generally not a big deal for natural images at medium quality or higher, but blocking artifacts can be noticeable even at relatively high quality settings.

There are a lot of post-processing techniques to try and mitigate this, but in my experience they tend to do about as much damage as they fix. The proper solution is to overlap the blocks using one of the myriad techniques DCT-based audio codecs use.

It is bizarre to me that for all of the attempts to beat JPEG, nobody seems to have tried simply overlapping the blocks by 2 pixels. You'd have an implementation only marginally more complex than JPEG (in fact, you can even implement it on top of an existing JPEG encoder/decoder) with a slowdown of only 25%.

<comment><USER10> JPEG-XR has (optional) lapping. But lapping has the problem that you can't do spatial intra prediction, which is significantly more valuable. And no one figured out how to make frequency-domain intra prediction as good until Daala. Plus deblocking filters have gotten pretty good now that they're tuned based on the quantizer used.

But a bigger problem is that no one is really interested in designing a new still image codec that's better than JPEG, since JPEG can't be unseated. So video codecs are where the practical development goes. And avoiding in-loop deblocking filters there means OBMC, which is _extremely_ computationally intensive.

</comment></comment></comment><comment><USER11> For improving general-purpose gzip / zlib compression, there is the Zopfli project [1] [2]. It also has (alpha quality) code for PNG file format; since this functionality wasn't originally included, there are also third-party projects [3].

You might be able to shave a percent or so off the download size of compressed assets.

[1] [https://news.ycombinator.com/item?id=5316595](https://news.ycombinator.com/item?id=5316595)

[2] [https://news.ycombinator.com/item?id=5301688](https://news.ycombinator.com/item?id=5301688)

[3] [https://github.com/subzey/zopfli-png](https://github.com/subzey/zopfli- png)

</comment><comment><USER12> Now if only they'd do a mozpng.

(For context: libpng is a "purposefully-minimal reference implementation" that avoids features such as, e.g., Animated PNG decoding. And yet libpng is the library used by Firefox, Chrome, etc., because it's the one implementation with a big standards body behind it. Yet, if Mozilla just forked libpng, their version would instantly have way more developer-eyes on it than the source...)

<comment><USER13> There's already a "mozpng", it's called Zopfli. There's also AdvPNG which compresses PNGs with 7-zip's deflate implementation.

And if you want much much smaller PNGs, then try [http://pngquant.org](http://pngquant.org) or [http://pngmini.com/lossypng.html](http://pngmini.com/lossypng.html)

<comment><USER14> Or try ImageOptim, which bundles these and other tools with a nice GUI: [http://imageoptim.com/](http://imageoptim.com/)

<comment><USER15> ImageOptim also bundles PNGOUT, which I find to have exceptional compression when compared to the others (but it is a bit slower)

</comment></comment></comment></comment><comment><USER16> We've been using [http://www.jpegmini.com/](http://www.jpegmini.com/) to compress JPGs for our apps. Worked OK, although we didn't get the enormous reductions they advertise. However 5% - 10% does still make a difference.

We've been using the desktop version. Would love to use something similar on a server, but jpegmini is overpriced for our scenario (I'll not have a dedicated AWS instance running for compressing images every second day or so). Will definitely check out this project :)

<comment> rast-a Have u tried [https://kraken.io](https://kraken.io) ?

</comment></comment><comment><USER17> I noticed that optimizing JPEG images using jpegoptim ([http://www.kokkonen.net/tjko/projects.html](http://www.kokkonen.net/tjko/projects.html)) reduces the size by a similar factor, but at the expense of decoding speed.

In fact, on a JPEG-heavy site that I was testing with FF 26, there was such a degradation in terms of responsiveness that transitions would stutter whenever a new image was decoded in the background (while preloading).

It made the effort to save 2-4% in size wasted with a worse user experience.

<comment><USER18> Did you file a bug for this? This doesn't sound normal at all.

<comment><USER17> Honestly, no. libjpeg would show similar slowdown (interestingly, PNG decoding is slower than JPEG for the same size), and it make sense anyway.

The problem is that even if the bug would be fixed in recent FF versions, libjpeg is basically used in all other browsers as well.

</comment></comment></comment><comment><USER19> If my goal were to compress say 10,000 images and I could include a dictionary or some sort of common database that the compressed data for each image would reference, could I not use a large dictionary shared by the entire catalog and therefore get much smaller file sizes?

Maybe images could be encoded with reference to a common database we share that has the most repetitive data. So perhaps 10mb, 50mb or 100mb of common bits that the compression algorithm could reference. You would build this dictionary by analyzing many many images. Same type of approach could work for video.

<comment><USER20> Well, if we're shipping a common database and you're maximizing _transmission efficiency_ then yes of course.

In information theory, my understanding (which is quite limited) is that when we talk of bits transmitted we can think of it as "uniquely identifying from within the _set_ of total possible messages". So, if you shipped the entire 10,000 image catalog "transmitting" an image from within that catalog would take you a mere, uh, let me count my fingers, 13 bits.

We could go one step further and find some way of hashing all the data together to remove any redundancies and so forth - but the problem alas is about defining arbitrary images :).

What you described tho is kind of what happens with all compression algorithms except on the "micro"/individual image level. You may already be aware but check out Huffman coding: [http://en.wikipedia.org/wiki/Huffman_coding](http://en.wikipedia.org/wiki/Huffman_coding) for a simple intro.

<comment><USER19> I am aware that compression algorithms kind of work on this general idea of referencing common bits. Of course I am aware of that.

How do you read my question and interpret simply it as "lets send all of the images in full and then give their index and call it compression?"?? What I suggest is that we take a standard encoding technique like Huffman, or some modification, but rather than creating a table based on data in an individual image, build this code table by analyzing many, many images.

I have read the Wikipedia article on Huffman coding before. However, the details are not really important in regards to my point.

What I am suggesting is that rather than looking at just the bits in individual images and using them to construct a Huffman table or some other kind of reference, look at the bits on many, many images and create a larger reference table. And then of course you may need a local table for things in the image that don't quite correspond to the larger table.

Earlier compression techniques were much more constrained in terms of processing power, RAM, network connectivity etc. and so distributing and using a large table for compression was not practical. I am suggesting that someone who has knowledge of compression engineer a system where 10MB, 50MB, or 100MB of RAM is used and a large common bits file is transmitted, rather than starting with the idea that almost all of the data or all of the data has to be contained in one file. I am not suggesting that an existing compression algorithm could be translated directly into this general concept. I am suggesting an engineering effort starting with different constraints and trade-offs.

<comment><USER21> I thought about doing something like that for audio, but I think it gets really inefficient with higher sample rates because you end up with so little overlap.

Like this: imagine I've got an array of shorts representing audio data. If I've got two files with similar segments, I've saved one short:

[1, 2, 3, 2, 1, 2, 5] [7, 2, 3, 6, 8, 9, 3]

So I can say that [2, 3] is represented by a new value (z), and can shave a short off of both streams. Then what happens if a new stream comes along with no similarities:

[8, 2, 7, 1, 7, 3, 7]

...you still have to send each value.

Maybe I've just demonstrated that I don't know anything about compression, but I would be interested in working with you on this.

<comment><USER19> I think I would want to find an existing sparse autoencoder implementation, ideally a project already setup for encoding audio, and start from there. [http://www.stanford.edu/class/cs294a/sparseAutoencoder.pdf](http://www.stanford.edu/class/cs294a/sparseAutoencoder.pdf)

</comment></comment></comment></comment></comment><comment><USER22> Why don't they just contribute the jpgcrush-like C code back to libjpeg-turbo?

Edit: A good reason given in the reply by joshmoz below.

<comment><AUTHOR> This was discussed with the author of libjpeg-turbo. His priorities are different, it was agreed that a fork is best.

<comment><USER23> Listen, you're doing it all wrong. You're supposed to fork it, tell noone, do a years worth of work in secrecy, release it. palm it off to another FOSS community, then abandon it, then refork it, do another 2 years of work in secrecy and then release it again under a new name. Got it?

You'll never get to play alongside big boys like Apple and Facebook with this 'talk to upstream' attitude of yours. That's just not how the game is played.

<comment><USER24> It's telling you left off google.

</comment></comment></comment></comment><comment><USER25> Data compression and image compression is a great way to improve the overall internet, bandwidth and speed. Maybe as important as new protocols like SPDY and js/css minification and cdn hosting of common libraries.

As long as ISPs/telcos don't go back to the days of AOL network wide compression to reduce bandwidth beyond low quality I am for this at service level like facebook/dropbox uploads. I hope this inspires more in this area. Games also get better with better textures in less space.

Still to this day, I am amazed at the small file sizes macromedia (adobe now) was able to obtain with flash/swf/asf even high quality PNGs would compress. So yes we all have lots of bandwidth now but crunching to the point of representing the same thing is a good thing. With cable company caps and other bandwidth false supply shortage that focus might resurge a bit.

<comment><USER26> SWF has a very cleverly designed binary vector graphics format, which naturally lends itself to small filesizes. Much better than SVG or (E)PS, I think.

<comment><USER27> Can you share more on the "clever" part?

<comment><USER26> To quote from the format spec, "SWF uses techniques such as bit-packing and structures with optional fields to minimize file size." Many fields are variable-width numbers of bits, using only as many bits as necessary to encode the data. Coordinates are delta-encoded.

</comment></comment></comment></comment><comment><USER28> JPEG-2000 exists, but decoding is still too slow to be useful.

[http://en.wikipedia.org/wiki/JPEG_2000](http://en.wikipedia.org/wiki/JPEG_2000)

<comment><USER29> Seen a movie in the theater recently? JP2k lives on in the DCI [1].

[1] [http://www.dcimovies.com](http://www.dcimovies.com)

<comment><USER28> Yep. Theater projectors have roughly $1k in specialized decoding chips to make that work.

Software encode/decode of JP2k is the hard part. That's why there's little adoption of J2k outside of hardware solutions.

<comment><USER29> It's a really weird place to use JP2K -- they're functionally not space limited, or compute limited, so what's the advantage of wavelets? You could just gzip V210 or something.

<comment><USER30> The spec allows for 4K (4096x2160 at 24 FPS) or 2K (2048x1080 at 24 or 48 FPS) source material and projectors. The spec recognizes that 2K sources may be played on 4K projectors (where it leaves the task of upscaling to the implementer) and 4K sources on 2K projectors.

The advantage of the wavelet format is that they can implement progressive resolution decoding, so a decoder only needs to read half of the data from a 4K source to decode a full quality 2K image.

<comment><USER29> Interesting. I had known about the mosquito noise problem (as cjensen notes (and I was joking about gziping 10-bit 422)), but I didn't realize the progressive decoding was a part of the standard. I spent some time in the guts of the DCP in a previous life, and software decoding of JP2K was always a nightmare. So I'm a bit jaundiced.

But, as always, when you think an engineering decision is insane, you're _probably_ missing the context in which the decision was made.

</comment></comment></comment></comment></comment></comment><comment><USER31> What about WebP? Isn't that intended to be a eventual replacement to JPEG?

<comment><USER32> Only if other browser vendors adopt it, although IIRC WebP has hard-coded maximum file size limits that make it impractical for e.g. retina displays, let alone anything we might see twenty years from now.

<comment><USER33> Ah, that's interesting. I hadn't heard that before. It is a maximum size of 16383x16383[1], so it's more than practical for retina displays, but I can see the point about files in 20 years (for reference, jpegs can be 4x that in each dimension, 65535×65535).

I haven't heard that brought up as an objection to the format before, though. If it were really a fundamental stumbling block, there are likely ways to adapt the format around it.

[1] [https://developers.google.com/speed/webp/faq#what_is_the_max...](https://developers.google.com/speed/webp/faq#what_is_the_maximum_size_a_webp_image_can_be)

</comment></comment></comment><comment><USER34> I'm actually disapointed. I hoped they developed a still image format from Daala. Daala has sigificant improments such as overlapping blocks, differently sized blocks and a predictor that works not only for luma or chroma, but for both.

<comment><USER35> One of our listed intern projects is exactly that. It's not a high priority project for the Daala team because we already have good royalty-free image codecs, but not video codecs. We have finite time, so we choose to attack what we see as the more important problem.

Developing a still image format from Daala is possible but not as trivial as it looks (the same can be said for the WebP work from WebM).

</comment></comment><comment><USER36> I like that Mozilla is improving the existing accepted standard, but using modern (mostly patented) codec techniques we could get lossy images to under 1/2 of the current size at the same quality and decode speed. Or at a much higher quality for the same size.

The speed modern web concerns me. The standards are not moving forward. We still use HTML, CSS, Javascript, Jpeg, Gif, and PNG. Gif especially is a format where we could see similar sized/quality moving images at 1/8th the file size if we supported algorithms similar to those found in modern video.

In all of these cases, they aren't "tried and true" so much as "we've had so many problems with each that we've got a huge suite of half-hacked solutions to pretty much everything you could want to do". We haven't moved forward because we can't. WebP is a good example of a superior format that never stood a chance because front-end web technology is not flexible.

<comment><USER37> I think the surprising thing is that JPEG is as good as it is. WebP certainly isn't "1/2 of the current size at the same quality and decode speed". It's a modest improvement if anything.

GIF, sure. But, IMO, nobody should really be using GIFs anymore. We've had <video> and APNG for a while if you want animation.

</comment></comment><comment><USER24> I have heard similar things about GIF (that there are optimisations that most encoding software does not properly take advantage of). But I haven't seen any efforts, or cutting edge software that actually follows through on that promise. The closest I've seen is gifscicle, which is a bit disappointing.

What would be great if there was some way for an animated gif's frame delays to opt-in to being interpreted correctly by browser- That is, a 0-delay really would display with no delay, and so optimisation strategies involving the splitting of image data across multiple frames could be done- and when read in by a browser, all frames would be overlaid instantly, module loading time.

What other things can be done to further optimise animated gif encoding?

<comment><USER38> GIF should be killed. Animated PNG is where it's at.

<comment><USER39> Animated PNGs should be killed. Just embed a video with an alpha channel. It compresses way better.

[http://simpl.info/videoalpha/](http://simpl.info/videoalpha/)

<comment><USER38> Wrong use-case. Animated GIFs and animated PNGs were never intended for video. The analogy to still image formats is video:JPEG = web-animation:PNG. You don't save screenshots as JPGs and you don't save photos as PNGs. Same with video and web-animation, two different use-cases require two different formats.

<comment><USER39> except that's a false dichotomy

one format can support both lossy and lossless images and use better compression between frames to achieve 10x better results for animations than animated pngs without visible loss of quality

<comment><USER38> Which format can support both lossy and lossless images?

I spent 2 seconds in GIMP to make the following GIF: [http://i.cubeupload.com/kbrcfQ.gif](http://i.cubeupload.com/kbrcfQ.gif) Please show me the video format that can replace it.

<comment><USER39> webp supports animations, lossless and lossy images, etc. I already linked to a video with an alpha channel that can match your gif

</comment></comment></comment></comment></comment></comment></comment><comment><USER40> It's not clear from the article, in their "comparison of 1500 JPEG images from Wikipedia" did they just run through the entropy coding portion again or did they requantize? (I suspect they did jus the entropy coding portion, but hard to tell).

Getting better encoding by changing the quantization method can't be purely a function of file size, traditionally PSNR measurements as well as visual quality come into play.

Good to see some work in the area, I will need to check out what is new and novel.

That said, a company I worked for many moons ago came up with a method where by reorganization of coefficients post-quantization, you could easily get about 20% improvement in encoding efficiency, but the result was not JPEG compatible.

There is a lot that can be played with.

</comment><comment><USER41> If only JPEG supported transparency.

<comment><USER42> You can do a JS hack with JPEG for RGB and PNG for alpha, but it's not ideal.

</comment></comment><comment><USER43> When I optimize JPG or PNG I usually use ScriptJPG and ScriptPNG from [http://css-ig.net/tools/](http://css-ig.net/tools/)

They are shell scripts running many different optimizers

</comment><comment><USER44> "support for progressive JPEGs is not universal" [https://en.wikipedia.org/wiki/JPEG#JPEG_compression](https://en.wikipedia.org/wiki/JPEG#JPEG_compression)

e.g. the hardware decoder in the Raspberry Pi [http://forum.stmlabs.com/showthread.php?tid=12102](http://forum.stmlabs.com/showthread.php?tid=12102)

</comment><comment> kraken-io Hey everyone, after some testing we have just deployed mozjpeg to our web interface at: [https://kraken.io/web-interface](https://kraken.io/web- interface)

You can test it out by selecting the "lossless" option and uploading a jpeg. Enjoy!

</comment><comment><USER45> So... version 1.0 is basically a shell script that calls libjpeg-turbo followed by jpgcrush?

<comment><USER46> No, the jpegcrush functionality is implemented in C as an extension to libjpeg-turbo. But yes, I suppose a shell script would achieve roughly the same result.

<comment><USER45> So why wasn't this upstreamed to libjpeg-turbo? Why fork at all? libjpeg-turbo is still an actively maintained project after all...

<comment><USER47> Maintainer has different priorities, see [https://news.ycombinator.com/item?id=7349117](https://news.ycombinator.com/item?id=7349117)

</comment></comment></comment></comment><comment><USER48> Any chance of incorporating other psy improvements, instead of just targeting SSIM?

</comment><comment><USER49> At first glance this seems wasteful. I do not think anyone would have problem in using Jpeg. However, in many cases, before the the invention of a thing who has had no problem using old tools!

</comment><comment><USER50> Has somebody translated the jpeg library to JavaScript? Besides encoding and decoding jpeg, it has some useful modules that would be nice to have in the web browser.

</comment><comment><USER51> A bit to soon to start announcing the project. But I like the initiative hope the project manages to improve stuff.

<comment><AUTHOR> We would like to develop in the open, and hopefully with community participation.

<comment><USER10> BTW, please fix your lossy test methodology for when you do your tests on a future version of this.

[https://blog.mozilla.org/research/2013/10/17/studying- lossy-...](https://blog.mozilla.org/research/2013/10/17/studying-lossy-image- compression-efficiency/) was rather flawed and you never acknowledged this.

(see [https://news.ycombinator.com/item?id=6581827](https://news.ycombinator.com/item?id=6581827) \- only one of the four sets of test results _might_ have been valid)

</comment></comment></comment><comment><USER52> What license are they doing this under? Hopefully they're aiming to upstream this to libjpeg.

</comment><comment><USER53> This is so dumb, there are a million JPEG crushers in existence but instead of advocating the use of one of these Mozilla writes their own? Why not support webp rather than dismiss it due to compatibility and waste time doing what has been done before.</comment>


The ideal DevOps team structure - <AUTHOR> https://about.gitlab.com/2019/06/12/devops-team-structure/ <comment><USER2> Whenever I see articles about DevOps, I find myself wondering what exactly it is. It seems to mean different things to different people. Without any definitions in this article, I find it hard to understand what they mean by "Dev", "Ops", and "DevOps".

<comment><USER3> Devs are those who dabble in code. Ops are those who run and administer systems that run the code.

Devops is the act of codifying your operations. Automating deployments, ci/CD things like that. Putting alert and monitoring systems in place.

That's my understanding.

<comment><USER4> The following bullet points encapsulate DevOps to me:

\- Codifying infrastructure with tools such as Packer, Terraform, Ansible. The infrastructure should be just as auditable, repeatable, and transparent as the code.

\- Empowering and allowing the engineers who write code to deploy it.

\- Immutable servers/containers.

\- A solid suite of monitoring and alerting tools that auto-heal (preferred) or at least notify when things go wrong.

<comment><USER5> I think 1 and 3 are just certain ways of doing DevOps. Out of the 4, I think 2 seems most accurate.

<comment><USER6> #2 is the human combination of Dev and Ops

#1 is the technological combination of Dev and Ops

#3 is a corollary to #2

#4 is just good Ops

</comment></comment></comment></comment></comment><comment> k-ian At AWS there is rarely a distinction between dev and ops. Developers manage their service and join the oncall rotation for their own team. We also write all our own tests and manage deployments ourselves (albeit with the help of other AWS teams who maintain test frameworks and deployment tools.)

This seems a lot cleaner than the alternative. Sure the devs have greater responsibility, but there's no dev/ops disconnect here.

<comment><USER7> >>Sure the devs have greater responsibility, but there's no dev/ops disconnect here.

I feel this greater responsibility also leads to better code. Do devs in this model also have on-call responsibilities?

</comment></comment><comment><USER8> Recommend looking at [https://web.devopstopologies.com/](https://web.devopstopologies.com/). Good to understand the interplay between different team structures.

</comment><comment><USER9> Ugh ads</comment>


Report: Microsoft is going to try to make a cheap Surface tablet… again - <AUTHOR> https://arstechnica.com/gadgets/2018/05/report-microsoft-is-going-to-try-to-make-a-cheap-surface-tablet-again/ <comment><AUTHOR> I'd like the Surface to appeal better to people considering an iPad (Pro). This means it should match important specs: it should be no heavier, thicker or costlier than the iPad, have a HDR screen, have as good speakers, etc.

Some differences between the Surface and iPad are a tradeoff. Some people want the power of a desktop OS; others want the simplicity and touch-friendliness of a mobile OS. There's no one right answer for everyone.

But other aspects do have one right answer: nobody wants their tablet to be heavier, costlier or have a worse screen. There's no excuse not to match Apple's standard in these areas.

It's sad to see Windows tablets unable to compete with the iPad in that sense.</comment>


Ask HN: Would you use an email presorting service like this? - <AUTHOR>

Hi guys, at SquareOne we've been building a more manageable mobile email experience, and through all of our customer research we came up with an interesting service model that could live on top of the platform we've built.
This is a service for people who get 100+ emails a day and can't effectively stay on top of their inbox without spending hours a day inside it. We let the person with the highest incentive to have an email reach you - the sender - do a little bit of extra work and categorize their email for you. Concept landing page:
www.squareonemail.com/mailone
What do you think? <comment><USER2> Years ago I sent a message to a stranger. I received an auto-reply saying "I got your mail, spam is hard, click this link to deliver it and prove you're human".

For a second I thought it was a cute solution to spam, then I realized we'd be in a world of pain if we mailed each other, and both of us replied in such a fashion.

I never mailed him again.

If I have to do work to contact somebody? Well I'll just ignore that person. Sorry.

<comment><USER3> [http://en.wikipedia.org/wiki/Challenge%E2%80%93response_spam...](http://en.wikipedia.org/wiki/Challenge%E2%80%93response_spam_filtering)

Further reading if anyone is interested in challenge-response systems.

</comment></comment><comment><USER3> So is SquareOne supposed to replace my email, or work alongside it?

<comment><AUTHOR> SquareOne (squareonemail.com) is a standalone native iOS email client that works with your Gmail account. We're in public beta with the app now.

The experiment we posted about (we're calling it MailOne for now) would simply supplement the new email UX by an added service layer, where the sender of an email is challenged to categorize it for you as the recipient.</comment></comment>


Chinese University Tops MIT in Engineering Rankings - <AUTHOR> http://blogs.wsj.com/chinarealtime/2015/10/08/chinese-university-tops-mit-in-engineering-rankings/?mod=e2fb <comment><USER2> Seeing everyone on here so dismissive is simply shocking to me. If you don't like these results or don't believe them look at the overall trend folks. A decade ago there wouldn't have been the Chinese University in the top 10. Now they have half the top rankings. I take my hat off to them; I respect their achievements and if you follow the trend forward they look unstoppable.and of the people at these universities they or their parents grew up in villages without electricity with nothing. they've built out of that the ability to scratch the stars quickly. much respect let's hope this doesn't turn into a latent Cold War

<comment><USER3> BA/MA in Chinese studies here. I greatly admire China's accomplishments, starting back around 1,000 BC :)

China today is on a trajectory of becoming the premier world technological power, simply based on its tremendous energy and drive and respect for learning.

The U.S. and Europe are in decline, by comparison. The U.S. in particular is relying heavily on Chinese and other foreign students to keep its science, math, and engineering graduate programs going. The native U.S. population is lazy and unmotivated by comparison.

And tellingly, increasing numbers of Chinese grads are returning to China to work rather than seeking citizenship in the U.S. Even non-Chinese are flocking to China to start up technology companies, something unheard of a generation or two back.

I too hope this doesn't turn into some kind of cold war.

<comment><USER4> 'The U.S. and Europe are in decline, by comparison'. No argument about this. Where are the passionate national debates about why this is so and what can be done about it? Or maybe they are going on and I'm not seeing them.

<comment><USER5> [http://blog.samaltman.com/china](http://blog.samaltman.com/china)

</comment></comment></comment></comment><comment><USER6> "Each school's score is based on its number of publications and citations as well as its global and regional research reputation."

Oh lord... one is easy to game, and the other is a self-fulfilling prophecy.

<comment><USER7> Exactly, so MIT has clearly been ...

Oh, wait, you mean you were implying that Tsinghua University was gaming the rankings ...

MIT students, in my experience, have been overrated for years.

<comment><USER8> He is not claiming one or the other is gaming it, he is saying this is a terrible set of metrics to rank universities with

<comment><USER9> "Ok, they've outplayed us in our game. Lets call this game terrible and start a different one with new rules!"

<comment><USER8> See my other comment before attributing malice to my words. As a french, I've never in my life found the rules to be any good as they were biased against us and other non english speaking country (imho).

<comment><USER10> Except of course for the "Mines ParisTech : Professional Ranking of World Universities" :D [https://en.wikipedia.org/wiki/Mines_ParisTech_:_Professional...](https://en.wikipedia.org/wiki/Mines_ParisTech_:_Professional_Ranking_of_World_Universities)

Which is actually a really interesting ranking. Not based on academic research whatsoever.

</comment></comment></comment></comment></comment></comment><comment><USER11> So… Everyone is righty pointing out the uselessness of ranking systems. But, these are also the same kinds of ranking systems that place MIT, Cambridge, Stanford & the rest in the elite. Publication, reputation, etc. It's not about a specific ranking system either. Every ranking uses circular, subjective or self fulfilling measures. These reputations are also very long lived and carry outside of their context. Great research in biology does not really indicate great undergraduate teaching in the humanities.

Universities attract students, funding, press and all the other goodies with their reputations. Students then trade on these reputations. It's a big problem overall, and it doesn't have good feedback loops of the kind which will encourage universities to improve.

Also, the more attention the rankings get, the more they get gamed and corrupted.

Here's the thing, say you are looking for a place to study mechanical engineering, it will be difficult to find the new upstart schools that are doing great work. It's even harder to get employers, parents and colleagues to respect it. That's to say nothing of a course attempting to teach a subject like engineering outside the paradigm of Universities altogether.

It's a pickle.

The world is way too big for us to rely on a vague name recognition & reputation system anymore. It's too dynamic to wait for them to self correct and it's way too thirsty for better ways to train and educate to be without a dynamic system where new approaches and institutions can emerge.

</comment><comment><USER3> The Weibuo quote in the article sums it up. Would you rather have your child go to Tsinghua or MIT? The fact is, MIT remains the most prestigious scientific and engineering school in the world, and number of published articles be damned.

Another metric not considered by USNWR is academic freedom. Can students and faculty at Tsinghua use Google? I don't think so. No Google Scholar, no Google Books, not even Search. How ridiculous. Without equal access to the Internet, a fair comparison is not possible. And what if a student published an article complaining about lack of access to some internet resources? End of career.

That said, this evaluation should be a wake up call to American educators that our formerly great academy has deteriorated, brought down by decades of decaying performance in K-12 and lack of energy and commitment in the universities, where many or most of the science and engineering graduate students are foreign born.

<comment><USER12> They usually know how to jump the wall or get direct lines out for erm research purposes. ChinaSys, an acedemic/industry systems conference, is taking registrations in... Google docs.

I hope this is not a wake up call: what we don't need in academia is more gaming of the publication and citation system! This is a wake up call to not take usnews too seriously.

<comment><USER3> Interesting. I guess when it's a matter of national prestige and encouraging scientific accomplishment, the authorities might look the other way. But that's not really the same as complete and open access.

And I agree with you that we don't need even more gaming of the system; but we definitely need more motivation of K-12 students in the math and science fields.

<comment><USER12> The "motivation" should come from a genuine curiosity and interest in making the world a better place. Academia, even western academia, is often motivated by securing more funding so they can....secure more funding. That creates a closed loop that serves no one outside of it.

The US News report just shows who is better at gaming the system. Heck, even MIT has its problems, though you'll find more pockets of genuine interest and capabilities. Tsinghua also has a lot of great researchers, but usually this is inspite of, not because of, the games played to optimize for these metrics.

</comment></comment></comment></comment><comment><USER13> China only has two prestigious universities, Peking and Tsing Hua. They collect all financial and political support to make these two universities rank higher and higher, not only to save face on the international stage, but also to make sure all intelligence is under the control of Beijing. They may really have reached success in winning global reputation, but the chance for their research to turn into benefit for common people will be very small.

</comment><comment><USER14> American education based on sales. It sells knowledge and that is why it is going downhill. Being politically correct on races is another curse. And don't talk about creativity when you students cannot do math on paper!

<comment><USER15> It is? Can you provide some evidence?

</comment></comment><comment><USER16> If it's true, as a matter of national pride, let's hope other countries join the race to have the top ranked engineering university. We could use a dozen more MIT's in the world.

</comment><comment><USER17> It'd be more interesting to see a ranking by only the average competitiveness of their undergraduate school students, and I bet you will see lots of Chinese and Indian university on top and American Universities are not anywhere close.

America has so many good universities but at the same time just a small talent pool.

</comment><comment><USER18> For some Indians who may be invested in knowing the rankings, there are 9 of them with the first one starting at 100th position (IIT-D). Ouch.

</comment><comment><USER19> To those that believe that US system is rotten and ineffective and think that Chinese are doing something right because they are Chinese I can tell you only one thing. NEVER TRUST A COMMUNIST REGIME.

Even if you compare millionaires to millionaires the quality of life in China is far worse than in the USA. Never trust a communist propaganda. Never ever do that. The gov and chinese in general are known for trying to save face at every occasion and will fabricate anything that can benefit them.

If USA and UK is so bad then why thousands of Chinese are trying to immigrate there ? In Communist regime even if you are good and talented you will become a property of the state and they will use you for their benefit. Do not expect a good quality life but constant abuse for more propaganda and mind control.

No wonder that massive amount of people of Hong Kong are requesting british passports. Because in China you are nothing more than a property of a regime.

<comment><USER20> Ahahaha! China is not the Soviet Union. It's a capitalist society like almost anywhere else, and I'm positive that being rich in China rocks (apart from the air pollution).

<comment><USER19> Then you obviously have no idea what you are talking about. And the air pollution in China is something you can't imagine. The VW scandal is nothing compared to chinese air pollution.

And China is far worse than the former Soviet Union because they have more money and more control. Being lower class in China is far worse than in former states of the soviet union.

<comment><USER21> You talk a whole lot, but you've never addressed his point that "it would rock being rich in China". (I am not taking a side).

<comment><USER19> Being rich does not guarantee that you will have a good life anywhere. There are numerous other things you have to consider. Yes a rich man can buy a big land for himself and build a wall to keep the beggars outside. He can even move to the countryside and never go out or use a private jets / helicopters.

The thing is that China is a home for a lot of people and is too polluted and crowded because of wealth distribution. If people do not care about the environment, intellectual property and a countless of other things then being rich and filthy rich are too different things.

A wealthy man can walk the streets of european/american cities without worrying too much about its health or not having a space for himself without guards. Do not forget that european / us cities are dwarves compared to Asia in general.

The movie Elysium is a good picture (maybe too exaggerated but hey) of people being wealthy in China.

Yes you can have a good life in a golden cage but it's still a cage.

<comment><USER5> Compared to the US, China has a lot less of both violent crime and beggars. I'm sorry but it seems you don't know what you are talking about.

<comment><USER19> First of all you know nothing about me and that's fine.

Second just because you live in Hong Kong or Shenzhen it does not mean you know anything about China. Especially these 2 do not reflect anything at all about China. And that is fine that might be the reason why have you chosen them and not the other places in China.

Third. You do not seem to speak the Chinese language.

Fourth. Just because you have a positive prejudice towards China does not mean you are right.

Fifth. If you would know something about authoritarian regimes and especially China or NK you would know that hiding and punishing beggars is a common practice to save face.

Sixth. Please educate me how country of 1.3bn has a lot less beggars and violent crimes than US a country of 320m.

Seventh. If you have never lived in a communist country you surely can't be aware of propaganda and hiding of crimes. Ask some chinese friends of yours how hiding crimes, especially the violent ones goes in China. How it was in every communist state. You will be surprised.

Eighth. I understand you like China/Chinese more than Americans but do not close your eyes before the crimes of these regimes I am begging you. For all of those that died and suffered in gulags

Ninth. I recommend you to visit countryside and especially the northern region if you have the balls.

<comment><USER17> If you love American life just stay here and no one is asking you to move to China.

There are tons of people (not necessarily rich) prefer living in China than in America. They just care about different things.

As a Chinese citizen living in San Francisco right now, I would say the overall living quality in America is no where near as good as in China. If not for the prosperous software industry here, there is no reason to stay in America at all.

China has its problems but America has its own too. The same goes both for the society and the government. I do feel a lot safer walking down the streets in China, and air pollution has never been a big issue. What you see on news headlines are not what you see everyday.

<comment><USER19> I am not going to try to convince you that's none of my business. You are either coming from top 10% of talented chinese or your family was simply rich. 90% chinese can't afford what you can.

And air pollution is not problem in China ? Are you kidding me ? Just ask my any korean in Seoul how air pollution from China is not a problem. And THAT IS SEOUL that's being hit by chinese air pollution.

The pollution in chinese cities is too big to not consider it an issue.

<comment><USER17> I'm not saying air pollution is not a problem at all. It may had gone really bad in a few major cities a few times, but it's not something everyone gets to experience. The overall air quality is not top notch but it's also not so bad as many of the Americans think. There are many things I hate about China but air pollution won't even be in the list.

Comparing to that I'm bothered a lot more here about the prevalence of guns, drugs, high crime rates, the unfairness in education (how many ivy league students come from poor families?), ridiculously expensive medical system, so many bullshit in the law, government wasting so much money while getting almost nothing done... I'd happily trade air pollution with solution to any of these problems. Sometimes it's just funny to see how Americans could be okay to accept all of that really well, some may not even realize these are all appalling problems, while they still give such a big fuss about air pollution in China.</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment>


Synthesizing thunder using JavaScript - <AUTHOR> http://blog.kaistale.com/?p=1340

<comment><USER2> It killed Chrome and made my laptop irresponsible until I managed to close the page. Something in the thunder generation is bringing my system to its knees.

I'm not complaining about the page or anything, but I'm really surprised how unreliable and fragile are web browsers to pages with javascript. It's using a web worker, but other than that I can't see anything special.

<comment><USER3> _It killed Chrome and made my laptop irresponsible_

I know you meant _unresponsive_ , but this conjured up a delightful image of your laptop jumping around the room snapping its lid at people like a set of autonomous false teeth.

<comment><USER2> Well, that would be amusing. English is my third language and sometimes I trust too much in the spell checker suggestions.

My graphologies ;)

</comment></comment></comment><comment><USER4> Andrew Glassner has written about this a lot [http://www.glassner.com/computer-graphics/graphics- research/...](http://www.glassner.com/computer-graphics/graphics- research/lightning-and-thunder/)

(See his references section)

<comment><AUTHOR> Great, thanks for this, I'll have to check those out when I'm at a computer with access.

</comment></comment><comment><USER5> Sounds pretty real to me and the theory behind it is written up very nicely. I wonder: would this work the other way round? Take a (stereo) recording of thunder and an exact time measurement between the flash and the first sound and plot the flash from that. If that worked, you wouldn't need a huge high speed camera mounted on a truck in order to record how a flash propagates...

<comment><USER3> Besides issues like temperature, you'd need to to control for interference from geographic features like hills and valleys. It might work in wide open flat spaces like the prairies of the American mid-west.

If you're interested in this, there was a very good program on PBS's science show _Nova_ a month or two back, called 'the edge of space' or something similar, which involved photography of lightning at altitude and later from the space station, resulting in confirmation that lightning interacts with the upper atmosphere as well as the ground.</comment></comment>


Your call is important to us - <AUTHOR> http://www.economist.com/displaystory.cfm?story_id=10789441

<comment><USER2> Seems like it would be cheaper and easier to just hire people of local nationality to the callers instead of trying to create a computerized system to make cultural and psychological inferences based on key words which are then fed to foreign phone support personnel.</comment>


Devops needs to be a two way street devs and ops - <AUTHOR> http://mike-woome.tumblr.com/post/1691265243/devops-devops What strikes me right now, that the devops community, at least the bits I am meeting seems to be full of devs believing their bringing a revolution to the poor unwashed sysadmin masses. Well, you have some nice ideas and I think we can improve the whole community with some of this stuff but seriously drop the attitude because I'm finding it quite condescending and I'm not the only one. <comment><USER2> The relevant XKCD: <http://xkcd.com/793/>

Frankly, being an ops person who was dev trained (CS degree, doing ops since) what I've seen from other ops people once exposed to a devops mindset one of a few things:

1\. Fear - mostly people who haven't even scripted anything before (common in Windows and pre-OS X Mac environments)

2\. Confusion - I'm supposed to learn what with the who now? Mainly this is a tool choice issue - there's an overwhelming number of options out there.

3\. Relief - finally, someone ELSE understands what I've been trying to do for the last N years, and I can talk to!

My guess is that, for ops people to accept this is to get the #3's to work with the the #2's on tool choice and familiarity, and then have the #2's convince or prove to the #1's that it's a good idea.

Dev interaction would probably be with mainly the #3's, as that would be the best opportunity for dialogue that would be meaningful to both sides.

</comment><comment><USER3> I have to completely agree with this blog post. I'm a sysadmin living in a world of developers and what I'm being shown is nothing new. I think there are a lot of companies in which dev and ops is split to such a degree that the devops movement becomes relevant in aiding them to bring together both sides, but for the rest of us we've seen it and done it. I agree with the author, there's a lot of condescension coming across. I am open to learning and refining the processes by which we all collaborate and work but I'm not a layman. I understand my field and enough of yours to keep systems running in the evenings, middle of the night and times when you are not around to look at bugs. I understand the scaling problems. I understand the need for continuous integration testing and deployment systems.

We are the league of sys admins. We know what we are doing.

<comment><USER4> Hear hear.

Personally, I find the whole devops notion a bit bizarre, since, from what I can tell, the theme is one of two.

One is trying to make sysadmins into programmers (or hybrid programmers/sysadmins). This may be the one-way street the blogger is referring to. I _could_ be a programmer if I wanted to, but that's not what brings me joy. There's a reason I've been a sysadmin for my entire career, and it's not some deficiency (i.e. I'm not a layman).

The other is, to my mind, just wishing for better sysadmins. Sadly, I don't think there are enough of us who _do_ know what we're doing, not to the depth you describe. Much of this may be for lack of experience and/or mentorship: they haven't seen it and done it.

 _when I cut my teeth on sysadmin work in the late 17th century, ops guys drew a line in the sand_

This comment was the revealing one to me. Its inference belies long experience. My own start at the profession was in 1988, though, arguably, not in an environment with software developers until '92. Not only did we[1] not put artificial limits on our responsibilities, but I had to fight a couple battles to keep developers from running roughshod over a shared environment made stable through my concerted efforts.

Agreed: We are the league of sysadmins. We know what we are doing.

I would only add: Not all problems are best solved by more code.

[1] originally just me, but then a total of three of us in that environment

<comment><AUTHOR> devops makes so much sense to me because it's what I've been doing my whole career. The trouble is with the devops movement, as the blog says, the old tools and processes are sometimes still relevant. Sysadmins in that category 2 from the first comment need to be persuaded that a lot of their old skills are absolutely still usable. The example from the article, of bash scripts getting version controlled is a really good one I think.

And of course, sysadmins have a wealth of experience to bring to the way development is done. It wouldn't hurt if a few more developers learned bash and the different levels of RAID.

</comment></comment></comment><comment><USER5> As someone who has a CS degree but who is professionally a systems person, I've been flipflopping continuously on if I even like the term "devops". I'm looking for someone for my team to fill a "devops" role, and sometimes people talk about it like it's "developer/operations" (someone who can be both a developer and do systems work) and sometimes it's "developer operations" (someone who is a toolsmith _just_ for developers, does release engineering, deployment related work, monitoring and maintenance, doesn't deal with the nittygritty of production systems).

Personally, give me a developer who also knows operations and systems maintenance and debugging, or a operations guy who also groks development and the product development lifecycle. Having these roles be distinctly separate people or teams, who don't have experience on the other side, is a gross inefficiency.

There's a tendency, as a company grows larger, to separate the concerns. In a smaller company, it's almost necessary to have the same one or two people do both roles (especially in a startup -- your startup's chances of even getting out the gate are reduced if your technology arm _only_ does one or the other). I think some of the increased elitism from the purely developer side comes from the recent perception that "you can get rid of your systems and ops guys because developers can now deploy to the cloud". Elitism exists in some form on the systems and operations side because technical debt is more obviously visible to them (but they are unaware of the history of the reason for much of the technical debt, not having been involved with the product side). The ones who are legitimately elite, because they are on both sides, are also seen as humble because they concentrate on actually solving problems using whichever tools are available from either camp.

<comment><USER6> I'm not really sure why "you can get rid of your systems and ops guys because developers can now deploy to the cloud" is elitism, if you take the long view. (Today, yes, you're jumping the gun a bit if you don't think you need ops folks.)

We're likely less than 5-10 years away from a world where most operational tasks can be passed on to trusted cloud providers, similar to how you don't wire up your own phone service or plumbing any longer. You still need talented ops people, but they'll be working for a service provider not your own organization.

<comment><USER4> _I'm not really sure why "you can get rid of your systems and ops guys because developers can now deploy to the cloud" is elitism_

I think you answer your own question:

 _you don't wire up your own phone service or plumbing_

Although linemen and plumbers aren't seen to be in jobs that are _menial_ , you'd have trouble classifying them as elite, compared to their customers[1].

Regardless, the analogy strikes me as tenuous, since plumbing and telephones aren't _fundamental_ to most businesses. On the other hand, computer systems, for a software company, are very much the fundament of what it does.

Perhaps a better analogy would be the likes of FedEx or UPS outsourcing the maintenance (and perhaps all operation) of its trucks and planes. I'm pretty confident they keep that in-house.

[1] Presumably, this is, predominantly, homeowners and landlords.

<comment><USER6> I have a feeling the reason we think racking servers and configuring Apache are fundamental elements for software companies has less to do with reality and more to do with the status quo. The world is going through a phase shift, and each day that goes by trends _away_ from the current model of in-house operations. It's just a matter of how much farther things are going to get before it slows down and stops.

<comment><USER5> It definitely trends away from in-house operations for companies who view their systems and operations to be cost centers that _are not_ intimately involved with the product/service. And companies that do keep in-house operations will have the same kind of competitive advantage that companies that don't outsource their developers have (whatever that may or may not be).

<comment><USER6> Honestly, how many companies fall into the category of having their data centers be intimately involved with their product/service? Not very many, to me.

<comment><USER5> None, but the point of the OP is that operations is a lot more involved than racking servers and hitting power switches. There's a wide range of stuff that between "developer" and "server racking". The things on this spectrum that are closer to developer are the things that I'm saying are not worth outsourcing, my previous comments in this thread touch on that.</comment></comment></comment></comment></comment></comment></comment>


What happened to changing the world? - <AUTHOR> http://venturebeat.com/2014/04/27/what-happened-to-changing-the-world/

<comment><USER2> This article suffers from presentism.

The people who set out to change the world are still in the Bay Area, probably in the same rate per capita as the past. What has changed is that tech has become more accessible and therefor more pop. Basically, the number of software engineers per capita has grown dramatically, but that most of the growth is happening in the type of companies the author laments. 20-30 years ago these newcomers would not have been founding frivolous tech companies because they would probably be in some other line of work instead of in tech founding any company at all.

</comment><comment><USER3> I work for a company who "move fast and break things", and strangely customers don't like it when we break things they rely on.

This is about professionalism now, and professionals get what is due. A few snacks is nothing compared to what founders or VC get. We are the people who bring your plans to fruition, hook us up with a couple snickers or even artisanal coffee, the expense is worth it.</comment>


Startup Weekend Comes to NYC - <AUTHOR> http://www.centernetworks.com/startup-weekend-nyc

<comment><USER2> Not to thread jack or anything, but here's some stuff about Startup Weekend Atlanta coming up in November.

<http://news.ycombinator.com/item?id=52796> (article from the guy bringing Startup Weekend to Atlanta) <http://news.ycombinator.com/item?id=52696> (my post about what SW is all about)

<comment><USER3> Thanks for posting this -- I was just thinking about what I would have to do to organize one of these in Atlanta.</comment></comment>


Rob Pike on Go interfaces - <AUTHOR> https://groups.google.com/group/golang-nuts/msg/8de71de10ea48f96 <comment><USER2> I probably have a parsing problem here. Can you help me solve it?

Knowing not a lot about Go I followed the links and saw that interfaces are 'implemented' implicitly, the quacks like a duck way. If I fulfill the contract of an interface I can be used as something of that interface. Cool.

But - what is Rob's comment now?

"It's always bothered me in OO languages that we can make Circle, Square, etc. subclasses of Shape (say), but that's the one design decision we get to make. What if we want to align those things along other axes (so to speak), like topological genus or plant genus, if you're a landscaper? You might lose your way or dig yourself into a hole with multiple inheritance." (goes on, imo down the same route)

From what I can tell there's no issue here. Except for the requirement to be explicit (implements Foo, Baz, Baz) instead of implicit (just add those interfaces' methods on your type) the very same thing can archived in the biggest (market share?) OO languages. Java? Check. C#? Check.

What exactly is leading to this final quote:

"Go's interfaces aren't a variant on Java or C# interfaces, they're much more. They are a key to large-scale programming and adaptable, evolutionary design."

I'm honest here. I don't get it. Duck typing? Yeah, fine. I think I'd like that. But what am I missing? It seems in Go you get two things: No need to explicitly annotate the types with the classes it implements (good or bad, probably depends on taste and general complexity of the code) and, perhaps bigger?, you can probably extract interfaces later which are now magically implemented. But both features seem to be ~refactoring/design~ details. Nifty maybe, if they fit your style. But not a leap in any way?

<comment><USER3> Ok, so you're missing the main point. Suppose you have a library that isn't yours, and you don't have the source code. But, it has a type "Foop" that doesn't implement any of the interfaces you'd like it to for your purposes. What do you do? In the Java world, you make a subclass and implement those interfaces. We probably call that the Adapter pattern or something.

In the Go world, or the Haskell world, or even the Clojure world, we say, "No problem. I'll just _extend_ Foop with this interface by adding these 3 methods."

Extending/adding those methods doesn't change Foop though. Foops are still Foops and can still be used as is with the library that Foop was bundled with. They just now implement a new interface for my purposes.

<comment><USER2> Just to be sure: You suggest to write interfaces ('extract' in my original post as that seems to be the refactoring name you'd use for this kind of stuff in my world) that define a contract for pre-existing characteristics of 3rd party types?

Or are you, as I read your sentence in quotes, talking about some kind of monkey patching of the original classes?

I understand your point, but I don't understand the original post/mail. Interfaces per se still seem to be the exact same thing, simple contracts, and types can fulfill multiple of those. I guess my main problem is that for me this is not a discussion about interfaces, this is a discussion about orthogonal language features which might happen to allow you to use interfaces a little more liberal.

That's why I started with my assumption of a parsing error: Reading the mail I'd summarize that as 'Go interfaces are awesome and much better than C#/Java equivalents' while the little amount of understanding I have right now lends itself far better to 'Go interfaces are - well - just like any other interfaces out there. But! We can weave them magically (java: AOP?) and implicitly in at runtime'

Still a nice feat.

<comment><USER4> _"Just to be sure: You suggest to write interfaces ('extract' in my original post as that seems to be the refactoring name you'd use for this kind of stuff in my world) that define a contract for pre-existing characteristics of 3rd party types?"_

No, in Go you can write new methods for pre-existing 3rd party types that you do not have the source for. It has the feel of monkey patching, but there is no classes in Go. A method is just a function that takes objects of certain type as a parameter. Knowing an object's type does not tell you what actions can be performed on it. To know that, you want to know what interfaces the object implements.

This is a one way to solve the so called expression problem, which cannot be solved by simple use of classes and interfaces in a language like Java.

</comment></comment></comment></comment><comment><USER5> Go OO is like Java only they took stuff away. They took away class extension and they took away the requirement that you declare interface implentations. So if you have all of the functions required for an interface then you implement the interface, no need to be more complicated.

So it's hard to see why people from an OO background (assuming an OO background like Java or c++) would have a problem with it. It's like that only simpler.

<comment><USER6> I guess go interfaces are like c++ template implicit interfaces except the interfaces have names instead of just protocols. I think the reason that it can be difficult to grasp is because it isn't as explicit. Maybe OO programmers are afraid of the crazy gcc error messages you get when you incorrectly instantiate a template in c++ and think you might have the same difficulty. Whereas explicit interface implementation declarations give you better error reporting. I have very little go experience but when I first learned about go interfaces that is where my mind went.

</comment></comment><comment><USER7> I've seen a lot of criticism of the choice to use interfaces in Go, but I don't understand why. Can someone explain to me why some people seem to dislike them? From my point of view, all they are is effectively the static variant of duck-typing.

<comment><USER8> That's pretty much exactly it. Duck-typing in dynamic languages is just "I don't care what this _is_ as long as it doesn't break when I do stuff to it," and Go's interfaces are "I don't care what this _is_ as long as I can call these methods on it." It's basically just a way to duck-type and guarantee that the duck-typing will work at compile time.

I don't know why people dislike them, because the concept and the possibilities involved are quite impressive. (Combined with generics, they would be even more impressive...but that's another topic.)

<comment><USER9> "I don't know why people dislike them, because the concept and the possibilities involved are quite impressive. (Combined with generics, they would be even more impressive...but that's another topic.)"

Me neither, they seem great to me (same with optional typing in Dart). For me the big advantage isn't going to be so much the compile-time checks, instead its the documentation you get in the code from it (as in you can see that an argument is a shape and quickly find out what you can do with a shape).

Have to say I've done no more than play with either language though so there could be big real-world disadvantages that I'm missing...

</comment></comment></comment><comment><USER10> The way Rob is describing this reminds me of Haskell's Typeclasses, of which Go's interfaces are basically an ad-hoc informal version (sans default implementations) without parametric polymorphism (but that's another topic entirely).

<comment><USER11> They're similar in that they both allow a type to be associated with a supertype post-declaration, which is unusual in an OO language; they differ in that Go uses structural subtyping rather than nominal subtyping which is much more prevalent.

<comment><USER10> I had not heard the terms "structural subtyping" and "nominal subtyping" before, so thank you for that (Wikipedia was able to define them). And yes, I realize that Go's interfaces are implicit whereas Haskell's typeclasses are explicit.

I was thinking more along the lines that Haskell's typeclasses provide the exact same type of functionality that Rob is talking about wit Go's Interfaces, e.g. a type Foo can be a TopologicalGenus and a GrassGenus and a Square and an Area and a Symmetry all at the same time, by declaring the appropriate typeclass interfaces.

They're also somewhat similar from a usage standpoint, where in both Haskell and Go I can take a type Foo and declare methods on it that conform to a known type Bar (e.g. declaring an interface Bar Foo in Haskell, or simply declaring the methods from Bar in Go) and now my type Foo can be treated as that type Bar even if I don't control the source to Foo.

</comment></comment></comment><comment><USER12> In general I like the way Go interfaces work (we'd like to do something similar in Gosu as well); they let you decouple different parts of a system much better than you can if you have to explicitly declare interfaces.

When discussing putting something like this in Gosu, the main disadvantage we came up with was around tooling. Probably not a big issue for Go, given that I haven't really heard of too much focus on IDE/refactoring tooling there, but if one were to implement such tools, the implicit interfaces make things a bit harder. For example, if I have a Nameable interface with a getName() method, and 20 classes in my system have a getName() method but only 3 of them are actually used as Nameables, if I want to refactor the getName() method in a system like Java, I'll know which classes explicitly implement Nameable and thus which classes to refactor. In Go, it's a hard problem, because you have to know which classes are ever used as Nameables; that's no longer a property of the class itself, but rather something you have to derive from your code base. It similarly makes IDE functions like "find usages" or "find implementors" much harder to implement. (Technically they're not 100% intractable, but they're certainly much harder to do well.)

Those tooling issues are really the only drawbacks we could come up with, though. Otherwise I think the Go approach is much more flexible than the Java one.

<comment><USER13> wouldn't a bigger problem be that getName() may refer to getting a Person 's name or an enumeration of the platform or worse. ie. that you use the same names for unrelated functions which can happen and does make a certain amount of sense sometimes.

<comment><USER12> It's certainly a related problem: with Go-style implicit interfaces, neither a person nor a machine really has the context to know if something that conforms to the interface is actually intended to be used that way, or if it just happens to have the same method signatures. As you say, getName() is a pretty common sort of method name that could mean a bunch of different things, and not everything with a getName() method is necessarily interchangeable with everything else. Explicit interfaces require more work by the programmer, but make those intentions explicit to both other programmers and to automated tools.

In Gosu, we already have Java-style explicit interfaces, but we're likely going to add in Go-style implicit interfaces as a language option as well, since they're not completely interchangeable. The more I think about it, the more I think that you often want implicit interfaces for method arguments, and explicit interfaces for return values. That's oversimplifying, but as a library author you often want to say "this method works with anything that has foo() and bar() on it," so you want an implicit interface so people can use your library with existing code without changing that code to add explicit interface declarations. But if you're providing objects back as the result of a function, you might want to be more explicit and make harder guarantees about exactly what sorts of things they return.

</comment></comment></comment><comment><USER14> > Moreover, you don't have to work them all out ahead of time; the whole > design can evolve without invalidating early decisions.

That is a _very_ important point IMHO. In my experience, the Achilles heel of classical C++/Java-style OO is that you tend to lock yourself into a complicated design that is difficult to modify. This goes a long way toward accounting for the recent-ish surge in popularity of dynamic languages, duck typing, etc. Being able to apply a similar design mindset in a language with static typing sounds pretty cool. (I've looked into Go a little; I think I'll look into it some more.)

</comment><comment><USER15> Structural subtyping is not OO? Somewhere Luca Cardelli is scratching his head.

<comment><USER16> Maybe Go interfaces fit a theoretical formulation of OO, but this isn't the OO most people learned for Java or C++.

</comment></comment><comment><USER17> Go is a truly beautiful language; it really feels like a modern C. I hack Python and C++ for a living but would much rather hack go.</comment>


H-1B Visa Reform Bill - <AUTHOR> http://blogs.wsj.com/briefly/2015/11/16/h-1b-visa-reform-bill-at-a-glance/?mod=e2fb <comment><USER2> This is a good start, but it doesn't address the root of the problem with H-1B visas. The only way to fix the H-1B visa is to switch from a lottery system to a bidding system where employers bid up the wages of the H-1B employees.</comment>


Apple market share tops 10%, Windows share lowest since tracking began - <AUTHOR> http://www.tuaw.com/2009/01/02/apple-market-share-tops-10-windows-share-lowest-since-tracking/ <comment><USER3> If I were head of product development in Apple right now, what I would do is:

1\. Change the behaviour of Mac OS X to match Windows in terms of Window Minimizing, access to desktop, file sorting and so on. Simply, make the transition a lot more painless

2\. License out Mac OS X for specially designated high end machines by big vendors. For example, make a VAIO version of Mac OS X that works only with those laptop

3\. Expand the Apple App store to also become an OS X App store.

4\. Add a feature to the iPod and iPhone that only works with Mac OS X. For example, some type of 3D file manager that 'requires' OS X to work

5\. Offer some type of open source framework that allows virtualisation across platforms like the iPhone, Web and other OSes

<comment><USER4> I don't like how you're being downmodded for going against the grain - <USER2> post was cogent and appropriate, though I still disagree :)

1 - Mac users _like_ the behavior differences, and as one myself I can say there are many ways where Apple's format is superior to Windows. One of the major complaints for recent switchers is the lack of a "minimize" button, and IMHO this should not be "fixed". The lack of a full-screen button allows people to focus on sharing screen space between apps, which is something Apple does extraordinarily well and where Windows fails horribly. The inter-app cross-compatibility (allowing robust drag/drop features) is key to the Mac platform.

2 - No way. Apple makes most of their money off hardware, OS X is merely a selling point for their laptop and desktop products. Not to mention Apple is in a very good position to push hardware/software synergy like nobody else. I recently saw a demo of the Windows 7 multi-touch features, and the main thing that struck me is: "this is useless, the quality of touchscreen on 95% of tablets out there is not enough to make this useful". Apple is in a unique situation where their hardware can work precisely as well as they need, and vice versa. There's no waiting for either side to catch up like MS does.

3 - Agree, though not necessary. I wouldn't put much effort in it, but rather maybe build something that increases exposure for small-time app builders. App distribution on the Mac platform is already problem-free (unlike Windows, with is myriad of non-standard installers). This seems like a solution looking for a problem.

4 - That's going to piss a lot of people off - and I doubt anybody makes a $1500 purchasing decision based off a single "missing" feature.

Can't comment on #5, my brain is too fried :)

<comment><USER3> 1 - Mac users may like the difference, but windows users are not familiar with the difference, making it difficult for them to switch initially. It does not matter what is superior - if Mac were to minimize the small differences, it would be way easier for the users to switch. I'm not talking about what mac users are used to, I'm talking about making a switch completely painless. Windows users I've observed who switch to macs don't use this drag/drop thing like you think they do - they continue to use the osx like it were windows.

2 - Hardware costs money, software is free. This is why I say, if they allow Vaio produce hardware that is very similar to the Mac OS X hardware, they can quickly expand their market, while maintaining a high end brand. It's up to Sony then to make sure their hardware always fits.

3 - It's not problem free till I can click and icon on my mac select what I want and click buy.

4 - People will indeed make such a purchasing decision based off 1 feature. All other things balance themselves out, so one snazzy thing makes the world of difference.

<comment><USER5> _It does not matter what is superior_

It does to Apple users, and to Apple itself.

 _It's not problem free till I can click and icon on my mac select what I want and click buy._

The problem is that by doing that Apple would be shutting out other venues. Did you see my link to Apple.com/downloads? It does basically what you're asking for.

 _People will indeed make such a purchasing decision based off 1 feature. All other things balance themselves out, so one snazzy thing makes the world of difference._

What about the people purchasing the other way around? The people who'll buy a Zune instead of an iPod because it doesn't have the same crippled feature?

</comment></comment></comment></comment><comment><USER6> only started keeping stats 4 years ago, I would think since then any decline would be windows lowest; not to mentin the decline is less than 1 percent</comment>


The coming $2 trillion underfunded public pension crisis - <AUTHOR> http://volokh.com/2010/01/05/underfunded-public-pensions-as-stranded-costs-two-trillion-dollars/

<comment><USER2> The mere fact that it is legal for any pension to be underfunded is absolutely beyond me. But I can't say that I blame any government for doing such, most of the time they're essentially being extorted by public workers unions for pay and benefits that far exceed the market rate. What the heck is wrong with a traditional 401k contributions like the rest of America gets.

I know a guy who worked as a janitor, mopping floors at a state building for 10 years and because of this I'm going to end up paying him 25 to 30 grand a year, every year after he turns 65? Its outrageous that the state tax payers are being fleeced.

Eventually municipalities and states are going to go bankrupt and the pensioners are going to get pennies on the dollar.

<comment><USER3> The determination of whether a pension is underfunded is based on actuarial studies that, by necessity, must make assumptions about both the longevity of retirees (and survivors) and the return on invested funds. A fund which is fully funded under a 6% return assumption which then takes a 20% hit to its assets due to a sudden market downturn then becomes underfunded, and the sponsoring entity is required to make catch-up contributions. There is no difference between the public and private sectors in this regard. However, as stated by the author, the pension beneficiaries of a public system have much more powerful tools to force their interests to come first in the event of underfunding than do private pension beneficiaries.

There have also been cases of manipulation of the actuarial numbers, including a case in New York or New Jersey (can't remember which) where the actuary doing the pension study was also a paid consultant for the public sector unions. Not surprisingly, he concluded that significant benefit increases could be managed at the current funding level.</comment></comment>


Ask HN: How do you incorporate virtualisation into your dev workflow? - <AUTHOR>

I've always felt that I don't take enough advantage of Virtual machines for my dev process.
Nice sandboxed environments, testing on different operating systems, developing locally in production like environments etc.
I use Vagrant and get a lot of benefits from that, and I'm interesting in integrating similar tools  into my daily workflow.
Interested to hear your thoughts. <comment><USER2> "I use Vagrant and get a lot of benefits from that" \- I'm curious about that. What do you do specifically that benefits from Vagrant or VM's in general? I've never had the need (or opportunity) to have to use them.

Typically if I'm building a Rails website I'm using RVM and gemsets. An ASP.Net MVC website I'm using Windows 8 with Nuget.

So what does something like Vagrant solve?

</comment><comment><USER3> Mainly for testing.

But without fancy Vagrant/Chef stuff.

Just a bunch of VMs. Everyone has a mounted network directory with the test scripts and code. Jenkins runs the scripts from every VM and this is it.</comment>


In Star Trek, does the transporter conserve the momentum of transported objects? - <AUTHOR> http://scifi.stackexchange.com/q/39295/63

<comment><USER2> Depends on the writer.

</comment><comment><USER3> Afaik yes and no.

</comment><comment><USER4> Perhaps the Heisenberg compensators function in some fashion by stopping the momentum of the object being transported?</comment>