Google exploits loophole to help employees evade taxes on $36,500 / year - <AUTHOR> https://email@example.com/google-exploits-loophole-to-help-employees-evade-taxes-on-36-500-of-income-8fade7901434 <comment><USER2> I was sort of with the author until I got to the recommended policy change they want to see from legislators:

"We should abolish all tax advantaged savings accounts. If the $158 billion in savings were then distributed evenly, every household could receive $1,250 per year and companies like Google would no longer pay HR tax specialists to exploit our tax code."

The author appears to literally suggest we just take everyone's 401(k)s and IRAs for the year, give them away, and abolish the whole system. How is this a reasonable policy?

<comment><USER3> Tax reduced savings accounts effectively flatten the tax curve because whilst they are available to everyone only those being paid more can effectively make use of them.

On the other hand it does incentivice the higher paid to save more for the future rather than spending it now. This can save governments huge amounts when personal savings are used to pay for elderly care and medical treatment as opposed to the government.

</comment></comment><comment><USER4> These are important tools for saving for retirement, and have enabled many people to build wealth. I know people who never made a lot of money in any given year, but used these tools to save money over decades, and they now can enjoy a good retirement. Calling these things loopholes, tax avoidance, or what have you is misguided and counterproductive.

We already do not have pensions, taking away private pension options such as these would condemn many to not ever being able to be retired, or to poverty.

<comment><AUTHOR> Even if you don't abolish these, you could at least cap the total contribution (and even lift the annual contribution limit). A cap of $1.0 million per individual would allow for a safe withdrawal rate of approx. $40k / year (social security income would provide a further boost to that). You could move the cap higher or lower as you see fit.

There are over 150,000 individuals w/ a 401k balance over $1.0 million; there is little social benefit in providing millionaires with even more in a tax advantaged savings.

[https://www.investors.com/etfs-and- funds/retirement/retireme...](https://www.investors.com/etfs-and- funds/retirement/retirement-savings-more-americans-than-ever-have-1-million- or-more/)

<comment><USER5> $40k year is below the cost of living, or will be, in many regions. People also use 401ks for home loans, and for other lie events. I took a 401k loan to make the down payment on my first house.

Most high income truly rich people aren't using 401ks for retirement, it seems to me that once again, people are going after the professionals who are squeezed in the middle, who already pay high tax rates because most of their income is wages -- doctors, lawyers, engineers, and avoiding the top 0.1% and others who amass vast returns and mostly avoid income taxes.

Maybe we could try going after the big fish first before we start raiding retirement savings?</comment></comment></comment>


A Third of Wikipedia Discussions Are Stuck in Forever Beefs - <AUTHOR> https://motherboard.vice.com/en_us/article/3k9zjv/a-third-of-wikipedia-discussions-are-stuck-in-forever-beefs <comment><USER2> I'm in a forever beef with Wikipedia on the cheesesteak page re: "vegetarian imitation cheesesteak" (which the Wikipedia editors do not believe exists) and "vegetarian* cheesesteak" (which I maintain, by definition, cannot exist). I'm both proud and ashamed to admit this has been going on for several years now.

*edit: spelling</comment>


Now is a great time to be unfundable - <AUTHOR> http://bryce.vc/post/2151529046/now-is-a-great-time-to-be-unfundable <comment><USER2> I think what the author is talking about is having too much money to the point where a lot of it gets wasted before a solid business model is proven.

However, I still believe that seed funding is important to most startups for the simple fact that if you have to worry about paying for groceries and food, you're not going to be able to focus 100% on your startup.

A 30k - 150k seed round could go a long way for most startups and I think is almost required just to get the basics covered.

<comment><USER3> You have a good point. I voted you up. But I also have a counterpoint.

The most valuable asset you can build is a resilient founding team. Facing important problems while still building your product is good practice for what happens when a company grows. Not being able to focus 100% on all important aspects of your startup is a good situation to gain experience with.

Another upside to not having funding is it forces you to seek economical ways to grow your userbase. I'm focusing on this intensely now. There are a lot of things my users would like, but I'm picking the ones that help them show my service to others.

</comment></comment><comment><USER4> I speak from my startup's experience. We thought we were a lock for angel funding around 4 months ago. Finally, all the answers came back, "no". We believe it was because it's our first startup, and we did indeed look like the cardboard rocket ship to investors. They loved the cardboard rocket, but weren't about to give us a quarter of a million dollars to turn it into a real rocket.

We have since gone through 3 _major_ pivots (possibly 4, depending on how you split it). One of them was to consult in order to bootstrap -- something through which we've learned a lot about our customers. 2 of them were pretty big changes to our core business.

Interestingly, our original concept is still the long term goal. Hearing "no" from investors just made us realize that we need to build certain foundations before we can build our mansion of a company. This is likely a consequence of us being overly ambitious at the start of the project.

</comment><comment><USER5> Okay, am I the only one who's a bit tired of hearing that every rejection is a badge of honour and the like? I mean, okay, learn from failures and all that, but they aren't badges, they're things to avoid. There's no need to sugar-coat it, sometimes your idea just sucks or is unworkable. Take the feedback and do your best with it, but being proud of it is a bit of a stretch.

<comment><USER3> This one sounds different. It's by an investor, not a rejected entrepreneur.

<comment><USER5> I know, but still, it sounds a bit patronising. Sometimes you just fail and it amounts to nothing, just ignore it and go on to the next thing. I know it's hard to admit we've wasted time on something, but sometimes it just happens. Sure, you gain experience, but that doesn't mean it's better than if it had actually succeeded.

</comment></comment></comment><comment><USER6> hey all- thanks for taking an interest in the post, I've really enjoyed reading your feedback.

the impetus for this one was a series of conversations with entrepreneurs who said they were glad we hadn't funded them when they initially approached us. In retrospect, they felt that funding the wrong direction for them would have validated their bad idea and reenforced their belief that it was worth pursing. The push back ended up forcing them to really think about where to spend their time and limited resources.

The other conversations have been with entrepreneurs and founders expressing concern that its too easy to raise seed funding right now. That there's a growing sense of entitlement and arrogance particularly in the bay area. That half baked ideas aren't getting push back; rather, they're getting validated and compounding the entitlement problem.

No one knows how all of this is going to play out over the long aul. Just trying to lend a little perspective to those going through their own fundraising process.

</comment><comment><USER7> Well a 'no' is definitely a lot better than the silent treatment</comment>


Atlassian Launches the New JIRA Software Cloud - <AUTHOR> https://techcrunch.com/2018/10/18/atlassian-launches-the-new-jira-software-cloud/ <comment><USER2> "It's important to note that the next-gen experience will not replace our classic experience, which millions of users are happily using,"

"Happily" might be too strong a term here. :)</comment>


Not One Artist's Album Has Gone Platinum in 2014 - <AUTHOR> http://www.forbes.com/sites/hughmcintyre/2014/10/16/not-one-artists-album-has-gone-platinum-in-2014/

<comment><USER2> from what i understand, the benchmark measurements for things like this are based on units _shipped_ not necessarily _sold_. with that in mind, the context of the lede has an entirely different meaning.</comment>


Kill Your Dependencies - <AUTHOR> http://www.mikeperham.com/2016/02/09/kill-your-dependencies/ <comment><USER2> I disagree, and this quote I've seen floating around the internet sort of sums the idea up to me (albeit with a music analogy):

> _I thought using loops was cheating, so I programmed my own using samples. I > then thought using samples was cheating, so I recorded real drums. I then > thought that programming it was cheating, so I learned to play drums for > real. I then thought using bought drums was cheating, so I learned to make > my own. I then thought using premade skins was cheating, so I killed a goat > and skinned it. I then thought that that was cheating too, so I grew my own > goat from a baby goat. I also think that is cheating, but I'm not sure where > to go from here. I haven't made any music lately, what with the goat farming > and all._

<comment><USER3> I don't think that's what he's advocating: he's just saying that every dependency you have is another thing you have to worry about, so why not try to limit them as much as possible? Obviously it's impractical sometimes, and that's OK, as long as you understand the consequences.

<comment><USER4> Every dependency you have is another thing that people besides yourself can worry about with you or for you

Need to do X? If you write your own library that does X, chances are you'll be the only one to ever work on it. Need a new feature? You have to stop working on your actual project and implement that feature. Found a bug? No one else will fix it for you.

If you depend on a library that thousands of other people also use that does X, if you do find a feature you need that it doesn't have, open a ticket, someone will likely do that work for you. More often than not, that feature already exists, all you have to do is read the docs. If you find a bug, report it and and wait for it to get fixed, but there's also nothing stopping you from fixing it yourself and submitting a pull request.

<comment><USER5> A better way to view this is that if libraries are kept small, then you can pick a more granular set of dependencies than say a larger library that includes the kitchen sink.

I agree with most comments here that it's not a good idea to reinvent the wheel everywhere, but when all you want is a wheel, and not the entire car, it's ideal that we have a way to just include the wheel.

</comment></comment></comment></comment><comment><USER6> A large number of dependencies is only a problem in environments that aren't amenable to per-function static linking or tree-shaking. These include dynamically typed languages like Python, Ruby, and JavaScript (except when using the Google Closure Compiler in advanced mode), but also platforms like the JVM and .NET when reflection is allowed. Where static linking or tree- shaking is feasible, the run-time impact of bringing in a large library but only using a small part of it is no more than the impact of rewriting the small part you actually use.

Edit: Dart is an interesting case. It has dynamic typing, but it's static enough that tree-shaking is feasible. Seth Ladd's blog post about why tree- shaking is necessary [1] makes the same point that I'm making here.

[1]: [http://blog.sethladd.com/2013/01/minification-is-not- enough-...](http://blog.sethladd.com/2013/01/minification-is-not-enough-you- need.html)

<comment><USER7> Not always. Dependencies were a huge problem at Google, even in C++ (perhaps especially in C++), because they mean that the linker has to do a lot of extra work. And unlike compiling, linking can't be parallelized, since it has to produce one binary. At one point the webserver for Google Search grew big enough that the linker started running out of RAM on build machines, and then we had a big problem.

There's still no substitute for good code hygiene and knowing exactly what you're using and what additional bloat you're bringing in when you add a library.

<comment><USER8> That's a pretty significant special case though. I'd be willing to go with the advice "If you get as big as Google's codebase, be sure to trim the dependencies on your statically-bound languages too." But you probably have a ways to go before that's an engineering concern for your project.

(... note that one could make a similar argument for more runtime-dynamic languages. I won't disagree, other than to observe that as a lone engineer, I've managed to code myself into a corner with dependencies in Rails ;) ).

<comment><USER9> ^ THIS. I wish I had more up votes.

The amount of time I've seen wasted trying to scale to Google is insane. People should worry about what Google does when they work for at least a billion dollar company.

For most projects import as many dependencies as you can as you are getting free labour. Sure, once in a while you'll fuck something up and waste a week or two, but it pales in comparison to the months you didn't spend reinventing the wheel.

No one really ever notices that it's all the companies with boat loads of cash that have massive technical debt. Even with the example at Google the first thing I'd try is jamming more memory in those machines, keep going until the linker needs more than 256 GB.

Fuck, Facebook still uses PHP, the stock market doesn't seem to care.

</comment></comment></comment></comment><comment><USER10> As everything, I think a bit of balance is needed.

You're doing a quick MVP to demonstrate that your idea is working? Fuck it, just throw in dependencies for everything, just care about solving the problem you're trying to solve and proving/disproving your point.

Once you verified it, then go and kill your dependencies. But don't do it just because you want to do it. If in the end the users doesn't benefit from you optimizing your dependencies, why do it? (Speaking from a product side rather than a OSS project used by other projects)

Not sure KILL ALL DEPENDENCIES is helpful, but I'm not sure that MAKE EVERYTHING A DEPENDENCY is helpful either so...

<comment><USER11> That'd be good advice if those MVPs didn't so often become the actual product themselves. If industry and management understood that these things were proof of concepts, and realized that the actual product is going to have to be rewritten, then I'd agree with you.

</comment></comment><comment><USER12> To me, this seems more like an argument for optimizing beyond your own stack. Don't kill your own dependencies.

Your app uses to much memory? Improve a dependency, you have now improved other peoples apps, too.

Your app uses to much dependencies in total? Try to get all your first-level dependencies to standardize on the best http-client. (Which he is partially doing with his post.)

Dependencies may have problems, but shared problems are better than problems only you have.

<comment><USER13> I agree 100% with this.

I used to bring in dependencies with the "don't reinvent the wheel" mentality. Then I realized how much trust I'm giving to the authors of all dependencies I pull in. Now I tend to do my best to understand the dependencies I bring so I can improve them if I can.

The only problem I find with this decision is when I make an improvement/fix a bug on a dependency, and the project is either inactive or the authors don't give a crap about your work.

<comment><USER12>
        The only problem I find with this decision is when I make an improvement/fix a bug on a dependency, and the project is either inactive or the authors don't give a crap about your work.

True, but I think a temporary fork, which will eventually be merged back in, is still better than your own code with its own bugs.

</comment></comment></comment><comment><USER14> Another benefit to minimizing your dependencies is security. The less external packages you are using (especially packages without active, security-conscious maintainers) the less likely you are to suffer a surprise vulnerability due to something deep down in your dependency hierarchy.

This goes for client-side JavaScript too. XSS holes are one of the worst web app vulnerabilities out there and could easily be introduced accidentally by a simple mistake in a library. And this stuff is incredibly hard to audit these days thanks to the JavaScript community's cultural trend towards deeply nested dependencies.

<comment><USER15> but otoh, if you try to reinvent something instead of using a tried & true library, you might as well just add new bugs.

I.e. I'd 100% use libxml to sanitize xml rather than trying and reimplementing xml parsing myself.

As always, trade offs.

<comment><USER8> Yep.

OpenSSL has major security issues encountered on a relatively regular basis.

Do _not_ do your users the disservice of rolling your own SSL implementation. ;)

</comment></comment></comment><comment><USER16> I'm not 100% sure I agree with this as stated. Sure if the functionality is in core lib, use it but... it depends...

Consider these three statements:

\- No code runs faster than no code. \- No code has fewer bugs than no code. \- No code is easier to understand than no code.

For a language like scala where there is no json processing in the standard lib, if there is a json library that is battle tested, then by removing my own json code and leaning on that well tried and tested code for serialization/de- serialization, I've removed a whole bunch of code from my own library. The whole point of having modules as abstractions is to keep concerns neatly tucked in their own places to to increase re-use. By subscribing to the idea that my module should implement all of the functionality it needs, we're loosing the benefits of modularization.

I just went through this exercise myself in a library I maintain - I removed my own json code and put a library in. I removed a bunch of code and made the whole thing simpler by leaning on that abstraction.

<comment><USER17> > ... I've removed a whole bunch of code from my own library

You removed a bunch of code you understood, and added a bunch more code you don't understand, along with whatever technical debt, edge cases, and performance issues which are lingering in that library.

Adding a library is never removing code from your project, it's adding code you don't yet understand to your project. It can still be a net win, but it's not less code for you to maintain.

<comment><USER18> > > ... I've removed a whole bunch of code from my own library

> You removed a bunch of code you understood, and added a bunch more code you > don't understand, along with whatever technical debt, edge cases, and > performance issues which are lingering in that library.

Not all code you've written is good code. Hell, not all code you've written you actually understand. Libraries and dependencies make sense in many cases. Don't write yet another JSON parsing library unless you _really_ need to.

> Adding a library is never removing code from your project, it's adding code > you don't yet understand to your project. It can still be a net win, but > it's not less code for you to maintain.

It's referencing code that you don't maintain. If the maintainer is bad, use a different library.

</comment></comment></comment><comment><USER19> "The mime-types gem recently optimized its memory usage and saved megabytes of RAM. Literally every Rails app in existence can benefit from this optimization because Rails depends on the mime-types gem transitively: rails -> actionmailer -> mail -> mime-types."

It seems like this could also be cast as a major success for "semi" standard dependencies.

</comment><comment><USER20> This article would be more accurately written as "prefer the standard library over 3rd party solutions" since all the examples given still required dependencies, but ones that are shipped as part of the language runtime (Ruby in this case).

However when discussing languages with no specific standard library or languages who's standard library is missing feature _y_ , then it's quite understandable to use a 3rd party battle tested dependency. In fact I'd go further and say it would be advisable to use a respected 3rd party library when dealing with code which handles security or other complex concepts with high failure rates.

<comment><USER21> Matter of fact, sometimes it's better to be using a respectable 3rd party library. Requests vs. urllib2 in Python springs to mind, and I'm sure there's more examples.

</comment></comment><comment><USER22> Avoid shims.

There are lots of libraries that just put one interface on top of another interface. They don't do much actual work. Pulling in shims, especially if they pull in lots of other stuff you're not using, should be avoided.

If the dependency does real work you'd otherwise have to code, then use it.

<comment><USER23> _cough_ Mongoose. Been moving away from it on my projects. While it does provide a nice interface, it just creates more work down the road.

</comment></comment><comment><USER24> Perl apps have thousands upon thousands of dependencies. It's intentional - reused code in CPAN means less downloading, more efficient code, and less bugs as the codebase gets refined. An app that relies mostly on dependencies is essentially an app with free support by hundreds of developers. That's the case with CPAN anyway; I don't know how Ruby people do things.

Bugs happen, though. If you see a bug in a dependency, _it is your job to report it_ at the very least, if not make an attempt to fix it. Without this community of people helping to improve a common codebase, we'd all be writing everything from scratch, and progress would move a lot slower.

</comment><comment><USER25> This reminds of of this article

[http://www.joelonsoftware.com/articles/fog0000000007.html](http://www.joelonsoftware.com/articles/fog0000000007.html)

Apparently Microsoft's Excel team had even written their own C compiler.

</comment><comment><USER26> Obligatory essay from PHK on the effect the author describes:

[http://queue.acm.org/detail.cfm?id=2349257](http://queue.acm.org/detail.cfm?id=2349257)

History continues to repeat itself. Fake reuse and proliferation of unnecessary bloat are two of those recurring themes. Fight it whenever you can. The old TCL, LISP, Delphi, REBOL, etc clients and servers were tiny by modern standards. They still got the job done. Strip out bloat wherever you can. Also, standardize on one tool for each given job plus hide it behind a good interface to enable swapping it out if its own interface isn't great.

</comment><comment><USER27> Gems I use fall into three categories.

A lot of my projects are just wrappers around one main gem. Rails, Nokogiri, Roo, API wrapper gems. These are 'project gems'. If they give me problems, I'll re-evaluate the scope of the project and perhaps pick another gem to orient the project around. Once the project reaches maturity, I'll default to fixing the problem rather than re-engineering it unless the problems run deep.

Sometimes I'll use gems like Phoner to handle datatypes that are too tricky to do with regular Ruby. I'll call these 'utility gems'. When I include a utility gem, generally it has one job and one job only, it's invoked in exactly one place in the code and gets included in that file. I can generally replace a utility gem with stdlib Ruby code if I really need to.

I also have what I call 'infrastructure gems'. These are gems like pry, capistrano, and thor that I tend to include in every project where it seems they would be useful. These are gems that are worth getting to know very well because they solve really hard problems that you don't want to use stdlib for. If these give me problems I will do whatever I need to to resolve them and understand why the problem exists, because the costs of migrating off of them would be _steep_.

The decision to use a gem should not be taken too lightly, but nor should it weigh large on the mind. Be quick to try it out, but also quick to take it out.

</comment><comment><USER28> I was just thinking about this today. But from the point of view of growinga community around a platform!

Would you want to have one namespace for "official" modules and heavily influence everyone to use them? That's centralization (of governance). But, it's not centralization of a process that requires high availability. So the "drawback" is only that you centralize control and can make certain guarantees to developers on your platform.

When you're starting an ecosystem, you can choose a "main namespace" as yum, npm etc. does or you can choose the more "egalitarian" convention of "Vendor/product" as github and Composer do. I think, in the end, the latter leads to a lot more proliferation of crap, and as the articls said, multiple versions of everything existing side-by-side.

I have to deal with these issues when designing our company's platform ([http://qbix.com/platform](http://qbix.com/platform)) and I think that having a central namespace is good. The platform installer etc. will make it super easy to download and install the "official" plugins. You can distribute your own "custom" plugins but the preferred way to contribute to the community would be to check what's already there first and respect that. If you REALLY want to make an alternative to something, make it good enough that the community's admins protecting the namespace will allow it into the namespace. Otherwise, promote it yourself, or fork the whole platform.

</comment><comment><USER29> This is a great read that can be applied to node.js very much. I've seen apps that include 10, maybe 20 dependencies but when you flatten out the full dependency tree? Thousands. It's incredible and if one of those dependencies screws up semantic versioning or just screws up in general it can be a nightmare to debug and fix.

This is why every 1.0 product I work on I include every dependency that speeds up my development. In 2.0 the first things to do is prune all unnecessary dependencies and start minor rewrites when a dependency can be done in house (yeah yeah reinventing the wheel is a problem but most npm dependencies are small and many can be recreated internally without issue).

This is even more important if you're creating a library / module. My msngr.js library uses zero dependencies and yet can make http calls in node and the browser because it was easy to implement the minimal solution I needed without bringing in dependencies to support a single way of calling http.

<comment><USER30> The worst offender, IMO, is request. I've seen more than a few projects pull it in just to make a single HTTP call. Just look at its package.json:

[https://github.com/request/request/blob/master/package.json](https://github.com/request/request/blob/master/package.json)

<comment><USER31> Yes, the npm "request" module is extremely bloated and badly written, to boot. [https://www.npmjs.com/package/needle](https://www.npmjs.com/package/needle) is a good alternative.

<comment><USER32> Can you elaborate? How do I know Needle is any better?

</comment></comment></comment></comment><comment> yoz-y > No code runs faster than no code. > No code has fewer bugs than no code. > > No code uses less memory than no code. > No code is easier to understand > than no code.

The dependencies you decide to implement yourself in a minimal fashion are code though. I generally agree with the article, but in the end It Depends™

<comment><USER33> And are generally worse tested, worse supported, and you have to maintain it on your own

</comment></comment><comment><USER34> It sounds like this is advocating NIH syndrome. If a library is going to make my job easier I'm going to use it, unless there is a very specific benefit of doing it myself.

</comment><comment><USER35> Perl takes a pragmatic take on this (as well as other takes...) with the collection of ::Tiny CPAN modules that just do one thing pretty OK. Things like Try::Tiny that help immensely with exception handling - something you don't want to really roll you own.

It itself does not have any dependencies that aren't in core:

[http://deps.cpantesters.org/?module=Try%3A%3ATiny;perl=lates...](http://deps.cpantesters.org/?module=Try%3A%3ATiny;perl=latest)

</comment><comment><USER36> So, I've been writing a home automation system using the .NET Framework (with Visual Basic, I'll wait until you finish laughing).......... Okay.

I've made a point not to add any third parties references and packages I can avoid. I went ahead and got a third party scheduling engine, and the SQLite provider, but beyond that, I'm writing everything else myself so far.

First of all, I'm learning a lot in having to write stuff myself. At the very least, it's a great educational experience. I've worked with a lot of code samples, so I'm not going totally from scratch, but they're all at the very least tailored to my needs.

But for me, the big thing is keeping everything thin. The program loads in milliseconds. Almost all of the reference data for what it's built on is in one place (the .NET Framework Reference). And key, is that the features my program supports are the features I want and need, not the features some dependency has told me to have.

The biggest dependency I have, Quartz.NET, is actually the most confusing part. It's not structured like the rest of my program is, it's documentation leaves some things to be desired, and it does a lot more than I need it to. There's a lot of bloat I could cut out if I wrote my own scheduler, and maybe someday I will.

</comment><comment><USER37> Double edge sword. Deps are _great_! Functionality added quickly. Deps are _terrible_! They broke my app.

If your app has a long shelf time, the less deps you rely on, the easier to manage from what I've seen.

For some reason Golang _feels_ like it makes sense here. Pretty much everything you need is in core. *Disclaimer, I don't have any Golang apps in prod but I'd love to hear from those that do.

</comment><comment><USER38> A lot of apps (old-timey Windows apps, for example) have this philosophy, leading them to reinvent things like crypto and image decoding. Naturally, this leads to tons of bugs, including security bugs.

I would revise this to: Don't bring in more code than you need. But if the choice is between writing something yourself and using someone else's well- tested, heavily-used library, always go for the latter.

<comment><USER39> As an architect, you need to be able to do a cost/benefit analysis of each option. That is what software architects do, why they have experience. For example:

      How much time will it take to implement each option?
      How much time will it take in the future to support it?
      What security risk does each option incur?
      What is the risk of the project being abandoned?
      What is the risk of the project changing in non-backwards compatible ways?
      What are the performance characteristics of each option?

NIH is a disease, but so is import-mania. With experience, you can make a good decision.

<comment><USER40> One thing I've gotten into the habit of doing is looking around the commit history and issue list for any package I import. Was it something somebody wrote in a hurry and hasn't really touched since? Is it something that has a solid set of regular contributors? Are there a lot of outstanding issues relative to how heavily used it is?

I also spend more time actually reading through specs to see how well they exercise the code.

That's probably standard procedure for a lot of people, but it's something that I had to learn to always do.

<comment><USER39> That's a good idea

</comment></comment></comment></comment><comment><USER41> Your programs shouldn't do things you do not understand. You do not have to be an _expert_ in cryptography, memory allocation or b-trees, etc, but if this is what your app requires, then you should take the time to read up on it and carefully research what is out there if you suspect it is beyond your abilities to implement.

If you take the time to do your research, the choice between rolling your own, copying or adding a dependency will become clear. If it's not becoming clear, then you haven't finished your homework. Learning is a good thing, yes it takes time, but it's time well spent, and it's _fun_ above all.

You may discover that this thing that you thought was hard and needed a dependency is really a few lines of code (a good example is a graph implementation). It might even change your career path. At least that's been my experience in the nearly two decades of writing software.

</comment><comment><USER8> Incidentally, though the author says this can apply to any ecosystem, finding it applies to Ruby too often is what pushed me out of developing Rails apps. At least at the time I was using it heavily, the Rails space just wasn't stable enough to trust that I wouldn't have to learn an entirely new wheel to get my work done every time I went in to fix a relatively small problem.

"Can I implement the required minimial functionality myself? Own it" is advice one gives if one can't trust the libraries one depends upon to stay healthy, performant, and applicable to your use-case. Nobody'd recommend re- implementing readline or printf; if you have some heavy-lifting mathematics to do in Python, use numpy.

</comment><comment><USER42> How long did it take anyone to realize this nightmare? Since we are on the path to major discoveries, let's talk about runtime, runtime-dependencies and all that. Every Ruby app ever created is stuck somewhere on the time axis, before its origin.

</comment><comment><USER43> Your app/library inherits the technical debt of all its dependencies.

There's a natural tension between code reuse and avoiding dependencies. If you can avoid a big dependency by writing a couple hundred lines of low- maintenance code, its probably worth it.

</comment><comment><USER44> I find this argument sort related to the framework vs library and opinionated vs agnostic.

Being an old fart Java developer I generally prefer things where you can plugin your own implementation (ie agnostic).

That is there is an extreme for killing your dependencies of either extreme copy'npaste OR which every library offers a plugin SPI (ie inversion of control) (or a combo of both).

The problem with the dependency injection above approach (aka Spring prior to Boot) is that you have developers doing lots of custom crap, bloated/overly engineered libraries, increased ramp up time, and configuration hell.

But I still think this is probably better than ole copy'n paste.. most of the time. I do hate dependencies though.

<comment><USER6> You're probably more experienced in java development than me. But maybe a good rule of thumb is that most Java libraries shouldn't use reflection or dynamically loaded classes (i.e. using Class.forName or ClassLoader). That rules out most dependency injection frameworks.

<comment><USER44> That is correct that the libraries should not have DI but I should be able to wire up the library on my own and not let the library do its own static initialization. What is far worse than Class.forName and other crap is libraries self imposed singletons.

Take for example Hystrix. I'm just now fixing that the thing loads up its own configuration framework (Archaius) which uses static initialization. Archaius needed like 10 other dependencies. This is all really because Hystrix uses static singleton (HystrixPlugins) and many frameworks need this or else is incredibly difficult to get an implementation up (ie using pseudo singleton to avoid excessive passing of a context).

[https://github.com/Netflix/Hystrix/pull/1083](https://github.com/Netflix/Hystrix/pull/1083)

</comment></comment></comment><comment><USER45> One issue I had with Ruby on Rails was getting MySQL drivers to even cooperate on both Windows and Linux. At the end of the day I wound up sticking to Python and other languages instead. I don't mind using any language, but if the language is fighting me due to native dependency hell then I can't really do much. Even to just use SQLite was a bit of a painful experience, yet on Python SQLite works out of the box without any effort on my part (on Windows). Oddly enough.

I'm looking to getting back into Ruby at some point later this year, but I might ignore Rails altogether so I don't miss out on learning a new fun language.

</comment><comment><USER46> This is not only an issue for running software but also a huge issue for compiling/building software. Each dependency adds the potential to break your builds in new ways. As the software your program depends on evolves the risk increases that it will change the way your program executes or cause it to fail to build. Many devs will insist you do not reinvent the wheel by writing things like JSON parsers but you always have to weigh the cost of adding a dependency. It's not free.

</comment><comment><USER47> This reminds me of an example I ran into yesterday. I haven't used webpack yet but I saw a question on SO of someone wanting to use some package called glslify. I thought I'd take a look and maybe learn webpage in the process.

From the description all glslify does is look for files with the extensions .glsl, .frag, and .vert and lets you get their contents with `content = require(filename)`.

Sounds like it would be at most 10-30 lines of code. Nope

        npm install --save glslify-loader
        email@example.com /Users/gregg/temp/webpack-glsl-test
        └─┬ firstname.lastname@example.com
          └─┬ email@example.com
            ├─┬ firstname.lastname@example.com
            │ └─┬ email@example.com
            │   ├── email@example.com
            │   ├── firstname.lastname@example.com
            │   └── firstname.lastname@example.com
            ├─┬ email@example.com
            │ ├── email@example.com
            │ └── email@example.com
            ├─┬ email@example.com
            │ ├─┬ email@example.com
            │ │ └── firstname.lastname@example.com
            │ ├── firstname.lastname@example.com
            │ ├── email@example.com
            │ ├─┬ email@example.com
            │ │ ├── email@example.com
            │ │ └── email@example.com
            │ ├── email@example.com
            │ ├── firstname.lastname@example.com
            │ └── firstname.lastname@example.com
            ├─┬ email@example.com
            │ ├── firstname.lastname@example.com
            │ ├─┬ firstname.lastname@example.com
            │ │ ├── firstname.lastname@example.com
            │ │ └── email@example.com
            │ ├── email@example.com
            │ ├── email@example.com
            │ └─┬ firstname.lastname@example.com
            │   └─┬ firstname.lastname@example.com
            │     └── firstname.lastname@example.com
            ├── email@example.com
            ├── email@example.com
            ├─┬ firstname.lastname@example.com
            │ ├─┬ email@example.com
            │ │ ├── email@example.com
            │ │ └── email@example.com
            │ ├─┬ email@example.com
            │ │ └── firstname.lastname@example.com
            │ ├─┬ firstname.lastname@example.com
            │ │ ├── firstname.lastname@example.com
            │ │ ├── email@example.com
            │ │ ├── email@example.com
            │ │ └─┬ firstname.lastname@example.com
            │ │   └── firstname.lastname@example.com
            │ ├─┬ email@example.com
            │ │ ├── email@example.com
            │ │ ├── email@example.com
            │ │ └── email@example.com
            │ ├─┬ email@example.com
            │ │ └── firstname.lastname@example.com
            │ ├── firstname.lastname@example.com
            │ ├─┬ email@example.com
            │ │ ├── firstname.lastname@example.com
            │ │ └─┬ firstname.lastname@example.com
            │ │   └─┬ email@example.com
            │ │     └── email@example.com
            │ ├── firstname.lastname@example.com
            │ ├─┬ firstname.lastname@example.com
            │ │ └─┬ firstname.lastname@example.com
            │ │   ├── firstname.lastname@example.com
            │ │   └── email@example.com
            │ └─┬ firstname.lastname@example.com
            │   └─┬ firstname.lastname@example.com
            │     └── firstname.lastname@example.com
            ├── firstname.lastname@example.com
            └── email@example.com

> 4 meg of source files

\---

update: I think maybe I misunderstood the description. glslify actually parses GLSL and re-writes it in various ways so maybe this is a bad example.

I've seen other though. Like 40k+ lines of deps for an ANSI color library or 200k+ lines deps and native node plugins for launching a browser from node.

<comment><USER48> NodeJS developers ought to be embarrassed at how absurdly huge their dependency trees are.

<comment><USER49> I think they are a proud bunch.

</comment></comment></comment><comment><USER50> Slightly related, I was packaging up a webapp in docker, and one thing the application did was form-fill PDFs. I had been using pdftk to do this, but it turns out pdftk is written in gcj, and gcj pulls in a _lot_ for its runtime libraries. I wrote a small program using the mupdf libraries and cut the size of my docker image by over 400MB.

<comment><USER51> Congrats, you just violated the GPL. You were already violating the proprietary license if you're using his for commercial purposes without paying.

Not all free software is free.

<comment><USER50> I just saw the commercial redistribution clause for pdfTK if that's what you're talking about. It does not affect me since this was not a commercial application, but it would seem to me that that clause itself is a violation of the GPL, since pdfTK links to GPL software and it is also a contradiction of a separate place on the site that claims the same software is licensed under the GPLv2. Preventing commercial redistribution is not compatible with the GPL.

</comment></comment></comment><comment><USER52> There is a counter example of his reasoning in the Python world. There is a HTTP client library "urllib" in the standard library, but nowadays everyone rather pulls in the external dependency "requests" because the urllib API is terrible. It is mature, well tested, good documented code though.

<comment><USER53> just as a data point: _I_ don't use request, I onestly prefer the urrlib (was urllib2) API offered by the standard library

</comment></comment><comment><USER54> Looks like the mime-types upgrade has some sort of hard dependency on Rails 5? I seem to be stuck on 2.99

<comment><USER55> Its the other way around actually: ActionMailer 4.2 depends on mail ~> 2.5, >= 2.5.4, so you'll get 2.6.3 now. That version depends on mime-types <3, >= 1.16, so you'll get 2.99.

The big change in mime-types 3 is using the columnar store by default, which is where the memory savings come from. It's opt-in from mime-types 2.6 onwards because it's a breaking change. Mail and afaik most other gems have opted in already.

</comment></comment><comment><USER55> Excellent article. I tried to develop on GitLab once but the sheer amount of gems it pulls in (~100 directly declared, 350+ including dependencies if I remember correctly) with a bunch of installation problems made me decide it was not worth the hassle.

<comment><USER56> I'm sorry to hear you had installation problems trying to develop for GitLab. Have you tried the GitLab development kit [https://gitlab.com/gitlab- org/gitlab-development-kit](https://gitlab.com/gitlab-org/gitlab-development- kit)? If still are interested and experience problems please email firstname.lastname@example.com and reference this comment for help.

I agree with the article, the less dependencies the better. GitLab's gemfile.lock [https://gitlab.com/gitlab-org/gitlab- ce/blob/master/Gemfile....](https://gitlab.com/gitlab-org/gitlab- ce/blob/master/Gemfile.lock) has over 1000 lines and GitLab uses a lot of memory. We try to be careful what we pull in but if anyone has suggestions which can be removed please let us know. Recently we found out that we still had to remove Redcloth as a dependency, it will be gone in GitLab 8.5.

</comment></comment><comment><USER57> If there is an analogue in the standard library you should have a compelling reason to use an alternative. Wish this could be filed under "common" sense. Thanks for articulating and presenting this principle, among others. Great writeup.

</comment><comment><USER58> if you're running multiple rails processes on a server like this, couldn't you somehow do the initialization in one process, then fork off the new processes? wouldn't that prevent the base libraries from being copied in memory?

<comment><USER59> Yes, [http://unicorn.bogomips.org/](http://unicorn.bogomips.org/) popularized this for ruby / rack / rails with its forking model and preload_app option. [http://puma.io/](http://puma.io/) does the same thing, but additionally runs multiple threads in each process.

The garbage collector in Ruby 1.8 / 1.9 negated the benefits of copy-on-write forking, but that's fixed since Ruby 2.0

</comment></comment><comment><USER60> This is a huge mistake if applied without care. Building things from scratch necessarily will introduce more bugs, more maintenance costs and leave you with a codebase that suffers from a lack of maturity.

<comment><USER61> _Building things from scratch necessarily will introduce more bugs, more maintenance costs and leave you with a codebase that suffers from a lack of maturity._

Unfortunately, in some programming language ecosystems where having many small and transitive dependencies on modules from an non-curated repository is common, none of those three things is necessarily true.

Code reuse is not a trivial problem, and you always have to weigh the benefits against the costs and risks to decide whether it's worth it. If we're depending on GitHub repositories with a dozen files and three subdirectories just to provide some simple functionality that any junior programmer could implement directly in five lines of code, we've probably lost the plot. On the other hand, if we have a full in-house implementation of encryption algorithms we use to throw sensitive customer data around between the browser and our servers, we've also probably lost the plot.

</comment></comment><comment><USER62> I remember one nasty bug where someone has included one function from bootstrap.js library and someone else had included the entire library. So both functions were running causing an issue.

</comment><comment><USER63> This is not just true for Ruby but also for the entire npm ecosystem.

I wonder how much traffic could be saved by optimizing npm packages... probably on terabyte scale at github alone, methinks.

</comment><comment><USER64> Yes, this is the sort of thing that scares me away from Ruby.

I'm worried this sort of "screw it just add a library" is going to spread further in my language of choice: Java.

In my time doing open source programming on the side, I've found that it has become more common with the advent of things like mvn and gradle to just slather on layers to your stack even for the simple tasks.

Need a function to turn a byte buffer into a string? Download these 3 Apache commons libraries and their dependencies.

I understand if you are relying on a large portion of a library and you need to use it, but why bring an entire library in for one function.

<comment><USER65> There are ideas floating around that make it appealing to do just that. For example, the commons library might be considered "battle-tested," and who really knows what could happen with your own custom byte-buffer-to-string function? Maybe you missed something? Maybe there is some "best practice" that you didn't follow? Maybe the commons library is optimized? And writing your own thing doesn't add business value. Developer time is more expensive than dependencies. And so on.

Me, I very often prefer to write things myself, in a way that can get labelled as NIH. My inclination is based on bad experiences with trying to debug external libraries. Sometimes I look at open source library code and find staggering complexity that I have no need for. Yes, maybe the library is great, but if its combinatorial size is 10,000 times the functionality that we need, then depending on its correctness becomes scary to me. And when I need to customize it, due to some requirements alteration, I will find it difficult and tedious.

Black-box type libraries for isolated complicated tasks like codecs and crypto I will happily use.

Otherwise, I'm a fan of the "design patterns" approach to reuse, which is all about learning from others, but without creating reusable formal abstractions in library form. So if you teach me how to write an URL router, I can then use your insights without depending on your code base, and I can adapt the idea so it fits my application perfectly.

<comment><USER66> Urgh. I find design patterns the worst approach. It's just copy-paste at a slightly higher level. If you really understand a pattern, you should be able to express it formally - i.e. as code.

<comment><USER65> Richard P. Gabriel's book _Patterns of Software_ has a chapter about that. (It's free and out of print.)

Bluntly, it's kind of like: if you really understand a style of house building, you should be able to deliver it as a prefab. Maybe true in some way, but also neglects the drawbacks of standardized components.

</comment></comment></comment></comment><comment><USER67> Interesting to note that the Stripe gem removed one of its dependencies seemingly in reaction to being called out at the end of this article.

</comment><comment><USER68> This should be one of the perks of Go since the compiler won't let you include anything that you aren't using.

<comment><USER69> That's an orthogonal issue. You won't accidentally bring in something completely unrelated because you forgot to remove it from the "include"s, but nothing technically stops you from having a deep dependency chain. Culturally the Go community is aware of the issue, though.

Still, it isn't hard to bring in a chain accidentally. I have a program than needs to do a query against the local LDAP system to extract members of a specified group. The LDAP library brings in five more libraries for parsing all the various bits of LDAP. Since this isn't C, I'm a bit less nervous about pulling in, say, a BER decoding library, because at least Go is generally memory-safe, but, still, that's a somewhat large stack for such a simple query. (Traditionally in C, you might as well just expect any library that decodes anything remotely binary-esque will have buffer overflows. C is a DSL for writing buffer overflows.)

And yet, I'd be insane to try to implement some sort of just-barely-minimal LDAP client to do it myself.

Looking at my local godoc instance's full set of packages that have gotten pulled in one way or another is still sort of intimidating. Some of them are cases where I'm just pulling in a subdir and got an entire large repo (the golang experimental repos do that a lot), but, still, I've got a lot of stuff in there. If you're a go programmer and you haven't run godoc locally and had a look at the packages page, have a look. You may be surprised.

</comment></comment><comment><USER70> _Does any of this sound familiar:_

 _\- Test gems loading in production._

That does not sound familiar! Is this a thing which happens with Rails?

<comment><USER71> No, but sometimes people are a bit careless in their Gemfile I guess.

</comment></comment><comment><USER72> mojolicious does a great job with this, supporting optional dependencies as progressive enhancement (installing EV will speed you up, but you don't necessarily need it)

[http://mojolicious.org/](http://mojolicious.org/)

</comment><comment><USER73> I have one thing to say about all of this...

 _Nokogiri_.

</comment><comment><USER74> They should sell sonatype for this.

</comment><comment><USER75> amen

</comment><comment><USER76> I find dependencies to be a very good indicator for how my code should be modularized. That is, rather than pulling a boatload of dependencies into "the application", pull a couple dependencies into a module, and then depend on the module. It makes it very easy for dependencies to be a "well, it gets the job done for now, and I can reimplement that myself if that changes" sort of thing.

<comment><USER77> I found this approach to work well for me as well. It has payed off many times. I try to wrap most of my dependencies so that if I later feel like I need to pick some low-hanging fruit I can implement some of the functionality internally while maintaining the original api.

<comment><USER76> Also, this is probably the single biggest difference for me when working with a static vs. dynamically typed language. With Static Typing, there's a "translate the dependency's types into the application's types" step that pretty much screams "put a seam here!". With Dynamic Typing, it's a bit less loud.

<comment><USER78> That's pretty much a C/C++ problem though. I don't think I've ever had to translate types in C# or F#.

<comment><USER76> I was referring to that, with Scala, I like to avoid leaking the innards of a JSON serialization library in the code for an API client, instead returning domain objects. Whereas with, JS or Python, the initial approach is to sling around the blob of JSON.

</comment></comment></comment></comment></comment><comment><USER79> So to summarize, to get rid of an extra 10 (or even 100) megs in Ruby (or 0 megs in some other languages) of memory usage (and disk usage don't forget!) spend weeks rewriting, testing, and integrating your own code instead of using already written, tested, and integrated code. Now, that I've clarified the article's point, how can anyone not follow this "best-practices" advice? </sarcasm>

</comment><comment><USER80> "Oh, I thought you said 'dependents'" — Abraham

</comment><comment><USER81> what's the point? none of them are realistic.

"no code" \- well, it's there for a reason. "own it" \- do i really want to write my own minimal implementation?

i understand that dependency is a pita but this post doesn't provide anything worthwhile.

</comment><comment><USER82> The problem with dependencies is that developers approach it from a top-down approach. The question they answer is: I need an HTTP client, JSON API, monitoring tool, logging framework.

Never do they ask the opposite: what kind of foundations do I need? What elementary blocks do I need to have or learn in order to make a JSON parser in 5 lines of code? Is it possible to do logging without all the cruft? Can I write the library in the same amount of time as I can read the docs? Could the code I write be the docs?

Similar line of reasoning: can I leverage my OS to do scheduling/IPC/monitoring/security? If it can't, should we lobby for better OSes (that might scale over multiple machines?) Does Linux/Docker offer the right fundamentals?

Dijkstra was truly right: the art of programming is the art of managing complexity.</comment>


New York Times Study Calls for Rapid Change in Newsroom - <AUTHOR> https://www.nytimes.com/2017/01/17/business/new-york-times-newsroom-report-2020.html?smid=tw-nytimes&smtyp=cur&_r=0 <comment><USER2> To be honest, I never appreciated the scale of the NYT until I read those revenue numbers. $500 million in digital media revenue is nothing to sneeze at.

I wonder if at some point in the future, there will be a reaction to this push for a subscription-based news model. Perhaps a website who's value proposition is that it only focuses on news, with no clickbait articles and titles.

<comment><USER3> I think the future is micro payments. I don't want to pay for a magazine, I want to pay for quality articles, wherever they come from.

<comment><USER4> In this future, how do you determine quality articles before reading them?

<comment><USER3> Quality of the publication/author, recomendations, and relevance of the subject matter. It's a proven model: it's the same way I determine quality movies or books before I pay for them.

</comment></comment></comment></comment><comment><USER5> The most important area to focus on is accuracy. There's a huge prevailence of rumors masquerading as news. Plus, there's an entire political party who's base has lost faith in the mainstream media. If the NYT starts taking shortcuts, they will fall into a dangerous trap and will lose a lot of readers.

<comment><USER6> What do you even mean? NYT has a price sheet for sponsered content/native ads. If you have the funds, you can put any "fake news" you want in the NYT. Much more up front about it than back in the day.

All "news" is fake to some degree or another, it is a fantasy to think otherwise.

<comment><USER7> What a load of garbage. If you're talking about what I think you are, you're mentioning those half/full page ads that are marked "advertisement" and have slightly different fonts than the normal news. It's hardly a conspiracy theory to allow Trump to buy reporters.

</comment></comment></comment><comment><USER8> The internal report itself is interesting for a few reasons:

[https://www.nytimes.com/projects/2020-report/](https://www.nytimes.com/projects/2020-report/)

1) The graphic on top doesn't show a laptop or desktop.

2) " On a per-dollar basis, our freelance-written journalism attracts a larger audience on average than our staff-written journalism."

Wow. Hopefully, they have automated internal metrics to track this.

3) " Today, department heads and other coverage leaders must organize much of their day around print rhythms even as they find themselves gravitating toward digital journalism. The current setup is holding back our ability to make further digital changes, and it is also starting to rob the print newspaper of the attention it needs to become even better."

Until the focus is changed to digital first, I am unsure how much more they can grow.

I still marvel at how it is cheaper for me to get a NY Times Sunday paper delivery with digital access halfway across the country than it is for me to get just the digital access. lol

<comment><USER9> Re: 2, Does the freelance-written material attract more readers, cost less, or both? The statements in the report are ambiguous.

<comment><USER10> If you drop all staff writers, what's your differentiator (I guess, maybe in editorial)? Assuming what they're saying is true, could staff writers be a loss-leader to keep that brand and core audience used to reach the wider readers?

</comment></comment></comment><comment><USER11> >The report coincides with a series of broader changes at The Times, including a reimagining of the print newspaper; an aggressive international expansion; a heightened emphasis on graphics, video, virtual reality and podcasts; the $30-million purchase...

God, please no. Don't invest heavily in VR for news reporting. I agree with reducing multiple layers of copy editing for a digitally focused company, but VR is a misdirection of funds to get the facts straight. Or, maybe that was just a buzzword that didn't get edited out of the final press release.

<comment><USER12> An investment in VR might be premature because of the immaturity of the technology, but assuming that VR tech does grow to be commonplace, wouldn't e.g. immersive VR video shot in a warzone be more affecting than still images or 2D video when communicating the scale of atrocity/warfare/protests to the viewers back home?

<comment><USER13> That's actually one of their current VR stories. "The Fight for Falluja": [https://www.nytimes.com/interactive/2016/08/14/magazine/figh...](https://www.nytimes.com/interactive/2016/08/14/magazine/fight- for-falluja-vr.html)

Personally, I used the cardboard VR thing with my iPhone 6 to watch a short during the election. I was underwhelmed by the experience.

</comment></comment></comment><comment><USER14> This is anecdotal, but just today I cancelled my print subscription to the NYTimes because they kept messing up the delivery (not showing up, going to the wrong house)... and every time I contacted their customer service reps (who were obviously outsourced) they had absolutely no power to do anything about it.

I wonder how much the simple things like "taking care of your customers" factors into their business struggles. (Probably "not much", but just seemed strange to me that they'd drop the ball like that for honest-to-goodness paying customers).

<comment><USER15> > _but just seemed so strange to me that they 'd drop the ball so much on the > kind of customer they should be very much desiring_

Don't get me wrong, I'd have done the same, but:

What makes you think you're such a desirable customer? What sort of effort did _you_ put into solving the problem? I can easily imagine a scenario where the delivery guy is so far removed form the news business (doesn't care a wink), that it would be night impossible to ensure the paper "gets there" without incurring costs beyond your subscription fee and satisfaction. This is reality.

<comment><USER14> I was a desirable customer because I was paying for their most expensive offering (delivery of the print paper, as opposed to just digital access).

And I'm baffled by the rest of your response... What sort of effort am _I_ supposed to put into solving _their_ logistics problem?? Other than repeatedly telling their customer service people about the issue, how or why would I investigate any further?

(Honestly I would have loved to get in touch with the actual delivery company in my area to ask them if they have the wrong address or can't find the house or something... but this is one of the many ways in which the customer service reps could not help me because they either didn't know or wouldn't tell me who the delivery company was!)

I think perhaps you misunderstood what I wrote or think I'm actually a delivery person?

</comment></comment></comment><comment><USER16> This report started well before the real meat of the recent election and I think it really shows some lack of appreciation of what's become pretty clear in the last few months, is a rather spectacular failure of "the Web" and the ridiculously over-hyped "digital journalism" as means of providing people realistic information about people, events, policies and other things that fall under the umbrella of "news."

As much as many of us have maligned newspapers as "legacy," "dead tree," or "rags," or as people that just don't "get" technology, all we have built, from news perspective, is a world wide network spreading silos of disinformation at the rate of terabytes of nonsense per day.

I think if anything, for me, we have seen that the legacy dead tree rags are far more valuable to an individual, a nation, and really the world, than blog engines and social networks. Newspapers, as with anything, have their faults but are much more preferable to "choose your own propaganda" aggregators or Alex Jones meme factories.

So, please, don't give the "visual storytellers" a "primary role." Let's get to real investigative journalism and journalism itself. It's not the medium; it's the content.

</comment><comment><USER17> My take away from this article is that they acknowledge that a lot of trouble lies ahead. Ironically, they don't have the courage to state their current state in a more factual manner. Instead they offer some vague statements and occasionally some outdated PR-speak

> "The world is changing really rapidly," David Leonhardt, a columnist who led > the group's work, said in an interview. "We have to keep up, and even get > ahead of it."

> In a note to the newsroom, Mr. Baquet and Joe Kahn, The Times's managing > editor, endorsed the group's recommendations, saying they outlined an > "opportunity we have to produce an even more vital, more authoritative, more > indispensable[0]'' news report.

Wouldn't it be a great ad for their product if they were demonstrating that it is possible for the Times to even report about themselves in a blunt, factual manner? While some admissions are there the overall article is quite humble- bragish.

[0] more indispensable - what does that even mean?

<comment><USER18> The Times is one of the most successful papers at weathering the digital transition. As more people get their news online it's the handful of mega- papers that profit at the expense of local and regional ones.

If you want to read gloom-and-doom about the newspaper industry there is plenty of that too.

</comment></comment><comment><USER19> Yeah, this absolutely feels like a case where content is driven by business model as much as the technology itself.

Going subscription-first and focusing on quality content makes sense if you're trying to differentiate yourself. The concern, of course, is that subscription baiting is just one step removed from click baiting. It certainly cuts out the worst offenders, but there will always be pressure for media companies to cater to their audience.

</comment><comment><USER20> The NYT is just can't shake the negative influence of their printing legacy with respect to their strategy.

The article quotes them as saying "Our future is much more digital than print". Are you kidding me, they still have to say things like this?

Of course the $1B in non-digital revenue is important and needs a steady hand to maximize it's contribution.

However their digital strategies have been so simplistic, so old fashioned, they just can't seem to catch up to what they really need to do.

<comment><USER21> And what do they really need to do?

<comment><USER20> For starters, the leadership has very little experience in innovating with tech product or business models. [http://nytco.com/executives](http://nytco.com/executives).

Another example is they claim reader engagement is important yet have made almost no progress on this front. Most articles don't even allow community comments. Rather than complaining about the quality of internet comments, they could use their influence to embrace new techniques that raise the bar for discourse.

Their selling of subscriptions is a mess. First the cheapest sub is $200/yr. They need to segment further and offer something at the psychologically important barrier of $9.95/month. This new tier could be made compelling with cannibalizing higher tiers.

The sell screen is a mess: [https://www.nytimes.com/subscriptions/Multiproduct/lp8RX3Y.h...](https://www.nytimes.com/subscriptions/Multiproduct/lp8RX3Y.html). It's confusing, not clean, has unnecessary messaging, and looks like it's never been through iterations of A/B testing to maximize conversions.

There are many more opportunities for improvement that are also not being taken up on. I wish I could meet with them for a couple days of strategy consulting.

</comment></comment></comment><comment><USER22> An interesting contextual point from 2009 that highlights what an odd situation this is:

Printing The NYT Costs Twice As Much As Sending Every Subscriber A Free Kindle

[http://www.businessinsider.com/2009/1/printing-the-nyt- costs...](http://www.businessinsider.com/2009/1/printing-the-nyt-costs-twice- as-much-as-sending-every-subscriber-a-free-kindle)

Also, their back of the envelope calculations were done presuming that a Kindle cost $359.

</comment><comment><USER23> I find it interesting that the report is full of renderings of NYTimes stories on phones.

I wonder what the bounce rate is among mobile readers - that is, the percentage of readers who read less than 5% or so of a given news story.

I suspect that it's very high. For me, it takes a significant chunk of radio- silence (that is, no calls, text, social media updates, app alerts, no need for map directions, etc.) combined with an equally significant amount of willpower (i.e. not proactively checking in on any of the above) to read an informative ~1000 word story on my phone.

The NYTimes might be the highest quality content I skim online, but really, that's what I do 95% of the time at my desk and close to 100% on a mobile device: skim. I'm pretty sure devices that are concurrently in use for critical communication and/or work functions are awful places to try to engage readers and convey critical, often nuanced information.

Print happens to be great for that. And while I'm not holding my breath, the fact that vinyl records are about to be a billion dollar industry does suggest that old good things can make a comeback.

<comment><USER24> I subscribe to both the WSJ and NYT and find that my bounce rate is probably pretty high. But I tend to save articles that are interesting and I might want to read later on my device. I'll later read the article on my iPad or laptop. I can't stand reading a long piece on a small screen.

</comment></comment><comment><USER25> It'd be nice to see more of a focus on news rather than advocacy. I'll decide my own opinion.

</comment><comment><USER26> Would poorer people have access to non-fake news? Or would they be force to read/consume fake-but-free news?

<comment><USER27> The larger problem is that many people can't differentiate between the two, and will be drawn to fake news if the message appeals to them (confirmation bias).

<comment><USER26> Agree with you observation.

I think this just makes things even worse.

Real news cost $ to produce. If publishers charge for access to real news, then what we are left with by default is the fake "news" that is so cheap to produce that it can be ad supported (or serves some other end that makes it worthwhile).

</comment></comment></comment><comment><USER28> The report aspires to the NYT becoming an "authoritative, clarifying and vital destination".

I'd like that. But it's not clear to me that they realize how badly they are currently failing at it.

Over the holidays, the PBS Newshour did a "free energy from water!" story. Really. And then apologized, saying they would review their news process. But even then, it wasn't clear to me they realized just how brainlessly they had f*ed up. The NYT usually manages to avoid such. And to spell names right.

But on topics I know about, NYT coverage often/usually manages to miss the point. (And yet I keep going back, implicitly assuming their coverage of other topics, that I know less about, is somehow less bad. Somebody-or-other's fallacy.) There are more ways to "get it badly wrong", than to merely fail fact-checking.

And then there's the old concept of news. Report the trivia of what happened yesterday, and if you're lucky, use it as a "hook" to squeeze in an inch or two aimed at providing context and understanding. From scratch each time, because you can't count on any one reader having previously encountered it.

Authoritative? Clarifying? Well, they manage to be less bad at these than many other venues. But that's a very low bar.

The report says "We have developed one of the most civil and successful comment sections in the news business". Talk about low bars. "but we still don't do nearly enough to allow our readers to have these interactions." Interactions... not say, opportunity to lend us a clue.

I was wondering over the holidays whether to get a NYT subscription. And then hit another article that was "well, sort of..., but no" getting it wrong, and gave up. News coverage, from every venue I know about, is just startlingly bad. Which makes "committed to holding powerful people and institutions accountable" a nice idea, but largely unimplemented.

NYT search... "groupthink trump": 17 hits (hmm, I expected more). "groupthink tpp": 0. ""regulatory capture"": 2 hits in all of 2016. Sigh.

</comment><comment><USER29> Only 37 % of NYT readers access the paper from the home page. The rest are coming from search, social media etc. [0] While I see a possibility of converting the first segment (37 %) into subscription paying users, I am curious about what they are doing to convert the rest?

[0] (As of 2015) [https://www.amazon.com/Innovators-Digital-News-RISJ- Challeng...](https://www.amazon.com/Innovators-Digital-News-RISJ- Challenges/dp/1784534161/ref=sr_1_1?ie=UTF8&qid=1484688819&sr=8-1&keywords=innovators+in+digital+news)

</comment><comment><USER30> News is now Global and readers are transient and not loyal.

Brave should take it's micropayments wallet model, but dumb it down and sign a business deal with all major news papers.

You pay $10 per month and get access to all of the news papers. Brave divides the royalties based on your reading habits.

Benefits:

* Easy for consumers to understand

* Easy for news orgs to understand

* Bitcoins and wallet hidden from mainstream

* Proves Brave wallet technology for wider adoption

<comment><USER31> It's been tried with magazines, e.g. Texture. My library lets me read some magazines through Zinio. So forms of such services exist but they're pretty niche.

<comment><USER30> It's a good point. An all you can eat Comic reader is somewhat niche, but I wonder if news isn't a more mainstream market.

Currently, Brave's wallet tries to "boil the ocean", but a more focused vertical seems more tractable.

<comment><USER32> Hi Austin! We are in beta, and not ocean-boiling. Our plan does not require publishers to agree before they start getting revenue shares in escrow.

That's part of the plan, because it turns out to be hard to get agreements inked and signed. Even at larger scale (we're growing), getting pubs to agree is not easy or quick. As always when bringing up a new protocol, Metcalfe's Law is a pain. So we decouple and start sending money first. Money always helps.

We are talking to publishers from big to small, and we have verified ~120 publishers as domain owners, with 70 already having filed the W-9 or W-8BEN (if overseas) form.

</comment></comment></comment></comment><comment><USER33> > The report, Mr. Leonhardt said, was written for The Times newsroom, but it > could end up providing guidance for other news organizations facing similar > challenges. The company is also releasing the report publicly.

This is so non business like (and the way things seem to be in today's day and age). You take your hard work and give it away for free to your competition. Now you could argue that if others do the same it's good for you in some way (more people who buy print papers means more business for all print papers) but for some reason I don't think that's the case.

Remembering all of the free publicity that newspapers gave the Internet back in the mid 90's and further down. Not realizing exactly how that would impact their business model and not seeing it as a threat in any way.

<comment><USER21> The NYT website has been online since 1996. You can't say they didn't see the significance.

</comment></comment><comment><USER34> One possible outcome is more than one NYT, for the laid off workers to join some other organization and make it better, I suppose. I think it's very possible a few of those who just contribute "low-value line editing" would do much more if they were just given the chance.

</comment><comment><USER35> The NYT have conducted internal investigations like this on a few occasions in my memory. I don't have specific examples right in front of me, but I vaguely recall reading of several other major internal reviews, all of them with the intention of being 100% honest, no matter what they uncover, to be sure, since 2007. Interestingly, I can't think of a single time where the conclusion was simply the quality of their editorial discretion, i.e. stories they cover, resisting telling a story according to an already familiar, established narrative, etc.

Many people on HN are probably familiar with a similar design pattern: hiring consultants to either a.) tell you something you already know or b.) support a competing position with data points from an outside source. In fact, I vaguely recall the outcome of these internal investigations being something to the effect of needing to ramp up in the digital age with more multi-media, fancier toys, and things of that nature.

I would argue that just as necessary as streamlining the editorial process to compete with online-only new sources, like Buzz Feed, would be to understand their major failings and differentiate themselves from their competitors by owning up to them with action. I'm in no way an expert on the subject, just a reader like anyone else, but off the top of my head I can remember the misreporting of the evidence behind the need to invade Iraq, and failing to report NSA mass-surveilinance before George Bush's reelection - the same revelations that came back years later from Snowden (along with more).

Granted, they did own up to these failings, and others no doubt, but what editorial changes have been made? Why aren't the actual people who choose what news gets reported and how the facts are presented ever included as part of the outcomes of these reports? If I had to venture a guess, I'd say it's for the same reason as when a consultant is brought on to figure out how to tease out data points in support of justifying some new (and bold) initiative to, say, dockerize a division's infrastructure vs. actually getting to the bottom of why the reliable tooling of yesteryear are slowing development and deployment down. You get praise for doing things, and so you pay someone to give you a path towards justifying that. In many cases, I see a parallel between the reporters who lead this investigation and the ones from the NYT who are out there working in the field: to them the truth is all the news that's fit for their editors to print.

<comment><USER36> > Many people on HN are probably familiar with a similar design pattern: > hiring consultants ...

The two major reviews I'm aware of, including this one, were conducted internally.

<comment><USER35> > The two major reviews I'm aware of, including this one, were conducted > internally.

Good point, I wasn't very clear. For what it's worth, I know that they're exactly the same, but I should have pointed that wasn't the main thrust of my point. I was trying more to say "Hiring people to tell you what you want to hear". Probably not the most helpful analogy in retrospect.

I should point out now that I don't disagree with their conclusions either, but was arguing rather that they could have been made more comprehensive by examining the editorial discretion.

</comment></comment></comment><comment><USER36> > The New York Times has deftly adapted to the demands of digital journalism

EDIT: And from the report itself: [0]

> The Times employs the finest staff of journalists in the world

> [Elsewhere in the report, in case you forgot:] Our staff is made up of the > world's best journalists

> We have defined multimedia storytelling for the news industry and > established ourselves as the clear leader. Yet despite our excellence ...

> The print version of The New York Times remains a daily marvel

> We continue to be the most influential news organization in the country

First, I'm not a hater: I think the NY Times is one of the more important institutions in the world and read it daily.

IMHO they have been dragged into the digital age kicking and screaming, and their execution has been barely adequate to keep afloat. Financially, they've been cutting staff. Their digital products are clearly the product of old- school newspaper people trying to adapt their product to a web browser and to claim cred by doing some clever tech that has little impact or value (such as 'Snowcrash').

Look at their homepage: Does it look like, A) Any digital-native news site you've seen? B) Something optimized for the user and interface? Or C) Is it an old paper layout adapted to digital. (Answer: C). Look at their branding, the typeface: Overdone custom typefaces may have been a symbol of status and resources when printing presses were used, but now anyone can make a typeface on my laptop. The branding says loudly _We have come to you from the world of paper_. If someone used that typeface for anything else on the web, it would look bizarre; I can imagine the HN response. When the NY Times gets rid of that typeface, I'll believe they are all in for digital and aren't just trying to adapt. (EDIT: They even say it themselves in the report: _Our largely print-centric strategy, while highly successful, has kept us from building a sufficiently successful digital presence and attracting new audiences for our features content_ ; but then still can't let go: _We need to reduce the dominant role that the print newspaper still plays in our organization and rhythms, while making the print paper even better._ )

Finally, I fear the NYT suffers from narcissism, similar to the U.S. President-elect. This report (see the quotes above) and prior internal reports I've read (very few, a small sample) repeated many times how the NYT is the greatest. These reports aren't for marketing purposes, but for internal consumption. I'd be worried if I saw that in my organization. Narcissists are fragile; they keep telling themselves how great they are because they can't handle reality.

EDIT: A few edits after I read the report

\----

[0] [https://www.nytimes.com/projects/2020-report/](https://www.nytimes.com/projects/2020-report/)

<comment><USER37> > Their digital products are clearly the product of old-school newspaper > people trying to adapt their product to a web browser

I think by far their main concern is how to get people to pay for reading their digital product in the same way that people pay to read their paper product.

Given that every one else on the digital spectrum that extends down to 'random blog page' gets paid zero or near zero for their writing (interesting exceptions aside), making your web site look like something that people currently pay for seems like a good idea to me.</comment></comment>


Coronavirus can travel twice as far as official 'safe distance' - <AUTHOR> https://www.scmp.com/news/china/science/article/3074351/coronavirus-can-travel-twice-far-official-safe-distance-and-stay <comment><USER2> At the top of the article: " Note: The study at the centre of this article on the transmission of the coronavirus was retracted on Tuesday by the journal Practical Preventive Medicine without giving a reason. The South China Morning Post has reached out to the paper's authors and will update the article."

</comment><comment><USER3> If no masks and one needs to go to supermarket, what measures would help? eg wearing bandana/kerchief is completely useless?

<comment><USER4> Order groceries delivered wipe down all surfaces with alcohol. Avoid supermarkets if possible

<comment><USER5> I've been using bleach in water.

> "Sodium hypochlorite required a minimal concentration of at least 0.21% to > be effective."[1]

[1] [https://www.sciencedirect.com/science/article/pii/S019567012...](https://www.sciencedirect.com/science/article/pii/S0195670120300463/)</comment></comment></comment>


Microsoft Analyzed Data on Its Newly Remote Workforce - <AUTHOR> https://hbr.org/2020/07/microsoft-analyzed-data-on-its-newly-remote-workforce <comment><USER2> I'm surprised and skeptical about some of this. In particular:

      We also measured networks across more than 90,000 Microsoft employees in the United States. Frankly, we expected to see them shrink significantly, given the rapid shifts in environment, daytime rhythms, and personal responsibilities. Instead, we discovered that most employees maintained their existing connections. Even more encouraging, most people's network size increased. We had assumed that in a time of crisis, employees might strengthen networks within their own work groups in an insular way. In fact, we saw network growth not only within existing work groups but also across different groups, indicating that to adapt and thrive teams sought to build bridges.

This does not square with my lived experience. Close connections at work have stayed relatively strong, perhaps even strengthening as we shore them up with recurring 1:1s and social team meetings. Peripheral connections have gone down the toilet. Team and role homophily run rampant.

I wonder what evidence they actually used to back up this statement. I suspect the effect is actually in the opposite (and more intuitive) direction, and that their finding is actually a result of the McNamara fallacy.

<comment><USER3> "Network" is measured by Outlook emails, Yammer posts, and Teams conversations and meetings. I'm willing to wager money most peoples' network metrics increased simply because offline conversations never happened, and they moved to online conversations where necessary.

<comment><USER4> Not only this: online, the cost of "growing" your network by adding more recipients is (nearly) free which is very much at odds with offline growth. So my guess is the quality of weaker connections actually diminished even if the number of connections increased.

</comment></comment></comment><comment><USER5> I would like Google to publish the report from its statisticians on this. They have an _amazing_ team who collect data and _really_ rigorously dig into questions that are important for the company to know answers to. They then publish some of their findings company-wide (I guess some are only seen by execs, since there are relatively few public reports coming out of that team). The quality of their work would always be quite obvious, and I always wondered how come other large companies I worked for do not have a team like this.

<comment><USER6> I know this is happening. Without any insight into what the teams are doing, my guess is that results will be some time as they are likely focusing on engineering outcomes that require multiple survey styles and longer timelines, e.g. code metrics, engineering happiness, release frequency, bug trends etc etc rather than the easily collected metrics presented in the OP.

</comment></comment><comment><USER7> > The share of IMs sent between 6 PM and midnight has increased by 52%.

While I like the idea of "social meetings" (non-work virtual meet ups), this trend of IMs outside of core work hours would make the change to remote work a net negative for me.

<comment><USER8> Curious to know if anyone in a fully remote or mostly remote company: does your employer have even informal etiquette about "off-hours" messaging or is the stance more ad-hoc and left up to the employee to enforce those sorts of boundaries?

What's the proper approach between the two, just looking for general opinion.

<comment><USER9> Automattic: off-hours contact is normal, just don't expect a response until they get back to work. My team is spread out around the world, there's always someone asleep while you're working. If it's an emergency (like you deployed broken code and closed your computer) you can probably expect a phone call and/or text message.

Edit: one of the benefits of getting a late-night IM is to read it and have plenty of time to think about a response. Though, many people do not have Slack installed on their phones and only on their work computers.

</comment></comment></comment><comment><USER10> Remote work is right for some people, but the past few months have convinced me that it's not right for me.

A thing that puzzles me is how adamantly some of the people who were already fully remote try to impose the remote-is-always-better mentality on their fellow employees. Different people have different circumstances/preferences.

<comment><USER11> I think remote work is most strongly preferred by very experienced individual contributors. It is not the best setting for many people learning the ropes and it is an absolute crap for me as a people manager and I do not mean "I want to see your behind in the seat" style of management.

When I was an individual contributor and the work was very well specified, well contained, and known, remote work was a breeze.

Teaching new and junior employees is a constant struggle, something very critical is missing in remote work. It could be that people are wired better to learn face to face or there is a significant non-verbal component.

Managing people as in - getting them organized and synchronized is also a challenge. There is an immense value of getting everyone into the same room. I am not surprised articles points out that managers bear the brunt.

<comment><USER12> Well you are making it seem like how well remote work depends on the type of work, which while must be true to an extant, is besides the point the parent was trying to make: that it depends more on the type of person. I agree with the parent, as I have seen some of my colleagues flourish, and others flounder in the current environment. We all have similar tasks, the difference is the individual.

</comment></comment></comment><comment><USER13> It's important to note that remote work in March/April/May (and perhaps even today) was uniquely coupled with remote life, since most knowledge workers stayed home due to social distancing guidelines. It should not be surprising that the boundaries between work and life blurred when life itself came to a standstill.

I'd be surprised to see if the same tendency to blur life and work, such as by working on evenings and weekends, persists once life returns to its previous form.

<comment><USER14> Many companies are now shifting to full time remote work. The new normal is the new normal, things may never go back to the way they were. And companies might be able to save a lot of money on office space as well. I know in Seattle and Portland, there are quite a few firms who are allowing remote work because of issues of safety as well, chaos and disorder combined with a pandemic creates a new set of challenges we've never had to deal with before in concert.

<comment><USER13> Correct, which is why I'd like to see this same analysis redone once social distancing is relaxed and the situation on the ground (e.g. in Seattle/Portland) stabilizes, but people are still working from home.

</comment></comment></comment><comment><USER15> What about effect of saving on disgusting commute time? That itself trumps everything else imho.

<comment><USER16> Forgive my bluntness, but I have never understood the people who buy or rent houses very far away from work so they can save on a monthly housing payment when their hourly rate at work times the additional commute time is larger than the "saving".

<comment><USER17> You are assuming that people can increase thier income by working more hours. Most salaried employees can't do so at all and hourly employees don't often get overtime hours. It's quite possible that commuting really is the only way to "earn" the bigger house.

Regardless, there are other factors at play. A big one is that the amenities you want aren't near your work place at any price point- see all those people who commute from SF to Mountain View. Another is that your job changed but you don't want to move- I once rented a place near the downtown core thinking I'd always be close to work, and then somehow ended up with 45 minute commute to a satellite location. Once someone buys a place, get settled, and kids start school it gets a hard to move your residence as well.

</comment></comment></comment><comment><USER18> My experience ([https://taoofmac.com/space/blog/2020/07/11/1830](https://taoofmac.com/space/blog/2020/07/11/1830)) matches this to a T, from the longer hours to the shorter (typically 30m) meetings.

It bears mentioning, however, that I have been remote for a good while, but now have _so many more meetings_ that it isn't even funny anymore.

The switch to all-on video (rather than audio conference calls, which had its own etiquette and "dances") and the increase in context switching has made these very frequent meetings a lot more tiresome and impacted my productivity to the point where I have blocked out large swathes of "focus" time so I can actually work rather than constantly "sync" across multiple teams.

It's not about being remote (the "old" asynchronous remote working was awesome), it's about all the newbies wanting synchronous, high-intensity face- time.

(I get very few IMs, too. Only from other "old timers", now that I think about it.)

<comment><USER19> >>It bears mentioning, however, that I have been remote for a good while, but now have _so many more meetings_ that it isn't even funny anymore.

This ^^^. The girlfriend and I have both worked remote for years. Since her company did wfh company-wide, there has been a huge uptick in meetings. Also, for some reason companies seem to think 'video' calls are better then voice. Given we did business for almost a century with just voice (telephone) I find it rather odd. Maybe its just the novelty factor of playing with 'zoom' backgrounds ??

</comment></comment><comment><USER20> All the findings in that research completely mirrors my observations at my current employer.

My personal observation of forced remote work seems somewhat bipolar. If you have large responsibilities, lots of work, or the freedom to take initiative the new time flexibility allows you to increase personal performance. If you have a lot of time availability the new flexibility allows you to do other things than pretend to work or mindlessly surf the web. Sometimes people see that gap and try to fill it with more meetings to force engagement.

</comment><comment><USER21> The issue with the study is this was an "unexpected and unplanned" work from home situation.

If a team or employee had better preparation I would expect mostly positive results (ie tools, processes , and employee had time to prepare workspaces, kids at school, etc)

</comment><comment><USER22> What's with the blank white pop up on the website?</comment>


Ask HN: Simple static type registry in C++17 - <AUTHOR> I'm working on a side project where I'm also learning modern C++. As a part of the project I've created a static type registry class that I'm using for associating type names with types, and for instantiating types from type names. It's part of my serialization logic.
It's not big, but it seems to do what I need it to do, currently.
Can someone with a deeper C++ knowledge take a quick look at it, and see if there are any obvious mistakes in my current approach to this problem?
https://github.com/bjaastad/simple-static-type-registry-cpp/blob/master/simplestatictyperegistry.hpp <comment><AUTHOR> Link to the code:

[https://github.com/bjaastad/simple-static-type-registry- cpp/...](https://github.com/bjaastad/simple-static-type-registry- cpp/blob/master/simplestatictyperegistry.hpp)</comment>


AI potentially 'more dangerous than nukes,' Musk warns - <AUTHOR> http://www.cnbc.com/id/101892104

<comment><USER2> "The AI will chase us there pretty quickly." Erp.</comment>


A Simple Explanation of How Image Recognition Works - <AUTHOR> https://medium.com/@ageitgey/machine-learning-is-fun-part-3-deep-learning-and-convolutional-neural-networks-f40359318721# <comment><USER2> This is an excellent introduction to neural networks, deep neural networks, and the recent use of GPUs to speed up training them.

Good explanation of Type 1/Type 2 errors at the end, too: 'it correctly identified birds 95% of the time, but only spotted 90% of the birds in our test data'.

</comment><comment><USER3> Zeiler's ECCV14 paper is good follow on reading from this. He was one of the first people to actually visualise what these networks were doing.

[https://www.cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf](https://www.cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf)

Have a look at Figure 2.9. What you see is that the upper layers tend to be very general, basic filters that act as edge, corner and circle detectors. When you move to lower levels, the filters get more complicated and you start seeing things like animal parts, faces and so on.

</comment><comment><USER4> This is a really good tutorial series. If you're just getting started learning about this stuff, I can also thoroughly recommend [http://neuralnetworksanddeeplearning.com/](http://neuralnetworksanddeeplearning.com/) by Michael Nielsen.

</comment><comment><USER5> Google also has something else than lots of people's images. It has a reCaptcha system where users have to identify things in images. I assume Google is crowd sourcing training its AI systems.

</comment><comment><USER6> Hey Aymeric, thanks for posting this. I'm a big fan of your work on tflearn!

</comment><comment><USER7> I really liked the visual representation of the "8" bitmap! It's kind of neat how `0` has a fairly pixel density compared to `255`, so even as numbers the `8` is still recognizable.

<comment><USER6> I'm the author. I just saw this post.

I actually cheated to make that illustration better. In a real grayscale image, 0 is black and 255 is white. I inverted the image to make the numbers easier to see as an "8". I swapped it so that black was 0 and white was 255 :)</comment></comment>