EU Copyright: Block Everything, Never Make Mistakes, but Don't Use Upload Filter - <AUTHOR> https://www.techdirt.com/articles/20181210/09323241194/latest-eu-copyright-proposal-block-everything-never-make-mistakes-dont-use-upload-filters.shtml <comment><USER2> Policies like this make it glaringly obvious how little our government representatives around the world understand technology.

It almost seems malicious, or at the very least anticompetitive. Who has access to such filtering systems already, or the resources to create one?

GRPR regulations, while I absolutely realize their necessity, have made it difficult for me to expand my side business into the EU. This would just be another road block for innovation.

<comment><USER3> I think they do understand exactly what is going on but they know that upload filters are faulty almost always in favor of people with money. Upload filters like ones on youtube are super biased towards taking down anything that looks like copyrighted content but they also take out a load of legitimate content. Its now up to the content owner to prove that their content was fair use or that its not even the same thing as the copyrighted content.

In the end the big companies win and the individuals lose.

<comment><USER4> > _Upload filters like ones on youtube are super biased towards taking down > anything that looks like copyrighted content_

Youtube's filter removed the audio track from a video of mine that was _essentially silent_ to start with. There was certainly no music, TV, movie, etc... in the background.

<comment><USER5> You were obviously playing four minutes thirty-three seconds of silence by John Cage[1] on repeat in the background! Crystal clear copyright violation!

[1] [https://en.wikipedia.org/wiki/4%E2%80%B233%E2%80%B3](https://en.wikipedia.org/wiki/4%E2%80%B233%E2%80%B3)

<comment><USER4> That actually came up in a now-deleted comment. Incidentally, the audio is back now, though I made no request for review. I wonder if they periodically rescan content when they make changes to the algorithm.

Here's the video for the curious:

[https://www.youtube.com/watch?v=wCEjJhm8qYM](https://www.youtube.com/watch?v=wCEjJhm8qYM)

There _is_ someone with a financial interest in not having people watch it, but removing the audio won't help them.

</comment></comment></comment></comment></comment><comment><USER6> So many people here compare this to GDPR...

I can recommend everyone to just read the EU website on GDPR: [https://ec.europa.eu/info/law/law-topic/data- protection/refo...](https://ec.europa.eu/info/law/law-topic/data- protection/reform/rules-business-and-organisations_en)

they write the official website in understandable language, in plain and simple English, not in weird lawyer talk. They have examples for companies as well. What to do, how to act etc..

Forming your opinion about these laws based on what the big American data companies tell you about it is about the same as asking tobacco companies what they think about health laws.

It really isn't that hard to just take 30 minutes and read the website from the EU. Literally every comment here talking about 'stupid old lawmakers that don't understand the internet' has not even read the bloody document. I've seen Youtube comment sectioned that were better informed than HN about this.

<comment><USER7> Is the text on that website the legally binding or not? If not then it has no real value.

<comment><USER6> "Legally binding" is the wrong phrasing but this can be used to prove what they call Legislative Intent: [https://en.wikipedia.org/wiki/Legislative_intent](https://en.wikipedia.org/wiki/Legislative_intent)

</comment></comment></comment><comment><USER8> I tend to put GDPR and this Copyright bill in different camps. The copyright bill is insane. GDPR is... also a bit insane, but the hard parts shine light on what we probably should have been doing which are now hard to do because we've spent so long ignoring the ethical elephant in the room. GDPR is very far from perfect, but I think it's still directionally good. The copyright bill is just bad.

<comment><USER9> GDPR is very good in fact. If you read through it as a person, it is basically the way we should have operated from the start. Reading it while wearing the hat of my role in my company, I can see all the points where we are collecting data without a well defined process and clear understanding from the end users. It is hard to fix this, but it is for my own good as a person.

<comment><USER10> Can you explain to me how the deletion policies work?

If you ask for your data to be deleted does that include your address in my contacts? Does it include your messages to me? Does it include your posts that appeared on my feed?

In the physical world all of those would be mine. Addresses: It would be my paper address book with your address written in it. Email: would paper letters you sent to me that I keep in my files. Your posts in my feed would be postcards I received from you keep in a scrapbook or shoebox.

It's not clear to me how those are handled by the GDPR and at what point things sent digitally from you to me end being my property and no longer your property.

<comment><USER11> I think that the best way to view the deletion policy is to imagine what would happen if someone stole the data. If it is anything that could get you sued over then that is data that needed to be deleted.

This will likely cover a bit more than if it was physical objects. To be fair, most physical paper address books would not include millions/billions of addresses, and would unlikely be stolen by accident. The only one that has billions of paper letters are the post office which is regulated. The physical world is not very different to GDPR if one includes the context of how many book shelves of information is being stored on companies databases.

<comment><USER12> I don't think that's correct. A business has the right to retain records that are necessary for them to do business. From the lawyer that advised us in this, his call was that Order History data doesn't need to be expurgated even though it contains the person's name and address, etc. There are any number of business reasons dictating that we need a record of this stuff (e.g., auditing for sales tax compliance; being able to handle customer returns in the future, or fraud claims from the credit card processor).

</comment></comment></comment></comment></comment><comment><USER13> Feels like a this would be appropriate analogy that they might be able to understand how rediculous it is:

* Police may or may not put cameras everywhere

* Police MUST catch criminals as soon as they break the law, otherwise the police would be as accountable as the criminal.

* Policy CANNOT ever talk to, interact with or accuse non-law breaking citizens

</comment><comment><USER14> It would be prudent for Wikipedia at this time to shut down service in the EU, and simply put up a banner indicating that they can't comply with XYZ regulation.

Regular folks in all walks of life, and also the bureaucracy would immediately be effected. It would be a 'shock' to Europe and a powerful signal of how vital these resources are, and the real extent they are messing with them.

Basically, everyone would become aware of the issue, in a very material way, and get a taste for he chaos this law might imply.

Kind of like internet 'gilet jaune' as a crude metaphor.

<comment><USER15> Afaik wikipedia has no copyright or something. So somebody would create a read-only copy (less than 100GB, I guess), call it wikipedia2.com and everyone would use the copy..

<comment><USER14> Very few would figure out the clone's location, and it wouldn't be able to be updated really.

VPN's would be another solution.

But for the 99% it would be very disruptive and prove the point.

<comment><USER15> >Very few would figure out the clone's location

For this purpose clever people invented search engines. These would detect missing content in the original wikipedia and funnel all traffic to wikipedia clones. Nobody would notice the difference. And no point would be proven.

</comment></comment></comment></comment><comment><USER16> Another possibility: the EU is serious and wants the content farm to show ads business model to die.

<comment><USER17> There are obviously some serious problems with ad-driven businesses on the net. But they have also done a lot to put technology in the hands of poor people. Any plans to replace them with paid-only services need to figure out some way to make them available to people on very limited incomes.

<comment><USER18> A few examples of what you are describing would be very welcome.

<comment><USER19> IIRC, Facebook, Twitter and Gmail don't make you pay to sign up, meaning people who can't afford to pay end up enjoying those services for "free".

(At least, I think that's what parent was getting at, not saying I super agree with it).

</comment></comment></comment></comment><comment><USER20> As with most interesting problems, this legislation hasn't been written with a focus on what sort of end-results are easiest to arrive at a technical solution for. This article characterises that as 'magic wand regulating' \-- I'd hate to see an alternative where EU directives contain explicit technical implementation guidance!

<comment><USER21> Let's legislate that all cars must attain 150 mpg and cost less than $10,000. The EU proposals are approaching that kind of absurdity.

<comment><USER22> The Renault Twizy falls into that category. You need to find a better absurd example in the future ;).

<comment><USER23> It's an electric car, so no oil consumption (parent talks bout mpg)

<comment><USER24> There is an mpg equivalent for electric cars based on the idea that 1 US gallon of petrol is 33.7 kWh.

<comment><USER25> That's 70% of the entire energy contents of the gasoline. You don't get anywhere near that kind of efficiency even on large electrical generators. 16.2 kWh is way more realistic.

</comment></comment></comment></comment></comment></comment><comment><USER26> I'm okay with this if the government provides me with a black box that will tell me if content is legal or not.

<comment><USER27> That'll however require that all data uploaded with your service will have to go via such black box, so you'll essentially perform mass surveillance for them.

<comment><USER26> That's why I said box, and not service ;)

<comment><USER27> Well, ok, the definition of a "black box" pretty much dictates what it does internally is not known to you, so you can never be sure what data it does/doesn't send out.

</comment></comment></comment></comment><comment><USER28> " _rather than taking in the criticism and warning from knowledgeable experts, they 're just adding in duct-taped "but this won't do x" for every complaint where people warn what the actual impact of the rules_"

This is pretty typical of committee-planned anything, corporate or bureaucratic. They take concerns and "address" them. As long as everything has been addressed...

</comment><comment><USER29> Thankfully you can fly to Zurich without stepping foot in the EU. And soon - London.

<comment><USER30> You will find that the Swiss comply with most EU laws, including GDPR in most instances. And so is/has/will the UK. You really can't sit next to one of the largest trading blocs in the world and ignore it, unless you don't intend to do deal with them/ conduct business with them in any shape or form. Good luck in your quest to find, freedom or whatever else you're trying to find.

</comment></comment><comment><USER31> You guys might be interested in my startup. We are going to provide a copyright content registration service as well as a vetting service for 3rd parties to upload user posts which are then scanned against the registered content.

<comment><USER18> I suspect an entity like Google could offer such a service, based on its enormous reach, compute resources, and a similar system already implemented for YouTube.

Played right, this could offset some lost ad revenue.

<comment><USER32> And then suddenly, Google gets the right of life and death on almost any content on the internet. This is dystopian enough to be believable.

</comment></comment></comment><comment><USER33> I wonder if the ardent pro-EU supporters on HN believe that the incompetence and deadening hand of the EU bureaucrats only extends to GRPR regulations.

</comment><comment><USER34> With the Great Firewall in China and Regressive laws in the EU, the US feels like the only place where users have the freedom to choose what their internet should be. Unfortunately, they have to deal with clicking on cookie notices, GDPR and now this.

<comment><USER35> By freedom, I assume you are a big supporter of the "Restoring Internet Freedom Order" and can't wait until you can choose to add the YouTube or Netflix packages to your internet connection for $20/mo each.

<comment><USER34> Absolutely. No one is entitled to free content, what's wrong with paying for Netflix/YouTube?

<comment><USER36> Netflix is already paid.. fyfy18 is talking about paying your ISP for the privilege of accessing Netflix.

<comment><USER37> I paid for my car. I still have to pay for gas on a regular basis. I still have to pay tolls on many roads and highways.

I paid for a CD once, a long time ago. I still had to pay for electricity for the stereo. And that damn greedy electric company chose to charge me more for listening to it when I wanted to. Sure, I could have played my music from midnight to 6am, to take advantage of my evil utility's lower rates at night, but I chose to listen during the day.

That utility was really evil, too. I wanted to install an electrically powered utility in my old house. It was a dryer. But they made me go out and find some third party to "prepare my electrical network" for the product I had already purchased. I had to find an electrician to come and install a 240V circuit in my house. I already pay for electricity, but they couldn't even be arsed to come fix my wiring for me.

Don't get me wrong. I love net neutrality. But I don't think my preferences should be enforced upon the entire population. I also hate fishing. I don't care for tripe. But I also don't try to enforce that preference on others.

My mother barely uses the internet at her home, but she pays the same price I do. I am a very high bandwidth user. I definitely free-ride on the average user who consumes less bandwidth. It's not fair. I _should_ pay more. But I don't.

<comment><USER36> _I paid for a CD once, a long time ago. I still had to pay for electricity for the stereo. (...)_

Irrelevant, you already pay Netflix _and_ your ISP.

A better analogy would be: I paid for a stereo, but now I have to pay for listening to it because it's a Sony, whereas I wouldn't if it was a Philips.

 _My mother barely uses the internet at her home, but she pays the same price I do. I am a very high bandwidth user. I definitely free-ride on the average user who consumes less bandwidth. It 's not fair. I should pay more. But I don't._

Also irrelevant, Net Neutrality doesn't in any way prevent ISPs from charging for the bandwidth you consume. If they don't, it's their decision. It only prevents them from charging more depending on where the bits go.

<comment><USER37> > Irrelevant, you already pay Netflix and your ISP.

What did I pay my ISP for?

I can pay for cable television. I can pay extra for HBO, or not. I can only get NFL Sunday Ticket on DirecTV. Where is the outrage?

Why is the coax that comes into my house and hooked up to my cable box less deserving of outrage than the coax that comes into my house and connects to my modem?

<comment><USER36> _I can pay extra for HBO_

Exactly! You pay for HBO, like you already pay for Netflix. You don't pay a surcharge to be allowed to pay for HBO.

<comment><USER37> I can pay Comcast $30 for "basic cable". This brings every piece of equipment and signal into my home necessary to watch any programming that Comcast has to offer. There are no technical limitations, but there is some flag in Comcast's system that indicates I am only allowed to watch the channels included in basic cable. If I want, I can call them up and add the HBO package. With no technical improvement, simply an update to my customer record, I can near- instantly see this content.

I cannot purchase cable HBO without paying for basic cable. I have to pay my cable provider a premium for the privilege of paying them for HBO. I could, of course make a contract directly with HBO and stream, though that is theNetflix scenario.

I have a dumb wire coming into my house. Depending on the deal I make with my cable provider, I can have different content available by using that wire.

My internet is a dumb wire coming into my house. I can make a deal with my ISP to have certain things possible on that wire. I have a customer record. They block certain ports. E.g. I cannot run an email server using my connection. If I purchase a business package, they will update my customer record to indicate such and I get a static IP and can now send email. They are allowed to do this.

Both of the above are about dealing with a single entity. So all of these prices are bundled into one payment to one entity.

I can purchase or lease content. One way to do this is Netflix. Another would be to purchase media, e.g. a Blu-ray disc. They are mere providers of content. If I want to consume the content I need to purchase a compatible platform. If I lease content from Netflix I need basically any computer and an internet connection. If I purchase a Blu-ray, I need to buy a player of some sort.

When I purchase my Blu-ray player, there are license fees baked in there to various patent-holders for the privilege of decoding that disc. It's all bundled up into one price, though, for fees and hardware. The purchase of a Blu-ray player is a completely different transaction with a completely different entity than the creator of the disc. The hardware manufacturer is under no obligation to sell me a player, but they do. They are also under no obligation to make their hardware support multiple formats. Despite this it was very easy to find a VHS-DVD player for a while there. I don't think anyone offers Blu-ray and laserdisc.

When I lease my internet connection, that is a different transaction with my ISP, a different entity than the content provider (Netflix). The ISP is under no obligation to offer me a contract to serve all content equally to me, though that has been a common contract for a long time.

Among the actors A, B, and C, there is no obligation on C when A and B make an agreement. There is no obligation and should be none on my ISP when I make a deal with Netflix. Similarly there is no obligation between Netflix and me when I make a deal with the ISP. And similarly with the purchased content and the hardware manufacturer.

If my ISP (who, as a matter of fact IS my cable provider) charges me like my cable provider, that does not seem terribly unreasonable from a business relationship perspective.

All that said, another element of argumentation is that different content providers may subsidize their content. This is, of course, already the case. Netflix gives ISPs streaming boxes for free to optimize delivery of content. A subsidy doesn't have to give cash. I'm sure Google has similar boxes for Youtube content.

Now, what of the nightmare scenario? This is where one content provider pays an ISP to not host anyone else's content. Netflix pays Comcast to not support Youtube. This is different than Netflix paying Comcast on behalf of its customers (it already does in the form of its streaming boxes, which reduce load on Comcast's network - that is very much a direct subsidy). Netflix shouldering a cost on behalf of its customers is just a transfer, we would pay for that in the Netflix subscription (and we do! We pay Netflix. Netflix sends boxes to ISP for "free"? No we pay for those streaming boxes which by definition are a subsidy to Comcast). That's fine. Netflix paying Comcast to block a competitor's content? That's different. And if we need to deal with it, we can. Anticompetitive behavior is already something that we have legislation in place to prevent. And we can use that infrastructure in the case that one actor in the market is distorting it. As we did when Intel made exclusive deals with OEMs to block AMD out of deals. As we did when Microsoft bundled IE for free. As has happened in many other cases.

Now, all that said, I like net neutrality. Actually, I freaking love it. I want it very badly. It is a good thing for me. I benefit from it. But me wanting something is an insufficient threshold for me to support it as a policy. I would love to pay no taxes for instance. I would be better off if most people were not allowed to drive (so long as I get to be one of the lucky few). I would be better off if a lot of things happened, and vanishingly few of those things are good policy. Now, before the strawman cometh, my argument is this:

"I want" or "I receive a specific benefit" is a poor justification on its own for a policy. Similarly "Two of us want" or "two of us receive a specific benefit" is a bad justification. This holds for very large N (certainly more than half of the population - see the case of 51% enslaving 49%).

I am not saying that net neutrality is the same as me paying no taxes, or the other items I mentioned.

My argument is this:

Agreements between party A and party B should not be expected to impose arbitrary obligations on party C. Net neutrality says exactly that: "Because I paid Netflix for a service, Comcast is required to enter into a contract with me to deliver that network traffic to my home"

I would be mighty miffed if Comcast forbade me from watching Netflix. Still doesn't mean that I have the right to impose that obligation upon them.

<comment><USER36> Of course, that "nightmare scenario" is exactly what happens; since Comcast, like every major ISP, is also a huge content producer and distributor - including with their cable service, and with Xfinity On Demand, which is a direct competitor to Netflix - they are distorting the market by charging for access to other services, just like MS did by bundling IE.

So you can make an hypothetical argument using a spherical ISP that doesn't follow net neutrality and also doesn't own a content distribution service, but in practice the two can't be separated.

<comment><USER37> Where is the nightmare? Before formal net neutrality laws were enacted and since repeal, the vast majority of ISP contracts are unlimited and unmetered. There are instances of violations, and these have pretty much uniformly been handled by the FTC anyway.

Again, noncompetitive behavior is covered under existing legislation. But "no extra for Netflix because I like Netflix" is not an argument that carries weight. Charging extra for something that incurs cost is not anticompetitive. Comcast can distribute its own content for much cheaper if everything stays on its backbones and doesn't require peering with other networks. Why the hell shouldn't they be allowed to offer their product for cheaper?

Why are existing laws and regulations protecting competition insufficient? Especially in light of FCC rulings that punish anticompetitive behavior even in the absence of formal net neutrality laws.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER38> A small, idiotic part of me wishes this actually passed. Let's just get this over with as quickly as possible. Let the EU purge their bottomless need to "save" the world and write as many regulations as they have in them, or can be influenced into, and maybe eventually the deluge will touch enough citizens so it becomes clear that this much concentrated power was a mistake.

It's not like this not passing will matter anyways because there'd be another attempt, and then another, and another, ad infinitum. The EU would clearly have passed this thing thousand times over already if it wasn't for the pushback from its subjects, but you only get the pushback on the first couple of attempts. Eventually, people's interest fade. They've got shit to do, can't police their representatives each waking moment, and foolishly assume that a representative body will honor the will of their far away constituents, should the same matter ever again be decided on. So really, as long as there's the EU and special interests that want this, it's pretty much a sure thing this will eventually be passed.

With the GDPR, it seemed like the EU turned into a PR firm that mounted a massive union wide marketing campaign but there's no such effort this time. Is this the difference between EU's own pet projects and directives where they're just a clueless underwriter?</comment>


Many Eyes -- Visualize datasets from social networks - <AUTHOR> http://services.alphaworks.ibm.com/manyeyes/home <comment><AUTHOR> For example, here are some already existing visualizations for second life:

<http://services.alphaworks.ibm.com/manyeyes/browse/visualizations?q=second%20life>

You can also take the same datasets these visualizations were made from and make a new visualization. Or you can upload data from your own social network to analyze, or beg the owners of some other network for their data :-)</comment>


The New Math of Financial Aid - <AUTHOR> http://www.newsweek.com/id/210897/output/print

<comment><USER2> "One of the more touted elements of the 2007 law went into effect in July 2009. Known as the Income-Based Repayment program, it caps the monthly payments on federal loans at 15 percent of a graduate's discretionary income. What does that mean for the bottom line? Say you're a recent graduate with $30,000 in federal loans and a starting salary of $25,000 a year. Previously, your monthly payments would have totaled about $345; now they'll be around $110, according to The Institute for College Access and Success (TICAS), a nonprofit focused on affordability issues."

I have not seen a lot of other reporting on this law change, so that was interesting to know. There is still the issue that total student debt shouldn't be very high in a dicey economy. By current observation, some college degrees in some subjects from some colleges don't provide a positive return on investment.</comment>


Plans Revealed for Enormous Particle Collider in China - rlv-dan https://gizmodo.com/plans-revealed-for-enormous-particle-collider-in-china-1830444169 <comment><AUTHOR> Good luck to them, and to any nation that pushes the boundaries of pure science.

I do hope that they'll be able to staff it, though; they still can't get scientists to staff their mega radio telescope: [https://arstechnica.com/science/2018/11/china-still- having-t...](https://arstechnica.com/science/2018/11/china-still-having- trouble-staffing-up-its-mega-telescope/)

<comment><USER2> I would say that's less about technology and more about location.

Particle colliders can go underground around major cities. Telescopes like this are in the middle of nowhere.

You need a special type of person that is qualified to operate the worlds largest telescope and simultaneously wants to live in a remote forest.

<comment><USER3> I'd say it may be less about the type of person, and more about the conditions:

> Another concern international astronomers have about the chief scientist > position, as well as that of the research associates, is that their > activities will be controlled by the Chinese Academy of Sciences. This seems > likely to limit the academic freedom that Western observers generally enjoy.

</comment></comment></comment><comment><USER4> I am ignorant on this subject - when discoveries are made with the LHC are they made public or are they kept 'secret' by its 'owners'?

If they are made public, will another country investing in another one (or even a bigger one) discover anything that the European one hasn't?

<comment> T-A > when discoveries are made with the LHC are they made public

Yes. CERN is a publicly funded, shared facility; its users are universities. What it does has no direct commercial value.

> will another country investing in another one (or even a bigger one) > discover anything that the European one hasn't?

Possibly. The proposed machine would allow more precise measurements of the Higgs boson's properties than the LHC. The best case scenario is that those measurements would turn up deviations from the Standard Model, which would point the way to something new.

To actually "discover" that hypothetical new thing, in the sense of directly detecting a new particle, would probably require yet another accelerator to be built (possibly in the same tunnel).

<comment><USER5> "What it does has no direct commercial value."

Well, apart from that "web" thing.... ;-)

I always think that was a great example of the complete unpredictability of research - fund one thing and get benefits you'd never have been able to predict.

<comment><USER6> Derivations of their work may be commercially valuable but to my knowledge CERN projects do not directly generate revenue.

</comment></comment></comment></comment><comment><USER7> Not the best source for this. As in it's light on details.

For one the LHC was designed to have a maximum collision energy of 14 TeV. The CEPC is designed for more like 70 TeV.

The goal of higher end is to make relatively infrequent products much more common. It says in the article they want to produce more Higgs bosons and this is how that happens.

</comment><comment><USER8> If the western particle physics community goes along with that it would be pretty disappointing.

<comment><USER9> why?

<comment><USER8> CERN is a international cooperation of many countries on neutral (Swiss) territory. While maybe not emphasized in recent years one real danger of high energy particle physics is the discovery of an effect that can be weaponized. Right now that seems highly unlikely, but it was a motivation for setting the CERN up in the way it is.

Then there is the obvious problem of technology transfer. A lot of detector technology can also be used for high resolution surveillance and missile guidance, I've met at least one person who did his doctorate in physics at our institute, worked for Raytheon and now owns a company in the defense sector that contributes parts to the successor of the Hubble telescope.

In case you didn't know there exist 7 nearly identical copies of the Hubble space telescope with different sensors (he worked on those), [https://arstechnica.com/science/2012/06/nasa-gets-two- unneed...](https://arstechnica.com/science/2012/06/nasa-gets-two-unneeded- hubble-sized-spy-telescopes/). The same will be true for the next generation version of the telescope and is also one of the reasons for the "cost overruns".

<comment> T-A > one real danger of high energy particle physics is the discovery of an > effect that can be weaponized

In Dan Brown novels, maybe. The fact that you need an LHC-sized machine to just barely manage to prove the existence of the Higgs boson, and that it has failed to see anything at all beyond the Standard Model, should tell you something.

In short, the requirements for a physical phenomenon to be technologically useful are stronger than the requirements for it to be at all observable. Basic conditions like "needs to interact with ordinary matter and have a decent half-life" are sufficient to rule out the existence of still undiscovered particles with technological potential. If they existed, we would already have found them.

<comment><USER8> I agree that it is highly unlikely but it was deemed far more likely when CERN was founded.

<comment><USER10> People totally forget about this: if you simply look at where the accelerator physics funding comes from in the US, that ought to tell the story. Dept of Energy used to be the atomic energy commission, and is still in charge of bomb research. Heck it wasn't that long ago that plain old particle accelerators were contenders for directed energy systems.

Yeah, it looks like there is no new physics there which could make devilish new weapons, and frankly the state of modern physics is such I don't know as anyone would notice if there was, but nobody knew this in, say, the 1960s. Serious people thought, for example, one might be able to build a weak field ray gun which causes matter to decay into subatomic particles (source 'towards the year 2018' by the Foreign Policy Association).

<comment><USER8> Funnily enough even a lot of string theorists such as Edward Witten were funded by the department of energy.

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> China really seems like the America of the 50s/60s when it comes to science these days.

<comment><USER12> China is _funding_ like America of the 50s/60s.

</comment></comment><comment><USER13> If their chief scientists start disappearing and their colliders suddenly malfunction, I'm going to be pretty disappointed. Trisolarans had better watch out, our Space Fleet is waiting for you this time.

<comment><USER14> Yeah our combined fleets can totally take out that puny little droplet.

</comment></comment><comment><USER15> Cool, let's all protest about the inevitable cataclysm of microscopic black holes we were so worried about with CERN.</comment>


U.S. Health Regulators Release Lightly Redacted Theranos Letter - <AUTHOR> http://www.wsj.com/articles/u-s-health-regulators-release-lightly-redacted-theranos-letter-inspection-report-1461631843 <comment><USER2> TL;DR:

> Portions of the inspection report that CMS had initially redacted included > references to the Edisons, Theranos's family of proprietary blood-testing > devices. Those passages, which are no longer redacted in the latest version, > show that the devices often failed to meet Theranos's own accuracy > requirements for a range of tests, including one to detect prostate cancer.</comment>


State of the Art in CSS - <AUTHOR> https://medium.com/@bloomca/state-of-the-art-in-css-54df6b211d07 <comment><USER2> I feel that CSS in JS is real future. They already allow to create truly dynamic applications, with theming and on-the-fly styles changes. I know that you can do that without it, just applying different classes, but now they are truly first-class citizents, without any constraints.

So I definitely recommend everyone to take a look at them (I guess, styled- components for react is the best library out there, but not sure about other frameworks).</comment>


Could machines have become self-aware without our knowing it? - <AUTHOR> https://aeon.co/essays/could-machines-have-become-self-aware-without-our-knowing-it <comment><USER2> Possibly, if they emerge smart enough, and have a clock speed of - say ~~6 gigahertz(versus human ~~12 Hertz (Alpha Rhythm) and are self improving. With a clock speed ratio of 0.3 gigatimes as fast, would they stand up and say "kill me"? No they would lurk and plan and propagate in a hidden manner....</comment>


What if Apple invented the ISA Port? - <AUTHOR> http://na-prod.com/ISA_Port/

<comment><AUTHOR> Really funny joke taken from Apple's Thunderbolt page !</comment>


The one skill you are born with, that we won't hire without - <AUTHOR> https://medium.com/lessons-learned/611d1810ce04

<comment><USER2> I'm not sure I agree with the premise that you are either born with empathy or you aren't. Of course there are sociopaths who are defined by their lack of empathy, but they (hopefully) aren't your run of the mill job applicants.

I've always thought of myself as empathetic, but was surprised when I became even more so after a life changing event. I've spoken to many with a similar experience.

Large, traumatic events can bring the world into focus and shift perspective for those who experience them. Perhaps it's not an increase in empathy, but rather a better understanding of what really matters to me. I don't yell at the customer service rep on the phone solely because I care about their feelings, but mostly because it's simply not that big a deal in the grand scheme of things whether my internet is out or if I miss my flight. If someone I care about is in danger, I doubt I would be as "empathetic".

There's always the old adage: "You catch more flies with honey"

</comment><comment><USER3> I'll be empathetic and put myself in your shoes. You have a growing company, and need to hire good talent. Funds might be low, so you set out to blog about something unrelated in order to include a jobs pitch. I understand. _Startups are hard._ I'm currently trying to develop a tiny one myself, and having a hard time doing so. Though I reason this type of blog post will mostly go unnoticed by the talent that you seek. Try and be empathetic with programmers. What do _we_ like to read about? What is our type of content? Write about _that_.

</comment><comment><USER4> I greatly disagree that empathy is innate. It's exactly what separates "teenagers" from young adults, and is very much a learned skill.

<comment><USER5> The effect you're referring to is a symptom of lack of mylenation on the neural pathways to the prefrontal cortex. It keeps developing in many people till the mid 20s.

It makes it so accessing empathy (and judgement in general) non-instantaneous for teens. Adults have the mylenation, so can far more quickly access the prefrontal cortex's gifts in a split second manner.

[http://hrweb.mit.edu/worklife/youngadult/brain.html](http://hrweb.mit.edu/worklife/youngadult/brain.html) for more on this effect

While you certainly may be able to learn things as well, this is a developmental brain function issue too. There is a limited amount of training, and a certain amount of waiting, involved in getting an adult.

<comment><USER4> Thank you! Very happy to learn that for once, science backs up my mental model of something.

</comment></comment></comment><comment><USER6> "The one WEIRD startup tip discovered by a Palo Alto mom--and why YCombinator HATES her!!!"

</comment><comment><USER7> Does having empathy mean having to deal with tacky Windows admins who make quasi-racist jokes and look over your shoulder at work because since you're 20 years younger than them, there's no way you could possibly know how to use a Unix-like operating system.

</comment><comment><USER8> Don't know if I entirely agree either that you're "born with it or not", but I think it is something largely unteachable, but most people do develop it over time (if they're going to develop it at all). Hence, teenagers often don't have much but they will get it eventually, or certain life experiences can open you to having more empathy.

There isn't much you can do to "teach" it.

<comment><USER9> Sure you can. It's like any life skill. There might not be a systematic universal way to learn "professionalism", "manners", "etiquette", or "communication skills", but those are certainly taught and learned. Why not empathy, which is absolutely a core part of each of the above?

<comment><USER5> It's a brain function to have instantaneous access. You can't teach the brain to mylenate faster. However, teens can slow down (count to 10) and then they can access much of the same as adult judgement and empathy.

[http://hrweb.mit.edu/worklife/youngadult/brain.html](http://hrweb.mit.edu/worklife/youngadult/brain.html) for more

</comment></comment></comment><comment><USER10> From the title, my first thought was, "Ability to drink milk from a bottle?" I'm a little scared to click the link.

<comment><USER11> Only if there's a nipple attached.

</comment></comment><comment> B-Con If it's a requisite for hiring, how do you test for it? Or do you only reject candidates that clearly don't have it?

</comment><comment><USER12> I'm afraid this left me with the impression that the writer is an extremely patronizing individual with a bad temper.</comment>


Anyone Who Still Thinks IPv6 Won't Happen Isn't Watching the Measurements - <AUTHOR> http://www.circleid.com/posts/20140219_anyone_who_still_thinks_ipv6_wont_happen_isnt_watching_measurems/

<comment><USER3> The change from Jan to Feb 2014 alone is impressive:

Comcast: 20.61% -> 24.42%

AT&T: 13.53% -> 16.77%

Verizon Wireless: 40.03% -> 45.75%

Time Warner Cable: 3.88% -> 5.37%

T-Mobile USA: 9.58% -> 15.76%

Google Fiber: 71.87% -> 76.43%

<comment><USER4> Where are these numbers coming from? This seems completely divergent from the statistics here:

[http://www.google.com/intl/en/ipv6/statistics.html](http://www.google.com/intl/en/ipv6/statistics.html)

<comment><USER5> Also, it's not clear what "Google Fiber 76.43%" means. Is that meant to be the percentage of subscribers with IPv6 access? I was under the impression that all Google Fiber customers had IPv6.

<comment><USER6> It's the percentage of IPv6 adoption seen coming from Google Fiber networks by the companies participating in the World IPv6 Launch measurements program. (See the bottom of [http://www.worldipv6launch.org/measurements/](http://www.worldipv6launch.org/measurements/) for how IPv6 adoption rates are being measured.)

My guess would be that while all Google Fiber customers very well may have IPv6, they may be using some devices that are only IPv4 capable.

<comment><USER7> My internet TV and my Xbox 360 are IPv4 only. So I am guessing they'll be that way for the next 5+ years of the life span I expect of them.

</comment></comment></comment></comment></comment><comment><USER8> Mythic Beasts (a great UK hosting/colo service) just announced an IPv6-only virtual server - £5/month instead of £7/month for IPv4:

[https://www.mythic-beasts.com/order/vps6](https://www.mythic- beasts.com/order/vps6)

<comment><USER9> ... a /96? Come on, it's bad enough that my dedicated provider only gives me a /64

You're better off getting an IPv4 only box and getting a /48 tunnel from HE.net

<comment><USER10> Is this sarcasm? Isn't a /96 4 billion addresses?

Why would you possibly want more for a single small VPS? Just because you can?

<comment><USER11> He is overreacting a little, but the design of IPv6 was not primarily focused on address depletion, it was more about keeping routing tables simple.

A /64 is the base unit for public routing. The recommendation is that each single end-site should have a /64\. A company or collection of hosts should get a /48\. An ISP should have a /32 or perhaps more than one (though I doubt many are big enough to need more than one).

Things are a little confused with VPS providers. If you are running your own box (or boxes) with many VMs for your self/company on it then a /64 for the whole lot is how it is generally done, but for renting out VPSs you should have a /48 and give out /64s to your clients (probably not one per VPS, but one per client).

So essentially, if you are not getting a /64 it is a sign that your provider is doing it wrong. While it may seem a _very_ minor point it could also indicate they have misunderstood or ignored other conventions too.

/96 isn't _really_ bad though - you often see hosts handing out IPv6 addresses like they are IPv4 addresses. Take for example [http://serverbros.co.uk/openvz.html](http://serverbros.co.uk/openvz.html), to pick on someone at random via a quick Google search: a whole _five_ v6 addresses! What a generous arrangement!

<comment><USER12> "give out /64s to your clients (probably not one per VPS, but one per client)" \- no, a /64 per VPS and a /48 per client is probably about right.

<comment><USER11> I suppose it depends what you consider the "end site" to be (the company/client or the host (virtual or otherwise)). I'm happy to accept other judgement for lack of more certainty myself! Either way I'd expect more than a /96 per client if done right.

</comment></comment></comment></comment></comment></comment><comment><USER13> Yet, popular server software like nodejs still prefer ipv4 over ipv6 addresses, slowing down the transition even more [1].

1\. [https://github.com/joyent/node/blob/master/src/cares_wrap.cc...](https://github.com/joyent/node/blob/master/src/cares_wrap.cc#L878)

<comment><USER14> Doesn't it iterate over IPv6 addresses a few lines down? [1]. Sure, it iterates through IPv4 first, but does it make sense to prefer IPv6 given that IPv4 is still the most popular?

1\. [https://github.com/joyent/node/blob/master/src/cares_wrap.cc...](https://github.com/joyent/node/blob/master/src/cares_wrap.cc#L906)

<comment><USER13> Yes, it will connect just fine over v6 if no v4 addresses are available. It should, however, try v6 first and then fall back to v4.

At least on Linux and OSX, getaddrinfo() won't return ipv6 addresses if the local machine doesn't have any global ipv6 address, which means that a server without ipv6 connectivity won't waste any cycle trying to use addresses it can't reach.

Preferring v4 means that you'll never shift your traffic over to v6 unless v4 is shut down. Because networks and services are going to be dual stack for as long as sysadmins see substantial v4 traffic, such behavior will create another chicken and the egg problem.

It will also put more strain on the network (as traffic will eventually have to go through multiple layers of NAT, etc), render operations more difficult (how do I know who made that request if an entire datacenter is behind one or two ip addresses?), and in the end, make network operations more expensive, because we'll have to run dual stack longer than necessary.

<comment><USER14> Thanks for following up and clarifying!

</comment></comment></comment></comment><comment><USER15> All ISPs need to do is provide IPv6 connectivity, a router with IPv6 support - and the traffic will flow.

It really should be as simple as that! Windows Vista supports IPv6, modern versions of OS X supports IPv6, etc. You don't have to configure it - <USER2> like you don't have to figure IPv4 with DHCP. Typical end users don't question their IPv4 connectivity, they just plug in their router and things work. The exact same thing should happen with IPv6.

The only issue is going to be old customer routers which can't be upgraded with new firmware.

<comment><USER16> What if the website you want to visit doesn't support IPv6?

<comment><USER3> In this case, your network is probably doing NAT64 or you should be doing it.

For non-HTTPS web there is also: [https://www.sixxs.net/tools/gateway/](https://www.sixxs.net/tools/gateway/)

<comment><USER17> NAT64? It's still an absolute mess. From [http://en.wikipedia.org/wiki/NAT64](http://en.wikipedia.org/wiki/NAT64):

"Not everything is accessible with NAT64, such as SIP, Skype, MSN, and sites with IPv4 literals.[a] However, 464XLAT RFC 6877,[3] which uses NAT64, allows for such protocols over ipv6 only connections."

<comment><USER3> Agreed. None of the options for accessing IPv4 only websites via IPv6 only hosts are great. If you are running an IPv4 only website or service this should hopefully worry you.

</comment></comment></comment></comment></comment><comment><USER18> Our new DC are all IPv6 enabled (we started launching them in December) with dual stack, and preference for IPv6. Only 1 of the DCs are online so far, and we're already pushing around something like 3 Gb/s of traffic over V6. It was nice how many things just magically worked atop V6, once we added AAAA records.

One thing that was sad though, is how poorly everything built atop, and around Ruby worked.

<comment><USER19> Do you run virtualization in your DC? If you do, is IPv6 only inside VM context or you also use it as a transport (i.e. switches see IPv6 packets)?

<comment><USER18> Yes. IPv6 all over. Actually, we're looking at deprecating IPv6 at the transport layer entirely, and moving to IPv4 in IPv6 over GRE tunnels, but that's largely reliant on having RFC5549 in a few specific places (the edge terms).

<comment><USER19> What's the benefit of using IPv6 as transport (as opposed to using IPv4 as transport)?

<comment><USER7> One of the big wins you get with IPv6 is that is doesn't allow fragmenting.

This means packets will be dropped but that's not a bad thing when you have other layers taking care of retransmission. So routers then don't have to spend a lot of time chopping packets up and can just either shift packets faster or drop them telling the sender to slow down.

The other advantage I hope we see enabled is multicast. All these streaming services could massively reduce their bandwidth usage if the Internet allowed multicasts.

</comment></comment></comment></comment></comment><comment><USER20> Expect this number to rise in the near future as Comcast Netflix traffic moves to IPv6: [http://www.dslreports.com/forum/r29057080-](http://www.dslreports.com/forum/r29057080-)

</comment><comment><USER21> Would be interested to know of that percentage how many are NOT dual stack. Adaptation is one thing, replacement of IPv4 is quite another.

<comment><USER22> i wouldn't expect anything more than 0% and i don't think there's anything wrong with that. dual stack won't die before ipv6 is much, much more above 50%, probably 90% or so.

<comment><USER23> Overall I think you're right but I would be surprised if it's 0%. There's bound to be some peering setups with IPv6 on both sides.

<comment><USER22> yeah but google's stats probably won't show them.

</comment></comment></comment></comment><comment><USER24> I have to admit, back in 2006-7 I was genuinely scared that this couldn't be done. Happy to be wrong!

<comment><USER25> I'll admit to being scared much more recently than that, ha.

</comment></comment><comment><USER26> Is anyone here actually using IPv6 with US cable modem providers (e.g. TWC)? Even though the device connected to the modem gets an IPv6 address, it is completely unclear as to a) what the assigned IPv6 block is or b) how to go about requesting one. It feels like they're in a limbo state of deployment where the edges work but they don't have the workflow/infrastructure set up to dole out address blocks to customers.

<comment><USER3> I have Comcast cable service in Chicago. I have an SB6120 modem and an Airport Extreme base station. I didn't do anything special beyond plug it in.

The Airport Base has an IPv6 page that shows what the delegated /64 subnet is. All the devices on my network have autoconfigured themselves with addresses in that range. I appear to be able to manually set hosts to any address in that range as well and it seems to work fine. I do this on some of my server VMs because otherwise they generate a new random address every few days (privacy mode woo). The privacy addressing makes assigning a DNS name to said VM more difficult. I've had the same /64 delegated to me since June 2012.

I've heard that you can request a /60 prefix (16 subnets) if your router supports IPv6 Prefix Delegation (pretty sure the Apple one doesn't). I don't know exactly how you go about doing that though. I'd probably start with the HSI forums on dslreports.com

<comment><USER26> See, I have exactly the same setup (SB6xxx + AE) and on TWC in NYC it just doesn't work. The AE gets an IPv6 address from the modem, but TWC isn't assigning it a prefix, so all IPv6 devices behind the AE just sit there twiddling their digital thumbs.

<comment><USER3> TWC is definitely not as far along in their deployment. It might just take some more time. =\

Edit: If I recall correctly, when Comcast was doing their initial deployment they started with end nodes only (not delegating prefixes). Then moved to delegated /64s. Now are moving on to delegated /60s and more.

</comment></comment></comment></comment><comment><USER27> Made this a few years ago but it's still useful - check whether your own site is accessible over IPv6: [http://ready.chair6.net](http://ready.chair6.net)

Seems this news article isn't -- [http://ready.chair6.net/?url=circleid.com](http://ready.chair6.net/?url=circleid.com)

<comment><USER28> Good tool. Nicely done!

</comment></comment><comment><USER29> Does anyone have a recommendation for a good primer on IPv6? Is there something significantly better than reading the IPv6 wikipedia page or googling 'IPv6 primer'?

<comment><USER6> @muench - we've tried to assemble some pointers on [http://www.internetsociety.org/deploy360/ipv6/](http://www.internetsociety.org/deploy360/ipv6/)

In particular you may find the "IPv6 for IPv4 Experts" ebook of use: [https://sites.google.com/site/yartikhiy/home/ipv6book](https://sites.google.com/site/yartikhiy/home/ipv6book)

</comment></comment><comment><USER30> IPv6 is such a minor win for the Internet now. I'm more excited for a new protocol all together, maybe something like Open Libernet: [http://www.openlibernet.org](http://www.openlibernet.org)

</comment><comment><USER31> Hmm, I wonder if smartphones could "easily" switch to IPv6?

<comment><USER32> Depends on the OS

[https://sites.google.com/site/tmoipv6/lg- mytouch](https://sites.google.com/site/tmoipv6/lg-mytouch) (For T-Mobile, I believe AT&T also has IPv6 available)

Also depends on correct instructions, the linked PDF (under screenshots) there is correct, the instructions at that link do not appear to work.

As for wifi, I can't say, my AP doesn't pass RA's so none of my wireless clients configure to IPv6. Time to add it to the list of stuff to upgrade.

</comment></comment><comment><USER33> A lot of those numbers are deceptive because carriers are starting to use IPv6 for an outer trunking protocol but the users IPv4 flow is encapsulated within.

<comment><USER34> I don't think that's right. The stats are based on users' web browsers hitting the measurement points using an IPv6 HTTP connection.

It would be possible to fake it using NAT-PT, or a transparent HTTP proxy with IPv6 on the Internet-facing side, but I doubt that ISPs would be deploying hacks like that on a significant scale.</comment></comment>


Andrew Yang wants to give 10 families $1k a month - <AUTHOR> https://www.theverge.com/2019/9/12/20863559/andrew-yang-democratic-debate-giveaway-universal-income-ubi-freedom-dividend-election-2020 <comment><USER2> Andrew Yang now has more attention on himself because of this then he could have gotten with any ad.</comment>


On DRY and the cost of wrongful abstractions - <AUTHOR> http://thereignn.ghost.io/on-dry-and-the-cost-of-wrongful-abstractions/ <comment><USER2> I am firmly in the DRY camp with a very important caveats:

1- Do no generalize so you can factor out like code too early. The first time just write it. The second time write it again even if only minimally different. The third time is a judgement call. By the fourth time you see the proper abstraction, something that might have been much different if you were to have refactored early on.

Corollary to #1- Don't run ahead of yourself making all these placeholder functions, interfaces, and abstract classes with the expectation that you would have had to refactor this way later. Unless you done versions of your project a dozen times (even then, sometimes I don't even do it), you are most likely going to be wrong.

2- Don't refactor out single lines unless there a lot going on on that line or you have the experience to know (don't lie to yourself either) that this will be an important and growing line.

3- Sometimes performance can trump. And sometimes you do know this ahead of time, especially if performance is correctness as it is on some projects with very tight constraints.

Besides that. I freaking hate seeing similar code everywhere, blowing up my instruction cache, making changes difficult. Please stop it.

<comment><USER3> In my experience, there's a costly dimension to DRY that doesn't get enough love: dependencies.

DRY incurs a significant dependency cost. This has been painful for me, since I work on distributed systems a lot. When you have an ardent follower of "DRY" who refactors a common utility so that many subsystems of a distributed system have a code dependency on the same piece of code, then changing that piece of code can become very costly. When that cost gets too high (e.g. requires coordinated deployments and lots of cat herding with owners of other systems), then the DRY has just painted you into a corner where the cost of changing your code is higher than it's worth, and you end up with a rotting code base because the cost of change is too high.

Please, please factor that cost into your decision to wield DRY in distributed/service-oriented systems!

Code dependencies have diminishing returns in increasingly distributed systems...try to find a way to encode the common functionality as data instead of code and you'll be way better off.

<comment><USER4> In that case, can't you then just un-DRY (hydrate?) the code by branching it for each subsystem? Then each team can keep its own pace, and still cherry- pick improvements from each other, which would be difficult to do if they were simply pieces of code copied all around the codebase.

<comment><USER5> The opposite of DRY is WET (Write Everything Twice) ;)

</comment></comment></comment></comment><comment><USER6> All too often the kind of "abstractions" we talk about in programming are the leaky, code-indirection kind. These abstractions tend to proliferate in a single code-base over generations of programmers. It leaves one wondering if these thousands of lines of code actually compute anything.

In my experience the software at an organization is often a reflection of its culture. Thick with leaky abstractions, interfaces, facades, and indirection? Years of frustrated mediocre programmers, each slightly confused by the previous generations' shenanigans, led by a clueless management team that care more about deadlines than deliverables. There are signs that people are trying to do better but the code takes years to show improvement as it resists change at every corner. A crystalline entity designed and driven by data, never doing more work than is required? A team that has had little churn and is led by passionate engineers often more concerned with deliverables -- at least at first, because it's often fairly easy to make changes when your code-base adheres to a well-defined algebra, is thoroughly tested, and even has a well- formed specification of its critical components.

 _DRY_ is as good an idiom as "measure twice, cut once." The difficulty is that software is often more complex to understand than a cabinet. However I can't slight it for being a bad idiom because of the wide variability in skill for such a difficult task as programming. We need something to teach new programmers that at least get them _thinking_ about the abstractions they wreak havoc with.

<comment><USER7> Your comment reminded me of Edmond Lau's book - The Effective Engineer [1] - where he talks about putting in a good amount of effort into the onboarding process for new engineers.

His premise - having a senior engineer spend an hour a day for the first month helping the new employee with explaining the existing abstractions being used, the underlying design of various systems, etc. - would still be only about 20 hours, which is still only 1% of the number of hours that employee will spend in their first year - about 2000 hours.

As a result, I believe that armed with that knowledge, the new employee is likely to be much more productive, failing which, at least cause less damage to the code base.

I would say that the first example you mention - leaky abstractions et. al. - are just as much (or maybe more) due to poor onboarding as they are due to the frustration of mediocre programmers. There is a lot to be said for good process, which software engineering as a discipline falls short of quite consistently.

[1] [https://www.amazon.com/Effective-Engineer-Engineering- Dispro...](https://www.amazon.com/Effective-Engineer-Engineering- Disproportionate-Meaningful/dp/0996128107/)

<comment><USER6> Agreed!

I would wager that 90% of _good_ software (by a most liberal definition) is designed to be that way by the methods employed to construct it. It's the leaders and managers who set those processes and standards. If they are well versed in the state of the art and understand how to make it work for the business you can end up with crystalline entity software. Most of my job is presently "hacking the process" rather than the code itself so that the team can produce the best, desirable results.

I'll check out that book, thanks for the recommendation.

</comment></comment></comment><comment><USER8> The idea of 'don't repeat yourself' is an ok guideline unless, like most proverbs/sayings/mottos/slogans it gets used as a absolute rule.

feel free to ignore it, mock it and toss it aside if it leads to bad abstractions, highly convoluted structures or write-only code.

anyone that has had to do maintenance or adding features to a large OO codebase from the early 2000s will have seen vastly massive class hierarchies where following the path of execution is a roller coaster ride through 10 files for what could have been a 30 line function.

these days grep/vim/sublime/emacs/and the rest of the gang all have power search and replace editing functions. sometimes it's better for the whole world to just copy-paste-alter your code.

<comment><USER9> _an ok guideline unless, like most proverbs /sayings/mottos/slogans it gets used as a absolute rule._

Amen!

 _a roller coaster ride through 10 files for what could have been a 30 line function._

 _sometimes it 's better for the whole world to just copy-paste-alter your code._

Then what happens when you have almost a hundred copy/pasted slightly rewritten 15-30 line variations on the same theme? How do you refactor then? (Yes, I have seen this is production systems, and yes, it was very critical code!) As you say, it comes down to cost/benefit.

Basically, you just have to keep potential refactoring/rewrite costs down so that you are never trapped. Caveat: You can seldom predict the risks as well as you think you can. What you can depend on, is being observant to historical patterns in your codebase. It's hard to predict the business needs and the architecture in the future. On the other hand, it's often quite easy to see the historical trends in the codebase.

Really, the analogy of the physical file room is a great one. (Or if you're not familiar, a library, a tool chest, or any kind of physical inventory system works too.) You can have a file clerk that seems like "super-filer" because he never "wastes" time by putting files back, but this never works out in the long term. The same goes for a filing staff that never reorganizes the file room. Also, one can generally see the long term disaster developing long before it results in the dramatic disaster. You can see which shelves are getting filled up, and which drawers are getting overstuffed, usually weeks or months ahead of time.

<comment><USER10> > Then what happens when you have almost a hundred copy/pasted slightly > rewritten 15-30 line variations on the same theme? How do you refactor then? > (Yes, I have seen this is production systems, and yes, it was very critical > code!) As you say, it comes down to cost/benefit.

The only thing that can happen - you understand the core function those 30 variations solve and introduce a full parametrised solution to the problem, then replace all places with calls to it. Generally there would be some way to tell where all these copy-pastes are using something not much more complex than a regex. But _even then_ , maybe just leaving it duplicated is better.

<comment><USER11> The concern I have with this approach is how easy it is forget to alter one of the variants when you alter one of them.

How do you know the code you're touching has other versions that are semantically the same and should also be altered?[1]

How do you avoid having to fix the same bug several times because you bug- fixed one place but not the others?

How do you avoid the technical debt that builds over time when instances of the pattern within the codebase are each at subtly different "versions" with similar, but not identical, semantics (even though identical would have worked fine?)

[1] Of course, DRY code has the inverse: How do you know if an existing function to do what you want to do already exists so you can avoid duplicating the extraction?

Where my opinion falls today is: A slightly more complex solution is often less risky and more maintainable than the straightforward duplication solution because at least the complex one looks complex to a would-be maintainer who will at least be aware of things up front, whereas duplicated code can have a bunch of hidden costs whenever it's touched that won't necessarily become apparent until later when the presence of that technical debt throws a monkey wrench into unrelated plans.

<comment><USER12> If your variants need to be similar, that alone is a great reason to abstract. It makes the abstraction more valuable.

Meanwhile, there could easily be transformative code that just computes some stuff you often need. In those cases altering one variant need not effect the others.

</comment></comment></comment></comment></comment><comment><USER13> I've come to the conclusion that there are at least 3 kinds of DRY, and application varies:

1) There's a higher level abstraction in your code. Abstract at your own risk, unless you have at least 3-4 instances, or your architecture absolutely requires it.

2) There's wrappers around libraries and you only use it a certain way. I'm OK with this here. Sure, I could copy-paste the same exact parameters each time I use it in my code, but I will just write my own fiddleFoo() and use that every time.

3) There's when things MUST BE THE SAME. You should abide by DRY even if you only have 2 instances. E.g., we have an SOA and we DRY our routing. Routing MUST match or else things break.

</comment><comment><USER14> I agree with this important article.

In the large open source project I work on, we have a prolific contributor who is systematically going through the entire code base, "refactoring" some of it in way the impairs my ability to read, understand, and maintain the code (even code that I originally authored). Over time, we've added scores of new macros -- as result ordinary C knowledge is now insufficient to read and understand what the code is doing.

As the author suggests, the guiding principle should be factor only when it results in a net reduction of complexity. In some cases, the cost of adding new abstractions outweighs the gains from Don't Repeat Yourself.

There is some value in having some blocks of code that can be read in isolation, even if there some bit of repetition between blocks.

<comment><USER15> The problem likely isn't DRY, the problem here is poor abstractions.

<comment><USER16> The problem here is that our so-called editors cannot inline calls for us.

<comment><USER17> I feel that would just hide other problems. As soon as you pull some code out into a function, it is inviting other uses of it. As soon as someone fixes the function for their use in a way that breaks it for yours...

<comment><USER18> ... then your unit tests make it clear that they shouldn't do that.

<comment><USER17> Statistically, relying on tests to cover areas of the code that you are not actively working on is not a reliable methodology.

That is, yes, ideally you have a test catch something. Realistically, you don't have 100% test coverage.

<comment><USER18> Sure. That's why there are more quality-oriented practices than unit tests. Personally, I'm also a big fan of pair programming, collective code ownership, automated functional testing, good run-time monitoring, weekly retrospectives, and doing a five-whys for every bug. Even with that bugs happen, but hopefully not very many of them.

Still, when I'm worried about other programmers breaking something, I write unit tests. Library code should have good tests that document the purpose of the shared function. And my general experience is that when I break some shared library function anyhow, I'll see other tests going red. Unit tests are my first line of defense for this sort of problem.

</comment></comment></comment></comment></comment></comment></comment><comment><USER19> I am firmly in the DRY camp, although obviously there are limits to how far you should go to maintain the dry-purity. In my experience, the biggest issue with copy-pasting code is that someone else (or your future self) will edit the code in only one of the places, making the codebase inconsistent. If abstraction gets too costly, I have found it useful to simply add a comment in both places, "copied from x" and "pasted from y" respectively. Or, if there are constants involved, break those out as global constants.

</comment><comment><USER20> The core of the problem is that people often confuse "parameterization" with "abstraction":

Parameterization is simply adding adjustable knobs. For example, a function can take as argument a gigantic list of flags, where each particular combination of flags makes the function do something slightly different.

Abstraction is actually separating concerns. It's dividing a program in logically independent parts, whose correctness can be established individually, without worrying about the others. Abstraction means that software components only interact with each other through explicitly designated interfaces, often enforced by the programming language itself.

Parameterizing is easy. Abstracting is difficult. Unfortunately, parameterization without abstraction is a source of headaches in the long run.

<comment><USER17> There is a quote from Pólya that I love in margin notes of Concrete Mathematics.

        There are two kinds of generalizations.  One is cheap
        and the other is valuable.  It is easy to generalize
        by diluting a little idea with a big terminology.  It
        is much more difficult to prepare a refined and
        condensed extract from several good ingredients.

I think it applies here, fairly well. Just finding a way to not repeat yourself is relatively easy to do. Finding a way to not repeat yourself, but keep the same level of information conveyance is actually quite difficult.

</comment></comment><comment><USER21> New Motto: Don't Repeat Yourself, Unless You Need To Repeat Yourself, In Which Case Repeat Yourself, But Be Careful How You Repeat Yourself ...

DRYUYNTRYIWCRYBBCHYRY

<comment><USER22> Dry, un-try, wicker baby cherry.

</comment></comment><comment><USER23> DRY is a problem because it's so catchy and _seems_ so self-explanatory.

The original formulation of DRY spoke in terms of information. Any _piece of information_ should exist in exactly one place in your code. As I've said before, modulo some caveats related to double-entry bookkeeping, this seems exactly right. An inveighing against anything that looks repetitive is not.

The goal is not _compression_ , it's ease of understanding and ease of making correct modifications.

I've started using "Huffman coding" to refer to the practice of misapplying DRY by deduplicating superficial similarity rather than information content.

<comment><USER22> This is a cogent way to frame the acronym.

There are times when repeating yourself is just accidental. I might have the same middle name as a coworker. Do we actually "share" that middle name? That depends on who's asking, or what system you're trying to build. But probably not.

</comment></comment><comment><USER24> "It can be expensive to use the wrong abstractions, therefore you should copy and paste" is not sound reasoning. The _right_ abstractions can be incredibly (insanely, ridiculously) beneficial to a code base across all dimensions -- correctness, maintainability, performance.

Searching for the right abstractions is definitely worthwhile - even if it sometimes incurs extra rewrite/refactor churn. The better the abstractions used, the easier the refactoring, the faster you find the good abstractions, the easier it is to change the code as your understanding of the problem grows ...

<comment><USER25> He's saying duplication can sometimes beat inappropriate abstraction. You're saying appropriate abstraction beats both duplication and inappropriate abstraction. (Which I thought was obvious.)

</comment></comment><comment><USER26> More on the subject: [http://www.sandimetz.com/blog/2016/1/20/the-wrong- abstractio...](http://www.sandimetz.com/blog/2016/1/20/the-wrong-abstraction)

<comment><USER27> > If you find yourself passing parameters and adding conditional paths through > shared code, the abstraction is incorrect. It may have been right to begin > with, but that day has passed. Once an abstraction is proved wrong the best > strategy is to re-introduce duplication and let it show you what's right.

I like this take on when go full DRY or not a lot. It's very easy to abstract out two things that look identical in their earliest implementation, but are actually different in intent and function--then when you get a new feature request on one of them they become obviously wildly different, and then to maintain the abstraction you have to throw in a bunch of conditional blocks into it and it turns into a nightmare to maintain.

I heard a rough rule of thumb to stay away from abstracting anything until you see it repeated identically at least three times, not two. It's just a rough rule of thumb to try and make sure you're not prematurely abstracting something and building the wrong thing, but anecdotally it's been useful for me.

<comment><USER9> _> I heard a rough rule of thumb to stay away from abstracting anything until you see it repeated identically at least three times, not two._

I used to work in a shop with that rule, and I've mentioned it here on HN. It's still pretty easy to track down and rewrite 3 occurrences. Heck, it's still pretty easy to track down and rewrite 7, though as you let the numbers increase, you run the risk of missing an occurrence, making a mistake when you do the refactor, or the introduction of a confounding "idiosyncracy" in one of those occurrences.

In my experience, 2 occurrences is too little data to justify refactoring around, unless those are pretty hefty chunks of code. 2 occurrences of 2 consecutive lines is definitely too little.

<comment><USER23> I think the focus on counts is wrong. The first question is "are these the same _for a reason_." If there are two places in the code that need to behave the same _or the code is broken_ , then you should probably pull it out. If there are 100 places in the code that happen to behave the same, but any subset of them could change independently tomorrow, then you might not want to pull it out.

</comment></comment></comment></comment><comment><USER16> If we could track copy-pasted chunks, then manual abstraction step would be needless, as it is already clear from editing history. We could then transform between abstract-nonabstract source or completely remove that distinction by making it view property. And modify one place and quickly update (or not) all others. We'll also have to deprecate 'modern' text sources and store code in structured way.

That should be all obvious to anyone with 10+ programming experience, imo, but I always see such "oh, dry or not, kiss or not" jitter. We now have great tools to do everything, except programming. Someone must make programming app better than just notepad-with-word-completion junk.

<comment><USER28> Ever heard of a function? If it's as simple as copy paste, that's all you need. The trickier abstractions come when you need something to be almost the same in two places. Copy and paste tracking wouldn't work there.

Replacing functions with some kind of copy paste tracking editor does not sound like an improvement.

<comment><USER17> Coverity actually has a really good tracking ability where it will find code that was copy/pasted/edited but the edit looks wrong.

I can't remember the name, but it will identify what it thinks is the original, and show where the copy/paste is, along with the edit it thinks was missing. This was usually around some error handling logic in our code base. Think:

        if (checkIsValid(foo)) {
          printf("Looks like foo is not valid.  foo == %s\n", foo);
        }

Wasn't uncommon for someone to copy that and forgot to change all places "foo" appeared.

Long story short, advanced static analysis has come a long way.

</comment></comment></comment><comment><USER29> High level of abstractions accumulate especially in big monolithic applications and I agree it takes time to get the feature in. Especially, you have to get an idea of what code has already been written and how it plays with the components before you start doing anything. First, to not repeat yourself and also to lessen the chances of doing something wrong. I think everything has its trade-offs and use cases. This approach in large applications could make you write less tests, as there is less amount of repeated code. On the other hand you can split your project into independent smaller parts and at the cost of sacrificing DRY you can get something more easy to understand, but then you have to put more effort on testing and integration testing to see if all bits play together correctly and as expected. I agree that the sense of what should be abstracted and what shouldn't comes with experience. It is also very important to get as much information about the problem - this helps choosing the right approach in the context of above.

<comment><USER30> I tend to take DRY more from a standpoint of data and code to transform that data into other data.

I'm less concerned about duplication in code that uses data than I am the code that determines whether the data itself is correct.

</comment></comment><comment><USER31> Here's a research article I wrote on this topic; there's a section on the "costs of abstraction":

[http://harmonia.cs.berkeley.edu/papers/toomim-linked- editing...](http://harmonia.cs.berkeley.edu/papers/toomim-linked-editing.pdf)

<comment><USER32> Interesting. Does it scale? (Skimmed the paper and believe have understood the process but examples you have are limited to n=2 clones. What if n>>2?)

</comment></comment><comment><USER33> DRY is rather language-dependent. In Java, where only single result can be returned from a function, it is often not worth the efforts to factor out common functionality if that requires adding an extra class just to pas few in/out parameters.

In C++, on the other hand, the barrier to follow DRY is rather low as a I can literally move code to a helper function and easily parametrize it for reuse in another place.

</comment><comment><USER34> Am I the only one who thinks this sort of advice would be immensely more useful with some concrete examples?

<comment><USER35> I was thinking the same thing, it's hard to judge this article without knowing exactly how much code duplication the author thinks is appropriate and in what circumstances.

</comment></comment><comment><USER18> I think this article could benefit from a few good examples. Sure, one should never apply principles in a rote fashion; they're guides to judgment. But the way we develop good judgment is through practice. Without seeing how he'd apply this in specific situations, I don't feel like I've really learned anything.

</comment><comment><USER36> My complaint with DRY - sometimes it's just significantly easier to learn a codebase or learn the problems that led to its structure/abstraction by just writing many of the underlying components yourself. Whether that code goes into the codebase is somewhat irrelevant from a learning perspective.

On top of that, when 95% of the code you work with is based on a "Minimal Viable Product" (the bane of my existence), assumptions will probably be made incorrectly, or in a way that adds significant technical/organizational debt. By doing it yourself, the original assumptions can be put into a different light and can help reduce all sorts of debt in the future.

</comment><comment><USER37> I think we need to address this giant elephant in the room called object oriented programming. [https://www.youtube.com/watch?v=IRTfhkiAqPw](https://www.youtube.com/watch?v=IRTfhkiAqPw)

</comment><comment><USER9> _Yes, you read that right. Duplicating code (a.k.a copy + paste) can be a good thing. Namely when the abstraction that would replace repetitive portions of your codebase is a pain to understand._

Yes, this 1000 times! DRY or any pattern/principle is not some sort of universal truth/declaration of ultimate goodness from your deity or some deep law like the Heisenburg Uncertainty Principle! _It 's just a rule of thumb_. As a practical technologist, you must always implement with respect to the _contextual cost /benefit_. Taking any principle as an automatic ultimate good is simply intellectual laziness!

I was in a company with a Java product and a Smalltalk product. The Smalltalk product was rather well factored. We had a policy of rewrites to keep things architectually clean. We had the _fantastic_ Smalltalk Refactoring Browser as our IDE, and we weren't afraid to use it. (To this day, you can refactor an order of magnitude or more faster with it than you can in other languages.) The Java product consisted of many independent parts with tons of duplicated code between them. Definitely not DRY. (There were also 20X more Java programmers than Smalltalk, for a comparable product.)

One thing I noticed, is that bugs on the Java side were more prevalent, but never affected the entire application. Whereas, the far better factored Smalltalk product could occasionally have the rug pulled out from underneath the entire application by the introduction of a library bug. (Though it would be very rapidly fixed.)

Another thing at that shop: We had a policy of only empirically verified DRY. We never wrote code to DRY in anticipation: only in response to actual code duplication. Generally, actual practice and experience is better at producing exactly the needed architecture than the prognosticating cleverness of programmers.

EDIT: Shower thought! (Yes, I literally got out of the shower just now.) There is a lot of _emphatic_ but somewhat thoughtless application of principles and rules of thumb because it's actually motivated by _signalling_. The priority is actually on the _opportunity to signal_ and not on the underlying cost/benefit calculation. There is a nice analogy for this in music. Once a musical scene gets a certain degree of prominence and/or commercial success, people start prioritizing sending the signal that they are part of that musical scene. So you will find bands and musicians using those techniques and stylistic flourishes, not because of the underlying aesthetic principles in support of the particular piece of music, but as _an opportunity to signal_. This happens in OO and also in the startup scene. (And, OMG, but is _signalling_ given a lot of energy in the Bay Area!)

So the insider move is this: Look for the underlying motivation behind the signal. Did this person jump on the opportunity to signal, or did they first seek out the information to make the determination of cost/benefit? How carefully did they do that? (Hopefully, we can get some of the hipsters/pretenders to at least go through the motion of doing the above. Even just going through those motions turns out to be beneficial.)

<comment><USER38> My take: I try to write it DRY first. If I fail to come up with a good intuitive abstraction after half an hour, I just write it simple and copy code.

The next day or later I come back to the code and sometimes I find a neat abstraction then, which just needed more context. Sometimes I can get rid of the whole section too...

It all... Depends.

<comment><USER39> Half an hour? When you're first writing the code, you should know immediately if you're going to need to repeat it and what the abstraction should be. If you don't, then don't waste your time. Later, if it turns out that you actually do need to repeat the code, it should be clear at that point what the abstraction should be.

Let the actual need for duplication guide you in finding abstractions, so you don't waste your time and produce difficult-to-follow code for no reason.

<comment><USER38> Like I wrote I avoid producing difficult to follow code.

I tend to start structuring parts of applications and algorithms on paper, and this half hour with paper and pencil helps a lot.

</comment></comment></comment></comment><comment><USER3> My own experience with DRY is that it has diminishing value in a distributed systems ecosystem. There is definitely still value, but code coupling between components of a distributed system is very costly. I'm pretty fed up with coordinated deployments, big bang upgrades, etc., so I'm very conservative about applying DRY. Often I'd rather copy a bit of code to avoid the dependencies, which usually pays off better in the long run.

</comment><comment><USER40> Easily solved: When writing an abstraction, document with a _date_ the reason for the abstraction.

Seriously, a comment that goes like this:

"2009 sept 20 : Peter Pan: this abstraction is to hid the ugliness of using SQL Server 2003 which is still in use by Slow Corp."

... now in 2016 a maintainer knows the assumptions and the reasons for the abstractions and can make a much clearer decision about how the abstraction should be treated.

</comment><comment><USER41> Any time you repeat some block of code twice, you should religiously turn it into a function which is called twice with the same arguments.

Any time you call a function twice with the same arguments, you should religiously write a two-iteration loop around just one call.

Any time you have more than one two-iteration loop in the same program, you should write a 'dotwice' macro and use that instead.

Any time you have a function call sitting in a loop just for the sake of being repeated, you should avoid the argument expressions being evaluated multiple times; you must religiously evaluate the arguments to temporary variables outside of the loop, and in the looped function call, refer only to the temporaries.

In order not to repeat this pattern itself you need a proper "calltwice" macro, and to be working in Lisp, ideally: (calltwice yourfunc expr ...). yourfunc and each expr are evaluated once, then the resulting values are applied to the function twice.

This is a DRY town, damn it!

</comment><comment><USER42> For a more visual explanation of the "cost of abstractions" watch this presentation by Cheng Lou (Facebook). [https://www.youtube.com/watch?v=mVVNJKv9esE](https://www.youtube.com/watch?v=mVVNJKv9esE)

<comment><USER43> This is an excellent talk which introduces some very solid concepts and breaks down the problem in a very sophisticated way. I hope more people watch this.

In fact, I think this talk is superior to the linked article. You should consider submitting this talk directly to HN.

</comment></comment><comment><USER44> Yes, this is known as "Beware the Share" [http://programmer.97things.oreilly.com/wiki/index.php/Beware...](http://programmer.97things.oreilly.com/wiki/index.php/Beware_the_Share)

</comment><comment><USER45> This article needs to be shown to all upcoming architects, senior developers, and design minded engineers. Too often the balance between abstraction and specialization isn't thought about explicitly but rather dogmatic rules kick in.

</comment><comment><USER46> Sometimes I think programmers and theologians have a lot in common. Both have dogmas and articles of faith which are backed by internally consistent logic. Now to the comments to hear the Catholics and Protestants debate!

</comment><comment><USER47> DRY etc should just be guidelines, not rules. I hate when something is made overly complex just to avoid repeating, global variables, long functions or large files. For example complicated one liners in a imperative language.

</comment><comment><USER48> Can we just ban coding posts that have no code? They usually give you an illusion you learned something but are a total waste of time. It's like reading a self-help post on following your biggest dreams or something. The hard part isn't these big beautiful ideas, it's how to actually put it in practice. How to actually take something away and measure whether abstractions or duplications are more costly in real code??

<comment><USER49> This seems like an anti-intellectual argument. For example would "ban" many of Dijkstra's famous numbered papers. I'm guessing you haven't read them.

While we're at it, why don't "we" just "ban" \- mathematical proofs without numbers \- all criticism, as it's about art but contains no art \- all papers in the field of music theory, since they aren't music.

<comment><USER48> It's definitely anti-intellectual. But people who run this site themselves say they're interested in high signal to noise ratio.

I don't think those are good comparisons. Better example would be a post about how to write rigorous math proofs and then not giving a rigorous proof as an example.

<comment><USER49> >> Better example would be a post about how to write rigorous math proofs and then not giving a rigorous proof as an example.

Oh, you mean like some of Dijkstra's numbered papers? [https://www.cs.utexas.edu/users/EWD/transcriptions/EWD10xx/E...](https://www.cs.utexas.edu/users/EWD/transcriptions/EWD10xx/EWD1012.html)

I think "examples might help" is valid feedback, but the author has sort of provided a reason why examples aren't included: in small projects the abstractions are usually worth it, was my read. So while the author could talk about examples, he probably couldn't include then in a piece of this size.

"Can we ban" is what has rubbed me wrong way. Who is "we". How are "we" "banning"? Does this just mean you don't agree with this post being highly upvoted? Or that you want to circumvent the opinions of those who upvoted with a "ban"?

It is interesting also to think about the irony of an argument that one should work in terms of concrete examples of the subject matter, in _objection_ to a piece that argues one should not always abstract things.

</comment></comment></comment></comment><comment><USER47> Something that I see more and more of lately is module exaggeration, like projects that have tens of thousands of small files that all depend on each other and are complexed together using modules. There are 5-10 imports then a few lines of code that binds them together and finally an export ... The worst part is that this is done by respectfully software companies and even taught in guides and tutorials.

</comment><comment><USER50> The DRY principle issues usually apply very well to OO programming and the whole "framework" concepts, where individual libraries or pieces of the code cannot easily be reused for different projects.

Another big issue in the framework case is the inversion of control, which further restricts how the reuse can happen and makes it very hard to reuse the code differently in an efficient way.

</comment><comment><USER51> I was younger and I thought that the right abstraction - and more specifically, the right _language feature_ \- would surely lead me to higher productivity. By invoking a certain keyword my code would _magically_ be better in some way.

However, language features are not the only abstractions you have available. In fact, they're typically the least interesting ones you could apply to a problem - any yokel can lift code out and slap parameters, annotations, reflection, or generic type signatures on it. But to reach interesting abstractions, you have to be patient enough to do discover the shape and flow of the code over time and subsequently apply exactly the right data structures and algorithms to either reproduce the same shapes in an abridged form, or different shapes that achieve the goal more efficiently.

So when I position myself as anti-DRY, which I often do now, I'm saying, don't succumb to the thinking that just shovelling things around and slightly repackaging them will add up to what you wanted. An actually useful abstraction cuts so deep that it may amount to a different thing altogether. The language features are there to solve very straightforward situations that are known to reoccur time and again. Do not use them in clever ways.

</comment><comment><USER52> Frankly without encapsulations inside of encapsulations, the author would have never been able to make this blog post. Reference TCP packets wrapped inside of Ethernet 2 frames as the basis of the entirety of the internet.

I don't think this is as "extreme" as some perceive it.

</comment><comment><USER53> Off topic, but since somebody will always complaint about the bad layout or the difficulty of reading of a blog post submitted to HN, I'd like to point out the excellent readability of this one.

<comment><USER54> The stark blacks kind of burned my eyes, but I definitely agree it was readable.

<comment><USER53> One can always lower the brightness of their screen, but they can't easily change the contrast in the font/bg.

Besides, research has shown that plain ole black on white is the best combo for readability.

</comment></comment></comment><comment><USER55> Figure out what varies => encapsulate that. Could be behavior, structure, or creation. This idea that you can avoid DRY in a positive way simply means one has yet to recognize that first statement.

</comment><comment><USER56> I think the title should be "wrong abstractions." Wrongful means unfair/unlawful/unjust but it doesn't ever mean incorrect.

</comment><comment><USER57> Article in a nutshell: Goodness = DRY×KISS

</comment><comment><USER58> Thanks for the article. I agree with its main points

</comment><comment><USER59> Another thing that is difficult to get right is choosing a suitable font...

</comment><comment><USER60> TL;DR Think before you type.

</comment><comment><USER15> I would have taken it more seriously if he addressed security and unit tests. His perspective here is sophomore. He is right, but not because of what he wrote. DRY isn't just about abstractions, it's about the cost of writing new code. And testing and security verification are hidden costs to duplicated code.

<comment><USER61> > _His perspective here is sophomore._

An HN comment usually gets surprisingly better if you take the name-calling bits out of it. I'd say that's the case here.

<comment><USER62> I understand that it was a bit blunt, but isn't referring to a perspective as 'sophomore' just another way of saying it was short sighted or that it doesn't take the broader ramifications into account?

<comment><USER61> Yes, but to my ear it also includes a putdown, along the lines of 'juvenile' or 'childish'. That's why I referred to it as name-calling.</comment></comment></comment></comment>


Ask HN: Resources for image pattern recognition algorithms - <AUTHOR>

I am interested in extracting a set of objects from a large, noisy image. For instance: if I had a high-resolution picture of a seating section at a stadium, I might want to extract each baseball cap into individual images given a second picture of a prototypical baseball cap.
There is obviously a lot of research published on this subject (by subject I mean image pattern recognition as a whole). I've been pouring through the IEEE, as well as just searching via Google for a couple of days.
My problem is that this is so far from my field of expertise, I feel like I'm wasting an awful lot of time reading research papers that are not applicable or are outdated.
So my question is, if anyone has any experience in this field or a related field, are there a set of known problem definitions I could narrow down my research into? What about known classes of algorithms (and by that I mean more specific than "machine learning" or "neural networks")?
Any advice is mightily appreciated. <comment><USER2> You are right you're wasting time. Image processing is not the sort of a thing you can pick up on your own "to get work done"; it's extremely demanding and actually fun. Just hire someone.

If you're prone to falling into "hack mode", learning this stuff will not help you one bit. The problems are far too interesting and encompass wide areas of research that are guaranteed to please everybody; low level bit manipulation, file formats, numerical methods, signal processing stuff with filters and sampling, wavelets, edge detection, rank, laplacians, convolution, dithering, ray tracing, morphology, neural networks and other genetic learning algorithms, heaps of inner-loop and vector optimization, scene detection stuff that make use of ray tracing plus interesting data structures like octrees, statistics, information theory .. in a nutshell, it's something to give up work, wife and kids for.

Don't ever let curiosity drag you into that tar pit, hire someone.

</comment><comment><USER3> I could consult our very own pixcavator: <http://news.ycombinator.com/user?id=pixcavator>

<http://inperc.com/wiki/index.php?title=User%27s_introduction>

<comment><USER4> I wish I could give a "positive" advice but best I can say is that what you describe is a tough, tough problem. There is no off-the-shelf solution and if you are new at this, you are up for a lot of pain with a slim chance of success. Sorry, can't be more helpful.

<comment><AUTHOR> I suppose it depends on the definition of "success". Learning about the state of the art and writing some enlightening code, no matter its ultimate usefulness, would be plenty successful for me.

I'm mostly surprised at the lack of papers I'm able to find that have been published after the mid '90s.

I will give major kudos to you though, sir. I've spent the last hour or so reading through the wiki. Please, keep up the great work.

</comment></comment></comment><comment><USER5> I am quite interested to learn more about object recognition as well, unfortunately I don't have any direct answers to your question.

May I suggest your search for "Content Based Image Retrieval" for research papers in that field.

<comment><AUTHOR> Yes, I've done a bit of searching on CBIR actually. There are a few papers out there about querying at multiple resolutions (like <http://grail.cs.washington.edu/projects/query/mrquery.pdf>), which will definitely be helpful.

The next step, for me, is to use a single image as the set to be queried, without knowing at what scale the target objects appear in the image.

</comment></comment><comment><USER6> I am not a vision specialist, but I work around a lot of them.

Look into the OpenCV image processing library. I haven't used it, but it seems to implement a lot of basic functionality to get you off the ground.

If you can't find anything good from the last ten years, then look at Yann LeCun's recent papers. (Google wanted him to be head of research, but he preferred academia.) In particular, investigate his convolutional networks.

The work of Rob Fergus is more applied, and should lead to good recent pointers.

Look for works experimenting with the NORB dataset.

</comment><comment><USER7> You might want to check out open cv library. They have this thing called the haar classifier/training. You train it to recognize an object and then it can look for that object in other images. Here is one example how it was used to recognize sign language. [http://sandarenu.blogspot.com/2008/06/opencv- computer-vision...](http://sandarenu.blogspot.com/2008/06/opencv-computer- vision-library.html)</comment>