License now displayed on repository overview - <AUTHOR> https://github.com/blog/2252-license-now-displayed-on-repository-overview <comment><USER2> ...but it is only visible if you are logged in. That inconsistency is a little weird.

<comment><USER3> And as with many other information (such as the bloody fucking search) it's only taking in account the repository's main branch and ignoring all others, if you decide to relicense in a branch for some reason that will not be displayed by github.

<comment><USER4> What would you prefer GitHub do? Display multiple licenses? I see that situation as not really solvable.

<comment><USER5> Why not apply the same solution already applied to branches? Have a drop down that lists the different branches and their licenses, with it sorted such that the branches with different licenses are displayed first.

<comment><USER6> I would think that would be UI overload for such an edge case.

</comment></comment></comment></comment></comment><comment><USER7> Dual-licensing seems to not be supported by licensee [1]; MIT/Apache-2.0 is what's used for Rust and a substantial fraction of the Rust ecosystem.

[1]: [https://github.com/benbalter/licensee/issues/57](https://github.com/benbalter/licensee/issues/57)

</comment><comment><USER8> Anything we can do to improve the usability and detection of licenses is a big win in my book. Even the best intentioned people have a hard time consistently doing the right thing -- whether that's attributing the right copyright owner, reproducing the right copyright message, or otherwise abiding by the license, it can be hard to be 100% correct.

It would be nice if this was more programmatic. If licenses had an identifying string they carried so they could be clearly and consistently identified, and machine parseable metadata. If we make it easy to build automation around license files, we make it easy to do the right thing -- maybe even easier than doing the wrong thing.

I think about something like Webpack. If Webpack could just traverse your project structure and parse all your dependencies license files and generate the proper attributions, that would be amazing.

<comment><USER9> You might be able to write that as a Webpack plugin with a couple of hours work... could be neat.

</comment></comment><comment><USER10> This is really cool. One little detail I've noticed is that it doesn't seem to apply to private repositories, which seems like a bit of an oversight. The repo I'm checking clearly having an Apache 2.0 license text in the LICENSE file. Just because a particular copy of a repo is private doesn't mean that the code within it isn't still bound by an open source license.

EDIT: Before anyone replies with "Why would you want that?", it's fairly common to stage a project privately on GitHub before publicly releasing it. It'd be nice to see that the license is detected correctly before going public with it. As it is now, I don't know what'll happen until I go public, and my first public commit in the repo may well be fixing up something minor to get the license detected properly.

<comment><USER11> It seems to work fine on my org's private repos. Not sure if it's having trouble detecting the license in your case, or if it just hasn't worked through all the repos yet, or what.

</comment></comment><comment><USER12> This is nice, but the way they gather license data has _serious_ limitations if it hasn't improved in the last year.

[https://lwn.net/Articles/636261/](https://lwn.net/Articles/636261/)

<comment><USER13> I was thinking the same thing. Just a few years ago GitHub used to automatically "figure out" what programming language your project was in, and it was pretty bad.

<comment><USER14> Github uses an open source gem [0] to detect the license. You're welcome to contribute enhancements.

[0] [https://github.com/benbalter/licensee](https://github.com/benbalter/licensee)

<comment><USER12> Looks like although the README says it only looks at LICENSE, it has improved and checks more files; nice!

[https://github.com/benbalter/licensee/blob/3692df44ab32772a9...](https://github.com/benbalter/licensee/blob/3692df44ab32772a98a3aec8a44080f7dbe3d50a/lib/licensee/project_files/license_file.rb#L15-L25)

</comment></comment></comment></comment><comment><USER15> Another feature copied from gitlab. Not that that is a bad thing.

<comment><USER16> Gitlab is a dying product.

<comment><USER17> If it's dying why would GitHub implement their features? Do they want to die too?

<comment><USER18> It's called "inheritance". It's an OOP concept, you functional kids won't understand.

</comment></comment></comment></comment><comment><USER19> A nice small enhancement. While many projects mention the license in the README, not all do. So it is very easy to see at a glance which license a project is under. In this context, I really appreciate how github offers to add the license while creating a new repository with a quick list of the most common licenses. A good incentive to put a new project under a proper license from the start and making sure that the correct license terms are attached.

</comment><comment><USER20> It's a good idea to make this more visible, but they should also make it visible when a repository does not have a license. If they did the UI right this would encourage developers to license their code in the same way that Facebook encourages users to add a profile picture.

<comment><USER21> I don't think they should do that, for different reasons.

For example, lots of projects have more than one license, but GitHub seems to list only one. Incorrect information is worse than no information.

Besides current implementation only seems to pay attention to most popular licenses (?)... arguably CERN Open Hardware Licence is not amongst the most popular, but for some reason I was expecting GH to detect that one ;) So again: incomplete information.

Perhaps asking the repo owner to manually specify the licence could be an option, but if the repo owner cares about licences, I'm sure that information is available already.

</comment></comment><comment><USER22> I've noticed that it doesn't distinguish between the different versions of the CC licence. It just puts them all under CC BY 4.0

</comment><comment><USER23> Just checked a few of my own projects. Github doesn't recognize LGPLv3 properly, where the license is split into a GPLv3 "COPYING" file and an LGPLv3 "COPYING.LESSER" file. They ignore the latter and just slap GPLv3 on the project.

</comment><comment><USER24> > We use an open source Ruby gem called Licensee to compare the

> repository's LICENSE file to a short list of known licenses [...]

I have seen more people putting the license of their projects at the end of the README.md than in an individual _LICENSE_ or _LICENSE.md_ file which is what this feature analyzes. I noticed the addition of this feature a few days ago while checking the repository for the Atom.io project, but without an official blog post I was left thinking why none of my repositories _(which are MIT licensed)_ were missing it. I guess reading _LICENSE(\\.md)?_ is good enough.

<comment><USER25> The tool they use explains why they don't just scrape the entire repo. Basically it's because software licences are legally binding and it's not a good idea to put your license at the bottom of an FAQ in your readme. Licensing needs to be clearly deliberate.

<comment><USER26> I think a less charitable explanation is that they hope this will pressure developers into making their license info more consistently available to the advantage of everyone -- like how Google adjusts its search ranking algorithm to punish certain kinds of behaviour/problems.

... which I would be perfectly okay with.

<comment><USER25> Yeah, that's also a valid point. I've contributed license PRs to several projects, just so that the next person doesn't need to trawl through the readme or source code to find the one line that specifies the license in the most ambiguous way possible.

</comment></comment></comment></comment><comment><USER27> This is a really good development. Programmers shouldn't have to worry about copyright; in the world of software, it does far more harm than good and should be done away with entirely. But for now we _do_ have to worry about copyright, and the proliferation of "open-source" Github repositories with no explicit license puts us at risk of being sued in the future if our projects are successful. Little nudges like this will help a lot.

<comment><USER25> I think that developers _should_ think about how they licence their software. Software licences are a cornerstone of the free software movement, and people should really consider what they believe people should have the right to do with their code.

<comment><USER27> Software licenses are a cornerstone of the free software movement because _they are our only defense against copyright_.

I agree that we need to keep thinking about what one person has the right to do with another person's code, but copyright is a terrible starting point for that thought process, centering as it does the business models of eighteenth- century stationers at the expense of the much broader interests of political dissidents, scientists, historians, patients dependent on medical devices, teachers, students, librarians, archivists, whistleblowers, anybody trying to fix broken things, tinkerers, and journalists.

</comment></comment></comment><comment><USER28> How long does it take to index the file? I've added LICENSE.md to my project ([https://github.com/tananaev/traccar](https://github.com/tananaev/traccar)), but the license doesn't show up at the top bar.

<comment><USER29> I think it is because of the .md extension. I have LICENSE files (no .md) and they are picked up correctly.

<comment><USER30> I have my license in LICENSE.md, LICENSE.rst, etc. and they are picked up correctly. For example, [https://github.com/susam/uncap](https://github.com/susam/uncap) and [https://github.com/susam/ice](https://github.com/susam/ice).

</comment></comment></comment><comment><USER31> Is there any way to disable this, or set it manually at least? I'm concerned there may be a situation where the license is detected incorrectly, and then Github brands your repository as being under a certain license, when it really is not.

Also, it seems to be missing a lot of licensing cases. You can have files under multiple licenses in one repository and dual licensed projects. There doesn't seem to be any "GPLv2 only" and "GPLv2 or any later version of the GPL", which is also a big deal. For example Git and Linux are under "GPLv2 only", but are just listed as GPLv2 just the same as they list a license that's "GPLv2 or any later version".

Even more conspicuous is the lack of support for licenses with additional clauses. For example, many compilers like gcc have additional exception clauses allowing you to link the compiler runtime to your library without incurring the GPL requirements. Swift has a similar exception clause for their runtime library. Even though it's Apache v2 which is permissive, Apache v2 still requires distributing the license/copyright info with all copies binary or source for the software, which would be extremely annoying for a compiler to require. LLVM is discussing switching to a similar Apache v2 plus exception as well.

There's also a lot of licenses missing. And I'm not talking about ones that are obscure or equivalent to MIT, I'm talking about for example the Boost license, which is designed specifically to deal with the binary distribution requirements problem, and is popular in the C++ community. [0]

Just looking at the little badge isn't telling you enough useful information. For example, say you are writing a GPLv3 program, and you want to use a library. Looking at the github page shows you its has a GPLv2 badge. But if it's "GPLv2 only" instead of "GPLv2 or any other license" then you can't use it with your project. And the only way you can find that is by reading the license, or at least the README if the author explains the license there. Similarly, just looking at the badge doesn't tell you if the authors have added any additional exception clauses that could be useful in legal compatibility for your program, so you'll have to click through for that as well.

In the end, I think this just provides potential for confusion, and doesn't really make the job of seeing what license the software is under any easier. There could be advantages in listing the license in a standard place and way, but it should allow the developer to manually select the license, and for arbitrary text input for additional notes, or for licenses that github hasn't added. If they insist on keeping it automated, then there should be a way to opt out (of course it'd be best if it was opt-in, but that's not going to happen, so I won't even go there).

0: _The copyright notices in the Software and this entire statement, including the above license grant, this restriction and the following disclaimer, must be included in all copies of the Software, in whole or in part, and all derivative works of the Software, unless such copies or derivative works are solely in the form of machine-executable object code generated by a source language processor._

<comment><USER32> Hey, Tim, I've talked a number of times with GitHub about the issue of lack of nuance in their license monikers on GitHub. I haven't gotten very far, but this change will cause me of course to raise it again.

As others have commented, kragen is jumping to excitement a bit too quick. GitHub's license data is not curated, and I'm quite sure determining a license of software is an undecidable problem; it just requires human judgment, and many self-report their own license incorrectly to GitHub.

GitHub has a lot of problems to solve before the license data they are presenting can be trusted. Even many very common programs have licenses that can't even be described with an SPDX moniker, so the "badge method" just isn't going to work.

</comment></comment><comment><USER33> I noticed this a few days ago and I think it's a great addition. I hope they keep this feature.

</comment><comment><USER34> This is great! I often find myself clicking through to the license file and trying to remember which license text it looks like. This'll be both easier and more reliable.

</comment><comment><USER35> Neat.

</comment><comment><USER36> Interesting that they're devoting screen real estate to something that can be found in the data section.

<comment><USER10> What is the data section? I don't see that on GitHub.

<comment><USER36> Err, the file browser area.</comment></comment></comment>


Looks like Bootsnipp.com has been hacked (and defaced) - <AUTHOR> http://bootsnipp.com/ <comment><AUTHOR> But I wonder why? I wasn't aware that Bootsnipp had any political/ideological message that would clash with ... well ... anything?

<comment><USER2> Most of the times, hackers hack the websites just to convey their hatred for something. Website being linked to an ideology or political organization hardly matters to them.

OR

Sometimes they hack a website on a server and if symlink is ON, other websites on the same server can be compromised.

</comment></comment><comment><USER2> Yes looks like it has been hacked and defaced.</comment>


One Reason It's So Hard to Become a 'Creative Superstar City' - <AUTHOR> http://www.citylab.com/work/2015/05/one-reason-its-so-hard-to-become-a-creative-superstar-city/394256/

<comment><USER2> It's worth noting that this article is written by Richard Florida ([http://en.wikipedia.org/wiki/Richard_Florida](http://en.wikipedia.org/wiki/Richard_Florida)), whose 2002 book _The Rise of the Creative Class_ ([http://www.creativeclass.com/richard_florida/books/the_rise_...](http://www.creativeclass.com/richard_florida/books/the_rise_of_the_creative_class)) introduced the idea that struggling cities could find new prosperity by reinventing themselves to become more attractive to the "creative class" \-- Florida's term for the mostly young, mostly white, mostly techie types who have risen to wealth and prominence in San Francisco and New York.

And that Florida raked in tens of thousands of dollars in speaking fees per speech and hundreds of thousands in consulting fees per gig from lots of those struggling cities, all on the premise that he could show them the way to convince the members of that "creative class" to upend themselves from SF and NYC and move there. (See [http://www.city- journal.org/html/14_1_the_curse.html](http://www.city- journal.org/html/14_1_the_curse.html), [http://www.newrepublic.com/blog/plank/105746/gay-bars- bike-p...](http://www.newrepublic.com/blog/plank/105746/gay-bars-bike-paths- ka-ching-the-creative-class-10th-birthday))

And that after all the checks had cleared but those cities failed to turn around, Florida did some turning around of his own and declared that they had really been beyond hope to begin with (see [http://prospect.org/article/ruse- creative-class-0](http://prospect.org/article/ruse-creative-class-0), [http://www.newrepublic.com/article/114013/richard- floridas-d...](http://www.newrepublic.com/article/114013/richard-floridas- detroit-flip-flop-save-or-not-save)).

And that after a decade of working this circuit, he eventually admitted that programs to attract the "creative class" \-- the programs he'd been selling those struggling cities for all those years -- mostly just ended up making life better for the members of that class without benefiting anyone else (see [http://www.thedailybeast.com/articles/2013/03/20/richard- flo...](http://www.thedailybeast.com/articles/2013/03/20/richard-florida- concedes-the-limits-of-the-creative-class.html), [http://beltmag.com/austin- cleveland-and-the-cost-of-cool/](http://beltmag.com/austin-cleveland-and-the- cost-of-cool/)).

So... yeah.

</comment><comment><USER3> Surprised Austin did not make the list.

<comment><USER4> And New York

<comment><USER5> Many of the artists are priced out.

Moving to NYC in the 70's and getting a brownstone for cheap seems like it would have been an incredibly interesting time.

<comment><USER4> They're getting priced out of Brooklyn too. Not sure what the ideal answer is.</comment></comment></comment></comment>


Kobalt a new build system in Kotlin by Cedric Beust - <AUTHOR> http://beust.com/kobalt/home/index.html <comment><USER2> Just gave this a try on my project, it was surprisingly painless and the build file is really tiny and readable. Very interesting.

Hope this takes off, I'm ready to migrate away from Gradle.</comment>


The Technical Evolution of Vannevar Bush's Memex (2008) - <AUTHOR> http://digitalhumanities.org/dhq/vol/2/1/000015/000015.html <comment><USER2> coincidentally i am watching these treasure trove of videos:

[https://archive.org/search.php?query=subject%3A%22Vannevar+B...](https://archive.org/search.php?query=subject%3A%22Vannevar+Bush+Symposium%22)

after reading the above, highly recommended to watch those!

<comment><USER3> Wow, thanks for the link.

</comment></comment><comment><USER4> After reading "As We May Think", the thing that struck me wasn't so much the predictions that Bush got right (which were many), but one particular technology that he missed: digital image processing and printing.

Everything he describes is based on photographic reproduction of handwritten or traditionally printed material, with an extremely small number of annotation bits encoded by exposing film to individual lamps. He more or less describes barcodes, but misses that they can be printed on the product packaging without special equipment. The idea that a computer could _draw_ things was completely unimaginable.

I actually think that this oversight helps sell how visionary his ideas were. This really isn't a paper that could be written today, because there's this really fundamental thing that everyone takes for granted that's conspicuously absent— It helps anchor the text in the time it was written.

</comment><comment><USER5> I just posted this in another thread, but there are some very similar ideas I am implementing here: [http://onemodel.org](http://onemodel.org)

(Its strength is not in document storage, though it can. Also not really about version control but that could come later: it is about brainstorming _fast_ , rearranging things fast, exporting as outline documents, and not being as limited as if the info were all textual.)

A hierarchical organizer: AGPL, desktop- and keyboard-oriented, highly efficient, I use every day because org-mode was more awkward to me: [http://onemodel.org](http://onemodel.org) . (My progress has been slow lately but I have plans: feedback welcome, especially if on the mailing list.)

Anything can be nested anywhere, so I put effectively all my notes in it on everything. In college it would have been a huge help. It exports to text outlines (w/ or w/o legal numbering) which I've found useful when sharing info before the real sharing feature is implemented.

(The idea behind it is to reduce knowledge to an atomic level, as an object model: things we know have relationships to other concepts, can be expressed as measurements, etc. But it is cumbersome to create an object model anew, for every app. Right now, it is really good at supporting a big, efficient list of lists (based in postgresql), with attributes, and very minimal support for defining classes on the fly as a side-effect of use. But I really hope to be able to add anki-like (spaced repetition / flashcard-like) features, internal scripting, hosting, and secured sharing of info between instances, so it becomes like a wiki in convenience, but more efficient and computable rather than piles of words for everything).</comment>


Exploring Silicon Valley's High-Tech Heritage Trail - <AUTHOR> https://medium.com/swlh/exploring-silicon-valleys-high-tech-heritage-trail-e9a664c2d2b4 <comment><AUTHOR> "Silicon Valley doesn't care much for its history, high-tech or otherwise. It should." — David A. Kaplan, The Silicon Boys and Their Valley of Dreams (1999)For those who do care about tech history, the itinerary in this story connects some of the key sites and buildings in the 30-mile corridor from Stanford University to the former IBM disk-drive campus in south San Jose.</comment>


Show HN: HIPAA compliant file storage - <AUTHOR> We (TrueVault), released our HIPAA compliant file storage today. Our BLOB Store is not a cloud file backup service like Box or Dropbox but instead, it's meant to be integrated with healthcare mobile apps, web apps and wearable devices. During our beta period in December, early adopters are using the BLOB Store to store X-Rays, CT Scans, MRIs, PDFs, scanned medical records, images, and videos.
You can upload, update, delete, and download any binary file via our REST API.
We are also beta testing a browser-to-TrueVault direct file upload and download web form. If you would like to join the beta, please email us at firstname.lastname@example.com
API: https://www.truevault.com/rest-api.html
Blob post about its release: https://www.truevault.com/blog/hipaa-compliant-file-storage.html
Love to get your feedback. <comment><USER2> AWS is also HIPPA compliant.

[http://aws.amazon.com/compliance/](http://aws.amazon.com/compliance/)

<comment><USER3> TrueVault does more and costs less than AWS. We also save you hundreds of dev hours.

The HIPAA Security Rule requires appropriate Administrative, Physical, and Technical Safeguards to ensure the confidentiality, integrity, and security of protected health information (PHI).

AWS will sign a BAA, but they only cover the Physical Safeguards (e.g. facility access controls, etc.). TrueVault handles both the Technical and Physical Safeguards.

With AWS you still need to build your own HIPAA compliant application stack. The technical requirements include: -encryption and decryption -key management -key rotation -access control -unique user identification -emergency access -automatic logoff -audit controls -mechanism to authenticate electronic PHI -person or entity authentication -transmission security -integrity controls

<comment><USER4> I've been working on a mobile application for doctors and healthcare teams the past few months. We are a startup and will be going into beta testing with real users near the end of Q1.

I was interested in your last post on HN, as I've found it nearly impossible to sift through the bureaucratic cruft of HIPAA, HITECH, HL7, and all the other standards. A blog post on the topic would be amazing -- "What does HIPAA-compliant mean?".

The main fear I have about using your service is trusting a brand-new company for hosting. What reassurances can you provide that relying on you for my infrastructure will not screw me over if something happens to your company?

<comment><AUTHOR> Hey vonseel - we are reaching out to you via email. We can share with you a few details about TrueVault that'll surely give you confidence.

</comment></comment></comment></comment><comment> health-techie First things first, and this is more for the edification of those commenting, not the group posting, the correct spelling is:

H I P A A

The acronym is the "Health Information Portability and Accountability Act."

Again,

H I P A A

I can't stand how many people mis-spell it - it exemplifies the fact that really and truly, people don't care about the regulatory standard enough to really learn about it ...

Not to hijack the thread, but I'd love to hear people's opinion on this question:

Which type of data is more important, health data or financial data, and why?

I'll start, financial data is more important than health data. Why? Because we all use money at least 100+ times a day to get things done - outside of a doctor's visit health data is virtually worthless.

Outside of obtaining your soc number in a medical record, would you be more afraid if a person got your bank's routing number, or your insurance ID?

Your credit card number, or the fact that you have erectile disfunction?

<comment><USER5> An interesting read on this topic at [http://www.ncbi.nlm.nih.gov/books/NBK9579/](http://www.ncbi.nlm.nih.gov/books/NBK9579/):

"When personally identifiable health information, for example, is disclosed to an employer, insurer, or family member, it can result in stigma, embarrassment, and discrimination. Thus, without some assurance of privacy, people may be reluctant to provide candid and complete disclosures of sensitive information even to their physicians. Ensuring privacy can promote more effective communication between physician and patient, which is essential for quality of care, enhanced autonomy, and preventing economic harm, embarrassment, and discrimination (Gostin, 2001; NBAC, 1999; Pritts, 2002)."

I disagree with "outside of a doctor's visit health data is virtually worthless" for the above reasons.

<comment> health-techie Okay. Thanks for linking to that article. I concede health data isn't "worthless," but I'd like to get people's personal opinion on which scenario is scarier:

Target losing the credit card data of 40 million customers, or a dermatology office losing a USB drive: [http://www.fiercehealthit.com/story/dermatology- practice-fir...](http://www.fiercehealthit.com/story/dermatology-practice- first-be-hit-hitech-breach- penalty/2014-01-03?utm_campaign=AddThis&utm_medium=AddThis&utm_source=twitter#.UswqvlK4CmA.twitter)

A malicious party getting access to my banking data I feel is more scary than knowing my co-workers are teasing me because of my erectile disfunction.

Disclosing important health data to a person's employer, insurer, law enforcement etc, is a health data stewardship and education problem, not a 256-bit secure socket layer, 1048-bit RSA key-encryption problem, IMHO.

<comment><USER6> I think it may be helpful to break perceptions of health data into at least two camps:

1\. Societal stigma: this may not be something you agree with but it's just something that exists. People in the U.S., for whatever reasons, for right or wrong, feel like they should be protective about their health data, even if there's nothing potentially damaging or embarrassing in the data. It could be an ingrained feeling trained in them from years of visiting doctors who never gave them access to their data or it could be a learned response from watching CSI. Whatever the cause, societal stigma to by default be very protective of your health data is a real thing in the U.S. (I'm not saying they're necessarily more protective of health than financial data, but pointing out that stigma towards health data on its own is at least a thing.)

2\. Legitimate fears: I think if you changed your example away from erectile dysfunction this would be more apparent. There are 133MM chronic illness sufferers in the U.S. so let's use a different, very common example: congestive heart failure. If you're 45 years old and trying to apply for a job and your potential employer finds out you have CHF it's very possible (in an unregulated environment) they'd consider not hiring you because you have a high risk of missing work or even dying. Or imagine you're a CEO of a powerful company and someone leaks the fact that you have pancreatic cancer — what would happen to your company's shares, let alone your employees' morale? This doesn't even get into the notions of genetic discrimination, which is already a real thing: [http://scholar.google.com/scholar?q=employment+genetic+discr...](http://scholar.google.com/scholar?q=employment+genetic+discrimination&hl=en&as_sdt=0&as_vis=1&oi=scholart&sa=X&ei=937RUseQMcXEoASO6IGIBg&ved=0CDkQgQMwAA)

EDIT: Also thanks for mentioning HIPAA spelling. I twitch every time I read HIPPA.

</comment></comment></comment></comment><comment><USER7> Awesome work! I have a nascent project in the healthcare space and I will definitely consider using this for all HIPPA related issues.

One tidbit - will you have some kind of "HIPPA Compliant" image that can be placed on partner sites using your API? Sort of like the "Secured by VeriSign" graphic that shows up on eCommerce sites to give the end user more trust that their data is secure, etc.

<comment><USER8> "Secured by VeriSign" style badges are pointless advertising for the vendor. I'd love to see an A/B test with them that _isn 't_ sponsored by the vendor - they all tout conversion with internal tests that could easily be biased. Given that I can just grab the image and slap it on any old website, they offer no benefits.

<comment><USER9> I thought the same thing as you and had the same reservations when a marketing exec asked me to put one on the purchase page for a startup I used to work for. I was shocked when the A/B test we ran showed a lift. I wish I had more details but this was about 5 years ago.

I think people just want some assurance, even if it isn't backed by anything they understand, that a shows them that a decision they really want to make is ok. Putting a shiny badge that says it is safe is enough to sway some people.

</comment></comment></comment><comment><USER10> Per their site: "Example monthly cost:

If WebMD Answers was a TrueVault customer, their monthly charge would only be $1,604. Here's the breakdown: WebMD Answers has 5,036,000 questions, so that's 5,036,000 JSON documents stored . According to Alexa, they have 1,042,705 page views a month, so that's 1,042,705 requests. And presto, $504 + $1,100 = $1,604/mo."

To save money WebMD should store their json documents as compressed binary objects and get way more than 10k documents per $1.

<comment><AUTHOR> That's absolutely okay with us.

Though the developers at WebMD team may not be very happy if every piece of data is stored in a compressed BLOB. For each record lookup, you'll have to retrieve the entire compressed binary object, decompress it, find the record you need from some kind of a data structure. And if you are updating a record, you'll have to do all that, plus send the compressed binary object back. Performance _might_ be an issue :)

All kidding aside, our system is optimized for fast lookup of secure data. For a site as popular as WebMD, our JSON Store really is the way to go.

</comment></comment><comment><USER11> How is TrueVault better than AWS?

<comment><USER3> Our customers are typically deciding between using TrueVault or a HIPAA compliant hosting provider. The customer is facing a build vs. buy decision.

The HIPAA Security Rule requires appropriate Administrative, Physical, and Technical Safeguards to ensure the confidentiality, integrity, and security of protected health information (PHI). AWS, FireHost, and Rackspace are great! But, HIPAA compliant hosting providers like these only provide a HIPAA ready environment. They will sign a Business Associate Agreement (BAA), but they only handle the Physical Safeguards mandated by HIPAA. If you use a HIPAA compliant hosting provider you still have to spend months developing a HIPAA compliant application stack within that environment.

In contrast, TrueVault provides all client-side and server-side functionalities required by HIPAA, and works just like any other API service. The typical TrueVault integration takes days and saves months of development time.

Plus, if you want AWS to sign a BAA you need to use dedicated instances and each instance hour is 10% more than the standard fee. So your meter starts at $1,500/month if you want to become HIPAA compliant with AWS. FireHost starts at $1,115/month and you are charged a $250 premium for each HIPAA ready instance.

<comment><USER12> "In contrast, TrueVault provides all client-side and server-side functionalities required by HIPAA"

Maybe I am misunderstanding, but you seem to be saying that a customer's application is automatically HIPAA compliant by using TrueVault. I get that you take care of the HIPAA storage piece, but how do you make their entire application HIPAA compliant?

</comment></comment></comment><comment><USER13> HIPAA is actually a pretty fuzzy, nonspecific standard. As long as you take 'adequate' technical/physical/administrative safeguards you are basically fine.

Literally a sealed envelope (when dealing with paper documents) satisfies all three.

<comment><USER14> Would you be so kind as to expand on 'adequate' safeguards that make 'basically fine'?

</comment></comment><comment><USER15> BTW, Box is HIPPA compliant

<comment><AUTHOR> Box is HIPAA compliant, but:

Box is cloud backup for files on your computer.

TrueVault is an application backend that mobile apps and web apps are built on top of.

We have customers switching from Box to TrueVault because 1) our API is specifically designed to be called from a mobile/app rather than targeted for file backup use cases and 2) Box has a user based pricing that doesn't make sense if you are building an app rather than backing up files.

</comment></comment><comment><USER16> A company I work with are looking into using TrueVault's document store to keep PHI off of our servers, however TrueVault still does not fulfil the requirements of our application. More specifically multiple document get & document search.

Other than that, the team over at TrueVault is extremely friendly, reasonable and nice to work with.

<comment><USER3> Thank you for your kind words!

Search functionality will be released by the end of January.

</comment></comment><comment><USER17> I don't quite get the USP here. S3 is also HIPPA compliant, and costs < 10% as much.

<comment><USER18> Well, according to much of Branding/USP theory, focus is a good starting point for a USP. The other providers (S3, box, etc) may also be HIPPA, but this company is driving the point home -- we only do HIPPA. I have no idea if this is a strong enough differentiation, just pointing out it's not necessarily anti-USP. And price is never a factor in government-compliance and government contracts.

</comment></comment><comment><USER19> Seems like they're running out of an apartment unit:

TrueVault 801 Church St. #1328 Mountain View, CA 94041

<comment><USER3> Dell "Beginnings" [http://youtu.be/Ja61fxmY77Q](http://youtu.be/Ja61fxmY77Q)

</comment></comment><comment><USER20> How does HIPAA compare to PCI? Is it easier or harder or about the same difficulty?

<comment><USER21> The difference is that PCI DSS is an industry developed and maintained standard. HIPAA is a piece of legislation that created rules around privacy and security (and a lot of other things). There are some standards outlined in the rule. But there is no agency in place to perform validation or audits, and as such being "compliant" doesn't mean much outside of how much you trust the entity. But the penalties are very real, and very severe. Working for a small healthcare IT software company, we take HIPAA very seriously.

</comment></comment><comment><USER22> This looks great!

</comment><comment><USER23> much needed service</comment>


Ask HN: What is one boring task you automated? - <AUTHOR> and how you automated? <comment><USER3> I work at a construction company where they may have 15-30 projects active at any time ranging from 1 month to 3 years. On these projects the site superintendent is responsible for taking a minimum of 10 photos a day.

It was my job to take all their photos and rename them based on the date taken, resize them and store them by week ending date. I am working with construction guys and not computer experts so I set up a system with their phones/shared folders to easily allow them to drag/drop new photos into our file-server.

I do not know how to program directly, but found a nice utility called DropIt.

[http://www.dropitproject.com/](http://www.dropitproject.com/)

This utility allows me to take a batch/folder of photos and automatically rename them based on metadata I choose and output the results to another folder. I chose date taken. This can rename 500 photos at once to the date taken fast and easy.

I then take the files that have been renamed and reduce them in size using another software called Caesium.

[https://saerasoft.com/caesium/](https://saerasoft.com/caesium/)

This one allows me to take a folder of photos, specify a new output and I can typically get 90-95% reduction in file size while retaining the quality needed.

Taking it one step further using the folder replication in DropIT, I have it set so when a Superintendent drops photos into the shared folder on the network, it locally replicates those files into a folder on my computer - <USER2> drop it scans once and hour - if it finds photos in said folders it will then automatically rename it to the output for me.

The only thing I manually have to do is run Caesium, which is auto configured to scan/pull from the output folders of drop it and place back on the file- server.

My Mondays used to literally be photos all day, doing this manually with the way they showed me - now I can do all photos on all jobs in about 30 minutes (after the initial configuration).</comment>


Of Modes and Men (2005) - <AUTHOR> https://spectrum.ieee.org/computing/software/of-modes-and-men <comment><USER2> I wonder what Tesler thought about modern user interfaces in light of his "no modes" philosophy.

Today, each program/website effectively functions as a separate "mode" where everything potentially looks, behaves, and responds completely differently. There's no consistency and it's a huge part of the reason computers are frustrating to use for most people.

<comment><USER3> Bear in mind that the definition of "mode" includes a subjective part in its definition, i.e. an interface is in a mode if the user thinks of the input in a state as _the same action with different results_ than the same input on a different state. [1] For example, single click on a web link navigates to a new page, but single click on a desktop folder merely selects it.

But this does not necessarily applies to _content_. If you have two different apps, with very different media and data (e.g. one is a spreadsheet, the other a slide presentation), they don't represent different modes _if_ they respond the same to the same actions. You may have a zoom button in both which works the same, a similar color palette, similar scroll bars...

This is the reason why GUIs are composed with a small set of widgets, and these represent standardised behaviours that work essentially the same across all applications that use them (and also, this is why it's such a terrible idea to build widgets that deviate from these expected behaviours). Tesla should know, he was there when they invented them. [2]

You have a point that it's too easy to find applications that _don 't_ follow the standards and reinvent the wheel for no benefit, and also that the standards only extend to low-level interactions - before mobile OSs, there never was a unified metaphor for reading, storing and sharing information at the end-user space.

[1] [https://en.wikipedia.org/wiki/Mode_(user_interface)#Definiti...](https://en.wikipedia.org/wiki/Mode_\(user_interface\)#Definition)

[2] [https://web.archive.org/web/20040511051426/https://computerh...](https://web.archive.org/web/20040511051426/https://computerhistory.org/events/lectures/appleint_10281997/appleint_xscript.shtml)

</comment></comment><comment><USER4> They say "it paved the way for the one-button mouse we use today" but most people I know prefer two-or-more button mice, e.g. the classic scroll wheel design. A single button makes the mouse hand much less useful.

<comment><USER5> That's only because the system was designed for two-or-more button mice. The classic Mac OS was designed for a one-button mouse and it was far more intuitive than what we use today. It was also much more discoverable, since it didn't have modes, the thing Larry Tesler fought so hard against.

Right-mouse-button context menus are modes, since the functions you are shown depend on what type of object you click on/have selected. The classic Mac OS menu bar, on the other hand, is modeless since it always shows you the same set of functions (and greys out the ones that are inapplicable to the selected object). This is a tremendous boon to discoverability, a key feature of usability.

<comment><USER4> I'm going to have to disagree, not that discoverability isn't good, but that the addition button isn't worth the complexity.

It's a matter of expressiveness and bandwidth. You can do more pointer-related actions with multiple buttons, and they aren't confusing the way keyboard modes are, because you've got feedback on the display at the pointer location. Primary clicks do the important thing, secondary clicks give you a set of other options or an alternative action. It's very easy to pick up given its universality and its usefulness. A single button just requires you to press mode keys with the other hand to get that expressiveness needed in a reasonably intuitive, and powerful, pointer-driven UI.

Imagine if we didn't have multitouch and the only gestures we had on touch displays were tap, tap and hold, drag...

<comment><USER5> I think it's fair to say that everyone on Hacker News is a power user of their computer. Power users do not really know what it's like to be a regular user who is confused by multi-button mice, context menus, and all the rest. Try getting an older person to use a modern computer. They're completely baffled by all of the complexity. They don't know (and don't care) why they should be left-clicking in some cases and right-clicking in others. They prefer an iPad and generally don't use the multitouch gestures, besides pinch/spread for zoom. Simple taps and drags are highly intuitive for them.

I would argue that context menus are inferior to keyboard shortcuts anyway. I certainly don't use them once I learn the keyboard shortcuts. I believe we have truly lost something special, since the classic Mac OS is no longer with us. We don't even organize our computers spatially anymore [1], we just rely on search all the time. That's a terrible shame, as spatial memory is the fastest. When you walk into a room in your home, you don't even need to look at the light switch, you just flick it on because you remember exactly where it is.

Modern computers with all of their context sensitivity, browser windows, etc. are like a house where the light switches keep moving around to try to guess where you'll be next.

[1] [https://archive.arstechnica.com/reviews/4q00/macosx- pb1/maco...](https://archive.arstechnica.com/reviews/4q00/macosx-pb1/macos-x- beta-14.html)

</comment></comment></comment></comment><comment><USER6> Modes generally suck, but are often either unavoidable or better than the alternatives.

For example, every graphics program has a "mode" for creating e.g., curves, whereas the "default mode" is selecting. This is very much a mode, but it is clearly indicated to the user by (typically) a change in cursor shape. It's still a mode (affects what clicks of the mouse do), but it's tolerable because it is directly entered (by selecting an icon to enter the mode), and feedback is relatively clear.

There are modes, and then there are modes.

</comment><comment><USER7> I must remember this article the next time some Vim fanboi tries to mansplain to me why vim is better. (Note _most_ seasoned vim users are not in this category... But will happily press an emacs user, which I'm not either). I've heard it all before and even done the side by side comparison tests... >15yrs ago.

Ultimately, the results of Larry Tesler tests coincides with my own.

Thankyou Mr Tesler for No Modes editors!

<comment><USER8> Actual quote from the article for context:

> Modeless: Computer scientists when writing code typically worked in > different modes; you might have an insert mode, a delete mode, or a replace > mode. You would first select the mode, then select the point on the screen > at which the action was to occur, then perform the action. Tesler, in user > experiments, proved that modes were confusing for nonscientific users and > championed the "modeless" interface.

For non-technical users, sure modeless editing may be better. But for computer scientists, modeful editing may be better. The article mentions modeless editing to be better for no scientific users only, not necessarily for everyone.

<comment><USER3> _> But for computer scientists, modeful editing may be better. The article mentions modeless editing to be better for no scientific users only, not necessarily for everyone._

That's why they kept on investigating, and concluded that modes are bad for anyone; as long as they are human, using modes increase their rates of error. Jef Raskin went so far as to invent _quasimodes_ , to get the benefits of modal interfaces without the burden on short-term memory. ([https://en.wikipedia.org/wiki/Mode_(user_interface)#Quasimod...](https://en.wikipedia.org/wiki/Mode_\(user_interface\)#Quasimodes))

<comment><USER9> Calling people inhuman is obnoxious; please don't do that.

Quasimodes (chording) are physically difficult and distracting.</comment></comment></comment></comment>


Ask HN: What you automated using Python? - <AUTHOR> I am curious. Have you automated any task using Python? Please share link to Python scripts. <comment><USER2> Fabric is one of my favourite libraries, period. It's a remote deployment and task scripting library that uses SSH: [http://www.fabfile.org/](http://www.fabfile.org/) I use it to keep a small number of Debian VMs updated by simply defining a sudo('apt-get update && apt- get upgrade') task, then giving Fabric a list of hosts to run the command on, in parallel. There's infinite other uses, too. Very powerful.</comment>


New Hulu player includes volume normalization to fix loud ads. - <AUTHOR> http://blog.hulu.com/2010/05/13/pardon-our-dust/

<comment><USER2> <http://www.opencongress.org/bill/110-h6209/show>

<comment><AUTHOR> Good link, thanks. Think it will ever pass?</comment></comment>


Scientific Errors in Nat Geo Milky Way Photos - <AUTHOR> https://petapixel.com/2019/05/10/scientific-errors-in-those-nat-geo-milky-way-photos/ <comment><USER2> Reminds me a bit of the photo manipulation on Nat Geo cover photo of Egyptian pyramids.[1]

[1] [https://medium.com/engl462/visual-deceptions-national- geogra...](https://medium.com/engl462/visual-deceptions-national-geographic- and-the-pyramids-of-giza-3fee6d448d0d)

<comment><USER3> lol. You beat me to it, although I was going to be snarkier.

The funny thing is that even cropping can be an editorial decision. All media lies to a certain extent.

<comment><USER4> > The funny thing is that even cropping can be an editorial decision. All > media lies to a certain extent.

Don't let "perfect" be the enemy of "good." It's technically true that all media lies to a certain extent, but painstakingly modifying the actual content of a photograph such that no physical device could ever capture the result is obviously way worse than just cropping the image.

</comment></comment></comment><comment><USER5> > For starters, in real life, there is visible light pollution in several > directions around the forest and none of this is visible here on the > horizon.

It seems like in many of the pictures, including the one above this quote, the horizon is much lighter than the sky above. Isn't that light pollution?

> You can perhaps recognize...the bright star Vega and Polaris, which would be > located above the frame.

How do we recognize those? The arrow to "Polaris" is pointing out of the picture. And how was the bright dot in the picture identified as "Vega"?

<comment><USER6> Vega is pointed out because in relation to the galaxy in the picture it allows the author to locate where Polaris should be (above the frame). Polaris being above the frame is a problem because it is never above the horizon where the photograph is supposed to have been taken.

<comment><USER5> I get that, but if that bright dot isn't Vega, then the argument falls apart. How do we know it's Vega?

<comment><USER7> From the article "The night sky offers an unlimited source of markers (e.g. stars, constellations, Milky Way, etc.) that never lie about the time of year and location in the world you shot a night sky picture from."

The key word is unlimited. Vega is correctly identified by the many stars around it that are in turn correctly identified by the many professional astronomers and astro-photographers discussing these very pictures. The photo mentions polaris to point out the absurdity of the north star being in the sky of a picture taken in the southern hemisphere.

<comment><USER5> That's an argument from authority without even establishing the authority in question.

Evidence, please.

Edit: Well, that's a lot of downvotes. So you all see the pattern? Why not point it out?

Or are you merely accepting this blog post without question?

<comment><USER8> Are you actually in doubt that the star in question is Vega, or do you believe that the author knows what they're talking about but just think it would be an amusing thing to start an argument over? I'm hoping it's the former.

Here is a photo of the so-called "summer triangle" consisting of the bright stars Vega, Altair and Deneb: [https://astrobackyard.com/wp- content/uploads/2019/01/the-sum...](https://astrobackyard.com/wp- content/uploads/2019/01/the-summer-triangle-asterism.jpg)

And here is an animation showing that same photo, roughly superimposed on the photo in question: [https://i.imgur.com/MM37zt3.gif](https://i.imgur.com/MM37zt3.gif) (EDIT: Imgur seems to be redirecting to a badly converted version of this animation, try copy-and-pasting the link into the address bar.)

Even without being able to identify individual matching stars, you can clearly see that the shape of the stars and dust lanes in the Milky Way are the same.

<comment><USER5> Thanks for that, that's helpful. That's pretty damning evidence, and I can't find anything that might look similar from Namibia in Stellarium.

Since you mentioned my motives, I'll address that: I'm just tired of the Internet falsely accusing people and want to see more evidence before the pitchforks come out.

But in this case, I'm convinced.

<comment><USER8> Fair enough. For my part, I'm tired of the internet (and HN in particular) being quick to dismiss the opinions of actual experts in any given field for superficial reasons. Your original comment that started this thread strikes me as a good example of the so-called "middlebrow dismissal": [https://news.ycombinator.com/item?id=5072224](https://news.ycombinator.com/item?id=5072224)

Yes, one shouldn't blindly accept the words of an authority. That doesn't mean it's a good idea to constantly demand that those authorities constantly back up every little thing they say, even on topics about which they are more knowledgeable than you are. That gets irritating really fast (which I suspect is why you got downvoted), and more importantly, it tends to derail more interesting and productive discussions.

<comment><USER5> Usually I wouldn't even comment on an article about astronomy, but this was a public accusation of fraud. I see questioning such posts as a moral obligation.

What's the bigger risk: annoying a few people with persistent calls for evidence, or letting a false accusation spread for fear of annoying people?

<comment><USER9> The "false accusation" is trivial to verify. The stars never change, not on the scale of human lifetimes anyway. All you had to do is look up a chart and you could prove it for yourself.

<comment><USER5> It's not trivial if you don't know what to look for, and I obviously didn't.

<comment><USER10> The sky at night. Is quite big and appears fairly regularly, you can't miss it.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> "I'm not blaming the intern but it's completely their fault". Is her basic response, pretty shocking behaviour for a professional. Also, would t NatGeo demand the RAW files for any photo submitted? Or did they conveniently trust the photographers completely?

<comment><USER12> It's not a great one, but I think I can understand her response... I feel like she's between a rock and a hard place here. If you're accused of blatant dishonesty for someone else's work and demanded an explanation... would you make it appear as if you yourself were dishonest when you weren't, without mentioning someone else was involved? Wouldn't that itself be dishonest (and I'd argue unfair)? It's much easier when the company is the target, when you can just blame it on an error and move on, but I can't think of a much better response honestly when your name is the one personally on the hook, which it appears to have been here. What would you say instead?

<comment><USER13> I would apologize for putting my name on the work of an intern. Either they're lying and they were fully aware of the work being fabricated, or they were so uninvolved with the processing of the photographs that they had no idea they were faked.

<comment><USER12> > I would apologize for putting my name on the work of an intern.

How is that any better than what she already wrote? Or in fact how is that even an apology? It seems way worse and far more willing to blame the intern than what she has right now.

</comment></comment></comment></comment><comment><USER14> Would be a cool site or app if you could upload star images and it would give you locations/dates when that view was possible - or if the image was completely bogus.

</comment><comment><USER15> I've taken 30 second exposures of stars at night but they were too blurry for my liking. The earth never stopped rotating and I didn't have the gear to correct for the rotation. There's no way to get an accurate panorama on a moving subject unless all exposures are at the same time.

(Fifteen second exposures worked best for me.)

<comment><USER16> A good rule of thumb is (focal length / 500) to figure out your maximum exposure time before having blurry stars.

To get around a 20-30s exposure you're looking at somewhere between 14-24mm focal length, ideally with the largest aperture you can afford.

</comment></comment><comment><USER17> The way she mentions her father's death and then goes on to blatantly blame the image on a nameless intern while claiming she's not blaming it on her is just cringe inducing. This is a grown adult making childish moves. When you screw up, you stop digging, come clean and deal with it.

<comment><USER18> > When you screw up, you stop digging, come clean and deal with it.

This doesn't seem to be true. Are you saying this always produces the best outcomes, or that you wish more people would hurt themselves for your benefit?

<comment><USER10> Is often true. Also 'you wish more people would hurt themselves for your benefit?' is a rather unusual and somewhat fanatically zero sum way to describe someone's advice not to double down on a lie when caught out by competent experts.

<comment><USER18> > a rather unusual and somewhat fanatically zero sum way to describe

I'm not seeing where "zero-sum" applies. Zero-sum, intuitively enough, describes a system in which gains and losses sum to zero. That is almost certainly not the case here.

But it's still true that what you're hoping for is a gain to you and a loss to the fraudster. Somebody losing doesn't mean they lost an amount equal to some other party's gain.</comment></comment></comment></comment>


Locusts destroy crops in 'worst invasion in Sardinia for 60 years' - <AUTHOR> https://www.bbc.com/news/world-europe-48580182 <comment><USER2> Hope you all remembered to short sesame seed futures.

<comment><USER3> "Cicadas..."

<comment><USER4> Joke aside, Italy, Spain and France could get something similar near future

</comment></comment></comment><comment><USER5> Brown marmorated stinkbugs mentioned in the article are a problem where I live, southeastern US. Somehow they get into the house, then crawl into the light fixtures and die. I always wonder what natural predator is absent when there's an insect pest problem.

<comment><USER6> Apparently in Asia it has a natural predator, the awesomely named samurai wasp. It's a tiny parasitic wasp that lays their eggs in stinkbugs.

[https://www.sciencemag.org/news/2018/08/scientists-spent- yea...](https://www.sciencemag.org/news/2018/08/scientists-spent-years-plan- import-wasp-kill-stinkbugs-then-it-showed-its-own)

<comment><USER7> Parisitic wasps are some of my favorite insects to learn about as long as they aren't doing any mind control tricks on their hosts.

My favorite is the Megaphragma mymaripenne[1], the adults are said to have reduced the size of 95% of their neuron cells by getting rid of the nucleus.

[1][http://blogs.discovermagazine.com/notrocketscience/2011/11/3...](http://blogs.discovermagazine.com/notrocketscience/2011/11/30/how- fairy-wasps-cope-with-being-smaller-than-amoebas/)

<comment><USER8> Honourable mention for the microscopic trichogramma wasp, eater of clothes moth larvae. The Natural History Museum deployed them a few years back to deal with a moth outbreak. Supposedly the most successful way of dealing with an outbreak.

[https://www.telegraph.co.uk/news/science/11682039/When- moths...](https://www.telegraph.co.uk/news/science/11682039/When-moths-come- to-stay-theres-only-one-solution.html)

</comment></comment></comment></comment><comment><USER9> I'm scared to ask but why have I never heard about locust plagues in the USA?

<comment><USER10> North America used to have massive locust swarms back in the 19th century. But...

> The Rocky Mountain locust ranged through the western half of the United > States and some western portions of Canada. Sightings often placed their > swarms in numbers far larger than any other locust species, with one famous > sighting in 1875 estimated at 198,000 square miles (510,000 km2) in size > (greater than the area of California), weighing 27.5 million tons and > consisting of some 12.5 trillion insects.

> Less than 30 years later, the species was apparently extinct. The last > recorded sighting of a live specimen was in 1902 in southern Canada.

[https://en.wikipedia.org/wiki/Rocky_Mountain_locust](https://en.wikipedia.org/wiki/Rocky_Mountain_locust)

<comment><USER11> I find it fascinating that we dont know how they became extinct. I had originally thought it was simply habitat destruction but that hasn't really be proven at all.

<comment><USER12> Indeed, some thought it was simply farmers plowing fields or stock grazing, but their origin point were mountain valleys so it doesn't seem their demise would have been as rapid as it actually was.

[https://www.youtube.com/watch?v=DqyCO1eVC68&t=26m38s](https://www.youtube.com/watch?v=DqyCO1eVC68&t=26m38s)

</comment></comment></comment></comment><comment><USER13> Grasshoppers are pretty nutritious, and didn't the UN want people to use more micro-livestock? Instead of spraying poison everywhere, develop a new market.

<comment><USER14> Perhaps not everyone, especially Italians, actually want to eat insects

<comment><USER13> Lobster and shrimp were considered garbage, bottom-feeding poverty food at one time. Let fashion and marketing work their magic.</comment></comment></comment>


The data is in – stop the panic and end the total isolation - <AUTHOR> https://thehill.com/opinion/healthcare/494034-the-data-are-in-stop-the-panic-and-end-the-total-isolation <comment><USER2> This is written by a doctor at the conservative hoover institute. Good to keep in mind, doesn't mean he's wrong. In this case, he's wrong.

He has 5 'facts', I paraphrase them. I will just prejudice this by saying we need widespread testing and PPE, then we can consider opening up. Why don't we have these? Because we are acting like idiots here in the us and we can't organize ourselves.

1\. "The vast majority of people do not have a significant risk of dying from covid19". It's true but there will be young people dying who have no other risk factors. The problem is overloading the hospitals with those who are sick. It's the actual problem.

2\. "We can protect older at risk people and eliminate hospital overcrowding". Perhaps. But the entire world is failing to do this! Even Sweden, the place without shelter in place and allows open bars is expected to have over 10,000 deaths; similar sized Switzerland will have around 1,800 deaths according to IHME [1] (8.5 mil pop Switz., vs 10 mil in Sweden). In a sense, because they are similar countries with similar medical systems, There are about 8000 people dying in Sweden who probably wouldn't die if they had shut down society like Switzerland. How many extra deaths are worth opening the economy?

3\. Herd immunity is prevented by isolation - it's potentially right that more herd immunity could help later on. But we won't know how much immunity without being able to test for antibodies. Every western country trying this strategy seems to not be doing great (see Sweden).

4\. "people dying because hospitals stopped doing nonessential procedures". I'm in seattle. We got to the edge of hospital overload. All the docs were filling out their wills, they made plans if they died, there were medical workers married to another medical worker, they were afraid to go home. At least 3 med workers here in Seattle died. I think people died from being afraid to go to hospitals, but there were people with all the prep available (docs!) and they died.

5\. We know who the risk population is and we can protect this. This just seems so irritatingly wrong. We know we reduced covid-19 cases by staying home. We are failing to protect people at risk. If we stopped sheltering in place, we'd get more cases. We already can't protect people with the cases we have. Adding more cases does what - it kills more people.</comment>


Lessons learned scraping 100B product pages - <AUTHOR> https://blog.scrapinghub.com/web-scraping-at-scale-lessons-learned-scraping-100-billion-products-pages <comment><USER2> > As most companies need to extract product data on a daily basis, waiting a > couple days for your engineering team to fix any broken spiders isn't an > option. When these situations arise, Scrapinghub uses a machine learning > based data extraction tool that we've developed as a fallback until the > spider has been repaired.

I once worked on a spider that crawled article content and I ran into the same problem. I always wanted to try the following solution to it but never had the chance.

Assume you have a database of URLs and the fields you've scraped from them in the past (title, author, date, etc). If you ever fail to scrape one of those values from a _new_ URL, here's what you do:

\- Go back to one of the old URLs where you already have the correct value (let's say it's the title).

\- Walk through the whole DOM until you find that known title. At each node you will have to remove child nodes except for text, to deal with titles like "Foo Bar" which you want to match against "Foo Bar". So this is going to be an expensive search.

\- Generate several possible selectors which match the node you walked to (maybe you have ".title", ".title h2", ".content .top h2", etc).

\- Test each new selector on several other already-crawled pages. If any of the selectors work 100% of the time, there's your new selector.

Any thoughts on whether something like this would work?

<comment><USER3> That sounds like a very creative idea.

What I do is call a regression test every x minutes. If it fails, set a flag to save/store the html everytime we crawl pages. Now we can go back and process these saved pages when we fix our crawler

<comment> w0rd-driven I crawl a specific site somewhere up to 50 unique URLs a day. I store both the unparsed full html as a file and the json I'm looking for as another separate file. The idea is if something breaks instead of taking a hit to make the call again, I have the data and I should just process that. It's come in extremely handy when a site redesign changed the DOM and broke the parser.

I do the same at $dayJob where I'm parsing results of an internal API. Instead of making a call later that may not have the same data, I store the json and just process that. I feel like treating network requests as an expensive operation, even though they're not really, helped me come up with some clever ideas I've never had before. It's a premature optimization considering I've had like 0.000001% of failure but being able to replay that one breakage made debugging an esoteric problem waaaaaay simpler than it would've been otherwise.

<comment><USER4> Off-topic: I so wish I worked for a company where my work involves scraping and storing and analyzing data. :(

<comment><USER5> Now is a good time to work in this field since data science is hot and companies need web scrapers to provide the data for these models. Atleast that has been my experience in finance. Try applying!

<comment><USER4> I have zero experience in data science though. I am a pretty solid and experienced programmer and can learn it all but... don't know. Maybe I should just try indeed.

Do you have any recommendations for places and/or interview practices?

</comment></comment></comment></comment></comment></comment><comment><USER6> Ah yes, Challenge 4 (anti-bot measures).

At Blekko I developed a number of ways to deal with people that tried to scrape the web site for web results. The three most effective ways are blackholing (your web site vanishes as far as these folks are concerned), hang holding (basically using a crafted TCP/IP stack that does the syn/ack sequence but then never sends data so the client hangs forever), and data poisoning (returning a web page that has the same format as the one they are requesting but filling it with incorrect data).

We had a couple of funny triggers of the anti-bot stuff during the run, once when a presenter on stage showed an example query and enough people in the audience typed it into their phones/tablets/laptops and it all came from the same router that it looked like a bot. The other where the entire country was behind a single router and a school had all all of their students making the same sort of query at the same time (in both cases the trigger was a rapid query rate for the exact same search query from an address).

In Blekko's case since bots either never clicked on ads or _always_ clicked on the same ad (in both cases we got no revenue) keeping bot traffic off the site was measurable in terms of income.

<comment><USER7> I thought the number one anti-bot measure was a cease and desist letter :) seriously though, some of these websites clearly don't want to be scraped, what's stopping them from sending scrapehub a C&D letter and forcing them to comply?

<comment><USER6> Sure, if you can make a reasonable assumption it is them scraping you. As they point out in the article they invest in proxy networks to make their requests appear to come from a bunch of addresses that don't lead back to them.

One of the things we learned at Blekko was that people that run botnets often sell 'proxy service' as a thing, we identified several made out users of the Time Warner "road runner" service. That put us as a web site in a bind in that the proxy service that was running on an infected computer was violating our terms of service but the user might be completely unaware. If they were also a customer and we black holed their IP it would also cut off legitimate traffic. Since we didn't keep a logs that could identify these relations over time (privacy issues) we had to rely on other methods. We never got enough penetration into the search market to make this a huge concern however so the problem remained largely theoretical. We started a program of doing exponential banning where an IP would be banned and then an hour later unbanned, and if it resumed its bad behavior banned for 2 hours then 4 Etc. Once you get to 1024 hours it is pretty safe to assume they are lawfully evil as it were.

These guys fake their user agent, mask their IP addresses, and generally work hard to defeat anti-bot measures. They know they are over the line, but the law has yet to catch up to them.

<comment><USER7> My experience, when it comes to scraping airline websites, the airline's legal department usually doesn't wait to have proof that you were the one that actually scraped them. If you have their data on your website, they send you a C&D, and if they continue to find their data on your website, they will happily sue you. In other words, doesn't matter how you got the data, you must've broken the law if you got their data.

I'm thinking of RyanAir suing Expedia, United vs wandr.me, Southwest suing SWMonkey.com, I'm sure there's countless others.

<comment><USER8> But if you have Amazon turk workers 'scraping' the data is that illegal?

<comment><USER9> It's the reproduction of the data which they are suing over, not the method.

</comment></comment></comment></comment></comment></comment><comment><USER10> > Multi-threading is a must, when scraping at scale. The more concurrent > requests your spiders can make the better your performance - simple.

Intuitively I would think that this sort of problem would profit from using asynchronous ingestion at the edge pushing unprocessed contents to a multi- threaded/multi-process backend. (Because I'd expect that network latencies mean you need lots of threads to saturate I/O, which I'd expect would conflict with effectively using the available CPU power to do the actual document processing).

<comment><USER11> That's been exactly my experience. Most time is spent connecting or waiting for the server response (TTFB). Using an async I/O event loop approach in combination with EPOLL/KQUEUE you can handle thousands of concurrent connections. You then push the response to your worker nodes, which process the data in a multi-threaded fashion. Stream Processing Frameworks like Apache Spark or Storm work great for that.

</comment></comment><comment><USER12> Can I piggy-back off this submission to ask HN if you're running a scraper, have the recent wave of GDPR splash-screens caused you issues? How are you dealing with them? [https://news.ycombinator.com/item?id=17471599](https://news.ycombinator.com/item?id=17471599)

<comment><USER13> You can just remove them from DOM or hide with CSS.

<comment><USER12> What about e.g. [http://discourseontheotter.tumblr.com/](http://discourseontheotter.tumblr.com/) ?

Edit: In the UK I see this: [https://imgur.com/a/zlWOByh](https://imgur.com/a/zlWOByh)

<comment><USER14> I don't see a GDPR challenge on this page?

<comment><USER12> That's probably because it's being inconsistently applied and you're not in Europe. If that's the case it's all the more insidious! Or you accepted the Tumblr terms in the past.

This is what I get in the UK: [https://imgur.com/a/zlWOByh](https://imgur.com/a/zlWOByh)

<comment><USER15> :s/Europe/EU/

I'm in Switzerland (Europe), don't get the [https://imgur.com/a/zlWOByh](https://imgur.com/a/zlWOByh)

</comment></comment></comment></comment></comment></comment><comment><USER16> Just by chance we experienced a scraper bot on the site past week and we discovered some performance problems thanks to it. It literally fried our ancient caching system and we finally took the step towards using cdn for static delivery and redis for api responses. I wonder if there were those guys because it was some solid scraping.

<comment><USER17> Badly behaved scrapers should be blocked, not accommodated.

<comment><USER18> I'd agree. As somebody scraping content, what's so bad about increasing the timeout to like 10 seconds? That way the servers can handle the traffic easily and you're not being a jerk. If you have one async thread for each domain you can still get lots of data quickly. Causing a denial of service attack is very avoidable.

<comment><USER17> And you're going to be hitting millions of hosts anyway, so all you have to do is rotate from one host to the next and randomize your worker queues. It might take a little longer but you will not blow up someone's aging server. Being a good citizen of the net means to take into account that even if you have gigabits of bandwidth to burn the counterparty may not (and could easily be on the sharp end of a bandwidth capped contract).

</comment></comment></comment></comment><comment><USER19> As a side note, I have had quite a bit of experience trying to block automated scraping services. And I found that the best way is to quietly attempt to detect scraping. Then, serve up tainted data.

In our case, competitors were scraping pricing data in order to competitively price their products without having to do the work.

So we just randomly start to give them incorrect prices on every few products. Not only would it make the whole data set useless, they had no way of figuring out which data was correct without manually checking and since we didn't do it to everything and started at random intervals, it made it too difficult for them to figure out when their ip had actually be quietly blacklisted.

<comment><USER20> What's ironic is that most of the sites with anti scraping protection also do scraping of their own.

E.g. Amazon and Walmart both do a lot of their own scraping.

<comment><USER21> Really going to call for a [citation needed] on that "most"!

<comment><USER20> Maybe I should rephrase to "put the most effort into anti scraping".

Every major ecommerce site scrapes, it would be a competitive disadvantage if they didn't.

</comment></comment></comment></comment><comment><USER22> I hope you too got a chuckle from reading their anti-bot counter measures section only to see their form protected by Google's "I'm not a robot" CAPTCHA plugin.

</comment><comment><USER23> I've always wondered if it makes more sense to render the page as a jpeg and run some kind of machine learning to identify and read off the relevant details

<comment><USER17> I'd go the other way and say that pages that no longer contain relevant information in a normally digestible format should be dropped from search engines and other automated indices.

After all, the web was built on accessibility of information, not on purposeful obfuscation.

If you go so far as to essentially flatten the webpage to the point where you might as well print it out and then do OCR on it then you've thrown out the baby with the bathwater, you _had_ all that information when you started. Or at least, you should have had it.

Otherwise we might as well kiss HTML goodbye and render the web as pdfs, with or without links.

<comment><USER24> The biggest search engine doesn't have your best interests at heart and has been trying to make HTML and accessibility of information obsolete for years. Some pages now render only with javascript or require solving javascript challenge to even get to the rendering (hello cloudflare) and essentially kissed HTML goodbye.

</comment></comment></comment><comment><USER25> I always assumed Web Scraping wasn't something particularly challenging because of how many libraries existed for this purpose.

This article made me realize I assumed wrong.

<comment><USER26> It gets especially difficult with dynamic content or when trying to scrape sites written on very heavy frameworks like ASP.NET Webforms that require passing the view state with every request. I made a calendar aggregator for adult hockey times in my area that scrapes rink websites[0] and it was far more difficult than I had thought it would be because of the fact that the rinks all used Telerik Webforms controls to do their calendars. It turned a 30 minute job into a 2 hour job.

[0]: [http://dpscschedule.azurewebsites.net/](http://dpscschedule.azurewebsites.net/)

</comment></comment><comment><USER27> Wouldn't such scraping be under TOS of scraped sites?

<comment><USER28> Bot cannot consent to or understand TOS...

<comment><USER7> What if the site requires you to create an account and login? Can the bot create it's own account, and still claim to not consent?

<comment><USER28> Is bot aware of what it is doing? I don't think so.

</comment></comment></comment></comment><comment><USER29> I had to stop reading the otherwise interesting sounding piece when at the left bottom corner a "Get the Enterprise Web Scraping Guide" box popped up (second paragraph or so). Maybe I'll give it a second chance later.

<comment><USER13> It was a pretty thin self promoting post. You didn't miss much.

</comment></comment><comment><USER30> > However, our recommendation is to go with a proxy provider who can provide a > single endpoint for proxy configuration and hide all the complexities of > managing your proxies.

Can you provide an example of such service?

THanks!

<comment><USER31> I run [https://www.scraperapi.com](https://www.scraperapi.com) which does this!

</comment></comment><comment><USER32> > A large proportion of these bot countermeasures use javascript to determine > if the request is coming from a crawler or a human (Javascript engine > checks, font enumeration, WebGL and Canvas, etc.).

How effective are scraping countermeasures anyway?

<comment><USER7> They work pretty well for any scraper that's not using an actual browser with JavaScript engine. It keeps the riff-raff out.

A dedicated person will eventually work his way around all available counter- measures, though.

</comment></comment><comment><USER33> "Multi-threading is a must, when scraping at scale."

I disagree on this point. Starting with a single threaded model allowed my team to scale quickly and with little additional overhead. What we have lost with performance we gained in simplicity and developer productivity. That being said tuning and porting portions of the app to a multi-threaded system is slotted to take place within the next year.

Start with single threaded and simple, move to multi-threaded scrapers when the juice is worth the squeeze.

<comment><USER4> Or use a language where fully utilizing all CPU cores is transparent, like Elixir? There's zero complexity, you basically add 4-5 lines of code and that's it. Honestly, not exaggerating.

I've done several very amateur scrapers in the last several years, I am never going back to languages with a global interpreter lock, ever.

<comment><USER34> I'm assuming you're talking about Python, which is also "4-5 lines" to use multithreading or multiprocessing. Can you explain what's wrong with GIL languages?

Now that I think about it, it's even less than 4 lines:

from multiprocess.pool import Pool (or ThreadPool)

pool = Pool()

pool.map(scrape, urls)

<comment><USER4> When the pooled functions are I/O bound then the GIL is not a problem. Any GIL language will do.

However, for example when generating reports, try use the same instrument for serializing 4 pages of DB records to 4 pieces of a big CSV file, each working on a single CPU core. There the languages without GIL truly shine. And languages like Python and Ruby struggle unless their GIL implementations compromise and yield without waiting for an I/O operation to complete.

<comment><USER34> I'm not sure you understand how the GIL works in Python. If you're using multiprocessing, there's no locking across the code executing on each core. Also, if you're writing to the same file from four processes, you're going to need locking.

<comment><USER4> What I have last known is that GIL languages work well in multicore scenarios as long as all N tasks have I/O calls that serve as yielding points for the interpreter, and they do not use preemptive scheduling like the BEAM VM (Erlang, Elixir, LFE, Alpaca) do.

Am I mistaken?

<comment><USER34> As far as Python goes, yes. Multicore implies multiple processes, which means that each process will have it's own Python interpreter, each with it's own GIL.

If you were to use multithreading instead, you would generally have a problem if you were doing non-I/O work.

<comment><USER4> Then I think we have a misunderstanding of terms. To me "multicore" == "single process, many threads". Apologies for the confusion.

It seems that now we are both on the same page. Single process & many threads are problematic for GIL languages and that's why I gave up using Ruby for scrapers. GIL languages can work very well for the URL downloading part though.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER35> I tried several different queue systems best version I got is using Erlang Queue, Elixir & Kafka on top for doing high concurrent crawler, the project was to develop a realtime Amazon product ASIN price monitoring system for our company as a challenger prototype. Our main problem was basically proxies, we stopped buying them as managing thousands of proxies is a huge effort that we did not want to take, also lack of data means our Hadoop clusters gets thirsty and machines stops learning properly. Currently we are using a third party [https://proxycrawl.com](https://proxycrawl.com) on very high tiers > 10B with a great discount and we are happy to get that part solved. Other lessons learnt are like sometimes things fail and logs help a lot so you will need a highly available Logging and monitoring system.

<comment><USER7> Back when I worked for a very large tech company, building their web crawler, I had good success with Golang. On four servers, with 10 GigE interconnect and SSD, and a very fast pipe to the Internet, I was able to push about 10K pages / second sustained. At any given time, there were probably several million connections open concurrently.

I've played with Elixir as well, and it's also great for this type of thing.

proxycrawl.com looks very cool, I'm actually looking for a proxy service for my current scraping project. Are they also a good choice if you're doing lower tiers (like thousands of requests a day)?

<comment><USER35> Golang is a good choice too but in my experience its nothing compared to what you can do with Erlang Queue and Elixir. Regarding your question about proxycrawl, I do not know honestly, I tested the service for few days on some few millions per day and it was great too. I would say they are good for a very high volume, we are still using it, so that should be a good signal to try them.</comment></comment></comment>


Harvey Weinstein's Defense Lawyer and Unpopular Speech in a Cold Climate - <AUTHOR> https://www.newyorker.com/news/our-columnists/unpopular-speech-in-a-cold-climate <comment><USER2> She forgot to mention the enormous pay check Weinstein lawyer receives for representing him.

Her argument is weak. Her argument is that lawyers have a right to represent any client and should not face social stigma for doing so because these lawyers are just doing their job. What she doesn't understand is that people also have a right to protest. Her complaining just makes her sound whiny and entitled. And her bringing race into this to help her argument is worst. Him, being the first African American dean, does not matter. Being disliked for representing wrongdoers is literally part of the job - whether he is guilty or not. It's called having a thick skin.</comment>


Ask HN: How do you deal with snark on an open-source project? - <AUTHOR> I'm not talking about Twitter snark or HN snark where an all-out nerd war can happen in a snap and nobody really cares. I'm talking about snark thrown in a Github issue.
How do you deal with that while maintaining composure but not totally ignoring the poster? <comment><USER2> I'm not exactly good at it, but you need to ignore the snark as you see it.

Ignoring the snark, you then see if the issue really exists/is a fair point/is something you care about, and you respond as you would to any non-snark filled issue.

I don't see a point in trying to provoke the person, or 'educate' them, just respond neutrally and professionally, or happy and chipper if you can do it without it dripping in irony.</comment>


The Craft of Research - b-man http://www.press.uchicago.edu/Misc/Chicago/065685in.html

<comment><AUTHOR> Note this Amazon.com review which says the 2nd edition is better than the third: [http://www.amazon.com/review/R3E8JGKMTV2B2M/ref=cm_cr_rdp_pe...](http://www.amazon.com/review/R3E8JGKMTV2B2M/ref=cm_cr_rdp_perm)</comment>


Rails 3 jQuery Setup Script - <AUTHOR> http://github.com/lleger/Rails-3-jQuery

<comment><USER2> This would probably be better done as a template, but nice job

<comment><AUTHOR> You are totally right about that. I just committed a template to do that, but left the script for posterity.

</comment></comment><comment><USER3> Anyone know of an equivalent for rails 2.3.5?

<comment><USER4> jrails will let you use jquery rather that prototype but it doesn't generate clean code like rail 3 does. unless you have some dependencies that are not compatible, the bump to 3 is really worth it.

<http://github.com/aaronchi/jrails></comment></comment>


The Nobel Prize in Physics 2008 - <AUTHOR> http://nobelprize.org/nobel_prizes/physics/laureates/2008/

<comment><USER2> It's a shame how little Nobel gold there is to go around - so many deserving people get left out. For every winner there's a long line of those equally worthy IMHO.

In this case, I'm surprised they didn't give some of the prize to Goldstone.

<http://en.wikipedia.org/wiki/Nambu-Goldstone_boson></comment>


Show HN: DocentEDU – Turn any website into a lesson - <AUTHOR> https://docentedu.com/demo/ <comment><USER2> I probably have very little value to add except to say that I played around with it for a couple of minutes. This is a really cool solution to a problem that I have had for a while and while I cannot reward you financially for it or provide a helpful technical critique, allow me to congratulate you on getting stuff done. If for nothing else, then remember this comment for trying to boost your morale!</comment>


Skype on Linux: Is It Functional, or Falling Behind? - <AUTHOR>

<comment><USER2> Well, doesn't seem too private or secure :(

[https://www.eff.org/secure-messaging-scorecard](https://www.eff.org/secure- messaging-scorecard)

Not even open-source.

</comment><comment><USER3> I use skype all on the time on Ubuntu 14.04LTS, seems fine, no issues! :)</comment>


When and why to clean up your code: now, later, never - <AUTHOR> https://codewithoutrules.com/2018/11/02/when-clean-up-your-code/ <comment> d--b I used to be a full time developer at a company where we only shipped new features. At some point, I wanted to go remote and part time. I've asked them if they'd be down if I worked in the shadows refactoring shitty code and replacing crappy modules. Which is what I'm doing right now.

For them, they pay me a fraction of a developer's price and get a constant improvement of the codebase.

For me, I work at my own pace, remote and part time. I live in a place where I don't need much money.

It's awesome.

<comment><USER2> That's really interesting. Compared to what you could get on the open market, what's "a fraction" mean? 1/2? 2/3?

It's kind of hard for me to imagine moving out of the Bay Area, but I'm sure there are major cities outside California that one could easily live on 1/2 a software developer's income.

<comment> d--b I took 1/4th, but I was working for a very successful finance company in New York. If you don't count the yearly bonus, I took about 2/3rd.

But the calculation should not be about the fraction, it should be about what you need. If you make $100/hr and work 5hr/day, that's $110k with 5wk vacation. I make more than this and I moved to France, so I make a very decent living compared to the local standards.

</comment></comment></comment><comment><USER3> _The Boy Scouts have a rule: "Always leave the campground cleaner than you found it"._ \- Robert. C. Martin

<comment><USER4> To this I would add: _When in Rome, do as the Roman 's do_.

When you are repairing other people's code, don't change it to your style, fix it in theirs.

<comment><USER5> And this depends on who "owns" it. If you don't have communal ownership of code, don't make drastic changes without consultation. This is like rearranging someone's kitchen for them "because it's more efficient". It's not, because now they don't know their code anymore (I had this happen to me a few times, I know it needed work but the work done made it harder for me to make other changes later that the "helper" didn't know was planned).

If you'll be the one owning the code, do what you want. If it's not yours, check first.

If you do have communal ownership, coordinate and establish style guides, ideally using tools to establish conformance (rather than eye-balling it, this also leads to greater consistency).

<comment><USER6> That's why a team should have code reviews; and everyone should keep an eye on what's changing.

This way if someone decides to rearrange something, the stakeholders can hold up the review until they're happy.

I've certainly blocked code reviews because of planned changes.

That being said, you only "own" code if you own the business. If you think you "own" code, you might need to re-evaluate your ego.

</comment></comment></comment></comment><comment><USER7> In my experience the only two real choices are now or never, because later almost always becomes never under pressure to do new things.

</comment><comment><USER8> In my ideal environment, each time I would load a source file in my editor, it would be reformatted according to my own preferred formatting rules. And upon writing, it would be saved in some common "standard" format good for git diffs. This way each programmer can be presented the code in the best way he can read it, and the tools can work on a standard representation that they can best deal with.

I will note that some projects such as FreeRDP, already have an autoformat script that does the "saving" part (unfortunately, not entirely automatically, since the precommit-hook is not set to run it).

</comment><comment><USER9> it's kind of frustrating, cracking open old code for a change or bug fix and seeing some really gross helper function you have to use. the temptation to take 30 minutes and refactor is too irresistable sometimes.

I do agree with the basic idea of the article though. maintenance/sustaining = touch the code as little as you can (helps with diffs) greenfield = take the time to make it right for as long as you can

<comment><USER10> As I gain experience, I find myself more and more just gritting my teeth and using the nasty function as-is.

The business types don't care. What they care about is having something to sell.

The engineers who are even capable of understanding what I'm doing don't care because they've got their own deadlines on a totally unrelated part of the codebase. They probably won't even work here any more by the time someone has to touch this code again.

</comment></comment><comment><USER11> Remember that updating references is usually a global update. Now you have to test EVERYTHING that used that reference.

I've had so many projects bitten by an overzealous dev who decided to update the version of a library because newer = better and then we spent the next three weeks chasing bugs/differences between the versions.

Edit: Curious about the down votes. Would someone who disagrees care to illuminate me?

<comment><USER12> Not a downvoter, but when I saw your reply it made me think that you may need a better process around maintaining dependencies.

At least IMO, best practice is to continually evaluate and adopt updates to dependencies. I'd rather have a slow trickle of problems associated with continuous updates, than have a total firefight when you realize you _have to_ upgrade, now, and you have neither solid plans and practice in place to do so, nor the time because you're in the middle of yet another feature drop that is pushing tech debt to the breaking point.

Setting and forgetting dependencies is like saying you have a backup somewhere, but never actually testing if you can recover from it. That's the kind of thing that could turn a routine outage into a crippling emergency.

<comment><USER11> Fair point.</comment></comment></comment>