(Un)Informed Consent: Studying GDPR Consent Notices in the Field [pdf] - <AUTHOR> https://www.syssec.ruhr-uni-bochum.de/media/emma/veroeffentlichungen/2019/09/05/uninformed-consent_Yl7FPEh.pdf <comment><USER3> When I browse the web, it feels like 100% of websites do it wrong. Consent needs to be active and cannot be enforced unless technically necessary.

Telling people "by continuing to browse, you accept our cookies" is wrong. You need to clearly offer a decline option and you can NOT force people out the door when they dont want to be tracked.

Nobody does this right.

<comment><USER4> Consent form is not needed when cookies are _technically necessary_. If user's login state or shopping cart would break if they didn't store a _first-party_ cookie, you don't need to ask them for consent for that cookie (as long as it's limited to that purpose).

In other words: if a site displays a consent form, it's _already doing something wrong_. Not horribly wrong - that's why it's a consent form requirement, and not immediate fine or jail time for the site operators - <USER2> wrong nonetheless. GDPR is purposefully structured in such a way that consent is needed only for things that are abusive to users, or were found to carry a significant risk of being used to abuse users.

<comment><USER5> > GDPR is purposefully structured in such a way that consent is needed only > for things that are abusive to users, or were found to carry a significant > risk of being used to abuse users.

What the regulators intend is one thing. What paranoid legal teams, particularly common law legal teams, think is another.

</comment></comment></comment><comment><USER6> Most of these opt-outs are nearly useless because they require so much effort. (And of course this is by design.) I personally prefer just using uBlock Origin and a DNS black hole. It's less effort, and it's more likely to prevent abusive behavior. (And it also cleanses the internet of most advertising.)

</comment><comment><USER7> It's true that most websites outside Google/fb etc just don't get it right. Makes you wish that google/FB could silo in all the web's content so peoples privacy can be better protected

<comment><USER3> I remember the FB consent process that was worded in a very specific way to create fear within the user when no consent is given. That was not cool.

</comment></comment><comment><USER8> The results explain why I keep running into Oath (and Q..something) popups that only present a single button "Confirm" to give consent, with minimal information on what that means. There's usually a de-emphasized link to change the hidden consent settings, with a hundred checkboxes, all on by default. It's downright insulting and malicious.

Choices (visible):

\- No option (27.8%)

\- Confirmation with no opt-out (68%)

\- Binary (3.2%)

So ~96% of websites sampled are not GDPR compliant, inadvertently or intentionally.

I agree with the conclusion of the study:

"The business model of online behavioral advertising, which targets ads based on large amounts of personal data, should be challenged, and alternative models like privacy-friendly contextual advertising or other ways of monetization for web services need to be developed."

</comment><comment><USER9> > Our results further indicate that the GDPR's principles of data protection > by default and purposed-based consent would require websites to use consent > notices that would actually lead to less than 0.1% of users actively > consenting to the use of third-party cookies.

<comment><USER10> Tough cookies.</comment></comment>


A tale of new censors - <AUTHOR> https://blog.torproject.org/blog/tale-new-censors-vodafone-uk-t-mobile-uk-o2-uk-and-t-mobile-usa <comment> mike-cardwell At the beginning of January, I wrote an article about T-Mobile UK spoofing RST packets to disrupt SSL connections on mail submission and other ports, and how to bypass it:

[https://grepular.com/Punching_through_The_Great_Firewall_of_...](https://grepular.com/Punching_through_The_Great_Firewall_of_TMobile)

</comment><comment><USER2> paulgraham.com is also blocked by Vodafone's filter.

Prepaid (pay-as-you-go in UK terms) users that look over 18 can show up at a Vodafone store and remove the filter.

I did it without giving away my identity by registering online with a fake name and confirming the name at the store. Your mileage might vary.

<comment><USER3> This is standard practice for mobile networks in the UK and it should be. As the OP says, it's removable on request. You don't have to give them any details other than prove you are over 18. Turning up in the store is usually good enough.

Without this, if you are in the UK an 8 year old can quite simply just buy a pay as you go mobile phone with cash from a supermarket, top it up using cash and use it to browse porn etc.

With respect to the providers, O2's ADSL/POTS broadband offerings have no censorship in place at all.

<comment><USER4> > Without this, if you are in the UK an 8 year old can quite simply just buy a > pay as you go mobile phone with cash from a supermarket, top it up using > cash and use it to browse porn etc.

I don't see a problem with that.

<comment><USER3> Well you can activate your 8 year old's phone and explain that to the social worker after a few weeks...

<comment><USER5> Do you honestly believe that giving an kid unfiltered internet is likely to land you in trouble with social services? That is so far out of touch with reality...

<comment><USER3> My sister is a social worker. It does. Regularly. More than you could imagine. People don't know what acceptable sexual boundaries are when they learn about sex from porn resulting in all sorts of messed up individuals.

<comment><USER5> I find it hard to believe it is the unfiltered internet connection that gets them in trouble, rather than the particularly bad parenting that accompanied it in those _particular_ circumstances.

An unfiltered internet connection itself is nothing for the government to be concerned about.

(You also likely have some pretty heavy selection bias tainting your perception of the prevalence of things like this.)

<comment><USER6> > find it hard to believe it is the unfiltered internet connection that gets > them in trouble, rather than the particularly bad parenting that accompanied > it in those particular circumstances.

How about a third possibility, that social services will screw with people's lives based on their whim.

<comment><USER5> I'm not sure the (real) problems with social services are on topic here.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> i'm a supporter of tor, but i'm not sure why they're making this out to be so alarming. t-mobile clearly presents its filtering as a family-safe content filter, and tor is software that makes it easy to bypass content filtering. this is only setup by default on pre-paid accounts because anyone can purchase a pre-paid SIM without being over 18. i would expect the same kind of filtering at a library or school, and any other internet service that one can purchase like cable or dsl service typically requires that someone over 18 be present during installation. i can just imagine the silly backlash over a parent complaining to the media because their child was able to look at porn by putting a pre-paid SIM in their android phone.

i recently purchased a pre-paid SIM from t-mobile US and was kind of surprised they didn't ask for any identification when purchasing (more so because i would have thought the government would have mandated some kind of tracking of phone numbers for going after drug dealers or something). once i logged into t-mobile's website, the form to disable this content filtering was easy to find and required giving proof of being over 18 (which they say is not stored or associated with the account after verification).

<comment><USER8> Requiring you to be over 18 to use "the real Internet" also requires you to present a government ID. Which means, effectively, anonymous or pseudonymous use of the Internet is dead. This may be a problem if you'd like to criticize your government.

Independently of whether or not you think that is a good idea in theory, it is unworkable in practice. There will always be coffee shops and open access points and neighbors to mooch off of for the kids to get ahold of porn. Your media backlash is now merely shifted from Vodaphone to the local mom & pop.

TL;DR - an Internet license is no substitute for proper parenting.

<comment><USER3> You don't have to have a "government ID" or present ID. That's a fallacy. You just have to provide reasonable evidence that you are a not a minor.

Pubic hair is usually enough.

It's a choice the providers made to protect their arses from lawsuits, not the government.

There are a lot of paranoid people here.

Edit: to explain the "pubic hair" comment is sarcasm relating to how lax the verification process is. Judging by the time of day, lots of humor-challenged Americans have woken up.

<comment> mike-cardwell I for one regularly use my pubic hair as proof of age.

</comment></comment></comment></comment><comment><USER9> T-Mobile blocks urbandictionary.com in the UK. When you want to access that site you need to give your credit card details, but only UK-issued cards work.

<http://texy.pl/2011/01/24/internetowi-psuje-cz-2-t-mobile/></comment>


Digital Ocean down - <AUTHOR> http://digitalocean.com/

<comment><USER2> All of NYC droplets are down from here... again.

Back to [http://linode.com/](http://linode.com/) or [http://vr.org/](http://vr.org/) \- AMS seems to be a lot more stable than NYC - but that's not where we are.

</comment><comment><USER3> one of my NYC2 droplet went down. It's like 2 or 3 times in the last 4 days. few mins downtime each.

</comment><comment><USER4> NYC2 has had lots of downtime... and there is never any contact from them about anything.

I guess this is the part where everyone says "You get what you pay for."

</comment><comment><USER5> For what it's worth, my droplet in nyc2 appears fine.

</comment><comment><USER6> It's online here on Australia.

</comment><comment><USER7> My NL sites are live but NYC DC seems to be completely offline. This is bad, only last Friday they had some more downtime.

As of now there's no news at [http://www.digitaloceanstatus.com/](http://www.digitaloceanstatus.com/)

</comment><comment><USER8> I've no issues with either my NL droplets or their main site.

<comment><USER9> Same for my Amsterdam droplet, all ok.

</comment></comment><comment><USER10> This downtime is getting ridiculous. Has been having a network issue nearly every day with the NY datacenters. Considering a move elsewhere pretty soon if there is no improvement in the next week.

<comment><USER2> So I'm not the only one feeling like an idiot keeping my servers there over the problems after the last 3 days... Wondering about the SLA, or refund at this point. Definitely moving once servers are back up.

</comment></comment><comment><USER11> DigitalOcean got 99,99% SLA.

<comment><USER2> Do you think they'll issue actual refunds, or service credit... toward more DO?

<comment><USER10> They are 'supposed to' but, will they.

Already requested a SLA credit for all these network issues as I have quite a few droplets in that region, however awaiting a reply from billing management.

<comment><AUTHOR> How did you go?</comment></comment></comment></comment>


He was fired after writing this article - <AUTHOR> https://hackernoon.com/the-myth-of-software-time-estimations-576a7466d91a <comment><USER2> I've spent my career living in sales and engineering camps. I've managed whole sales organizations and written full applications from the ground up. I could not disagree more with the author that time estimates should be eliminated. The business does not exist without revenue, and revenue is less likely to be retained (customers) or acquired (new customers) when the business is unable to anticipate product delivery timelines. Think about all of the other business functions that key their todos and deadlines off of a product release. There is also the question of accountability. I'm intrinsically motivated to be productive, but even I value knowing if I am tracking well on a burn down chart. It gives me daily insight into whether or not I should communicate a need to trim scope or coordinate a deadline change with the other business functions if scope can't be trimmed. Kanban is awesome for bug fixes that are hard to anticipate, but any dev worth their salt should be able to offer a time estimate. If something happens, then communicate why the estimate has to change and keep incorporating the lessons learned from into future estimates.

<comment><USER3> I think developers are kind of afraid to estimate, especially when these estimates are treated as commitments and not... well, estimates.

Evidence based scheduling [0] is an interesting approach to scheduling that may ease some of these pains but is not without drawbacks (e.g. estimation itself can take a lot of time).

[0]: [https://www.joelonsoftware.com/2007/10/26/evidence-based- sch...](https://www.joelonsoftware.com/2007/10/26/evidence-based-scheduling/)

<comment><USER2> This is awesome. Thanks for sharing this.

</comment></comment></comment><comment><USER4> previously: [https://news.ycombinator.com/item?id=14209141](https://news.ycombinator.com/item?id=14209141) (authors gofundme campaign)

</comment><comment><USER5> I'll talk about some counter points to the article. I don't mean to rustle feathers, but there are a couple of points I take issue with.

YAGNI/KISS are the two hardest concepts for developers to internalize. Ignoring them creates the brutally painful legacy systems we have to maintain today.

Requirements and feature priorities change all the time. I think building an over engineered foundation is always a terrible idea. Complex things are not necessarily better. You shouldn't be building frameworks or platforms. You should be building products. A framework or platform may fall out of your efforts, but it shouldn't be the goal. If you build a framework for the future, then you are wasting effort on a future that probably won't come. When the priorities change then every extra line of code becomes technical debt that the team has to work against. Now you have to have sprints to fix it. And you'll blame business for changing their minds and wasting your effort. That's not agile.

Shifting to Timelines.

Timelines are important because development, especially in larger organizations, does not work in a vacuum. Marketing, training, sales, customer support, implementations, existing customers, are all impacted by development releases. This impacts accounting's ability to plan for expenses or revenue. Things like, do we go to show X and plan for a big release blitz or not. Do we hire and prepare for a big lead generation campaign this quarter or the next. We have Y amount of revenue if we can get the release out. Is that revenue coming in now, or at the end of the year. We in technology are usually oblivious to these issues, but we shouldn't be. If we are asking everyone to understand our constraints, then we should understand theirs.

If development takes the stance "You'll get it when it's done", then it's really hard for everyone else to do their jobs.

The core of the friction is resource constraints. You can't add bodies and get more productivity quickly, so for a given release you are usually stuck with the existing team +/\- a small percentage.

The only dials left are time and feature set. Good orgs will as a group adjust both. Bad orgs will set both in stone and throw rocks.

This is why good agile is so important. The original concept of agile, not the "process in a box". I'll copy the tenets here:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

If you are building a platform/framework instead of delivering features then the dev team is prioritizng the plan over responding to change.

A good group will establish the MVF (minimum viable feature set). They will do so based on the needs of the company; sales opportunities, product vision (by an industry expert), operational needs (customer support tools, et al.), and technical debt management.

The MVF/MVP is not the place for business to include every bell and whistle, and not the place for development to build a MOAA (mother of all archtectures).

The most critical items are delivered first and readied for release. Then other features can fill in and enhance the big release date. If development is able to churn out features then marketing can pick the point where they want a formal "release" with fan fare.

All that said. It sucks his contract was cut. There might have been other issues and this was the last straw, or the company might feel he was airing dirty laundry, or it he was cut for something completely unrelated.</comment>


Better video quality through deep learning - <AUTHOR> https://mux.com/blog/better-video-quality-through-deep-learning/ <comment><AUTHOR> Author here: please feel free to ask any questions/comments on our methodology. I designed the neural net, which is basically video classification mapped to bitrate-ladders. Interested in any feedback!</comment>


Stanford Student Claims to Run Bootleg Covid-19 Testing Lab - <AUTHOR> https://mailchi.mp/fountainhopper/foho-102student-claims-to-run-bootleg-covid-19-testing-lab-50-stanford-affiliates-allegedly-tested-more-news-you-can-use?e=[UNIQID] <comment><USER4> It's incredible seeing this on hn. For context, while the FoHo is sometimes good for very, very breaking and recent stories, it's a little hard to overdo the niche it fits in Stanford culture: it's pretty much the main and only tabloid for fast-paced news and (sometimes) absurd 'takes' on things happening around campus.

While I don't doubt the story, I would always take it with a grain of salt. I don't personally know the person they claim is running these "under the table" tests, but the tests are very simple, so it wouldn't surprise me if it was actually true. (Most first or second year biology/bioengineering students can easily run RT-PCR without an issue, especially if they already have the primers available.)

EDIT: I was just contacted by one of my friends who knows of a few students and their labs doing this. Apparently there is more than one.

<comment><USER5> If I heard someone in my lab was running PCR on saliva samples potentially infected with 2019-nCoV, I'd lose my shit.

<comment><USER6> At the moment, you shouldn't be going to a lower than a biosafety level 4 lab at all.

<comment><USER7> This is incorrect. The CDC recommends BSL-2 for routine diagnostics and BSL-3 for isolating the virus in cell culture. ([https://www.cdc.gov/coronavirus/2019-nCoV/lab/lab- biosafety-...](https://www.cdc.gov/coronavirus/2019-nCoV/lab/lab-biosafety- guidelines.html))

</comment></comment></comment></comment><comment><USER8> The test is trivial. It's just RT-PCR.

\- Spit in a 2 mL tube

\- Purify the RNA from the saliva in that tube using qiagen's Viral RNA mini kit ([https://www.qiagen.com/us/products/diagnostics-and- clinical-...](https://www.qiagen.com/us/products/diagnostics-and-clinical- research/clinical-sample-processing/qiaamp-dsp-viral-rna-mini-kit/))

\- Get PCR primers against NCoV N by ordering IDT's CDC qPCR probe assay ([https://www.idtdna.com/pages/landing/coronavirus-research- re...](https://www.idtdna.com/pages/landing/coronavirus-research-reagents))

\- Run one step RT-PCR using your choice of one-step reagent (CDC recommends [https://www.thermofisher.com/order/catalog/product/A15300#/A...](https://www.thermofisher.com/order/catalog/product/A15300#/A15300))

\- Either do the reaction in a qPCR machine or run the PCR result on an agarose gel with positive control, if you see a band matching coronavirus positive control then the sample is positive, if not negative. Also helpful to have a human sample negative control to be sure as well.

There is no step here that requires critical thinking, it's just following instructions on the kits.

Also, this is not like a "bootleg" version of doing it, this is actually the official way the CDC does the test.

Edit: Please see official CDC instructions for how to run the test here: [https://www.fda.gov/media/134922/download](https://www.fda.gov/media/134922/download)

Edit 2: In terms of test shortages, it seems to be due to shortages in RNA extraction kits and swabs, only the former of which is really necessary, since the official guidance for sample collection has switched from upper respitory swabs to collecting sputum (phlegm) from the lower respitory tract (by just spitting it out): [https://www.ft.com/content/86efe246-692e-11ea-800d-da70cff6e...](https://www.ft.com/content/86efe246-692e-11ea-800d-da70cff6e4d3)

<comment><USER9> Agree it's trivially easy to do, but as anyone you does a lot of qPCR knows, they're rife with false positives.

Any environmental contamination will show up as a positive, particularly using the non-quantatative end-point method mentioned above. There's simple controls for that, but still....

<comment><USER10> > Agree it's trivially easy to do, but as anyone you does a lot of qPCR knows, > they're rife with false positives.

Honestly we'd be better off having a widely available test that gives 50% false positives. Worst case, a bunch of people quarantine unnecessarily, but we at least catch all the infected. The alternative is what we have now: widespread community transmission with no end in sight.

<comment><USER11> This is a classic statistical fallacy. A test that is 50% accurate serves 0 purpose. You might as well consider coin flipping a valid test. If you tested America with this test you'd end up thinking half the country is infected. And within the ones that tested positive only half would actually be and within the negatives half would be and you would have missed them.

A test needs to be materially more accurate than the odds of having the disease to be worth anything.

Classic thought experiment: if 0.0001% of the population has a disease and a test is 99% effective and you test positive, what are the odds you have the disease? Answer: 1%.

Complimentary thought experiment: if an expensive preventive drug was available in limited supply (for 0.1% of the population only) and the earlier you took it the more preventive it was should you give it blindly to as many positives as you can? Probably not because 99% of that would be going to waste. And you would run out of drugs to cover the actual sick. Only 10% of the sick would end up actually getting the drugs.

<comment><USER12> 50% accurate I'd agree is useless.

But the parent said 50% false positive, presumably with close to a 0% false negative would be _VERY_ useful and save potentially millions of lives. We need enough tests yesterday or so to avoid a repeat of Italy and a 50% false positive rate (with a very low false negative rate) could help do that.

<comment><USER13> Even the properly-conducted version of this test has a fairly substantial false negative rate, somewhere around the 30% mark, and if it's done by students using samples that might not be taken correctly on a rigged-together testing setup that's going to get worse. Seriously, you might as well flip a coin.

</comment></comment></comment></comment></comment></comment><comment><USER14> If anyone else wants to start a bootleg test lab, there's a disposition auction next week for a defunct biotech startup with most of what you'll need: [https://svdisposition.com/auction- detail?id=281](https://svdisposition.com/auction-detail?id=281)

<comment><USER9> I'd advise against this unless you really know what you're doing. Equipment is just one thing. You need a very well controlled environment that absolutely prevents cross-contamination and environmental contamination. My lab is a very clean molecular biology lab and it isn't good enough for that.

You need processes in place that ensure that these types of contamination are prevented, and constantly checked for. You need a very robust pipeline from sample collection to result dissemination that ensure no mix ups.

All these things are possible, but PCR is ultrasensitive and processes are hard; you could have a huge amount of false results if not very very careful.

<comment><USER15> It really isn't that difficult to avoid cross-contamination. As someone who has done literally 10s of thousand of PCR reactions (probably 100s of thousands) you just need to do your extraction and set-up seperate from where you run the reactions.

The real problem seems to be the kit-cowboys who have no idea how to do anything unless it comes out of a kit.

</comment></comment></comment><comment><USER16> Given the number of rich celebrities that apparently manage to get a test just because they're rich, I wonder if there is more illicit testing going on. Meanwhile a colleague of a doctor I know remains untested due to lack of tests despite symptoms entirely consistent with covid. Lack of tests are likely behind the comparatively high UK death rate.

Seems like there's a good opportunity for a nefarious get rich quick scheme here: setup a website, have hypochondriac rich people send in a swab, charge them lots, wait a couple of days and inform them of a random outcome with a warning about false positives. If this is going on, they should at least give people a positive so they quarantine them self.

<comment><USER17> I doubt the lack of test availability is directly increasing mortality.

The UK like many other countries the UK is advising self isolation for those with symptoms, and I imagine those who ignore this would not isolate even if they did test positive. There's no specific treatment. Those admitted requiring ventilatory support have that decision made on other grounds, and a CT can show a consistent viral pneumonia and help rule out other causes, which is good enough diagnostically when the pre-test probability is likely to be so high. I believe when PCR testing was not readily available in China they were using ct based diagnostic criteria.

Unless there's something I'm missing, the main benefits of a readily available test would be getting potentially symptomatic but actually negative key workers back from self isolation and perhaps catching the asymptotic if it could be rolled out widely enough.

<comment><USER16> That's not what I meant. It is affecting the published rates, i.e. the statistics.

Sorry, poor wording on my part.

<comment><USER17> Ahh, didn't think about it from that angle, apologies.

</comment></comment></comment></comment><comment><USER18> Almost completely OT but:

I'm almost certain I've had Corona, but it's almost impossible to get the medical system to test for that where I live (for good reasons).

Will it be possible to get a test kit like this [https://www.biomedomics.com/products/infectious- disease/covi...](https://www.biomedomics.com/products/infectious- disease/covid-19-rt/) as private individual (in the EU)? Will that test show if I have been infected previously? The graph on this PDF suggest it [https://www.biomedomics.com/?fldl=3001](https://www.biomedomics.com/?fldl=3001)

<comment><USER19> Out of curiosity, what changes if you have had it, recovered and been symptom free for 2 weeks? Is there something different in what you're supposed to do.

<comment><USER18> No, not really any enormous changes. But here in Sweden we don't have proper lockdowns (yet), and my office isn't completely shut dow, we are just strongly recommended to work from home.

If I can confirm that I've had it I would start going back to the office (better ergonomics) and start going out on lunches/dates, go to the gym, shop for clothes, meet older relatives etc with (almost) no fear of spreading it to other people.

<comment><USER20> Keep in mind that even if you have had it, people are infectious on average for 20 days from before symptoms appear [1], so it might be good to self- quarantine for a while.

[1] [https://www.scmp.com/news/china/society/article/3076022/coro...](https://www.scmp.com/news/china/society/article/3076022/coronavirus- infects-faster-and-lasts-longer-sars-raising-new)

<comment><USER19> There is also the possibility it is biphasic, although this could also be more of a problem with testing.

[https://www.nytimes.com/2020/02/29/health/coronavirus- reinfe...](https://www.nytimes.com/2020/02/29/health/coronavirus- reinfection.html)

[https://theguardian.com/world/2020/feb/27/japanese-woman- tes...](https://theguardian.com/world/2020/feb/27/japanese-woman-tests- positive-for-coronavirus-for-second-time)

</comment></comment></comment></comment></comment><comment><USER21> I'm a Stanford newbie. This is completely believable, but of the stories The Fountain Hopper breaks, what's the accuracy rate?

</comment><comment><USER22> If it's so easy, then is it so hard to ramp up testing?

<comment><USER23> If it's so easy to build a website. Why doesn't everyone scale to the size of Google?

<comment><USER24> I'd be curious to the answer to that too. I can think of quite a few reasons, but I'm sure there are nuances that are completely unknown to me.

I'm more interested in the parent question at the moment. I know almost nothing about the subject matter. What are the blockers for deploying a simple virus test at the scale we need?

<comment><USER25> It boils down to people, logistics, and efficiency:

Yes, we can produce the primers, develop the tests, ramp up the supply of RNA extraction kits, swabs, tips, etc. Individually easy, but together more challenging, especially when the entire world is also trying to acquire the same products.

Yes, large, fast and high-scale instruments exist. But, they need to be set up and calibrated. They need adequate power, cooling, water, space, etc. The technicians need training. They need network interfaces, etc.

Yes, it's a simple test. But there are still SO MANY steps needed to generate a result from a sample, and those take people. Just looking at the laboratory workflow: 1) Sample received in shipment -- unbox, scan manifest (if provided) 2) Unbag sample + requisition 3) Check sample label (usually handwritten) and verify accuracy to requisition (possibly handwritten). In an ideal case, orders are coming in as "interfaced orders" (meaning electronically) but many smaller clinics/hospitals will not have this. 4) Enter all demographics of the patient into laboratory system 5) Have second person verify the information 6) Re-bag and send sample to location in lab with test instrumentation 7) Technicians unbag, verify label + order accuracy 8) Decant sample from non- standardized tube sent to lab to standard tube used in lab 9) Scan into liquid handling robot... ...et cetera, et cetera

As I will probably find out in the replies there are "solutions" to all of these problems, and many labs utilize them. However, to scale such a complex system so quickly means that there is no room for a "test environment", and all changes have to be made "in production". Even simple changes to the workflow require days of prep to communicate to all of the staff across three daily shifts-- you get the idea.

source: i'm a laboratory medicine resident at the UW lab. We scaled from 0 to 3000 tests in 2 weeks and it's been an incredible all-hands effort.

<comment><USER7> I appreciate what's going on at UW regarding this and realize that it must have been a huge amount of work to say the least. However, I'd like to point out that most of what you've described is related to the bureaucracy and paperwork of certified medical procedures. Under ordinary circumstances that's all well and good, but given the severity of the situation it seems like it would have been better to eliminate much of that at least a month ago. Put another way, combat medics don't let someone die just because they're short some official FDA approved widget.

Why keep such detailed records? Barcode it on the way in, run the test, and report the result - forget demographic data and any other paperwork, whoever ordered the test can deal with that.

The supplies for RNA extraction are common and readily available (I think all the raw components are mass produced?), but only specific kits are FDA approved.

Only specific swabs are FDA approved. ([https://khn.org/news/as-coronavirus- testing-gears-up-special...](https://khn.org/news/as-coronavirus-testing- gears-up-specialized-swabs-running-out/))

> the government is considering expanding its recommended testing material > options to allow for more general nasal swabs to keep up with the increased > testing demand

Standard qPCR machines are incredibly common in both academia and industry and have 96 wells at minimum. The protocol published by the CDC has an 80 minute runtime. That's 1000 tests per day for a _single_ low-end machine.

The US response has been an absolute shitshow of bureaucratic dysfunction as far as I'm concerned, and people will likely end up dead as a direct result.

<comment><USER26> > Why keep such detailed records? Barcode it on the way in, run the test, and > report the result - forget demographic data and any other paperwork, whoever > ordered the test can deal with that.

Detailed records and redundancies ensure you don't mix up any of the tens of thousands of samples coming in. These systems are crucial for being able to scale up as they have without complete chaos breaking out in the lab.

You are right that barcodes are far more efficient, after implementation. But implementation takes months, and requires diverting resources that would otherwise be used to run more samples. The lab would face significant downtime putting a barcode setup in place and retraining. (See nkrumm's comment about labs not having a "test" environment.)

> Standard qPCR machines are incredibly common in both academia and industry > and have 96 wells at minimum. The protocol published by the CDC has an 80 > minute runtime. That's 1000 tests per day for a _single_ low-end machine.

That's only if we ignore the sample preparation time (orders of magnitude longer than analysis time), and the many control wells necessary to ensure the technique was successful. While I suspect these will also be dismissed as bureaucratic inefficiency, they're needed to make sure your newly-installed instrument running 24/7 hasn't broken down, and that the high throughput sample prep wasn't botched for a given prep batch.

The stakes for these tests are high enough that cutting corners on QA/QC isn't acceptable, because it means more lives lost. Doing a mediocre job for testing at this scale will have a very real impact on our population.

<comment><USER7> My calculation (1000 tests per day) left space for 12 controls per batch - <USER2> that not sufficient under the circumstances? Alternatively, at 500 tests per day (again, this is _per_ low-end machine!) you could run each sample twice in addition to the 12 controls. (And I would _never_ dismiss controls as an inefficiency, even the most trivial exploratory experiment is highly suspect without one.)

The point I was trying to make with those numbers was just how readily accessible the necessary instrumentation is.

Regarding sample preparation, I'm well aware that it's a time sink (I've done DNA and RNA extractions before). But it's fairly trivial and can be done fully in parallel by multiple people; the primary bottleneck is likely to be available bench space. (Unless the sample prep has been fully mechanized, in which case I'm really not seeing the issue.)

My point being that multiple, parallel sample prep pipelines can feed a single qPCR machine to keep it running just about 24/7.

Regarding barcodes, I didn't mean it had to be fully automated. Just drop the excess data entry and switch to a serial number scheme with built-in redundancy (or at least error detection). Many academic labs employ such schemes by hand.

Given just how dire the circumstances are, I'm afraid I'd have to strenuously disagree that cutting corners on QA/QC (as compared to standard medical diagnostic testing) would be unacceptable. None of what I described is at all out of place for handling research samples, and in my experience those are quite reliable.

I used the combat medic analogy in my previous post for a reason - <USER3> significant number of people are likely to end up dead due to our having blindly stuck to the rules. A bit of pragmatism could have saved them, and I view that as a tragic systemic failure on the part of the US government.

<comment><USER26> You're saying a lot of things here that don't make any sense.

> I'm afraid I'd have to strenuously disagree that cutting corners on QA/QC > [...] would be unacceptable.

> I would _never_ dismiss controls as an inefficiency

So, the QC in QA/QC stands for Quality _Controls_. So, you're strenuously insisting they're unnecessary, yet vehement that you would never dismiss the thing you just said wasn't necessary? Which one is it?

> But [RNA extraction is] fairly trivial and can be done fully in parallel by > multiple people

It takes longer than the instrument analysis was my point. That's the typical bottleneck in running samples, not your instrument sample throughput like you claim. So even if you can run 1000 samples on the instrument, if it takes longer to prepare them (which it always does), then that's your rate limiting step.

Unless, of course, you're working in this magic lab of yours where new, fully- trained techs suddenly materialize any time there's a spike in sample volume (whereas new benchspace doesn't? Might wanna take that up with your deity lab manager).

> Regarding barcodes, I didn't mean it had to be fully automated. Just drop > the excess data entry and switch to a serial number scheme with built-in > redundancy (or at least error detection)

With all hands on deck who do you think will have time to do this? Barcode migrations are a lot of work. Adding the word "just" to what I've described doesn't make something trivially easy.

Though apparently this whole process is trivial, from the sample check-in to the RNA extraction to the instrument analysis. Why not just fire the UW resident and all the staff, so that you can swoop in save the world, since it's all so terribly easy for you? I look forward to touring this fully- automated magic lab of yours.

<comment><USER7> I'm fairly certain that everything I said makes sense, so I'll try to clarify a bit.

First though, it seems a misunderstanding has developed. I was not calling the competence of the UW lab into question! They are bound by various state and federal regulations and I'm certain are doing the absolute best that they can under the circumstances. The examples I provided were to illustrate just how absurd the current state of affairs is, and to make it clear that bureaucratic ineptitude is to blame.

We obviously both agree that QA/QC is necessary; note that my back-of-the- envelope calculations included controls. I don't pretend to know precisely what requirements FDA regulations place on various diagnostic tests. What I do know, and attempted to illustrate in concrete terms, is that it is physically possible to scale testing in a sufficiently reliable manner using common instrumentation and bulk reagents. Any genuine attempt to explain US testing shortages _must_ account for this fact.

Yes, parallelizing sample prep requires a sufficient number of properly trained technicians. Presumably the US government is capable of locating and making use of qualified personnel in an emergency situation such as this? (To start with, literally any graduate student whose research includes running molecular biology protocols is likely over qualified for sample prep.)

Regarding serial numbers and labeling, it is indeed trivial. You can literally keep a list on paper and label tubes by hand (I did this for years in an academic lab). This step is fast compared to overall sample prep time.

I have no idea where you got the idea of a barcode migration from. Remember, I'm not describing a clinical lab operating according to FDA regulations; I'm describing a sufficiently reliable setup that closely resembles an academic research lab and would maximize throughput in an emergency situation. The only goal is to receive a package, label a corresponding tube, run a test, and report the result.

(BTW, I claimed that bench space for sample prep, not instrument throughput, was likely to be the bottleneck. In fact, instrument throughput _not_ being the bottleneck was one of my primary points.)

<comment><USER26> If you're basing your sample throughput calculations on instrument throughput (1000/day), you're implicitly assuming that instrument run time is the limiting factor, and the rest of the operations can match it.

I'm also confused by your apparent assumption that bench space is a universal constant that cannot be changed, even under a global pandemic, while lab personnel can be scaled up effortlessly.

> Regarding serial numbers and labeling, it is indeed trivial. You can > literally keep a list on paper and label tubes by hand (I did this for years > in an academic lab). This step is fast compared to overall sample prep time.

Scientists don't keep records on scraps paper for a reason, and I'm shocked your PI didn't read you the Riot Act for shoddy record-keeping.

With thousands of samples coming in, this is how you end up completely losing track of your samples. Once again: it's not trivial, particularly at this scale.

> I have no idea where you got the idea of a barcode migration from.

> Why keep such detailed records? Barcode it on the way in, run the test, and > report the result - forget demographic data and any other paperwork

<comment><USER7> I was attempting to illustrate the absurdity of the past month and a half by performing a back-of-the-envelope calculation, not provide a detailed business plan for launching a biotech startup. FFS

I never claimed that bench space was a universal constant, nor that hiring personnel is effortless. Rather, I attempted to make it clear that from both scientific and logistical perspectives the barrier to effective testing is quite low.

The point of the instrument throughput calculation was to make it clear that instrument availability can't possibly be the limiting factor because a single low-end one that many academic labs already have on hand can facilitate an incredible number of tests each day. That's important because you can't depend on being able to procure new instrumentation in a timely manner during a crisis so any estimates ought to assume that you're stuck with whatever you already have.

We see reagent shortages reported in the news, but that can't possibly be the limiting factor either because (as far as I know) only common chemicals (plus primers and probes) are required. (Actually you could probably dispense with the probes if you were willing to run an awful lot of gels, but that would further complicate the pipeline and require more labor so better not.)

So then we might suppose that personnel and training is the bottleneck, but that can't be it either because the government has deep pockets and just about any practicing molecular biologist is over qualified to perform such a trivial procedure.

Looked at this way, it should be abundantly clear that US policy is entirely to blame.

I also never said to use scraps of paper in lieu of proper record keeping. The obvious interpretation of what I wrote is to go line by line in a lab notebook, which is something I have in fact done before (obviously not on the scale described though). It works quite well in the long term provided that your serial numbers incorporate either a day or a page number and are strictly sequential. It's hardly the only way to go about it; I only mentioned it because the original post that I responded to explicitly called out record keeping as a necessarily time consuming and complicated part of the process. I was illustrating that once again, that is only true when complying with existing US regulations.

<comment><USER26> What was described above wasn't "US Administrative Bureaucracy" so much as "this is how busy labs are run." It can be challenging to understand that if you're coming from a small research lab where QA/QC boils down to recalibrating when you feel like it, and sample records are maintained by transcribing loose paper into Excel.

The bureaucratic aspects are to ensure you don't mix up the large volume of samples, and the quantitative QA/QC metrics are needed to objectively esablish what level of error is considered acceptable. Good record-keeping and QA/QC are all necessary when you're dealing with healthcare, particularly largescale healthcare.

That doesn't mean US policy or the administration's handling of this crisis was anything close to acceptable, but the issues there are pretty far removed from routine laboratory sample check-in and record-keeping.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER27> Edit: it seems the idea is to possibly catch asymptomatic people so they can self isolate (if they otherwise wouldn't) or tell their contacts to keep an eye out

I'm skeptical of the value add when weighed against the risk of introducing unregulated / unverified medical tests with unknown false positive/negative rates during a crisis situation, but obviously folks disagree, and I'm sure I could be missing something.

<comment><USER28> Either this is a genius hack (saving lives) or a genius scam!

<comment><USER27> How does it save lives? If someone took this test and popped positive, would it increase the likelihood of receiving effective treatment or of reducing transmission? I'm trying to figure out the value.

<comment><USER29> The best case scenario for controlling this epidemic is containment through contract tracing. That means we find people who have it, figure out who they got and from and gave it to, and get those people to quarantine. This strategy is failing in the US due to insufficient testing.

<comment><USER16> We're past that stage. It's purely about flattening the curve so the health system can cope while protecting the the vulnerable.

The official statistics have a lag in them due to only showing confirmed cases.

</comment></comment></comment></comment></comment><comment><USER30> What's the point of getting a "bootleg" unofficial test -- why would you trust the results (positive or negative)?

<comment><USER31> You trust the Stanford whiz kid with access to the lab.

</comment></comment><comment><USER32> Perhaps we need an open source effort for this? The main obstacle is sample collection and safety handling. given however the demand for testing, maybe a group of scientists could come up with specifications for ad hoc sample testing centers and perhaps recommend best practices and off the shelf products to minimize the risk. There are tons of graduate students who could volunteer and lots of unused pcr machines. It would be far better to organize these bootleggers as an open community sharing their standards rather than each person doing their own setup.

</comment><comment><USER33> People seem to be caught up in a frenzy of the importance of PCR testing.

In reality, even if _no one_ got tested, Doctors would still be doing pretty much the same things they are doing now. The same numbers of people would live or die. Hypoxemia and pneumonia are easy to diagnose. The treatment is standard. Any medicines are likely to work the same without regard to the exact RNA makeup of the virus.

Quarantine is still a good idea. Remember, if it mutates all these tests are going to be useless anyway. So a negative result has to be treated as dangerous anyway.

</comment><comment><USER34> I am buying a used RT-PCR machine for that, will get it home in the next few days. I have an account at Sigma-Aldrich as a hobbyist researcher, so I can order primers and other reagents.

</comment><comment><USER35> Kudos! Several dozen US labs in the US claimed similar capability in January. But ponderous accuracy certification from the FDA kept them unofficial until March.

Also this probably doesnt scale well to the thousands per day states need now.

</comment><comment><USER36> Hmmm I swear I heard a story about a Stanford student doing bogus blood tests before ... can't remember where though.

</comment><comment><USER37> _Please tell us about the time you most successfully hacked some (non- computer) system to your advantage?_

</comment><comment><USER38> Can we trust "Stanford Students" with medical tests? The school has a history of encouraging bad behavior in this area, as well as a history of letting people bribe athletic coaches to gain entrance.

I would not trust this organization. They're tainted.

</comment><comment><USER39> Elizabeth Holmes 2.0?

</comment><comment><USER40> > according to GroupMe messages obtained by FoHo

I remember coming to the bay area and it wasn't long before someone that felt they had a cultural affinity with me invited me to a GroupMe group. Man that was years ago. Surprised to see people still use GroupMe, still seems to be for cliques exclusively.

<comment><USER41> GroupMe is still extensively used at universities. I only know a few undergrads who prefer to use it, but most seem to use it as a compromise. I personally hate its UI.</comment></comment>


After the age of scarcity - <AUTHOR> http://ignoredbydinosaurs.com/2012/05/after-age-scarcity <comment><USER2> "If you're lucky you will have patrons, not customers."

Probably the best quote from the post. Soon the only way of making a living from a creative endeavour might be to be funded by patrons, then releasing your content for free. A generalized Louis CK model if you will.

Funny how that would bring us back to pre-18th century days where patronage was the only business model in town for creative types, with all the constraints that came with artists-patrons relationships.

Except in this world everybody would get to play Medicis or Louis XIV. I certainly hope that "crowd-patronage" will mitigate the limitations of artistic freedom that often came with traditional patronage.

<comment><USER3> Isn't 'crowd-patronage' essentially letting the market decide?

It wouldn't require that many fans either -- assuming each fan brought 30 bucks worth of merch and donations, you are looking at a middleclass wage at 2K fans. No matter how obscure your genre is, it ought to be possible to get 2k fans.

<comment><USER4> I would say it is _forcing_ the market to decide.

When I have to be a patron, I can't just lean back and honor good products by buying them. I have to look up names, sample their past work, invest money in projects that might suck.

I'd do everything that investors are paid for right now. Except I would feel I'm wasting my life away because I'm doing all this over €10, not €10m.

<comment><USER5> Maybe the future value for labels is to collect a subscription fee from customers and they would do the vetting for you. That's what labels do now but under this system, it'll hopefully avoid that whole fronting bands money in exchange for the bands' life thing.

</comment></comment></comment></comment><comment><USER6> Differential pricing is a retail strategy that predates the Internet. If you want to be one of the first people to read a new book by an author you like, you have to spend to get it in hardcover; if you don't like the hardcover price, wait a year and then buy it in paperback. You can watch a new movie in the theaters, or wait to pick it up on DVD. Pay sticker price for whatever groceries you want, or make the effort to collect coupons and buy the things that are on sale. Etc., etc. It's all artificial scarcity, in the sense that nothing prevented a publisher from putting out a book only in paperback from day one.

"Exclusive content" is the same idea: the only difference is that the discounted price is zero.

<comment><USER7> Yep, and time and time again has shown people pay for sooner. I really think people who assume "scarcity" is the only factor in economic models aren't seeing the whole picture. Exclusivity of group membership (I have a t-shirt, jacket, etc. with this awesome logo I can advertise my social status/beliefs), being "first" in a social group, and just plain impatience or apathy contribute greatly. If everyone was purely rational, I'm pretty sure we'll all use refillable water bottles, personal water filters, and buy food in bulk. Assuming the motivations behind every actor in the economy are purely rational is utterly naive.

</comment></comment><comment><USER8> This is absolutely true - for digital content.

For physical content, scarcity still sells. Take Amanda Palmer's Kickstarter - selling huge amounts of specially made CDs, books, personal visits, etc.

[http://www.kickstarter.com/projects/amandapalmer/amanda- palm...](http://www.kickstarter.com/projects/amandapalmer/amanda-palmer-the- new-record-art-book-and-tour)

<comment><USER9> Absolutely agree. But what if the next big wave is making physical 'content' abundant instead of scarce? 3D printing, the internet of things, etc. That's what I'm starting to think about...

<comment><USER10> There are a couple of interesting books about that topic [1, 2]

The premise of most of them is that innovation its outpacing the hability of the average person to keep him self employable. That will lead to massive unemployment, bigger wealth inequality and social unrest.

[1] Race Against The Machine: How the Digital Revolution is Accelerating Innovation, Driving Productivity, and Irreversibly Transforming Employment and the Economy

[2] The lights in the tunnel. Where will advancing technology, job automation, outsourcing and globalization lead?

<comment><USER9> I'm starting to think that the key is to help the unemployed, the '99%', the 'others', to learn how to build their own businesses, their own livelihoods. To stop depending on someone else to provide a livelihood for them. That's where we're heading, in my view. That's the only way out of the current mess. Not to get too hippie-ish, but it's about combining self-interest with empathy for others.

<comment><USER11> "combining self-interest with empathy for others" is basically the real-deal Adam Smith brand of capitalism. Not the hollywood/cliffs-notes version we're living in now...

</comment></comment></comment></comment></comment><comment><USER12> Creative works have always been purchased on a "Do I enjoy this enough to buy it?" basis. We're used to sampling art, and it's always been this way, whether it be via public art, radio, etc., then we go buy it if we like it. The internet is an evolution of this, but without the controls that distributors are used to (payola anyone?).

What RadioLab is trying to do here is using a "free gift" to goose the people who enjoy what they make into giving them some coin.

This is very common in public radio and TV, but usually comes in the form of a free t-shirt or coffee mug. In that case the "gift" has dual purpose - it also functions as advertisement and a physical reminder to go watch/listen to the thing you helped support.

</comment><comment><USER13> This goes beyond creative works, but to many other types of digital content.

I remember hearing John McAfee discussing trying to come up with a way to sell software, and say something like "here was this thing, that cost absolutely nothing to reproduce and transmit - and so I decided I could give it away for free, and charge for the updates."

Pretty much the Gillette Razor model for the 21st century.

</comment><comment><USER14> I often daydream about the future of products. I like to imagine what it would be like without fresh fruit, or water so scarce it's illegal to water the lawn. I also imagine the model's counterpart, that some things have become very easy to produce. The article refers to digital content, which can be infinitely reproduced without burden.

But I like to wonder, are there physical goods that are close to this digital model - are there some things we can make so much of, that we'll never have to really worry about their production? More to the point, are we entering a period where the baseline is that a person who refuses to work can literally not starve to death because of abundance? If most jobs eventually turn to digital production that can be so easily shared like the article suggests, could we ever see a planet where there are simply not enough jobs and that working is optional since you could subsist on free products?

<comment><USER15> At least in the USA...

We irrigate our lawns with _drinking water_.

We pay farmers to _not grow_ (or to dump what they do).

The greatest medical malady of our poor is _obesity_.

1 in 50 live on government-provided welfare.

The "poverty line" is _twenty times_ world median income.

Not quite the idealized form you have in mind, but getting there.

</comment></comment><comment><USER16> In this post-material economy, attention is what is scarce. Quality interactions are scarce. Of course we will are not quite there yet, and the material economy is becoming more influenced by externalities like carbon output. Both trends are making it difficult to use the price system as an efficient means of allocating resources.

</comment><comment><USER17> I understand what you are trying to say. I am curious though what you recommend as a monetization strategy if you take that route? I suppose ad revenue is always the old fall back, but I would imagine that for a lot of the content out there, that is not free to produce, there needs to be some way to pay the bills.

<comment><USER18> Well if the things you need are not scarce, then it doesn't take as much money to "pay the bills."

<comment><USER19> Let me know when you have a strategy to make housing into a post-scarcity good like software.

</comment></comment></comment><comment><USER20> There's scarcity in spending time with a particular person, for example. Meeting their producers sounds like a good "exclusive" bit of interaction, but I agree about non-exclusivity of content.

<comment><AUTHOR> Precisely. Great songs that touch people are exceedingly scarce. Recordings of great songs are not. Same goes for great books, great movies, great product design.

</comment></comment><comment><USER21> Convenience is still scarce. Digital content doesn't seem quite as abundant to less technically inclined people than HN readers.</comment>


The Problem Isn't Twitter. It's That You Care About Twitter - <AUTHOR> https://www.theatlantic.com/ideas/archive/2019/04/political-leaders-should-stop-caring-about-twitter/588004/ <comment><USER2> Twitter is a perfect example of why the squeaky wheel gets the grease.</comment>


Show HN: Generate a graph of any data from your terminal - <AUTHOR> http://tailgraph.live/intro <comment><USER2> What is the advantage of this over, for example, running gnuplot locally?

</comment><comment><USER3> Where's the data being sent to?

<comment><AUTHOR> The data is just being relayed through my server to a websocket. Nothing is stored, but I'm planning to post the source so people can see that clearly and run their own instance if they want.

</comment></comment><comment><USER4> Site appears to be down for me.

<comment><AUTHOR> whoops, looks like it had some issues with traffic, it's back up now!

<comment><USER5> aaaand it's down again

<comment><AUTHOR> it should be back up again

<comment><USER6> anndd it's down again

</comment></comment></comment></comment></comment><comment><USER7> What a great idea! This could be really useful, thanks for sharing</comment>


Show HN: Trigger – run a shell command repeatedly on file changes - <AUTHOR> https://github.com/sharkdp/trigger <comment><USER2> For people who are interested in doing these kind of things on other systems than Linux, check out fswatch, it's cross platform: [https://github.com/emcrisostomo/fswatch](https://github.com/emcrisostomo/fswatch)

<comment><AUTHOR> Interesting, thank you for the information!

</comment></comment><comment><USER3> I use anacron ([https://en.wikipedia.org/wiki/Anacron](https://en.wikipedia.org/wiki/Anacron)), it's like a cron on file changes.

<comment><USER4> Do you mean incron? ([http://inotify.aiken.cz/?section=incron&page=about&lang=en](http://inotify.aiken.cz/?section=incron&page=about&lang=en))

<comment><AUTHOR> That's probably it. I was trying to figure out if anacron could really handle file system changes ;-). Thanks for the link. It looks like it is bit more complicated to set up and rather suited for permanently running tasks.</comment></comment></comment>


Why no one uses functional languages (1998) [pdf] - <AUTHOR> https://dl.acm.org/doi/pdf/10.1145/286385.286387 <comment><USER3> FP is going mainstream. In the same way that mainstream OO wasn't pure OO in the manner of Smalltalk but instead hybrid languages like C++ and Java (and more recently Python), mainstream FP isn't Haskell but languages like JS (w/ Typescript), Scala, Kotlin, Rust, Swift, and friends taking core ideas from academic FP and presenting them in way that works better for industry.

<comment><USER4> Mmm maybe, but none of those languages (afaik) have explicitly pure functions which is the keystone of functional programming.

<comment><USER5> It's really really hard to get things right with pure functions. Haskell struggled with this for decades. You can't expect to be able to import this in some random language and just make it work.

So random languages just pick what they like (lambda expressions, weaker forms of algebraic data types, sometimes the complete typeclass concept, various forms of Monads from "unnecessarily specialized because whoever picked it didn't understand it" to "properly done"0.

And while having pure functions certainly forces you to get things right, you don't need to do this when copying ideas. And there are functional languages (some very old, like Lisp) that never had pure functions in the first places.

</comment></comment></comment><comment><USER6> Tooling is such a huge factor. I'd love a functional language that had build tools as nice as rustup and cargo.

Indeed I'd place Rust as an example of a language that got it right. Sure, Rust isn't functional, but it's confusing on first usage, so close enough :D

In all seriousness, the big issue that functional languages have to get over is that most programmers are not great mathematicians, so declarative programming feels very weird and confusing. With Rust the big issue is all the rules about memory. How did Rust solve it?

For one, they made the language as familiar as possible. Rust's syntax is extremely close to C/C++, even if the semantics aren't. As much as PL people claim syntax doesn't matter, it does. ReasonML is a project with a similar philosophy.

Next, they put a lot of work into documentation and explanations, especially at the compiler level. Rust's compiler messages are fantastic and often anticipate beginner mistakes. I'd love a functional language that detects when the code looks kinda imperative and gently guides you towards the functional option.

And finally, they had a killer app like Wadler mentions. Performant memory safety is pretty hard to argue against. Even a manager could see the potential upsides. I wonder what sort of killer app one could create for functional languages? Reason with ReasonReact could have robust UI as one, but that's still not as good of a sell as performant memory safety.

<comment><USER7> Performant memory safety is not that big of a feature. GCs are fast enough unless you have real time constraints.

If have real time constraints, you probably do not care about memory safety that much, unless your system is safety critical.

If your system is safety critical, you are using lots of tooling for validation beyond memory safety - and probably some form of model based code generation. That tooling just does not exist for newcomer languages.

<comment><USER8> Tracing GCs are only fast and predictable enough if you devote roughly half of your memory and a significant share of CPU/battery to them.

And of course garbage collection is only part of what's needed for memory safety.

<comment><USER9> That's not been true for a while.

GCs like ZGC can handle tiny heaps with no pauses, pretty minimal heap wastage and a ~10% throughput hit vs a more middle of the road GC. You're thinking of the Go GC and assuming they must be all like that.

<comment><USER8> I'll be very happy once claims about improved GC performance turn into actual reality for my workloads.

What is "pretty minimal heap wastage"?

As far as I can see, ZGC (which is currently marked as experimental) doesn't give any specific performance guarantees. It'll be interesting to try.

 _" How much headroom is needed very much depends on the allocation rate and the live-set size of the application. In general, the more memory you give to ZGC the better."_

[https://docs.oracle.com/en/java/javase/13/gctuning/z-garbage...](https://docs.oracle.com/en/java/javase/13/gctuning/z-garbage- collector1.html#GUID-A5A42691-095E-47BA-B6DC-FB4E5FAA43D0)

<comment><USER9> Heap wastage is tunable vs CPU time, but it's less than the 100% go uses. If you have spare RAM your app will run faster, is the usual tradeoff.

</comment></comment></comment></comment></comment></comment><comment><USER10> I think languages by themselves rarely become popular. There is some killer feature that a language offers which makes a certain class of problems very easy to solve. For example, Rails did it for Ruby, Go's stdlib makes it very easy to write performant network services, Java's promise of platform- independent code, huge stdlib and a free compiler and (later) free IDEs when almost all other compilers and IDEs were paid, C# - Microsoft's deeply integrated ecosystem, Rust's memory safety and the promise of fearless concurrency.

Functional languages are lacking such killer features which makes them 10x more productive for widely faced problems. I am learning Clojure to learn more about the functional paradigm, but I don't see any particular type of software that I could do with it which I couldn't achieve with Java, Go, JS or Python all of which I already know well enough.

<comment><USER11> I think you could argue that type-safe STM is such a feature for Haskell. It's something that I really miss when using threads or async essentially anywhere else. I would probably jump for Haskell over Rust as a result for heavily concurrent or multithreaded programs. Rust is memory safe, but it can't express purity and it can't enforce that STM transactions should be safely retryable.

I would also argue that Elm offers close to a 10x factor for certain types of front-end web applications. It's a very closed system, which is a matter of great controversy, but in areas where you can exist within that system it is a dramatic leap forward compared to anything else in my experience.

If you consider Erlang/Elixir functional languages, I would argue that BEAM can offer close to a 10x factor for certain types of distributed or concurrent applications as well.

You mentioned Clojure, though, and I don't know what the killer 10x feature would be there.

</comment></comment><comment><USER12> We should draw a distinction between FP as religion and FP as tool kit.

FP as religion has failed to gain acceptance because it imposes too much cost on the user. I have to rethink my whole stack in terms of category theory AND deal with your terrible ecosystem? Hard pass.

FP as toolkit, on the other hand, has been a smashing success. Most of the core ideas of FP are mainstream now and some of the latest advances in non-FP ecosystems (React, for example) are based on FP ideas.

<comment><USER13> FP ideas have been seeping into the mainstream for quite some time now.

The oldest: GC was originally invented for Lisp. It's common now.

Type inference was big in FP before it made the jump to language like Java or C++ much more recently.

Generics were a natural idea in a typed FP context. Mainstream languages got them, now.

I'm looking forward to algebraic data types becoming really common. (The simplest explanation is that they are C-style unions with tags to tell you which case you are in. The compiler enforces that the tags correspond to how you use them.) Some mainstream languages are starting to add them.

<comment><USER12> > The oldest: GC was originally invented for Lisp. It's common now.

LISP was so ahead of its time, its parents still haven't met yet. Not very FP, but another gem from SBCL: saving and restoring program state for later use. Now there's the CRIU [0] project for doing this with Linux and Docker containers.

> I'm looking forward to algebraic data types becoming really common.

I'm not too familiar with the full scope of algebraic data types. Wondering: does Typescript have this or is it still missing a few key components? Really like how it has Union types, which I wish Scala would have.

[0] [https://www.youtube.com/watch?v=LrHW7Vvbie4](https://www.youtube.com/watch?v=LrHW7Vvbie4)

<comment><USER13> Algebraic data structures mostly just means union types.

(That's the + in the algebra. The * comes from bundling multiple values together, like in a tuple or in a C-style record, virtually all languages already have that.)

There's also Generalized Algebraic Datatypes (GADT). They are a bit more complicated. So I don't expect mainstream languages to pick them up anytime soon.

About GC: you _can_ do pure functional programming without a GC. But it requires lots of big guns from more advanced theory. (Mostly stuff like linear typing.) However imperative programming without a GC is comparatively simple.

So it's no wonder that historically, GC was invented for FP first, and GC-free FP was only discovered later.

(And for general CRUD or web programming, or basically anything outside of low level systems programming, GC is more productive in terms of programmer time than other approaches. At least given currently known techniques.)

</comment></comment></comment></comment><comment><USER14> A more recent view: "Who Cares About Functional Programming?"

[https://thomasbandt.com/who-cares-about-functional- programmi...](https://thomasbandt.com/who-cares-about-functional-programming)

</comment><comment><USER15> > 'Much of the attractiveness of Java has little to do with the language > itself, but with the associated libraries ... (Much of the unattractiveness > is due to the same libraries)'

</comment><comment><USER16> Jane street is a very successful company that uses Ocaml for everything. Any other companies like that?

<comment><USER17> Brazilian nubank, they mainly use clojure

<comment><USER18> And they recently acquired Plataformatec, the company behind Elixir.

</comment></comment></comment><comment><USER19> Programs are incentivized by need, and businesses and users are the primary consumers of programs. If FP and OOP both satisfy those needs nearly indistinguishably, but one of them is easier for the vast majority of programmers, then guess which one everyone is going to use?

Working for a company has taught me that people will try to get away with doing as little as possible when it comes to satisfying a business need.

<comment><USER20> > Working for a company has taught me that people will try to get away with > doing as little as possible when it comes to satisfying a business need.

Aka productivity.

<comment><USER21> Sure but skipping all best practices in favor of fast delivery might bite you later on.

<comment><USER13> Functional programming is mostly touted as being able to keep productivity up in the long term.

(And detractors mostly complain that it takes too long to get started, not that the long term is unmaintainable.)

</comment></comment></comment></comment><comment><USER22> I think in the current crop of languages, the "functional language" concept has shifted its purpouse:

1 GP languages have adopted FP enabling features, so we can do FP just fine in many mainstream languages, and it is infact very common (see eg React, Ramda popularity on frontend and many recent FP features in Kotlin, Java, C++, etc).

2 At the same time FP leaning languages are more popular than ever, to the point it would be ridiculous to claim "nobody uses functional languages" given the visible positions of Clojure, Erlang/Elixir, ReasonML/Ocaml, F#, Scala, etc on the scene.

I think adding 1+2 together tells us that FP is on a real streak. People choose use FP langauges, not because of their capabilities, but because of the mindsets, ecosystems and culture they promote, along with promising to consistently pave the way for FP problem solving instead of often falling back to imperative.

<comment><USER9> (1) keeps cropping up but these new languages are hardly "FP" or even using "FP" features. I think we give FP research way too much credit.

Pure functions? No, none of those languages can even express the concept except maybe C++ with the const keyword (which is as old as Haskell itself).

Immutable data structures? Even very modern languages like Kotlin don't really have them. You can define a data structure where the fields are immutable after construction but it's not transitive. There are read only collections but not immutable collections, without doing extra work. Maybe immutable collections will turn up at some point, there's a proposal to add them, but it'll just be a library not a language feature.

Type inference? New languages don't use full blown Hindley-Milner TI, they do much more limited TI that's designed to strike a balance between readable error messages, documented APIs, IDE performance and so on. Meanwhile the concept of TI is obvious and required no inspiration from anywhere - <USER2> who ever looked at source code will be struck by the repetition required.

Generics? C++ templates started being discussed in 1985, and it's again a pretty obvious concept that would occur to anyone who wanted to offer type safe collections.

I honestly can't see the huge impact FP supposedly had on mainstream programming. All the ideas its advocates try to claim for their own are either not used, or are obvious but usually not implemented the same way. My guess is if Haskell had never existed modern languages would look exactly the same.

<comment><USER22> Many FP languages also lack these.

Try these: closures, first class functions, garbage collection, anonymous functions.

<comment><USER9> All except first class functions are found in Java. I don't think many people would claim Java is an FP language.

<comment><USER22> I think the discussion is in danger of going the way of a Möbius strip... Language features originating in FP have been adopted by GP languages, which was the point :)

<comment><USER9> But those features were found in Java 1.0 so the influence can't be that strong, given it was heavily advertised as an object oriented language at the time.

<comment><USER22> From the list, only GC was in early Java. GC was popularized by Lisp (was added in the 60s?) and other FP languages like ML (had gc from the start in 1973) decades before.

</comment></comment></comment></comment></comment></comment></comment><comment><USER23> Maybe another reason is: there is two kinds of software, one is programms which process input data and transform them to output data. And then there is software which operates hardware, like steering of the stepper motors of a robot, or the firmware of a washing machine. Functional languages are (rightfully) immediately dismissed for these kinds of problems. Then there is a grey zone: GUI programming, this is almost hardware progamming and FP have been trying but not doing well in this domain (might be wrong though). Hardware programming doesnt come up often IT departments, so maybe this is a blind spot, but basically you are loosing a big share of the market.

Ps: something special might be with the erlang telco system, but i have no idea about it

<comment><USER24> This is true, but it depends which language we're talking about. There are two definitions to FP. The stricter one is about having immutable data and pure functions. In that one, it's fundamentally only a computing language, not a programming language, in that by definition it cannot tell the computer to perform any action, as purity implies no side effects.

That means you always need an additional layer which then drives the side effect. For a lot of programs, that adds some complexity to the program. The programmer has to model their side effects with this layer in mind. The layer means the programmer loses control over the fine grained details of the side effect as well, which might hurt performance, or make certain things more difficult.

That said, there are impure FP languages. Where you can program the machine to do whatever you want, and also make computation functionally with immutable data and pure functions.

Clojure is such a language. I'm very biased towards it FYI, as it's the language I enjoy most as of now. But I think it's important to bring it up. Defaulting to immutable and pure computations, but letting you perform all kinds of controlled mutation and side effect really seems like a good future to me.

Rust in a sense is pretty similar. While it doesn't use a strict functional sense of immutability and purity of data and functions. The borrow checker tracks all changes to data and assigns ownership of access, which is kind of similar to immutability and purity. Yet it lets you do unsafe things when required, often for interopping with peripherals and the machine itself.

Erlang like you mentioned is another example of this.

I think those languages might have a better future, and are slowly starting to pick up on popularity. Even in JS for example, lots of people are adopting this style. So if not a programming language, at least the paradigm seems to become more and more popular.

As an aside, for example, there is a Clojure dialect called ferret [https://ferret-lang.org/](https://ferret-lang.org/) which is designed for use on microcontrollers. It has immutable data structures and support all the functional aspects of Clojure, yet targets real time control applications.

<comment><USER13> Immutable data and pure functions aren't as big as an obstacle as you might think.

Uniqueness types allow you to use immutable variables and still directly model side effects and state changes. Arguably better than mutable variables do so.

(Haskell is not a good language for that kind of control. Though you could make a language that looks quite a lot like Haskell that fits. And people have done so.)

<comment><USER24> Isn't uniqueness typing just a more restrictive form to Rust's linear types? And since Rust itself is often forced to rely on unsafe blocks for programming tasks (as opposed to computing tasks), then wouldn't that be true of a language with uniqueness types as well?

Also, I'm guessing you mean Clean or Miranda? Any idea why they aren't more popular than Haskell? Or why Haskell doesn't adopt uniqueness typing?

<comment><USER13> Linear types and uniqueness typing are so close together that people often mix them up. Yes, that's basically what I am talking about.

For general usage monads are much easier to deal with than Clean's "pass around the world" system. (I only used Clean for a bit, but haven't used Miranda, yet.) However, linear types for Haskell have been in the works for some time. See eg [https://www.reddit.com/r/haskell/comments/dpr276/what_is_the...](https://www.reddit.com/r/haskell/comments/dpr276/what_is_the_status_of_ghc_linear_types/)

(Similarly, GHC now support strict Haskell, too, on a per module basis.)

Uniqueness typing shines when you want to split up your 'real-world' object, so you can eg manipulate individual memory cells or ports etc independently.

Rust goes into unsafe mode for multiple reasons. You could eliminate most of those, if you were willing to take the complexity of dependent types. Ie more or less that means the programmer taking on a greater burden of proving code to be safe to the compiler; instead of relying on Rust's more automatic but conservative system.

</comment></comment></comment></comment></comment><comment><USER25> Just a couple gripes, they seem like non-problems today:

Training -- Schools should be covering the training problem in this PDF, and i think that would help a lot.

Availability -- GHC is no longer an 'adventure' to install

Packagability -- Can't you just build standalone programs with most FLs?

Popularity -- Aren't they getting more popular?

"They don't get it" \-- I don't think the example given of "i need some documentation and stability" applies anymore.

<comment><USER26> > Training -- Schools should be covering the training problem in this PDF, and > i think that would help a lot.

This just raises a chicken-and-egg problem: why should schools (primary, secondary or tertiary) teach a language that is not "mainstream"?

Is there a language that is more "functional" than Python or Lua that's designed to be a teaching language?

</comment></comment><comment><USER27> "[...] this allows Haskell to be used as a scripting language for Microsoft's Internet Explorer web browser". Interesting. Never heard of that.

[https://www.microsoft.com/en- us/research/publication/scripti...](https://www.microsoft.com/en- us/research/publication/scripting-com-components-haskell/)

EDIT: Add link

<comment><USER28> If WASM had access to the DOM, I imagine we'd start seeing a replacement of JS and a democratization of web languages. I however dread the day I get to read a full-stack C/C++ webpage.

<comment><USER29> You can get a full-stack .net webpage with Blazor already

<comment><USER30> I've been playing with it in the last two days, and while it's a bit annoying to have to deal with CSS for UI instead of XAML and the recompilation cycle, the framework is impressive. A lot of things can be done without touching JavaScript directly, and when in need the interop works well.

</comment></comment></comment></comment><comment><USER31> Conjecture: programming will become split between "just build my app" and an elite set of programmers who do the most demanding work, eg cloud infrastructure for Amazon, Google etc. The latter will mostly use FP, the former will use JS et al.

I think the reason most coders (and I include myself) don't use FP is that we don't have enough training in it, we often don't have a strong mathematical background, and we have an imperative mindset. It's harder to learn those things, it's a different way of thinking than we use in our day to day lives - much like mathematics is difficult for most people to grasp, because it doesn't come naturally to them. Humans are not theorem provers.

As competition for jobs soars and wages plummet (hello, remote work revolution!), more of the top X% of coders will be forced to learn FP to maintain their competitive advantage, and you'll end up with a lot more people using it. Some of that will trickle down, but most of us will be writing JS rather than Haskell for the rest of our lives.

<comment><USER10> >> an elite set of programmers who do the most demanding work, eg cloud infrastructure for Amazon, Google etc. The latter will mostly use FP, the former will use JS et al.

I don't think this is true. In fact Go seems to be at the front and center of most of the Cloud Native infrastructure projects such as Kubernetes, Docker, Prometheus, and many others.

<comment><USER31> Wasn't a comment on the status quo, but rather the future.

I think the Go trend confirms my point: an isolated group of programmers have been able to adopt Go for that kind of infrastructure, but outside of those shops most people are quite determined never to use Go and rant about it a lot.

In the same way, Go will be replaced by FP and app developers will not go near it.

</comment></comment></comment><comment><USER32> Here is a list of companies using using Clojure & ClojureScript:

[https://clojure.org/community/companies](https://clojure.org/community/companies)

<comment><USER33> Lists like this strike me as somewhat meaningless. Those companies probably also use bash scripts, hardly represents a significant chunk of the product. How _much_ of their codebase is in Clojure?

</comment></comment><comment><USER34> We have had these discussions for many years (even pg wrote about Blub IIRC), but the real reason (IMO) is that _language choice matters far less than we like to think_.

What business we choose to build with the language will outweighs the impact of language choice a hundredfold.

Our decisions on CI/CD, our discipline on building clean architecture with whatever language we like matters more.

In short, the things that make development a productive and enjoyable experience are linked less to the language than to the skill and professionalism of the development team - and "weird" languages tend to have more skilled and professional developers working in them, so the causality / correlation tends to get mixed up.

The upshot is, worry less about learning Lisp at work, and more about fighting your corner on why we should rewrite that 3,000 line monster engine module that we all call each day but making changes to it is a nightmare.

<comment><USER35> > " _What business we choose to build with the language will outweighs the > impact of language choice a hundredfold._ "

This is on one hand a truism that actually adds nothing to the discussion. In achieving success, of course the business matters. Even more so the best simplifications come from understanding the business domain, not from the tools used.

\---

On the other hand it's also patently false. Build a web browser in Ruby. Build an operating system in JavaScript. Build your average web app in C/C++.

The tools used have inherent limits, being optimized for certain domains and not others, can make us more productive, can lower the defect rate and can keep us happy enough to see the project released.

In my experience _some_ static FP languages can lower the defect rate, can make certain problems easier to tackle, can make the code easier to maintain and refactor and due to performance being reasonable, they tend to be more general purpose than other languages — this trait being a consequence of static typing, making performance good enough, coupled with the languages themselves still being high level enough.

\---

> " _learning Lisp at work_ "

Ah, I see the problem ... many of you thought Lisp is some sort of secret sauce. It isn't, it never was, Lisp is a freaking mess.

The real problem in our industry is not applying the scientific method for advancing it. We thus rely on fashion and salesmen to sell us magic pills.

<comment><USER36> > On the other hand it's also patently false.

The sentence "language choice matters far less than we like to think", which means something like "switching between _most_ pairs of languages in most situations doesn't make a big difference", and your examples which mean "there exist pairs of languages as well as situations where switching between them makes a big difference" are completely compatible.

In fact, I would say that the latter is not interesting because of course we know there are situations where high-performance languages are more or less appropriate, but the interesting question is whether languages that have similar aims and performance, and differ mostly in their linguistic features make a big difference.

> In my experience

Unfortunately, people will also support the effectiveness of homeopathic medicine based on their experience, and much more enthusiastically. I'm not doubting your sincerity, nor even the facts of your situation, but this doesn't count as evidence. On the other hand, the fact that different languages are tried in different places, in an industry with strong selective pressures, the fact that no single language or paradigm completely overtakes all others is evidence in favor of there _not_ being a very big bottom-line impact.

<comment><USER37> > the fact that no single language or paradigm completely overtakes all others > is evidence in favor of there not being a very big bottom-line impact.

Or perhaps because no paradigm is universally applicable to all cases. There will be a bottom line impact if the wrong tool is picked for the job.

<comment><USER36> Sure, but what I'm saying applies when comparing different languages that _are_ reasonable choices for a given domain.

Anyway, this isn't a poll, and while opinions -- one way or another -- shape the _popular_ opinion, they don't settle the empirical question.

</comment></comment></comment></comment></comment><comment><USER38> FWIW if you're not using algebra to write your software you're not quite doing FP just yet.

Doing math on software to make new software is the overlooked part of FP in my opinion. You hear about "first-class functions" and "referential transparency" a lot but not as much about "equational reasoning".

</comment><comment><USER39> I like framing FP and OO as technologies, and technology takes money and time to improve. The technological aspect comes from the ability of language and IDE teams to translate your programming style into an efficient program.

</comment><comment><USER40> I think there is a fundamental reason. The computer is inherently mutable. As a result, the basic abstractions of operating systems are mutable.

Thus, except for nicely contained problems such a text transformation (see Pandoc for an awesume use of Haskel), interfacing with the computer and the rest of the world becomes an impedence mismatch. Yes, monads are awesome, but you shouldn't need a math PhD, to print "Hello World" to the screen and understand what it is doing.

The other issue is performance. Yes, performance is generally good enough, except when it is not, and when it is not, it is not obvious why you are having issues or what you can do to fix them. This once again comes to the abstraction mismatch between the mutable computer and the pure functional language.

<comment><USER41> Monads are among the simplest abstractions you can find in computing. Compared to almost anything you need to understand in software development (how the hell do I make webpack do the thing I need?) they're ridiculously easy.

The fact that so many devs are scared of monads and believe outrageously silly things like you need a math phd to understand them seems to be similar to being afraid of drinking dihydrogen monoxide. Even though it's only water, it has a complicated sounding scary name, so it must be bad.

<comment><USER42> There's also a huge number of people who are fairly bad at explaining what exactly it is. I don't think I should fault them, per se, it's sort of trying to explain why water is wet.

</comment></comment></comment><comment><USER43> See also from 2014: [https://news.ycombinator.com/item?id=8082728](https://news.ycombinator.com/item?id=8082728)

</comment><comment><USER44> Interestingly written 3 years after the introduction of a functional language that eventually went on to become the most popular language there is.

<comment><USER28> Which language are you talking about?

<comment><USER45> The one that controls the web application you're currently using.

<comment><USER46> Controls the comment collapsing, everything else doesn't require JS.

</comment></comment></comment></comment><comment><USER47> My experience so far with functional languages (as an OOP-er):

* After a whole evening studying I am able to subtract 1 from 5.

* After spending hours checking what is wrong it just does not compile.

* After the hello world part there are no libraries that can be used for web development.

* Lack of good documentation.

* Lack of good tutorials.

But lately I discovered Elixir and I believe this could be a language that would change my mind about functional programming. Phoenix looks great and LiveView matches .NET Blazor (which I use a lot).

But then there is still the question: why even bother using a functional language? Because I have little experience in functional languages this is hard to answer for me.

Edit: _" After a whole evening studying I am able to subtract 1 from 5."_ I should have formulated this different. What I meant was: sometimes you follow a very long tutorial about a functional language. You will learn a lot about the syntax but in the end when you try to use it, you will only get to the basics of writing a function that subtracts x from y.

<comment><USER48> Funnily enough 5 - 1 works almost in every language the same way. Maybe with a different syntax but I cannot think about any FP language where I cannot do it.

<comment><USER49> I mean Lisp. But It's pretty debatable whether Lisps are functional languages. Imo, static typing is a requirement, at least culturally.

<comment><USER13> As far as I can tell at the International Conference on Functional Programming the Schemers are treated as full member of the functional programming community, despite lacking static types.

Static types are neat, and FP people tend to really like them, but I wouldn't call them a strict requirement even culturally.

You can make a statically typed Lisp, of course. Just like you can make a dynamically typed ML-like language. Haskell even sort-of supports that. See [https://gitlab.haskell.org/ghc/ghc/-/wikis/defer-errors- to-r...](https://gitlab.haskell.org/ghc/ghc/-/wikis/defer-errors-to-runtime)</comment></comment></comment></comment>