Working with Python subprocess - Shells, Processes, Streams, Pipes, Redirects - <AUTHOR> http://jimmyg.org/blog/2009/working-with-python-subprocess.html

<comment><USER2> Python's subprocess module is great. It is definitely the cleanest wrapping of a system program of any of the scripting languages I've ever used. When you just want output, you call .communicate() and be done with it. When you want to interact, the file descriptors are there for you to play with. It's scripting heaven.

There is one feature that he could have touched on more:

shell=False is the only easy, safe way to put user-supplied input in a command line. This stuff is just too damned easy to get wrong.

I've seen _heroic_ code that tried to work around shell quoting and escaping vulnerabilities. One colleague was trying to write a wrapper around the ldapsearch binary, and decided that filtering all "|" would do it. He'd completely forgotten that ` also triggers arbitrary command execution.

Don't be a hero. Keep bash away from your input.

</comment><comment><USER3> I had to use os.popen (Maya, MotionBuilder, and outside) (obsolete by now, but still available), and it seemed much more easier than subprocess.Popen because I simply wanted a file filter.

Basically a coworker had ways to read/write custom ASCII 3D animation and model format, and another one introduced a binary form for it. All I did was to replace his "open" function with "open_model" and "open_anim" (no monkey patching), with something like this:

      def open_anim(name, mode, *args, **kwargs):
     	if name.upper().endswith( ".BIN_ANIM" ):
     		if 'r' in mode:
     			return os.popen( "anim_bin2text <" + name, mode, *args, **kwargs )
     		if 'w' in mode:
     			return os.popen( "anim_text2bin >" + name, mode, *args, **kwargs )
     	return open(name, mode, *args, **kwargs)

<comment><USER4> popen might have been simple but don't forget popen2, popen3 and popen4, spawn, spawnvp, spawnlp. You've got to get rid of that cruft or you become php.

</comment></comment><comment><USER5> Great article. Chaining pipes together in Python with subprocess can get a little wordy, but the power is there to do anything you need to do.

It's especially excellent when compared to Ruby. Ruby has a bunch of popen calls, but each one has strange quirks and inflexibility to make it not good for building pipelines.

[http://whynotwiki.com/Ruby_/_Process_management#How_do_I_exe...](http://whynotwiki.com/Ruby_/_Process_management#How_do_I_execute_an_external_program.3F)

I'm finding myself using a ton more shell (bash) for pipelines, as this is what it was built to do, but when I need to add more logic its Python and subprocess.

</comment><comment><USER6> The python parts seemed OK. But the data flow diagram in the OP at:

[http://jimmyg.org/blog/2009/working-with-python- subprocess.h...](http://jimmyg.org/blog/2009/working-with-python- subprocess.html#what-happens-when-you-execute-a-command)

(I.e., the first blue box in the article, "what happens when you execute a command")

is wrong. It implies that the command (ls -l) is read by the terminal emulator, which gives it to bash, which gives it to the kernel.

This is wrong in several ways. The terminal emulator starts bash (or whatever shell) and it is bash that directly reads the string "ls -l", figures out what it means, finds the ls program in PATH, and runs the ls binary. It is ls that calls kernel functions when it's doing its job.

And of course, the kernel is not really of importance here. The selected command does whatever it does, which might not even access the kernel ("Hello, world").

And bash can and sometimes does directly call kernel functions without an intermediate binary (e.g., unlink(2) to overwrite a file for output redirection).

<comment><USER7> _...the data flow diagram in the OP..."what happens when you execute a command"...is wrong. It implies that the command (ls -l) is read by the terminal emulator, which gives it to bash, which gives it to the kernel._

It's accurate, the author just left off where the command is coming from (implying it's the user sitting at a keyboard). You type into the terminal- emulator, which passes that input to the program it's running, the shell, which figures out what it means.

A terminal (or something emulating a terminal) is always between the keyboard and the input of the program running. Even when you login on the console, the terminal driver is accepting your input, transforming it in some fashion, and giving it to the nxt level.

A terminal doesn't exist between this echo command and the sh program:

       echo ls -l | sh

And is, in fact, why things like expect(1) exist and that ssh can be configured to allocate a pseudo-terminal or not depending on how you want/need to interact with the remote command.

Incidentally, there seems to be a bug in yum whereby it detects if its output is a terminal or not tries to format it, but it gets the check reversed and pretty-print word-wraps output that goes to a non-terminal. Which is why

       yum search something

and

       yum search something | sort

produce different output and the latter has blank lines scattered throughout. (yum 3.2.28 on Fedora 14).

 _And of course, the kernel is not really of importance here. The selected command does whatever it does, which might not even access the kernel ("Hello, world")._

Every _Hello, World_ program I've seen displays text to the user (or sends it somewhere), which means a system call, which is calling "kernel functions". The kernel is the mediator between programs/processes, there are very few cases where it isn't directly involved.

 _And bash can and sometimes does directly call kernel functions without an intermediate binary (e.g., unlink(2) to overwrite a file for output redirection)._

Not really a good example, but the idea is sound. Bash has builtin functionality that emulate the functionality of system calls or other binaries. For this case, bash opens files for redirection when it's going to overwrite with the O_TRUNC flag set. This can be changed with various shell options (clobber) to have it not implicitly overwrite. You can see this with strace.

And technically, "overwriting" a file means truncation or seek-and-write, not unlink and recreate. The difference has implications for hardlinked files.

<comment><USER6> OK, I see your point regarding the terminal emulator. There are some keys that are intercepted by the terminal emulator and processed by it alone (say, whatever does "print screen" or "close window"). The shell never sees these key presses.

The thing I didn't (and still don't) like about the diagram is it implies the whole "ls -l" is delivered by the terminal emulator to the shell. Which isn't how it works. The emulator is transparent to all normal keystrokes and the shell is doing all the work, character by character, even for stuff like ctrl-H which, years ago, could have been (and sometimes was) intercepted and handled by the terminal driver.

And the presence of the kernel in the diagram is also confused. (You are right, system calls are all over, even printf() will bottom out in the kernel; I was trying to say that there is no _requirement_ for the user program to call in to the kernel.)

Here's a sentence in the OP that has the same confusion:

"When you execute a program from Python you can choose to execute it directly with the kernel or via a shell."

The kernel is not "executing the program directly". This is nonsense.

What they want to say is that you can use a system call (execve) to directly load and run the program, or you can use an interpreter like sh.

This same point of confusion is exhibited in their diagram.

<comment><USER7> _The thing I didn't (and still don't) like about the diagram is it implies the whole "ls -l" is delivered by the terminal emulator to the shell. Which isn't how it works. The emulator is transparent to all normal keystrokes and the shell is doing all the work, character by character, even for stuff like ctrl-H which, years ago, could have been (and sometimes was) intercepted and handled by the terminal driver._

I don't think it's as simple as this. When the shell spawns a program, it gets out of the way and the program spawned receives all the input directly, the terminal intermediates, but the shell doesn't. For example, if you want to turn off echo (the printing of the characters you type), you need to change the terminal settings. This is done with _stty -echo_ (go ahead and type that, you won't be able to see what you're typing), which communicates with the terminal driver to tell it to not print characters that are input. This is how password prompts are done.

Witness:

       $ od -c
       hello<backspace><backspace><backspace><backspace><backspace><control-d>
       0000000

_od_ sees no input. The input was deleted by the terminal driver. Usually, the terminal is in line-input mode, so this can happen. Now, in my case, my backspace key maps to ^?, so to actually get a control-h from my backspace key, I had to do _stty erase ^H_. Then, the above produces:

       0000000   h   e   l   l   o 177 177 177 177 177

because the backspace/erasure didn't occur at the terminal level (in fact, nothing got erased at all, nothing interpreted the input that way, and the input was passed to od plainly). Note the output of _stty -a_. There are a number of line editing control characters that modify the input _at the terminal level_ before the program/shell ever even sees it.

You can see this with _strace -o /tmp/s.out -f bash_ then running _od -c_ in that shell, typing the above, erasing it, and seeing that neither bash wakes up to handle the input and od sees no input (od's only read shows up as returning zero bytes, indicating end-of-file).

<comment><USER6> We're not in disagreement at all. Thanks for the careful reply.

It's definitely true that some spawned programs get line-by-line input (or some other buffered input). And some don't (e.g., bash itself, which when connected via a tty, interprets bazillions of control codes).</comment></comment></comment></comment></comment>


Court of Appeal bans Bayesian probability - <AUTHOR> http://understandinguncertainty.org/court-appeal-bans-bayesian-probability-and-sherlock-holmes <comment><USER2> This reminds me of the case of Lucia de B., a nurse once suspected of killing her patients.

There never was solid evidence against her. She was a suspect because nearly everytime someone died in the hospital she was on her shift.

The initial chance of her being present when these supposed murders happened, and being innocent, was estimated at 1 in 7 billion. This number caused the police to focus their research completely on Lucia de B.

Eventually she was convicted for the murders based not on hard evidence or witnesses, but on statistics. The chance of innocence was put on 1 in 342 million.

The econometrist Aart de Vos was the first to notice that the initial Bayesian analysis was plain wrong. For example they had presumed the murderer had to be found amongst the nurses, other possibilities were neglected. They also hadn't corrected for combined P-values. He reduced the chance Lucia was innocent to 1 in a million.

The court said they had abandoned the statistical "proof", but remained of the view that it couldn't be a coincidence. Possible murder cases were chosen when they were negative for Lucia, other possible murder cases were left out of the equation. This further reduced the chance to 1 in 50.

Statisticians Richard Gill and Piet Groeneboom further reduced the chance to 1 in 9.

In my opinion statistics alone can never be adequate for conviction. It can muddy research and lead to confirmation bias. And the difference between a Bayesian chance of 1 in 7 billion and 1 in 9 is so big as to doubt the initial use of statistics even further.

[http://en.wikipedia.org/wiki/Lucia_de_Berk#Statistical_argum...](http://en.wikipedia.org/wiki/Lucia_de_Berk#Statistical_arguments)

<comment><USER3> I do not think I understand your argument. In the case as you describe it the statistics are done incorrectly and cause a potential false conviction.

How is this different then an expert on autopsies incorrectly does the job which leads to a potential false conclusion.

In both case an expert made a mistake or did their job poorly leading to a potentially unfortunate outcome. Why should one be banned and not the other?

<comment><USER4> For a criminal trial, with a jury? People are hopeless at statistics.

Even on HN you'll find people making simple mistakes.

Someone confidant presenting a mathematical argument would be persuasive to a jury, even if their numbers are totally bogus.

> _In both case an expert made a mistake or did their job poorly leading to a > potentially unfortunate outcome. Why should one be banned and not the > other?_

People doing the job poorly should be banned. But the nurse case is interesting because poor statistics became self-fulfilling.

"We have some deaths, and this nurse is present when they happen, and so they are all suspicious", becomes "all the deaths that happen when this nurse is working are suspicious, and the deaths that don't happen when this nurse is working are not suspicious". And that sloppy thinking gets turned into "probability is 1 in a gajillion that these happen normally by chance."

<comment><USER3> >People doing the job poorly should be banned. But the nurse case is interesting because poor statistics became self-fulfilling.

>"We have some deaths, and this nurse is present when they happen, and so they are all suspicious", becomes "all the deaths that happen when this nurse is working are suspicious, and the deaths that don't happen when this nurse is working are not suspicious". And that sloppy thinking gets turned into "probability is 1 in a gajillion that these happen normally by chance."

Investigators have been known to be unduly influenced by pretty much any kind of evidence imaginable and then all further evidence is construed to support the original assumption/assertion. I need a more concrete difference or divider to promote other methods above statistics.

<comment><USER5> > Investigators have been known to be unduly influenced by pretty much any > kind of evidence imaginable and then all further evidence is construed to > support the original assumption/assertion.

A different way of phrasing this is that investigators have been known to incorrectly apply even standard tools that they've been explicitly trained in.

> I need a more concrete difference or divider to promote other methods above > statistics.

If by "promote" you mean that we should try to increase the statistical literacy of lawyers, police, judges, and the general public, than hells yeah. But encouraging armchair statisticians to try to determine causality and guilt from badly-collected observational data that's analyzed by financially- invested parties...? In the highest stakes setting we have? Really?

</comment></comment></comment></comment></comment><comment><USER6> The quotations in this article are taken out of context and presented incomplete. For example, the article quotes

 _The chances of something happening in the future may be expressed in terms of percentage. Epidemiological evidence may enable doctors to say that on average smokers increase their risk of lung cancer by X%. But you cannot properly say that there is a 25 per cent chance that something has happened: Hotson v East Berkshire Health Authority [1987] AC 750. Either it has or it has not._

And the judgement continues:

 _In deciding a question of past fact the court will, of course, give the answer which it believes is more likely to be (more probably) the right answer than the wrong answer, but it arrives at its conclusion by considering on an overall assessment of the evidence (i.e. on a preponderance of the evidence) whether the case for believing that the suggested event happened is more compelling than the case for not reaching that belief (which is not necessarily the same as believing positively that it did not happen)._

Which is _exactly_ the bayesian approach of 'probability as state of knowledge'.

<comment><USER7> The quotes are contradictory. The Bayesian approach agrees with the second and disagrees with the first.

I suppose you could weasel in some sort of consistency about the wording ... something like "You cannot properly say there is a 25 percent chance that something has happened, but you _can_ properly say you _believe_ there is a 25 percent chance that something has happened." OK, fine, but what's the difference? You still assign a percent probability to the chance that something happened, and you still rule on the basis of those beliefs. Or do you? (It's not clear to me from those quotes.)

<comment><USER8> Its post facto versus a priori. There is a 25% chance of something happening, but after the fact, it either did or did not happen. Averaged over many cases we can say that after the fact there is a 25% chance it happened, but criminal trials look at one specific instance, and its accurate to say that in an instance something either did or did not happen.

<comment><USER9> You should still use probability to quantify that belief. Whether it happened in the past or is going to happen in the future doesn't matter. If you throw a coin, and the coin is flying in the air, what is the probability that it lands heads? By your reasoning, the motion is already in place, and it's either going to land heads or it isn't. With a high speed camera and some physics we might even be able to predict which side it will fall. Even if we go back to a little before you threw the coin, the electrons in your brain are already in place and it's already determined which way you'll throw the coin. Would it be invalid to say that there is a 50% chance that it will land heads? So even when we are talking about the future, we use probability to quantify our belief. Or perhaps more accurately, we use it to quantify our lack of knowledge: in this case our lack of knowledge about how exactly the coin will move. Since we are already using probability not just for "fundamental randomness" but for quantifying lack of knowledge about the future, what's so special about the future vs the past? It makes total sense to apply the same reasoning to the past, even if we are talking about a specific instance.

<comment><USER8> You're shifting the argument. Sure you should, in some cases,[1] use probabilities to determine your belief about whether something happened or not. That's precisely what the court says in the second paragraph. Your beef was ostensibly with the wording of the first paragraph, but as I pointed out that when talking about things that have happened in the past, the wording is not incorrect.

[1] Of course you also shouldn't sometimes use probabilities. If we have statistics saying that witnesses lie 15% of the time, is it helpful to present expert testimony to the effect of "there is a 15% probability the witness is lying?" Such evidence is not useful and also impinges on the fundamental right of the jury to judge witness credibility.

<comment><USER9> My only point is that whether something is in the past or in the future has nothing to do with whether you can use probability. If you're consistent then either you don't use probability at all to model lack of knowledge, or you use it for both past and future. For example if you find it valid to say "when he tosses that coin, it has 50% probability to land on heads and a 50% probability to land on tails", then you should also find it valid to say "he tossed that coin, and it now has a 50% probability to be on heads and 50% probability to be on tails" if you did not yet see the coin. That's because in both cases, the outcome was already determined (by the laws of physics, disregarding quantum mechanics). The thing is that you just don't know which way it will land or has landed because you lack perfect knowledge.

I don't have a problem with not using probability to model lack of knowledge, but you have to apply that principle consistently (and whether it is ethically desirable to use probability is an entirely different and even more difficult question than whether or not it leads to accurate beliefs). That said, I do prefer to use probability to model lack of knowledge, because what is the alternative? Ad hoc gut feeling based reasoning?

</comment></comment></comment></comment></comment></comment><comment><USER10> You know, I think I'm OK with this.

Sherlock Holmes was a detective, not a judge. What was appropriate for his line of work isn't appropriate in the courtroom. Courts are not about what MIGHT happen (or have happened); they are about what DID (or did not) happen. Probability can be an excellent guide for further investigation into these things, but it should be left to the investigators. If they can't find anything harder than a set of odds, then judges have no place betting on them.

<comment><USER11> > If they can't find anything harder than a set of odds, then judges have no > place betting on them.

But this is how our entire legal system is set up. For criminal proceedings, we have to find the defendant 'guilty beyond reasonable doubt'. What is 'reasonable doubt' if not a (theoretically) quantifiable or parameterizeable representation of our belief that the defendant committed the criminal act?

We don't _explicitly_ quantify this threshold as, eg. a 95% chance, but that's still the exact same process we require a jury to undergo, even if we don't attach quantifiable numbers to the results.

The jury can never return a probability of 1 (in statistical terms, 'almost certain'), or else the entire appeal system wouldn't exist.

<comment><USER12> I believe that the purpose of randomize juries could be said to exist because while there are parameters, no one person or entity knows all the parameters.

</comment></comment></comment><comment><USER13> this seems like a complete over simplification and naive analysis of the situation - and the argument smells like a straw man and appeal to authority, comparing it to sherlock holmes - who incidentally /is obviously wrong/. the idea that 'whatever remains' is quantifiable and finite in the real world is a stark contrast with reality - and actually the fact that a probability is precisely 0 or 1 is very significant to its power in drawing conclusions. so not only does forbidding probability not forbid sherlock holmes style 'logic' but his logic is broken anyway in the vast majority of real world cases.

what the court says is common sense... its a shame it offends some academic view... oh, no, wait, it really isn't.

<comment><USER14> > _this seems like a complete over simplification and naive analysis of the > situation_

The key phrase in the linked article:

> _and so I must now tell them that the entire philosophy behind their course > has been declared illegal in the Court of Appeal. I hope they don't mind._

Is what reveals the misunderstanding. Judges don't "declare" things illegal, they rule on matters of _what_ the law is. That the law _requires_ judgements to be rendered in certain terms is not a statement of the legality of making Bayesian arguments.

The law requires a _decision_. The conceptual basis of the common law is a _promise_ that a court will _always_ render a clear and specific set of rulings or orders, and that there will _always_ be an explanation of those rulings or orders. Judges are not free to say to disputants that they are X% likely to win their claim.

This is one of those places where Bayesian probability isn't a good fit. Fuzzy logic -- so passé these days -- at least has an understood mechanism for "defuzzifying" in a fashion which lawyers would find quite familiar.

It also overlooks that judges have many other places to introduce flexibility and weighed judgement. For example, judges may assign blame in portions for some crimes or torts; they may reject, moderate or modify some claims in equity; they have leeway to combine multiple considerations and legislative constraints in handing down criminal sentences and so on.

One thing that bugs me _tremendously_ about outsiders looking in at law is the assumption that, since lawyers don't immediately and entirely embrace new idea X, they are fusty old fools who are an impediment to the good. It's an argument born of ignorance that lawyers are deliberately obtuse fools, or judges out-of-touch theoreticians. Lawyers and judges touch on more problem domains in more depth, with greater consequences, than pretty much every academic and every software developer.

<comment><USER3> >> and so I must now tell them that the entire philosophy behind their course has been declared illegal in the Court of Appeal. I hope they don't mind.

>Is what reveals the misunderstanding. Judges don't "declare" things illegal, they rule on matters of what the law is. That the law requires judgements to be rendered in certain terms is not a statement of the legality of making Bayesian arguments.

The write seems to be assuming that the court case sets a president that make it hard if not impossible to use Bayesian reasoning in the court room and sums it up as "illegal".

> The law requires a decision. The conceptual basis of the common law is a > promise that a court will always render a clear and specific set of rulings > or orders, and that there will always be an explanation of those rulings or > orders.

I thought this was a civil case and did not require "beyond a shadow of doubt." but rather just high probability a term I have heard is that it requires a preponderance of evidence. Bayesian statistics/reasoning is supposed to help come to a conclusion with a high likely hood of being correct which seems like the file state I want a judge to be in when ruling. What is not good or counter productive about this state brought about by Bayesian methods?

<comment><USER14> The problem is that the law _requires_ a statement of cause in order to apportion _responsibility_. It cannot abide uncertainty at the moment of judgement. It is accepted that, because in civil cases, the consequences of misjudgement are less severe, it is acceptable to relax the standard of judgement to "balance of probabilities".

Note the case that was mentioned inline that this case was affirming: _Hotson v East Berkshire Area Health Authority_ [1]. First of all, the linked poster should be railing at _that_ decision, this one merely takes it as precedent.

In _Hotson_ there was a question about causality based on the given probability of a child's recovery. Given that a child that fell out of a tree was estimated to be 25% likely to recover, medical staff seemed to have taken the view that the child was basically a hopeless case. Perhaps if they hadn't, went the reasoning, the child might have done better because the staff would have tried harder. In that were so, some responsibility could be apportioned to the Health Authority, and not just to the child falling out of a tree.

The Lords ruled that the only thing that _certainly happened_ was the child falling out of the tree. Counterfactuals based on probabilities couldn't be admitted because there's no way to reliably nail the damn things down. Anybody can come along with a different Bayesian network and give you a different estimate. What _mattered_ , in the view of the Lords, was _what could be verifiably said to have actually happened_.

A more-than-standard disclaimer:

I am not a lawyer. I was terrible at torts. Torts law is notorious tricky and can vary very widely from country to country -- I studied in Australia and these cases are in England. This post does not constitute legal advice. Hell, it doesn't even make for good reading.

<comment><USER3> Interesting, thanks for the pointe

> What mattered, in the view of the Lords, was what could be verifiably said > to have actually happened.

It would be interesting to see how "verifiably" and "actually happened" would be refined to produce a judgement. I could imagine at least refinement towards frequentist statistics, Bayesian methods, "I know it when I see it"(gut feeling, other non-quantitative standard of obviousness).

<comment><USER14> I would say that main things to remember about law in common law jurisdictions are:

1\. The judge does not seek evidence, does not provide evidence, and does not provide arguments. The presentation of fact and law is the job of the plaintiff/respondent or prosecutor/defendant (depending on civil or criminal law).

2\. The judge rules on what is introduced during court cases. Judges almost always follow judgements that have been made previously -- precedent, also called _stare decisis_ or "let the decision stand". In this case the precedent identified was _Hotson_. A lower court judge who presumes to modify a higher court's principles can expect that the higher court will accept an appeal on matters of law.

3\. The legal system's purpose is to render certain, verifiable judgements. Judgements can't be uncertain ("You are sentenced to a 95% chance of prison" or "between 20,000 and 50,000 is awarded to the plaintiff for damages" is not helpful), so as a step in the process of legal reasoning, information about uncertainty is deliberately destroyed. This is called "balance of evidence" or "preponderance of evidence" in civil trials, or "beyond reasonable doubt" in criminal trials. It is deliberately imprecise, because all attempts to introduce mechanical rules for evidentiary judgement have so far proved to be even more gameable than the fuzzy statements.

The legal system is profoundly introspective. Lawyers will introduce matters of legal interpretation in argument, these must be addressed. Appeals to higher courts are usually only possible on questions of law, meaning that very fine details of legal reasoning are constantly being teased out.

Judges must constantly weigh what the law _is_ , and they also at higher levels frequently decide _how_ the law is determined to be what it is. Given that subjects like jurisprudence and interpretation of laws can have massive practical consequences, it's an area that receives great scrutiny.

I became interested in the methods of analysis and judgement because of Australian constitutional law, where there are -- depending how you count -- between 4 and 6 basic schools of thought on how to interpret the Australian Constitution.

</comment></comment></comment></comment></comment></comment><comment><USER15> _"I teach the Bayesian approach to post-graduate students attending my 'Applied Bayesian Statistics' course at Cambridge, and so I must now tell them that the entire philosophy behind their course has been declared illegal in the Court of Appeal. I hope they don't mind."_

Classic. I was told (at same uni) that, in a case involving DNA evidence, I could ask to be dismissed from a Jury, citing training in Bayesian statistics (which was prohibited because reversing the scientific certainty to look at false positives, in cases with weak circumstantial evidence, essentially kills DNA evidence's usefulness. If someone was in the right city, you expect hundreds of DNA matches out of millions of people, but if you can track them to the right street at the right time, you would expect 0.001 matches, and thus the evidence points to them much more strongly.)

<comment><USER14> Everything you hear about law that was not conveyed to you by a lawyer, a law lecturer or a judge is probably horseshit.

People _love_ to think they know some obscure wrinkle, some cool loophole, some nifty curio about the law. But so many of the stories you hear are _just stories_.

ps. a better way to get disqualified is to have studied law.

<comment><USER15> indeed, this was my point. I didn't even contemplate trying it when I was called for jury duty.

</comment></comment></comment><comment><USER16> See also: R v T (Footwear Mark Evidence) <http://www.bailii.org/ew/cases/EWCA/Crim/2010/2439.html> (which is redacted to hide the appellant's identity, giving the judgment a delightfully dystopian feel).

There it was held that an expert witness shouldn't use Bayesian reasoning to calculate probabilities to tell a jury " _outside the field of DNA (and possibly other areas where there is a firm statistical base)_ ".

I _think_ that, by a 'firm statistical base', the court's getting at the sort of situations where the right prior is a widely agreed on (such as DNA, where the size of the DNA database is known), so there won't be much opportunity for different expert witnesses to disagree on probabilities due to having different priors. (N.B. IANAL)

This is still nonsense, IMHO. I can understand the court not wanting juries to be overly swayed by a spuriously precise probability that might have been very different if a different expert had been chosen. But there's no justification for restricting the statistical methods that the expert uses to reach their conclusion, however it's expressed to the jury.

Interestingly, that case shared one of the appeal judges from the case in TFA (Lord Justice Beatson, then Mr Justice Beatson).

<comment><USER17> I think the distinction being made here is the difference from using statistical methods for determining overall guilt versus using statistical methods for determining specific facts about the case.

We use statistics to say that someone was or was not at the scene of a crime (DNA testing doesn't 100% guarantee that a persons' blood is actually a match), for example. We don't make the leap that since they were there, "odds are" they did it, however.

<comment><USER16> The question in R v T that Bayes theorem was being applied to _was_ a specific, factual one: whether a footprint could have been made by a particular shoe.

</comment></comment></comment><comment><USER4> > * In upholding the first instance decision, the Court of Appeal reiterated > the principle in cases where there are competing explanations for a > particular loss that causation cannot be established only by a process of > elimination such that the 'least unlikely' cause of a loss is identified. A > claimant must demonstrate that the particular version of events that they > rely upon is more likely to have happened than not, in order for the civil > burden of proof to be satisfied.*

I'm not sure what the problem is.

"It could have been A, B, or C. It's really unlikely to have been A or B, and thus it must be C" is obviously flawed, because for all we know it could have been D, and even if it was C you need to show (on the balance of probabilities) that C is the cause. Not just that C is more likely than A or B.

<comment><USER14> It's not the judge's role to independently nominate D. It's up to the lawyers for the plaintiff and the respondent to present facts and make legal arguments.

The judge's role is to weigh those legal arguments, and in (most) civil trials to weigh the facts presented on the balance of probability, and to render a decision.

If you require judges to run a Bayesian network over the entire universe for each case, the legal system will get a wee bit slower.

<comment><USER6> _It's not the judge's role to independently nominate D._

But that is exactly why the judge can not assume all possibilities were enumerated, and requires positive evidence for C, rather than negative evidence for A and B.

<comment><USER14> Judges, oddly enough, decide cases according to the rules of legal reasoning. They cannot and won't introduce their own evidence. A judge that did so would be rightly removed from the bench.

<comment><USER6> So you agree then? I did not say a judge should propose any evidence.

Perhaps it's easiest to explain with an example. Let's say the defence argues for A, and prosecution argues for B. The prosecution shows that the probability of A being true is 1%. The judge can _not_ take that as a B having a probability of 99%, since it may be that C is true.

Thus the judge requires prosecution to show that B is 50%+ likely, not that A is unlikely.

Does that make more sense?

</comment></comment></comment></comment></comment><comment><USER18> This guy extrapolates wildly from the decision.

This is a question about whether or not a cigarette bud started a fire, not a rejection of bayesian probability as such.

This guy should stick to teaching statistics, not law.

<comment><USER19> Did you bother reading the post at all? The inference in the ruling is based on the rejection of Bayesian statistics.

<comment><USER18> Did you bother reading the ruling: <http://www.bailii.org/ew/cases/EWCA/Civ/2013/15.html>

The ruling is not concerned with statistics as a mathematical discipline. The "balance of probabilities" is a legal term.

Please read the court ruling, and then read the blogger's conclusion (summed up sarcastically in his last sentence) and tell me if you think he's right :-)

<comment><USER20> 'The "balance of probabilities" is a legal term.'

So is "reasonable doubt", but I am yet to see a perfect definition of it. Indeed, in the UK a jury questioned its meaning, the judge got snottily arrogant over it and ditched the case, but still not proper definition, either from the critical judge or anywhere else in the legal system. Interesting to me that the judge was so unable to answer the question, that he ditched a whole trial.

I am amused by legal arrogance that thinks every one else should fully understand all its weirdness and definitions. Especially having done some work in a solicitors office where I was the arrogant one getting irritated with most of the legal people not having even a basic understand of the computers on their desks...

<comment><USER16> > in the UK a jury questioned its meaning, the judge got snottily arrogant > over it and ditched the case ... Interesting to me that the judge was so > unable to answer the question, that he ditched a whole trial.

If you're talking about Huhne & Pryce, that's just wrong. The jury was discharged because after 2 days of deliberations they told the judge that they couldn't come to a verdict (which 10 of them agreed on). <http://en.wikipedia.org/wiki/Hung_jury> .

I don't know where you got the idea that the judge got defensive because he didn't know how to answer what reasonable doubt meant, but it's utter nonsense. It's a very common question for juries to ask. The appeal courts have debated several times what the best guidance is. The judge was following the current guidance, which is tell the jury to treat it as ordinary english words and refuse to give any more specific definition. If each judge gave their own favourite interpretation of what 'beyond reasonable doubt' means (e.g. p(false positive) < 5%, or whatever), then the standard of proof you'd be tried under would depend on the judge, which would be obviously unacceptable.

</comment></comment></comment></comment></comment><comment><USER21> If they make Bayesian analysis illegal only criminals will use Bayesian analysis.

<comment><USER22> If they make Bayesian analysis illegal, they've turned those using it into criminals (by definition). Still a great comment!

</comment></comment><comment><USER14> Please amend the title.

It is not correct.

The Court of Appeals has not "banned" Bayesian probability.

It does not have the power to do so.

</comment><comment><USER23> The problem probably lies in the fact that you cannot prove something by reduction to the absurd based on a probability distribution.

Because law requires positive proof (or it should) not just "this chain of events is so unlikely under other assumptions that our assumptions must be right". This is where Sherlock's quotation is misleading: he says "rule out the IMPOSSIBLE", not the improbable.

The judge is right epistemologically. Bayesian statistics has only a terminological issue.

</comment><comment><USER24> The whole basis of law is elimination of "epistemic uncertainty" and threshold of evidence for an event. If this is not possible, the case is not proved, therefore case dimissed.

Sherlock Holme's statement is a statement of method, not statement of proof.

In other words, the law is far greater than Bayesian probability or any method, including scientific, since the burden of proof is from the evidence with the primary legal threshold of _"innocent until proven guilty"_.

<comment><USER25> How many times in this comment page do people need to be reminded that the burden of proof in civil cases is only 51%, not "beyond reasonable doubt" like criminal cases.

</comment></comment><comment><USER26> Poker players understand that a lack of knowledge of past events is exactly the same as uncertainty about future events.

For example, if the first round of hold em has dealt two cards to each of 9 players, there are 18 cards out and you only know the state of two of them. The odds that the flop will contain specific cards that help your hand are calculated against the total number of unknown cards, regardless if the unknowns are held by other players or still in the deck.

For example, you get a pair of 3s. There are two more threes in the deck, and it is pretty likely that if the flop contains one of them that you will have the best hand at the table at that point. For the first card turned up at the flop, there is a 2/50 chance it will be a three, the next is 2/49, the next 2/48. It doesn't matter how many face-down cards have been dealt to your opponents.

All the unknowns are still in the pool of possibilities, and it is no different when you assess the evidence you have of any other type of event that has already happen. Each bit of evidence means what means, and the unknowns contribute to the pool of uncertainty.

</comment><comment><USER27> Why is this linkbait at the top of HN?

</comment><comment><USER8> If anyone finds this subject interesting: my evidence professor had a PhD in statistics, and he's written about the application of probability to the law of evidence extensively. See: [http://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=5157...](http://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=51570)

</comment><comment><USER28> I not sure what the OP expects the court of appeal to do. They are tasked with determining if there is cause for appeal; in this specific case whether the original judge erred in law. They determined that he did not. They aren't saying whether the judgement was right or wrong, rather that the judgement was arrived at in a lawful manner.

</comment><comment><USER29> As a frequentist I fully support this endeavour.

</comment><comment><USER30> For anyone curious about how this would play out in the US, federal courts and most state courts use the Daubert standard. TLDR: experts (including statisticians) can testify if they're using fairly standard methods and there are no significant gaps in the evidence-->analysis-->testimony chain.

<http://en.wikipedia.org/wiki/Daubert_standard> [http://en.wikipedia.org/wiki/Daubert_v._Merrell_Dow_Pharmace...](http://en.wikipedia.org/wiki/Daubert_v._Merrell_Dow_Pharmaceuticals)

</comment><comment><USER31> Accidents, murders, fires, etc... are all unlikely events by nature. Judging whether one of them can have occurred based on probability does not prove anything. The author post is wrong is the assumption that "an unlikely event"="impossible". Probability does NOT ascertain certitude. There is always an expression of confidence which is not equal to 100%, and therefore it seems logical that a court does not take the probability of an event as a tangible proof of what did or did not happen. Probability != science.

</comment><comment><USER22> What's the chance that the legal system would suddenly become coherent if we replaced all lawyers (and therefore judges) with mathematicians?

<comment><USER14> The law is highly coherent.

But it deals with the largest and messiest possible problem domain: everything humans do.

<comment><USER32> Well, "everything humans do" includes creating the law, so by that definition, the law is highly incoherent.

And a close reading of the law reveals this. In fact, it is often by close readings of the law (and precedent and every other historical quirk) on which controversial and contested decisions are rendered

<comment><USER14> > _Well, "everything humans do" includes creating the law, so by that > definition, the law is highly incoherent._

Which meaning of incoherent are we taking here?

> _And a close reading of the law reveals this._

Do go on.

</comment></comment></comment></comment><comment><USER33> The opening Holmes quote is being used improperly. Holmes used that as opening, then he went on to find further evidence to support that conclusion. He did not rely on logic alone to determine the culprit, but rather went on to find hard evidence.

</comment><comment><USER34> It is well known that 70% of all statistics are wrong.

</comment><comment><USER35> Elizier Yodokowsky is going to be pissed.

</comment><comment><USER36> As a Bayesian rationalist, wut.</comment>


Onyx BOOX Release 6inch Color E Ink Reader Running Android 9: Poke2 Color - <AUTHOR> https://shop.boox.com/products/poke2color <comment><USER2> It's from April, but a demo: [https://www.youtube.com/watch?v=-q_DDXpzQgw](https://www.youtube.com/watch?v=-q_DDXpzQgw)

<comment><AUTHOR> Yep, but they didn't put it on the store shelves until August 10. It's sold out now.</comment></comment>


Does money make you mean? - m-i-l http://www.bbc.com/news/magazine-31761576 <comment><AUTHOR> I'm not wealthy, I have to work, but I'm far more affluent than anybody in my family has been. I've given this some thought and here's my belief:

You grow up and for whatever reason, your ambition is to accumulate wealth. You make sacrifices. You invest yourself. You trade years; your 20s, your 30s. You make a fortune.

Your bank account is full, but to your surprise you're still hungry. The money you've been chasing your whole life, it turns out, does not have magical powers to make you happy. The thing that was supposed to offer you fulfillment and peace didn't work. And that sucks. You deny it, for years probably, or forever. And you get angry about it. Lashing out. High taxes and entitlements are the problem, or pan handlers, welfare and obamacare, all the people with their hands in your pocket. And it all just pisses you off.

None of this is fact, of course, just my belief. I'm trying to learn from these lessons as best as I can. So far I give myself a C+ at best.

</comment><comment><USER2> It's so tempting to engage this conversation on its premise, it's such a popular idea to say there must be something bad about people who have more money, but this article is so unscientific and has so many problems it just hurts. It's like eating bacon and marshmallows; so satisfying to do, but so bad for your health.

The very headline is a push-poll, suggesting the answer before a question is asked. The first piece of evidence is trash data. The consequence for breaking traffic laws is a fine that hurts poor people more than rich people. There's no direct evidence presented between breaking traffic laws and being "mean". The interactive questionnaire asks a bunch of questions about whether you would do things that hurt unspecified mega-corporations. Monopoly money is fake money that's part of a game where the rules are specifically designed to pit opponents against each other so they'll do mean things, it's impossible not to play monopoly meanly.

What does this all mean?

NOTHING.

</comment><comment><USER3> I've always been suspicious of these types of broad psychology studies. They always seem to make claims that go beyond what their data says.

Laboratory experiments where the people know it's an experiment don't seem like they should be able to say things like "rich people act meaner." There's just so many variables that could be different--for example rich people might be more competitive and a lab environment makes them feel more like they're playing a game than dealing with real wealth and well being.

For the car experiment, he's not really looking at rich people, he's looking at people who choose to buy an expensive looking car and live in the Bay Area. Notably, there are a number of people who live in the bay area, are quite wealthy, and drive a car that doesn't show off their wealth. He also says "hundreds" in the video when his paper[1] says 152, notably with n=5 for those with a car rated 1. The methodology also states that they didn't actually record the car type, they just guessed based the cost at the time based on what they saw.

They claim the observer was blind to the study's hypothesis, but the task was to record on a scale of 1-5 how expensive he or she thought the car was and whether they stopped for a pedestrian. It doesn't take half a brain to figure out what the hypothesis they were testing was.

[1] [http://www.pnas.org/content/109/11/4086.full.pdf+html](http://www.pnas.org/content/109/11/4086.full.pdf+html)

</comment><comment><USER4> Okay, so rich people act meaner.

But does that mean money makes you mean?

Maybe mean people are more likely to get rich.

Or maybe mean tendencies are evenly distributed, but rich people are more likely to think (quite rationally, I imagine) that they can _get away with_ being mean, so they tend to act more meanly.

<comment><USER5> Among the poor, there are compensating factors against the rich man's "I can get away with it", such as perhaps:

\- "I have nothing to lose"

\- "I'm too stupid/drunk/irresponsible to consider the consequences".

<comment><USER6> If only the poor stopped being so stupid and irresponsible, maybe they could pull themselves up by the bootstraps.

<comment><USER7> I know you jest, but there is a subset of the poor that absolutely could pull themselves out of it if they stopped being stupid and irresponsible. Certainly not a large subset, but a subset nonetheless.

<comment><USER8> From the small cross-section of lower income people I personally knew while working for a factory, there were several people held back by their record, several held back by various substance abuse issues (including some issues which were ongoing and others that led to direct problems at work/getting fired), and at least one person born to a well-off family who was completely unable to support himself due to various anger/substance issues.

The longer-term employees (i.e. the most responsible of the bunch, due to selection effects) were, in general, able to move up a little. Their kids have some hope of ending up with much better jobs than their parents. They really were moving on up and I did what I could to help those I could help that happen.

The less responsible folks were drinking away their paychecks for the most part. Several of them migrated to other, similar jobs after weeks or months after failing to show up to work after getting paid a few times. This is a common enough occurrence that people have little doubt about what a no-call, no-show means in those circumstances on average and they do not permit such things to become regular.

I'm hopeful that better access to internet education will allow the more industrious among those who have been trapped by their circumstances to move beyond them. But education is part of the problem for most of them. In spite of having completed high school, for the most part, a lot of them were simply unaware of all the great things on the internet these days that could help them.

But... that's true for a lot of people, not just the working poor. I mean, just the other day, I was telling a lifestyle business owner of all the simple things he could do to expand his business online (most of which I learned from reading patio11). It's surprising what teaching someone a few searches might enable.

</comment></comment></comment></comment></comment><comment><USER5> Hmm, where do more emergency calls related to domestic violence originate? Rich neighborhoods or poor ones?

(This is not rhetorical: does someone have the facts? It's easy to have an obvious stereotype about this.)

<comment><USER9> Does your definition of domestic violence include spanking? If not, why not? Is spanking not violence that happens in the home? Is spanking not violence against those who have even less choice than their parents of moving away?

If you do include spanking in your definition of domestic violence, then I think you won't find the numbers you're looking for.

<comment><USER5> The only 100% relevant statistic that gives us comparative meanness would be ... some sort of measure of meanness itself!

Since we don't have that, numbers like reported violence can serve as indirect estimators.

If spanking were reported, then it could certainly be used as an estimator of meanness. Spanking is aggression, but less violent than giving someone a black eye, obviously. Depends on the spanking too. Back of the hand versus palm? Head versus buttock? ...

<comment><USER9> Are you suggesting women are getting black eyes as much as children are being spanked? Is getting spanked 936 times a year as a 4 year old less violent than getting a black eye as a grown woman who has access to friends, cars, the police, weapons, means of negotiation, and so on?

Indeed the numbers we have are only indirect estimators. That was my point. Who cares what the numbers are if we are not going to do anything to protect the defenseless from being beat up daily and by the millions.

I won't even address your disrespectful disdain of this issue by suggesting it matters what part of the hand it is used to hit a defenseless human being, or where they are hit, but I will compare it to asking if the rapist used a condom, and which orifices were involved.

<comment><USER5> > _Are you suggesting women are getting black eyes_

/me rereads what /me wrote.

I don't see anything even remotely resembling such a claim anywhere, sorry.

</comment></comment></comment></comment></comment><comment> j-gibweb I'm way nicer now that I'm not broke any more

</comment><comment><USER10> Stupid article. Example, referring to the dictator game: "Rational economics would say the poorer person should keep more for themselves, the richer person should give more away." No, rational economics says the dictator should keep it all. If he really wants to give some of it away, it's better to give it to a person of his choosing after the experiment is over rather than giving it to an unknown person in the course of the experiment.

</comment><comment><USER11> Upon reading the article, I recalled seeing a Ted talk from Paul Piff himself about his study. So here it is, dug up from history: [http://go.ted.com/5D7](http://go.ted.com/5D7) [2013]

</comment><comment><USER12> Yes.</comment>


How Facebook Changed the Spy Game - <AUTHOR> http://www.politico.com/magazine/story/2017/09/08/how-facebook-changed-the-spy-game-215587 <comment><USER2> A real solution to this, which may seem indirect to many, is more aggressive antitrust enforcement. These internet giants have become the new (global) public square at the expense of all the old world institutions for civic discourse and if that were more fractured I suspect things would be better. Certainly newspapers, magazines and other web publishers would have more bargaining power (and they wouldn't have to ask for their own exemptions from antitrust law [1]).

I'm becoming more and more convinced that lax antitrust enforcement since the 1980s is the source of many problems in America today: income inequality, corporate profits at all time highs, capital expenditure seemingly no longer being necessary in the economy to obtain those profits, lack of inflation particularly in wages, and the "de-equitization" of America [2].

Concentration is a problem not just in pure economic terms in terms of higher prices or less innovation but also as a threat to liberal democracy itself (as I've gotten older I've started to take a more political economy approach to things).

[1] [https://www.cnbc.com/2017/07/10/newspapers-bid-for- antitrust...](https://www.cnbc.com/2017/07/10/newspapers-bid-for-antitrust- exemption-to-tackle-google-and-facebook.html)

[2] [https://www.wsj.com/articles/americas-roster-of-public- compa...](https://www.wsj.com/articles/americas-roster-of-public-companies-is- shrinking-before-our-eyes-1483545879)

<comment><USER3> Reading into your post, I think you're implying that the presence of a single monolithic target in the Federal Union of Heresland for The Republic of Adversaria to mess with represents a greater risk for carrying out the business of democracy, and that a multitude of smaller social networks would be harder to compromise.

I'm not sure how true this is though - inevitably those smaller organisations have lesser resources and capacity to detect and resist such attacks, and would in any case quite quickly balkanise themselves into clear demographics, leaving it obvious to the adversary exactly which of these smaller, more vulnerable organisations to target.

The solution in my mind is for these internet giants, regrettably, to get bigger - to become /more/ global, and to be present everywhere. Fragmentation of world society will not bring us progress - consolidation will.

<comment><USER4> If I understand your solution, it seems to suggest more global monopolistic size and behavior for the "Internet Giants".

Do you mean to define "Internet Giants" as the social media platforms or the infrastructure carrying these platforms and traffic?

Separating the social platform from the physical transport platform may change how enforcement is carried out depending on who owns which part, i.e. physical infrastructure being publicly owned vs private etc.

Without that type of change and distinction, I'm not sure how your solution is a solution without better understanding it.

<comment><USER3>
      Do you mean to define "Internet Giants" as the social media platforms or the infrastructure carrying these platforms and traffic?

Well, both. But the distinction is getting less clear as time goes on. And for Joe Public, the distinction doesn't even exist, of course.

</comment></comment></comment></comment><comment><USER5> I feel like the article boils down to "the Russians have a big advantage influencing the US because we allow free speech and they don't" with a strong implication that we need to give law enforcement more tools to crack down on free speech.

I vehemently reject any such notion. In the long term, the best counter to speech is more speech.

<comment><USER6> The article gave an example of FARA. Do you think FARA cracks down on free speech?

<comment><USER5> Depending on how it is used, my answer is "possibly".

Put yourself back in the days of Joseph McCarthy and the "red scare", but with today's laws (and hopefully we seek an outcome that is MORE friendly to free speech, not less). How would you feel about a law that required anyone supporting communism to register as a foreign agent? Or more precisely, one that allowed the government to arrest anyone who argued in favor of communism (a right protected by free speech) and force them to prove they do not have any ties to any foreign government at risk of imprisonment for 2 years.

I have little problem asking those actually employed by a foreign government to register. But if you stretch the definitions far enough ("you were eligible for a tax deduction, which counts as compensation so you are employed") then it can be abused.

</comment></comment></comment><comment><USER6> I didn't know about FARA(Foreign Agent Registration Act), "that requires any individual acting on behalf of foreign interests to register as a foreign agent with the State Department", also "passed in 1938 to counter Germany's dissemination of Nazi propaganda in the United States". That's genius.

The article is calling for FARA update to the age of social media.

</comment><comment><USER7> The linked piece about "The Agency" on the new york times also is a very interesting read.

\- [https://www.nytimes.com/2015/06/07/magazine/the- agency.html?...](https://www.nytimes.com/2015/06/07/magazine/the- agency.html?mcubz=3)

</comment><comment><USER8> > Any solution that we create will require a balance between national security > interests and constitutional rights

As I, as a European citizen, perfectly understand that FBI mission deals with internal US, and so the OP option is an internal point of view, wouldn't it be also wise to deal with it on an international way ?

Of course, with Putin and Trump, two strong virile personalities, dialog might seem hard, even impossible.

But any way, instead of trying to deal with it internally, "forcing" (and I honestly have no idea how it should be possible) American social media companies to be more transparent could be a breath of fresh air in this situation ?

Candid question.

<comment><USER2> That's already happening. Germany, for example, has enacted new legislation to force social media companies to take down hate speech posts or face fines [1].

[1] [https://www.nytimes.com/2017/06/30/business/germany- facebook...](https://www.nytimes.com/2017/06/30/business/germany-facebook- google-twitter.html?mcubz=0)</comment></comment>


Wrapped Bitcoin (WBTC) Is Now Live on Ethereum - <AUTHOR> https://blog.kyber.network/wbtc-is-now-live-on-ethereum-4b4e2d1ef76f <comment><USER2> The article talks about Dapps and decentralized finance, but WBTC is centralized and managed by a custodians.

Sure you could use it in your smart contract, but it has the same kind of risks as using Tether. I think it can be dangerous to build layers of decentralized apps on top of something that's not.

(Or maybe I'm not understanding it correctly)

<comment><USER3> It's technically possible to implement this in a trustless decentralized way: [https://en.bitcoin.it/wiki/Atomic_swap](https://en.bitcoin.it/wiki/Atomic_swap). I can't tell if this does it that way or in a centralized way though.

<comment><USER4> Atomic swaps don't make WBTC feasible in a trustless decentralized fashion. The swap itself is only a very small part of the problem. The article states they are using custodians to hold Bitcoins.

<comment><USER2> You can lock bitcoin up inside the BTC mainchain by using certain OP codes. This allows you to create a sidechain.

For example Liquid does this: [https://blockstream.com/liquid/](https://blockstream.com/liquid/)

<comment><USER4> Liquid holds btc using multisig not just OP codes.

</comment></comment></comment></comment></comment><comment><USER5> I don't understand why anyone would use this instead of the real BTC or ETH.

<comment><USER6> Because they can hold BTC for the purpose of using ethereum apps instead of selling BTC.

<comment><USER7> You don't actually hold BTC though, Bitgo does. It's more like a deposit certificate, if Bitgo were to lose the BTCs, all WBTCs would become worthless.

I much rather RSK's approach: bring easily programmable smart contracts to Bitcoin. Rather than bringing Bitcoin to smart contracts.

</comment></comment></comment><comment><USER8> It's difficult to keep up with this space...lots of innovation! But what's the adoption rate of these new tools? Dozens of users a week? Hundreds? I would find it hard to believe thousands of people will use WBTC anytime soon.

<comment><USER9> See comment by @askmike, this is not innovation, this is bullshit. And maybe even deceptive because they took the name "WBTC" to mimic "WETH", however, the latter is truly decentralized and trustless, while the former is not.</comment></comment>


Finding SARS-CoV-2 carriers by optimizing pooled testing at low prevalence [pdf] - <AUTHOR> https://www.medrxiv.org/content/10.1101/2020.05.02.20087924v1.full.pdf <comment><USER2> People seem to be missing the point of this article. Pooled testing is accepted and routine. The point of the article is that by constructing pools and subpools in a particular way you can dramatically speed up the process by exercising in parallel / up front all the possible testing paths that would normally have to occur in series so that the individual(s) who are positive are identified in only one round of testing - giving a crucial advantage in a time sensitive situation such as contact tracing for infectious diseases.

<comment><USER3> Yes, and it's already been done for covid. In may, China wanted to test all 11m people in Wuhan, so they used pooled testing:

[https://www.livescience.com/pooled-sampling-covid19-in- wuhan...](https://www.livescience.com/pooled-sampling-covid19-in-wuhan-and-us- cities.html)

<comment><AUTHOR> The algorithm presented in this paper is a significant improvement over what was done in wuhan. In a low infection population it reduces the cost of testing an individual to around $0.75 and can find infected individuals in a single step, without having to do subsequent assays to narrow down which samples tested positive.

<comment><USER4> Array testing is an improvement over Dorfman, but this has been widely known for decades. Blood banks regularly use high dimensional arrays for testing. Here they test the hypercube planes in serial instead of parallel, but this increases latency and decreases sensitivity.

</comment></comment></comment></comment><comment><USER4> I think to authors are overly optimistic in our ability detecting highly dilute samples. Pooling studies in Dengue Fever using RT-PCR found sensitivity fell faster than could be explained by dilution effects alone. At 100:1 or 20:1 pooling the false negative rate will be unacceptable.

Edit: false positive -> false negative

<comment><USER5> False negative rate maybe?

In general, titers of the virus in NP swabs should be high enough to withstand 10, maybe 20-plex pooling. However, as you allude to, there are plenty of other effects that can influence PCR. If there is one sample, for example, that contains a PCR inhibitor, like heme, it may then inhibit all the samples it is pooled with.

Additionally, swabs are not just swabs of virus, they also pull of highly variant amounts of bacteria and human material. These things may also affect the efficiency of PCR.

<comment><USER4> > False negative rate maybe?

Yes, thanks, fixed.

Check out Table 1 in this paper. It isn't very encouraging. [https://onlinelibrary.wiley.com/doi/full/10.1002/jmv.25971](https://onlinelibrary.wiley.com/doi/full/10.1002/jmv.25971)

</comment></comment></comment><comment><USER6> Aha! Someone must have read my comment on HN. [https://news.ycombinator.com/item?id=22616288](https://news.ycombinator.com/item?id=22616288). Where's my citation lol

</comment><comment><AUTHOR> I was listening to the bbc's science in action podcast[0] and came across an interview with the mathematician whose group had developed this.

By mapping individuals onto a hypercube they were able to reduce the number of tests needed to find unique positive individuals by a very large percentage relative to non pooled testing as long as the positive test rate was low in the population.

[0] [https://www.bbc.co.uk/sounds/play/w3cszh0k](https://www.bbc.co.uk/sounds/play/w3cszh0k)

<comment><USER7> A link for those who prefer reading: [https://www.nature.com/articles/d41586-020-02053-6](https://www.nature.com/articles/d41586-020-02053-6)

It seems like this method can be generalized to a detection problem. Anyone knows other applications of this technique?

<comment><USER8> [https://en.wikipedia.org/wiki/Group_testing#Example_applicat...](https://en.wikipedia.org/wiki/Group_testing#Example_applications)

<comment><USER9> Isn't this just an extension of one of those dumb brainteasers people sometimes ask in interview questions? (You have N rats and K vials, P of which contain poison. Can you determine which vials contain poison?)

</comment></comment></comment></comment><comment><USER10> Like so many clever puzzle answers, this doesn't take into account the effect of measurement error. The tests apparently have a false negative rate of 20% as it is. Now start mixing samples together, causing more dilution of the positive samples, and the false negative rate is bound to go up further. Now you have to do studies to figure out what the false negative rate is going to be, and put that into your model as well. This could get impractical fast.

<comment><USER11> I'm not a virologist, but from my understanding the false negative rate is caused when collecting swab samples and not related to the concentration during the PCR. There are also thresholds in the process (mainly the number of reaction cycles) that can be tuned to account for lower concentrations.

<comment><USER5> That is correct.</comment></comment></comment>


Ask HN: What kind of challenges do hardware startups (like Nest) face? - <AUTHOR>

I was reading the article posted today about how supposedly it's near impossible to have one's hardware solution accepted into a hospital. Folks mentioned that the challenge is indeed great for most hardware devices, not just the medical ones.
A few friends and I have been discussing a device + cloud hybrid solution, vaguely inspired by Nest. We were wondering how big of a hurdle a company has to overcome in order to get a physical device into someone's home.
We'd love to know what we're getting ourselves into before we spend too much time on that path.
Thank you! <comment><USER2> I learned a lot from the Twine kickstarter updates.

[http://www.kickstarter.com/projects/supermechanical/twine- li...](http://www.kickstarter.com/projects/supermechanical/twine-listen-to- your-world-talk-to-the-internet/posts)

I suggest reading through them.

</comment><comment><USER3> Bunnie has written some great posts about the adventures of hardware manufacturing: <http://www.bunniestudios.com/blog/?cat=7>

</comment><comment><USER4> You are going after a market that is not highly regulated like hospitals. Your biggest hurdle (without knowing specifics) would be convincing people that they need your stuff installed at their home. Not an easy task, unless you have good marketing.

</comment><comment><USER5> I don't know anything about hardware startups, but I'm curious to know if it's easier for them to recruit people than it is for software startups.</comment>


Why You Should Never Use Select * (Ever) - <AUTHOR> http://gavindraper.co.uk/2019/01/23/Why-You-Should-Never-Use-SELECT-Star-In-Production/ <comment><USER2> My braincycles are much more expensive than the DBs. It's fine to use select * if I do it.</comment>


Are we on the road to civilization collapse? - <AUTHOR> http://www.bbc.com/future/story/20190218-are-we-on-the-road-to-civilisation-collapse <comment><USER2> The question is how civilisation collapse must feel like from the inside and what it actually _means_?

Do you mean something like the end of the Roman Empire which in retrospect can be seen as a paradigm shift? Or do you mean we are going back to the stone age by e.g. destroying civilisation with nukes or similar (all of which could also happen any time with the right volcanic eruption or meteorite impact).

It is quite clear, that without education we are not gonna get more civilized than we are now. On top of that people realize one way or the other that they are ripped of by oligarchs, that masterfully played their power over the last decades to increase the share of money they drain out of the rest of the people. This won't go on forever and we definitly should think about the effects similar things hat last time when this was the case (pre World War era with it's rapid industrialization).</comment>


Inside Arizona's Pump Skimmer Scourge - <AUTHOR> https://krebsonsecurity.com/2016/09/inside-arizonas-pump-skimmer-scourge/# <comment><USER2> The first address I recognized is the Shell station just down the street from my Grandfather's house. It's not a very nice neighborhood anymore (a planned freeway just to the south was canceled back in the 1980's, and all the houses that had been taken by eminent domain were auctioned off. The new tenants didn't take care of their neighborhoods like the owners used to).

Some of the stations are in rather well-to-do areas. 4995 N Granite Reef Rd in Scottsdale is a Shell station too, and is pretty close to downtown Scottsdale. Lots of Circle K's - most of these stores are corporate, but a few are franchises.

I usually paid cash for my gas, and I haven't been skimmed. I brought up 'skimming' at my credit union the other day, when I deposited to savings rather than checking, and the teller said it's a problem for them in the Phoenix area.

Edit: Looking into it more, these aren't little Mom-And-Pop gas stations, they're mostly Circle K's (corporate) and franchises of big-name energy companies - lots of Shell and Valero franchises.

[https://en.wikipedia.org/wiki/Valero_Energy](https://en.wikipedia.org/wiki/Valero_Energy) \- for example: [https://www.google.com/maps/place/1925+N+Scottsdale+Rd,+Temp...](https://www.google.com/maps/place/1925+N+Scottsdale+Rd,+Tempe,+firstname.lastname@example.com,-111.9282363,17z/data=!3m1!4b1!4m5!3m4!1s0x872b096adbaace2b:0x857f89f063f036fc!8m2!3d33.4507166!4d-111.9260476)

[https://en.wikipedia.org/wiki/Shell_Oil_Company](https://en.wikipedia.org/wiki/Shell_Oil_Company) \- for example: [https://www.google.com/maps/place/4995+N+Granite+Reef+Rd,+Sc...](https://www.google.com/maps/place/4995+N+Granite+Reef+Rd,+Scottsdale,+firstname.lastname@example.com,-111.9007588,19z/data=!4m13!1m7!3m6!1s0x872b0a5213ae16ad:0x6149ce885316ee98!2s4995+N+Granite+Reef+Rd,+Scottsdale,+AZ+85251!3b1!8m2!3d33.5089587!4d-111.9000426!3m4!1s0x872b0a5213ae16ad:0x6149ce885316ee98!8m2!3d33.5089587!4d-111.9000426)

Edit 2: Mile Marker 27 (Last Stop) is a Texaco: [https://email@example.com,-114.51...](https://email@example.com,-114.5191791,11z/data=!4m8!1m2!2m1!1sfuel!3m4!1s0x0:0xdd2ce9b0d195bd2d!8m2!3d35.6843504!4d-114.4672394)

<comment><USER3>
      ... a planned freeway just to the south was canceled back
      in the 1980's, and all the houses that had been taken by
      eminent domain were auctioned off. The new tenants
      didn't take care of their neighborhoods like the owners
      used to.

This blows my mind. So the local government forced people out of their homes, then said, _" Nevermind!"_?

<comment><USER2> The freeway was also going to take out a private high school, which has many influential members of the community as alumni. It was not even a necessary freeway - parallel to I-10, but only a few miles to the north.

I might be wrong about the details (I was a kid at the time), but this page confirms that a freeway in this general area was canceled 'due to neighborhood opposition': [https://www.arizonaroads.com/arizona/az50.html](https://www.arizonaroads.com/arizona/az50.html)

</comment></comment></comment><comment><USER4> Family friend in CT was just telling me about this. He works for the Dept. of Weights and Transfers. Said they usually make sure that pumps give you exactly the amount you pay for, but lately they've been working with the Secret Service and finding even the insides of pump machines compromised.

Recommends preferring paying in cash, then credit, lastly debit and if you do so, move your money out of that account and into a savings account.

</comment><comment><USER5> This is a mostly solved problem in the most of the world. (but only mostly).

lock the cabinet, have cameras, have a non-flat shaped card reader slot (preferably with a picture of what it should look like stuck next to the reader), have a sealed emv/pci compliant reader/pin pad component, schedule regular checks of the equipment, remote alarm the pump enclosure.

and finally have credit card companies or head office tear shit out of the site if they don't have these.

<comment><USER6> Unless the legislature forces this, I doubt it will happen. The banks and credit cards have turned these exploits into a problem of the individual instead of a problem of the institution (which has shit security). So you and I get stuck with the fraud and they get paid, typically.

<comment><USER3> I've been the victim of a couple forms of fraud (fraudulent charges in TX, Macy's CC opened in my wife's name) and the CC companies bent over backwards to accommodate us. No _hesitation_ that the charges would be reversed, holds placed on the account, and new cards sent.

That and fraud detection is getting better. I went home to visit my parents (2 hours away), got gas, then exchanged USD for €300. That triggered their algorithms (as well it should have), and I got a text about 10s after I swiped my card.

<comment><USER6> That's interesting that my mom has had the exact opposite experience with getting their identity stolen and fraudulent cards opened in their name.

</comment></comment></comment></comment><comment><USER7> Why don't the operators add a daily check for skimmers while they do their daily pump checks? And with video so affordable now, even if it's only on a 7-day retention, they can turn over evidence to authorities to catch the perps?

<comment><USER8> It would help, but a lot of places just don't do it. They said thieves target places that neglect security, which there are plenty of.

<comment><USER7> Maybe GasBuddy or the like can begin identifying stations which have experienced compromises (and are complacent) as well as ones which maintain some operational security or at least actively seek to fight the skimmers.

I would certainly give preference to stations with better scores, in that regard, even if there were a modest price premium.

</comment></comment></comment><comment><USER9> Cash is king now. But just enough some wayward cop doesn't confiscate it because it's all shifty and suspicious looking.

</comment><comment><USER10> Will skimmers be rendered obsolete by chip cards? It seems like this will all be over when the last magnetic strip reader is shut down. Although I've never seen a chip reader at a gas station, so it may be a while.

<comment><USER11> It is impossible (yet) to hack chip & pin. The problem is that even chip cards have a magstripe that can be skimmed. EMV[1] enabled fallback funtionality by default, which is the biggest issue imho. Basically, if your chip is broken, a terminal goes through fallback mechanism and uses magstripe instead. This way you can clone a card with "broken" chip and copied magstripe. Some banks allow to disable (opt-out) magstripe for chip cards, so unless you are in US, you should do that. I've seen some people intentionally scratched magstripe, but I'm not sure it's a very good idea.

1\. [https://en.wikipedia.org/wiki/EMV](https://en.wikipedia.org/wiki/EMV)

<comment><USER12> Or, you know, take your fingernail and scratch the shit out of the stripe. Or play with some strong magnets. Or sand paper. Or... you get the point

<comment><USER11> Sure you could. But I'd like to have a backup options in case I end up in strange place that doesn't accept chip&pin or my chip is really broken while on travel. It could be opt-in, so in case I need magnet, I could call my bank and ask to enable it.

</comment></comment></comment></comment><comment><USER13> There have been incidents where in-store attendants have been caught skimming cards with handheld skimmers I would be surprised if done of these gas station attendants are not in league with the criminals or even install some of these devices on their behalf.

</comment><comment><USER14> Now I want my car to be able to detect local Bluetooth signals and warn me about skimmers.

<comment><USER15> The thieves will then switch to Wi-Fi, or Zigbee, or whatever.

</comment></comment><comment><USER16> Oh look, Krebs rides again ! :)

<comment><USER16> I wasn't sarcastic</comment></comment>


To Pay Attention, the Brain Uses Filters, Not a Spotlight - <AUTHOR> https://www.quantamagazine.org/to-pay-attention-the-brain-uses-filters-not-a-spotlight-20190924/ <comment><USER2> "If the doors of perception were cleansed every thing would appear to man as it is: Infinite. For man has closed himself up, till he sees all things thro' narrow chinks of his cavern."

Aldous Huxley had a similar intuition in his essay, _The Doors of Perception_ : the body perceives everything, and environment (including culture, traditions etc.) teaches the man to close some doors lest he be overwhelmed. Mescaline and other entheogens are a way to open most of the doors. Artists (such as Van Gogh) are people with a different set of doors open/closed.

<comment><USER3> If we're posting relevant quotes, Lovecraft is a favourite: "The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents. We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far."

<comment><USER4> The sciences, each straining in its own direction, have hitherto harmed us little; but some day the piecing together of dissociated knowledge will open up such terrifying vistas of reality, and of our frightful position therein, that we shall either go mad from the revelation or flee from the deadly light into the peace and safety of a new dark age.

^^ this part is what piques me because we are finally combining our knowledge from disparate fields and the indicators are truly that it's much more insidious than anyone can possibly imagine so the red pill blue pill moment is upon us as a humanity

<comment><USER5> > combining our knowledge from disparate fields and the indicators are truly > that it's much more insidious than anyone can possibly imagine

What are you specifically referring to here? QM?

<comment><USER6> The Internet and the www?

It's given humanity somewhat of a global consciousness, but not everything that comes out of that is benign or benevolent.

<comment><USER4> It's turning into the abyss that gazes back at us --

</comment></comment></comment></comment></comment></comment><comment><USER7> For what (little) it is worth, for a layman such as me it sounds like it make a lot of sense in hindsight.

As the common saying "Tuning out the noise" indicates, focusing on something feels more like dropping irrelevant info in the background than about magnifying the interesting bits.

Also, it seems to make sense to expect important functions for survival to be found in the "ancient" parts of the brain. And the attention capacity seems like a rather important function for survival.

On another topic, I wonder how this finding could serve as an inspiration in regard to the current trend of attention mechanisms in Artificial Neural Networks. The analogy with transformers is obvious, but the part about "blinking spotlights" made me wonder if Nature isn't hiding a nifty trick here.

</comment><comment><USER8> Which is why some psychedelic drugs are too much for people because it disables the filters for a period of time. You weren't aware of the pressure of your socks on your feet until reading sentence.

<comment><USER9> Especially psilocybin. "Hero" or "ego-death" doses are fascinating. Filters/barriers break down. Each room in my house had a special vibration/energy based on how I spent my time there. Artwork had deep meanings not previously noticed. Projected families and friends onto chairs in my empty dining room and had conversations with them (with cussing & sobbing). Started to think about my ancestors and had "past life" schizophrenic experiences based on memories and impressions of history. Felt powerful daemonic energy, radiating from body out of my hands, and felt amazed by the incredible ability humans have to create & destroy.

THC/cannabis is too much of a recreational drug... though it's useful for taking the edge off at the end of a psilocybin trip.

<comment><USER10> Not to take anything away from your experience, but I've noticed that in reading reports from various people taking larger trips, I get this impression that a big part of the experience is indistinguishable from schizophrenia. What I mean is nobody can properly define their terms in any commonly shared way, people believe and see things they can't possibly share in a relatable way. Not everything of course, but your feeling of your hands strikes me as something you certainly felt, but likely wasn't anything more than an emotional experience that your brain and consciousness imagined, rather than deduced from logic. But since it felt so real, it felt so 'true' but likely wasn't. I could be wrong though.

<comment><USER11> That's the thing with your brain: Everything is kinda like that! All the time.

Most of your brain literally cannot distinguish between fantasy, dream, stories, and actual experiences.

And the parts that do, will never be able to tell if it's all just made up.

Philosophically, there is no "reality" but that what you perceive. Which, due to relativity, is, well, highly relative too. There is no absolute reality either. You can literally have conflicting facts about even simple things, that _both_ are true, and scientifically verifiably so. (Like "Who shot first?". Or "What particle came first?".)

So be very wary whenever you are confident about such things. Because that is only possible with enough convenient ignorance.

<comment><USER12> Let's not confuse what's in our heads and what's outside of them.

Philosophy doesn't get us out of the gravity well, cure diseases, build computers, or feed the hungry. However, it does motivate people to prioritize those things.

<comment><USER13> >> Let's not confuse what's in our heads and what's outside of them.

Well, everything that is at the boundary of our knowledge is also on the boundary between our heads and outside of them.

Let's say you think your partner loves you. How well do you know that? Do you think you know them well? What's the difference between them realising that they should move on in life later on and having planned it all along (getting rid of you in a few years)?

Do you believe in marriage? Is it something enforced upon us by the culture or is it emergent from each one of us individually? What if I only speak about you yourself? Is happy marriage a 5, 10, 20 year long thing? If your partner dies, how long do you wait before your marry someone else?

I would even go as far as saying that "curing diseases, building computers and feeding the hungry" is something that people with a certain slave mentality tend to think about. They need a rational purpose to live because otherwise they would need to admit they are not rational players - but in the end it's them you lose in the game of life by blaming everybody else for being so _irrational_.

<comment><USER12> What?

</comment></comment></comment></comment></comment></comment></comment><comment><USER14> William James wrote, "The art of being wise is the art of knowing what to overlook."

He was also a proponent (later in his career anyway) of the idea that brains act as filters on an expanded field of consciousness. I don't recall if this specific detail comes more from him or from Myers, but the gist is that consciousness exists independently from bodies, and brains evolved to filter out all but the most useful perceptions that contribute to bodily survival. The last half of that agrees nicely with this article.

James did a lot of work on the nature of embodied experience and was rigorous—foundational to the field of psychology, in fact—when studying how cognition and sensation interact. And a hundred years later, we still don't have conclusive evidence that brains produce consciousness; we assume it to be the case because physicalist interpretations have worked really well everywhere else in the sciences.

</comment><comment><USER15> I'm more than a little surprised that this article makes no mention of ADHD. There is an entire group of people with a medical condition that causes problems with attention regulation and it is not used to test the various models for attention explained in here?

(Also speaking as someone with an ADD diagnosis myself)

<comment><USER16> It does:

>Halassa is particularly intrigued by what the connection between attention and the basal ganglia might reveal about conditions like attention deficit hyperactivity disorder and autism, which often manifest as hypersensitivity to certain kinds of inputs.

<comment><USER15> Ah, I did a quick CTRL+F for "ADHD" and found nothing. Thank you

</comment></comment></comment><comment><USER17> This makes sense to me and maybe I'm reading this wrong, but it explains why I can't find my keys when they're in my hand.

</comment><comment><USER18> I would love some kind of cure or treatment for misophonia. I've suspected my problem was not being able to filter sounds properly, but I can't think of any solution other than physically filtering sounds externally.

<comment><USER19> I've recently had a misophonia experience while waiting for my wife and kid. The sound was coming from a street party and while I was quite far from it, the sub-woofer bass gave me great discomfort and anxiety, there was no way for me to get away from it as the sound was everywhere. I tried covering my ears but that was not much help. I had to sit down and forcefully relax. When my wife showed up, it was hard to communicate, but she didn't seem as bothered as I was. I usually don't have anxiety or I've learn to control it quite well but this episode was quite concerning to me. Do you get bothered by hi or low frequencies or both?

<comment><USER18> It doesn't seem to be a property of the sound itself. It is more about me knowing that it is being caused by a person who has the power to stop, but doesn't. If you had the same sound being caused by nature and I knew for an absolute fact that it was outside of anyone's control it wouldn't bother me.

Bass sounds from down the hall or across the street definitely falls under the category of sounds that greatly irritate me. I would inevitably call the cops if I couldn't get them to stop even though I want them to be able to play their music as it makes me irrationally upset.

The kinds of sounds that annoy me on a daily basis are people eating loudly or talking too much. I bought some pretty nice sound canceling headphones so that I can avoid such sounds when I can. I would not be capable of working in an office otherwise.

</comment></comment></comment><comment><USER20> That's very interesting but I want to know more. The work described in the article only traces a likely attention circuit in the brain (... of mice, but OK). I want to know _how_ this circuit works. How is it that we know what needs to be attended to and what to filter out? What turns the knobs of attention?

When I'm looking for my keys, I know what I'm looking for so I can see how my attention may know what to focus on and what to ignore. Perhaps, when I'm driving down the road and an animal darts across it, I know to focus on it because I have a propensity to pay attention to sudden movements. I can see how that would be very useful to have indeed.

But when there is nothing guiding my attention, like an object of interest or a motion- it's still on and it can still focus. And it can still surprise me.

Last year I had two examples of this. I was on a boat with some friends and I suddendly noticed a cruise liner far off in the distance. It was blending with its surroundings (particularly the mountaints behind it) and so even when I pointed it out to my friends they could not see it for a minute or so, until it moved closer. Another time, I noticed a military helicopter flying across a hilltop. I pointed it out to my mother and again she couldn't see it. My eyes picked both objects out and I was just gazing towards them semi-distracted not expecting anything to be there... how? And why? And how about all the other times I've missed similar information right before my eyes?

How is it that we know what to pay attention to and what to ignore? How do we know what is _relevant_?

<comment><USER21> This study seems way too low-level for the type of knowledge that you're trying to extract. My intuition is that you're going to learn more (about your subjective experience which you're trying to do) from AI neural networks with attention mechanisms (there is a lot of info about it like [https://www.youtube.com/watch?v=W2rWgXJBZhU](https://www.youtube.com/watch?v=W2rWgXJBZhU)) than from this.

</comment></comment><comment><USER22> This is exactly why we all need to be incredibly judicious in what we expose ourselves to -- the people we surround ourselves with with family, work, and social life, the content we read, watch or listen to, and the emotions we absorb from the communities around us.

Sometimes our filters can handle it all, a lot of times it can't. And a lot of modernity, with always-connected internet devices, outrage porn "news", and sky-is-falling political hacks everywhere, can be overwhelming and frankly exhausting.

I always recommend disconnecting, taking a breather, and enjoying the serenity of quiet. Not working in toxic environments with crummy colleagues, and certainly not being dragged into the daily grind of content farm media companies. It's a huge benefit to one's mental health and emotional stability.

</comment><comment><USER23> You might be interested in this paper (of mine) which predicted most of the results of the paper in the thread title.

osf.io/k497u

</comment><comment><USER24> Another article reinforcing my belief that framing AGI research as a quest for "human-level" AI is misguided. The type of intelligence we are looking for is actually animal intelligence. The higher level human abilities are mainly just higher levels of characteristics that most animals have.

<comment><USER21> The idea is to make artificual human intelligence so that it can help with tasks that humans can do. If we make rabbit AI, it's not particularly useful, and while I am sure it can be a stepping stone (but we already have plenty of those, starting a model of a neuron), the goal is a human AI (and beyond). The human AI is one of the most important steps and points on the jorney, much more so than a rabbit AI. So not misguided at all.

<comment><USER24> Point is that animal AI is the stepping stone because it has almost all of the human abilities. Where as aiming for higher level human abilities is the wrong starting point.

</comment></comment></comment><comment><USER25> Filtering is in some sense deconvolution. It makes sense it would be deconvolution.

</comment><comment><USER4> This is innate knowledge that is finally being verified by science -- eliminating distractions is as much about removing the need to filter as it is about creating the circumstances for increasing concentration

For precisely this reason I am such a stickler for proper professional dress: I find printed T's and athleisure utterly annoying and the cost of filtering out my colleagues fashion choices over the long term leads to a huge cognitive load whereas if we could all work within the established paradigm we'd all attain a way better state of flow

Guess we'll have to wait another century for the science to prove what everyone already knew since antiquity before anyone in the tech industry deigns to wear adult clothes - apologies for the dig but I have points to burn and a desire to be validated as counterculture

<comment><USER26> If everyone else seems to be doing fine and only you have the problem maybe you should look a little closer to home to find a resolution.

<comment><USER4> This is exactly the reason why our industry suffers from gross diversity problems -- everyone who 'fits in' telling everyone else it's their fault for not fitting in -- oh sweet irony of ironies bring on the downvotes

<comment><USER27> You are literally taking a recent publication about attentional mechanisms in the brain and hijacking them to explain why you're annoyed with t-shirts, then turning it into a commentary on society.

I think you could use some perspective. This research is not so concrete that the application even makes sense.

<comment><USER4> Confirmation Bias! Tell me you never make the evidence fit the facts and not the other way around?

<comment><USER28> i do that sometimes, but i do my best to acknowledge that im going to be doing so before doing it, rather than after :)

<comment><USER4> My theory is that we've thrown out a large part of our humanity along with the bath water and technology of today is akin to the fire granted us by the mythical Prometheus. We're so busy dancing amongst the flames in our rags we don't notice that it's our humanity burning

<comment><USER28> i am not entirely disinclined to agree with that statement, at least as far as the uneducated masses go. but there are certainly those that know how to properly wield the fire and work towards the progression of humanity and i think it's somewhat silly to entirely discount them as they are the ones who have gotten us this far.

<comment><USER4> Yea, they're the ones wearing the puffy vests -> [https://www.businessoffashion.com/articles/intelligence/monc...](https://www.businessoffashion.com/articles/intelligence/moncler- canada-goose-patagonia-puffer-vest)

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> In lab animals!

Not necessarily applicable 1:1 in humans.

</comment><comment><USER29> This is also how God forgets it is the entire fucking universe so that it can have a human experience and come to know itself.

Edit: See the comments on psilocybin above and read them from a spiritual perspective, try it on. [https://www.youtube.com/watch?v=jmBQRb5iuMg](https://www.youtube.com/watch?v=jmBQRb5iuMg)

<comment><USER30> Since this account seems to have gone single-agenda and you've ignored our request to stop posting these links, I've banned the account until we get some indication that you want to use HN as intended, i.e. for intellectual curiosity on an unpredictable range of topics. That's not to invalidate your spiritual path in any way, it's just that repetition is the mother of offtopicness on HN.

<comment><USER29> Thanks for handling this so well.

Is there a place I can suggest features? I could have avoided a ban if I had seen your first warning.</comment></comment></comment>