Two envelopes problem - <AUTHOR> https://en.wikipedia.org/wiki/Two_envelope_problem <comment><USER3> You can even explicitly define the distribution as P($2^i) = 1/(2^i) and the 'paradox' remains. The problem is that you are reasoning about the expected gain of swapping but that expectation is a sum over a series which is not absolutely convergent ([http://en.wikipedia.org/wiki/Absolute_convergence](http://en.wikipedia.org/wiki/Absolute_convergence)) so it's value depends on the order in which you sum over the different cases.

In fact, if you repeatedly simulate the problem (make sure you use bignums) you will find that the mean gain from swapping is not well behaved at all and refuses to converge. The law of large numbers only applies if the expectation is well-defined.

The lesson here is: when in doubt, explicitly write out the probability space over which you are working. Problems like this and the Monty Hall problem are trivially solvable on paper. There is a reason that mathematicians get all hot and bothered about formalism - it gives you a solid base from which to build _correct_ intuitions.

EDIT Let's write this down properly.

        i | gain from swapping if I have the smallest | gain from swapping if I have the largest
        1 $2 -$2
        2 $4 -$4
        3 $8 -$8
        etc

There are two arguments.

The first is that the situation is symmetric so you can't possibly gain. That is:

        E(Gain) = (1/4 * $2 + 1/4 * -$2) + (1/8 * $4 + 1/8 * -$4) ...
                = $0 + $0 + $0 + $0 ...
                = $0

The second argument is that swapping from small to large is a bigger gain than the loss of swapping from large to small. That is:

        E(Gain) = (1/4 * $2) + (1/4 * -$2 + 1/8 * $4) + (1/8 * -$4 + 1/16 * $8) ...
                = $0.5 + $0 + $0 + $0 ...
                = $0.5

Adding up an infinite series is tricky :)

<comment><USER4> Well, yeah :) The problem is still in assuming something about the distribution without having any good reason to. In original "paradox" reasoning it was impossible uniform distribution. What you show is that even assuming possible distribution could still lead nowhere. It still doesn't mean there is any reason to assume it's P($2^i) = 1/(2^i). I may just as well assume it's exactly 100$-200$ and that switching from 100$ gives me guaranteed payoff. It would be as baseless as assuming uniform distribution or the one given by you.

I think the main lesson from this puzzle is that you can't assume stuff without good reason to.

<comment><USER3> I'm not _assuming_ that that is the distribution. I'm saying that the problem as presented is underspecified, but even if you give this specific distribution in the problem you can still cause confusion. Assuming stuff is bad but the original 'paradox' still doesn't go away if you nail everything down. It's a useful problem for education people about the subtleties of infinite sums.

<comment><USER4> Well, there are distributions which don't have well defined expected value and I agree that it's valuable lesson. I would argue though that this is not the best example to show it. The point of two envelope problem comes much earlier and specifically giving distribution defined by you would make it completely different problem and assuming it in original form would already be a mistake.

I think there are better examples to show how infinite sums and relying on expected value based on those might leads to problems. Like this one for example:

[http://plato.stanford.edu/entries/paradox- stpetersburg/](http://plato.stanford.edu/entries/paradox-stpetersburg/)

<comment><USER3> The expected value in the St Petersburg game is actually well-defined - <USER2> converges to positive infinity. That is a subtly different result than not converging at all. Both problems are useful.

</comment></comment></comment></comment></comment><comment><USER5> This is one of my favorite problems. Here is one variation can help clarify thinking about it:

"There's two envelopes. One contains twice as much money as the other. No envelope contains more than $N."

This changes the problem dramatically. If your envelope contains more than $N/2, of course you should not switch: the other envelope necessarily contains less. If it contains $N/2 or less, perhaps you should switch: the other envelope may or may not contain more.

Say your envelope contains $x, and you don't know what $N is. There's two possibilities:

1\. $x <= $N/2\. If you switch, your expectation value is $3x/2, by the original argument.

2\. $x > $N/2\. If you switch, you will get $x/2, since no envelope contains more than $N.

If we assume the distribution is uniform on the range [0, N], then these possibilities are equally likely. Therefore the total expectation value is the average of the EVs of the two possibilities (3x/2 and x/2), which is $x. So we recovered the naive expectation of "it doesn't matter" from this variation. Now we can take the limit as $N goes to infinity, and while the EV of $x approaches infinity, the fact that switching does not matter does not change.

As others have said, the underlying problem is the assumption that a uniform probability on an infinite set makes sense, which it does not. However, we can instead take the limit for finite sets, in which case we recover the intuitive result that switching does not matter.

<comment><USER3> Exchanging limits and expectations is not always valid (eg [http://www.stanford.edu/class/msande322/docs/app_B.pdf](http://www.stanford.edu/class/msande322/docs/app_B.pdf)). In this case the expected gain is not convergent in the limit. See [https://news.ycombinator.com/item?id=6387344](https://news.ycombinator.com/item?id=6387344)

This is one of my favourite problems too :)

</comment></comment><comment><USER6> That is a very long article based on flawed argument.

Given no other information, assuming someone gave you 2 envelopes and told you one has $40 vs $20, common sense dictates choose 1 randomly and walk away - with no other information it is illogical to reason any other way.

The chance you choose the lower value is 1/2.

Now, if you are allowed to look inside the envelope (which gets introduced further down) then it becomes a different game.

Get $20...well by swapping you may get $10 or $40...you should probably swap.

Get $2000...well by swapping you may get $1000 or $4000...you should probably swap.

I think this works all the way up...someone with a bit more background on game theory may be able to formalise it, but the realisation that swapping forever leads to $0 nullifies this "paradox"

<comment><USER4> Yes, it's not a paradox it's just seductive flawed reasoning. Yes, at any point EV of picking an envelope at random is 3/4n (n being higher amount of money out of the two). It is all there is to it. The "paradox" is introduced by silent assumption that distribution of amounts put in envelopes is uniform which is impossible (because you can't pick numbers from infite set uniformly even if there was infinite amount of money in "adversary" disposal). The assumption is then used for conditional probability calculations: "if we see 10$ there is 50% chance the other envelope contains 20$" \- BEEP, ERROR, THINK AGAIN.

Perhaps good exercise in clear thinking but not really a paradox. Good analogy is this: "If we pick random building and climb to the roof of it there is 50% chance first building we see is higher than the one we just climbed". This is obviously true, now following "paradoxical" reasoning we get: "If we climb a building randomly and see it's the Empire State Building there is still 50% chance first building we see will be higher".

This is exact analogy to reasoning about 2 envelopes problem which is supposed to lead to a paradox.

<comment><USER3> You can explicitly state the distribution and still run into the same problem: [https://news.ycombinator.com/item?id=6387344](https://news.ycombinator.com/item?id=6387344) .

The underlying problem is basically that probability theory in non-finite spaces has some gotchas - one of which is that the expectation of a random variable does not always exist.

<comment><USER4> Interesting point and nice read. Still the problem is in assumption about underlying distribution of amounts in envelopes (in original case impossible uniform distribution). The reasoning is based on this assumption and leads to nonsense. What you are saying (I think) is that assuming some other distribution (possible one, instead of impossible one) could still lead to nonsense or doesn't lead anywhere at all.

<comment><USER3> Not so much that it leads to nonsense as that naively applying expectations doesn't always work. This is a contrived example, but it's not uncommon in eg random walk theory to hit upon cases like this where the expectation does not exist at all.

People commonly think of mathematics as being purely about formal proof but the reality is an interplay between proof and intuition. Usually when a mathematician encounters a problem in a familiar area they immediately know the answer by intuition which then guides the production of a correct proof. When you first enter a new area of mathematics your intuitions are all completely wrong and you have no idea where to start with a proof. Good teachers will introduce edge cases like this problem to refine your intuition until it is useful enough to be a guide.

[http://terrytao.wordpress.com/career- advice/there%E2%80%99s-...](http://terrytao.wordpress.com/career- advice/there%E2%80%99s-more-to-mathematics-than-rigour-and-proofs/)

</comment></comment></comment></comment></comment><comment><USER7> I'd like to offer my own humble solution. Maybe it's flawed. Maybe you can embarass me. :)

Here it is: the goal is to choose a strategy which statistically maximizes our return. I.e. strategy A is superior to strategy B if it yields higher returns after, say, 1,000,000 iterations.

So there are two envelopes, X and 2X. You select one, then you're offered a chance to change your selection. What do you do?

Let's write out all the possibilities:

You select X, then you choose to stay, and wind up with X.

You select 2X, then you choose to stay, and wind up with 2X.

You select X, then you choose to swap, and wind up with 2X.

You select 2X, then you choose to swap, and wind up with X.

Those are the only four possibilities. You're forced to choose one of these possibilities randomly, because you have no information to guide your choice. Since two of them yield 2X and two yield X, and since your choice is necessarily random, then therefore all strategies will converge on the same expected value. In short, it doesn't matter what you do. You always have a 50% chance of X or 2X, regardless of your sequence of choices.

At first glance this is similar to the Monty hall problem, but the critical difference is that information is revealed during the Monty hall problem. No extra info is revealed here.

I assert that the envelopes could contain X and 1000X and it still doesn't matter what you do.

Ok, go, embarass me!

<comment><USER8> No. If I understand correctly, your last two outcomes are wrong. There are 6 possible outcomes, not four. This is because when you choose the 2X correctly, you may receive 2(2X) or X.

The 6 outcomes are:

You select X, then you choose to stay, and wind up with X.

You select 2X, then you choose to stay, and wind up with 2X.

You select X, then choose to swap and wind up with 1/2X

You select X, them choose to swap and wind up with 2x

You select 2X, then choose to swap and wind up with X

You select 2X, then choose to swap and wind up with 4X

<comment><USER7> Actually, you always wind up with X or 2X, never 0.5X and never 4X. Remember, the envelopes contain either X or 2X, and you always end up with either one or the other.

<comment><USER9> I myself understood the problem with Afforess's reply. You are explicitly pinning the value in the envelopes, but remember the other envelope has either half or double of the value than the one in your hand, and this is the crux of this paradox. Hopefully, I am thinking in correct terms.

<comment><USER10> But he has assigned the identifiers "X" and "2X" to the envelopes. He could have called them "A" and "B" and his logic still holds. The problem I see with the wiki article explanation is: while using actual dollar amounts as examples, it gives the impression that you get to open the envelope that you picked first to see the amount, but that you don't know if it's the smaller value or larger value.

So I choose "A" and I don't know if it contains $X or $2X. I don't even get to open it to know what amount is in the envelope. Whatever it contains, if I choose to switch, I get envelope "B" \- I always have a 50% chance of choosing the larger amount or switching to the envelope with the larger amount because no information is revealed after the first choice.

for A->$X, B->$2X:

        Choose A, stay with A, receive $X
        Choose A, switch to B, receive $2X
        Choose B, switch to A, receive $X
        Choose A, stay with B, receive $2X

for A->$2X, B->$X:

        Choose A, stay with A, receive $2X
        Choose A, switch to B, receive $X
        Choose B, switch to A, receive $2X
        Choose A, stay with B, receive $X

50% chance of getting either amount, regardless.

</comment></comment></comment></comment></comment><comment><USER11> The way I see it the flaw is in the first sentence of the "example":

> Assume the amount in my selected envelope is $20.

You can't just pull that assumption out of your ass. You can only assume what the problem states, which is that the values in the envelopes are X and 2X and you had a 50% chance of choosing either.

If you run the math without adding assumptions, it works out that swapping makes no difference, statistically.

Though it is a very clever trick :).

<comment><USER12> Taking out the specific dollar amount doesn't change the math at all. If X designates the amount in the envelope I've selected, than 50% chance the other envelope contains .5 * X and a 50% chance it contains 2 * X, so the expected value of the other envelope is .5 * .5 * X + .5 * 2 * X = 1.25 * X which is greater than X.

<comment><USER13> but you are implicitly turning the scenario into one with three values, 1/2X, X, and 2X, so something went wrong with what you're doing. There was only ever a universe of two values.

<comment><USER12> Sure, these are variables, so they correspond to different possibilities. X can be anything, and given X, the amount in the other envelope is one of two different possible values - 1/2X OR 2X.

So there are far more than three _possible_ values, but only two _actual_ values.

<comment><USER13> Exactly, that's why your model is broken.

</comment></comment></comment></comment></comment><comment><USER14> All the probability math therein, for a problem whose solution is highly intuitive (if you swap, you'd be just as inclined to swap envelopes indefinitely, is all you need to realize), reminds me of this quote:

"The intuitive mind is a sacred gift and the rational mind is a faithful servant. We have created a society that honors the servant and has forgotten the gift." \- Albert Einstein

Shameless plug for a blog I like that has little math, yet uses much intuition to solve five of the biggest outstanding problems of physics: [http://finbot.wordpress.com](http://finbot.wordpress.com)

<comment><USER15> Theoretical resolution of "intuitively obvious" paradoxes such as these are important.

If we cannot find a theoretical resolution, it can indicate a flaw in our theories, and that will likely provide a more accurate set of theories.

The problems around the speed of light gave rise to Einstein's theories of relativity. Another of those flaws ("the set of all sets that are not members of themselves") gave rise to modern set theory and formal logic.

Still a nice quote from Einstein though :-)

</comment></comment><comment><USER16> I don't see the paradox... If you have 2x and swap you lose x. If you have x and swap, you gain x. 0.5 _(-x) + 0.5_ (x) = 0, So you should be indifferent to swapping.

A I missing something? Not to say I don't make mistakes, but I have a BS in mathematics so maybe this is only obvious for people with a background in math?

EDIT: No need for dollar values.

<comment><USER17> I don't have a BS in maths, but the important bit, as I read it, is that you don't know the values involved. I.e. if you have 20, there is _either_ 10 in the other envelope, or 40. You have no way of knowing which is the case, so it's in your interest to swap since the benefits outweigh the risks.

This is totally counterintuitive, though, so I'm fully willing to accept I'm missing something! And I don't buy the 'indefinite swapping' argument, since the second swap must surely reverse any advantage gained.

<comment><USER18> Once you have the second envelope in hand, since you don't actually know what's in it, the exact same expected-value argument applies to switching back: there might be $10 or $40 in the other (first) envelope, "so it's in your interest to swap since the benefits outweigh the risk".

While the second swap would reverse any advantage _possibly_ gained, it would also reverse any _possible_ harm sustained You dont know. The situation is perfectly symmetric; picking and switching is the same as just picking the second one at first. So just pick one.

<comment><USER19> Env A has $20 (we have it and know). Env B has $10 or $40 (equal chance).

We swap. Now we hold Env B, which has $10 or $40 in it. We know the other envelope (Env A) is $20. Why would we switch again?

<comment><USER18> The original statement of the problem says you're offered the choice before you open the envelope. I guess I was sloppy about the $20 assumption too. As for the case you do know, I still feel wrong about the conclusion that you should switch, but I can't formalize it.

</comment></comment></comment></comment></comment><comment><USER20> Here's my contribution:

\</comment><comment></comment><comment>-----

Approach 1: Absent new information, we cannot improve our outcomes.

In the montey hall problem there is either obscure new information, or a obscure change in the rules between firs choice and second choice.

Montey hall collapses to an initial choice of the prize behind door A or the prizes behind both door B and C. When the true, collapsed, choice is revealed the common sense reasoning is correct.

In this problem there is no new usable information.

In the two envelopes problem the new information appears relevant but is actually not on it's own any more useful to reasoning about expected value than knowing that there is a red or green piece of paper in either envelope.

\</comment><comment></comment><comment>-----

Approach 2: Keeping the quantities symbolic.

There are two envelops with quantities x and y inside. We are told that 2x = y. After choosing it is revealed that our envelope has quantity z. It is not revealed if z=x or z=y.

Let's consider the universe of possibilities.

Possibility one (50% chance): z = y. Value of switching: -x

Possibility two (50% chance): z = x. Value of switching: +x

Therefore the value or switching is: .5 * -x + .5* +x = +/-0

The red herring here is that knowing value z feels like it is information about values x and y, but it isn't.

The slight of hand is in trying to say that the other envelope is worth either .5z or 2z. This is false because there is an unknown but fixed universal constant variable x. We don't know from the available information if we are in universe z=x or universe z=y.

In short: The two envelopes paradox mistakes an unknown constant for an unknown variable. Knowing that z = 20 doesn't change the universal constants x and y.

<comment><USER4> Approach 1: There is in fact new information when you look into the envelope it's also very valuable because it allows you make judgement taking into account your knowledge about the world and specific situation (who puts money in the envelopes, what are general preferences of people in such situations etc.).

Approach 2: You made the same mistake. Seeing the money is actually valuable and very real information. The problem is that original reasoning makes wrong use of it. It doesn't mean there is no information or that we can/should ignore it.

<comment><USER20> If the new information is relevant, how is this simulation code wrong? [https://gist.github.com/tedtieken/6567112](https://gist.github.com/tedtieken/6567112)

<comment><USER4> The code is wrong because there is an assumption that the distribution is 50% for 10$ and 50% for 20$. There no basis for this (how do you know you won't got 40$ if you see 20$ ?). See my other posts, I think this point is very well worth thinking about.

</comment></comment></comment></comment><comment><USER21> The paradox depends on the possibility of always being able to double the amount. This isn't true.

Swapping once will either double or half the amount; swapping back will just do the opposite.

<comment><USER22> Even better: if I put $10 into one envelope, and $20 in another envelope and get you to choose one, what is the probability that the other envelope contains $40? Zero.

</comment></comment><comment><USER23> This is a fantastic and subtle paradox, and not at all taken to quick resolution. A resolution follows, first to spot a problem (if there is one) gets a cookie.

Consider the amount in the lower to be f(x), the higher to be x, given f(z) such that f(z) < z for all z. In this way we generalize to _all_ distributions, as the problem clearly applies to _all_ distributions. You open the first envelope, which contains A. The second envelope contains B. The challenge is to calculate the value of B.

First we must calculate x. The chance that A = f(x) is 1/2, the chance that A = x is 1/2\. The average value of A is (x + f(x))/2, so the average value of x is [f+I]^{-1}(2A), where I is the identity function and [g]^{-1} denotes an inverse function.

Now we calculate B. The trick lies in this: _both calculations must lie in the same reference frame_. So B = x with probability 1/2, and B = f(x) with probability 1/2, giving us B = (x + f(x))/2.

The rest is plug-and-chug: B = ([f+I]^{-1}(2A) + f([f+I]^{-1}(2A))/2 --> B = [f+I]([f+I]^{-1}(2A))/2 --> B = 2A/2 thus B = A.

Therefore over all probability distributions that can be defined we have the average value of B equal to A in any case. A purely mathematical resolution is satisfying, but I am not in any case an epistemologist, so it may not satisfy people who take a different interpretation of math than me. It works, and I like it.

<comment><USER3> The problem here is that you are adding together two values for a specific case of x and then adding together the result for all values of x (is that clear? probably not).

It's analogous to another age old problem: what is the value of 1 + (-1) + 1 + (-1) + 1 + (-1) ...

You could argue that (1 + -1) + (1 + -1) ... = 0 + 0 ... = 0.

You could also argue that 1 + (-1 + 1) + (-1 + 1) = 1 + 0 ... = 1.

There are actually way to group the numbers in that series to get any integer answer you want :D

Since we haven't actually defined a distribution over x the problem is not well-defined anyway. But lets pick, say, P(x=2^i) = 1/(2^i) for all i>1.

Then the expected value of A is

        E(A) = (1/2 * $2) + (1/4 * $4) + (1/8 * $8) ...
             = +infinity.

Similary E(B) = +infinity.

Now for E(B-A)

        E(B-A) = (1/2 * $1) + (1/2 * -$1) + (1/4 * $2) + (1/4 * -$2) ...

Like the example above, we can add this up in different ways:

        E(B-A) = ((1/2 * $1) + (1/2 * -$1)) + ((1/4 * $2) + (1/4 * -$2)) ...
               = $0 + $0 ...
               = $0

Or

        E(B-A) = (1/2 * $1) + ((1/2 * -$1) + (1/4 * $2)) + ((1/4 * -$2) ...
               = $0.5 + $0 + $0 ...
               = $0.5

When adding up infinite numbers of things you have to be very careful. I go into more detail in this thread - [https://news.ycombinator.com/item?id=6387344](https://news.ycombinator.com/item?id=6387344)

Cookie please :D

<comment><USER23> In my original version of the post I had intended to argue for a geometric mean, which was sort of a joke, as it only works where f(x) is a constant k*x. So the statement "A false resolution follows, first to spot the problem..." appears in the post. Upon seeing this an enterprising and intelligent person may have clicked "reply"

While writing this I realized that inverting the definition of A gave a different resolution based loosely on operator theory, and posted that instead. A couple minutes later I realized the mistake and changed the beginning to refer to the first to spot "a" problem.

The argument you've presented is good, but you have attacked the well- foundedness of the problem itself rather than the logic I've applied. I'm not sure if you get the cookie.

<comment><USER3> Goddamn, what does it take to get a cookie around here? :D

</comment></comment></comment></comment><comment><USER22> I read through the common resolution but I just can't get why you need all this. The problem seems to be at the start: you select one of two amounts, then you say the other amount is either double or half what you _chose_.

That means you have 3 amounts in the equation: 0.5x, x and 2x. But in reality there are only 2 amounts: x and 2x.

So you have to state the problem like: you choose an envelope. The other envelope either contains x or 2x. It can't contain 0.5x because that amount never existed in the first place.

<comment><USER24> Unless you picked 2x to start with, in which case the other envelope does contain half of what you chose...

i.e. there are two envelopes, x and 2x, and you choose one. There is a 50% chance that the envelope you didn't choose is half the value of the one you picked (you pick 2x, and the other is x), or there's a 50% chance the envelope you didn't choose is twice the value of the one you picked (you pick x, and the other is 2x). I think it's misleading to introduce this "third" quantity of 0.5x... but perhaps I'm missing something subtle, in which case I'd be happy to be corrected!

<comment><USER22> _I think it 's misleading to introduce this "third" quantity of 0.5x_

Yes, it _is_ misleading, that's my point :)

Put it another way: you have two quantities in two envelopes, A and B. If you choose B, what is the probability that the other envelope contains a 3rd quantity, C? The probably of that is zero, because C was not present in the initial two envelopes.

Still another way of putting it: I will place $10 in an envelope, and $20 in another envelope, then ask you to choose one. What are the chances that the other envelope contains $40? The chance of that is zero, because the only two amounts present at the start of the trick were $10 and $20.

</comment></comment></comment><comment><USER25> Arbitrarily label the two envelopes A and B. Assume that no envelope is empty. Let «X» denote the amount of money in envelope X.

We are told that either «A» = 2«B» or «B» = 2«A». Because these propositions are exhaustive and mutually exclusive, we know that

        P(«A» = 2«B») + P(«B» = 2«A») = 1.      (1)

Since we have no knowledge that would make either proposition more plausible than the other, we also have (by symmetry) that

        P(«A» = 2«B») = P(«B» = 2«A»).          (2)

From (1) and (2) we can solve for the individual probabilities of the propositions:

        P(«A» = 2«B») = P(«B» = 2«A») = 1/2.    (3)

Therefore, from the initial conditions, we have no reason to prefer either envelope.

Now we are given the information that we have picked one of the envelopes at random (let's say it's A). We are further given the information that A contains $20, that is «A» = $20. How does this new knowledge affect the probabilities?

It has no effect because we can't say anything more about either proposition without also knowing «B» as well, and we don't know it. That is,

        P(«A» = 2«B» | «A» = $20) = P(«A» = 2«B»)

and

        P(«B» = 2«A» | «A» = $20) = P(«B» = 2«A»).

Therefore, our probability assignments from (3) remain unchanged, and we have no reason to prefer one envelope to the other, let alone swap A for B.

</comment><comment><USER26> This is a really cool problem. I don't claim to understand it, but here's one way of thinking about it. The paradox assumes that when we see the value of money in the envelope we open, we get no information about whether that's the smaller or larger amount of money. Now, in practice, is that possible?

Let X,Y be i.i.d. draws from some distribution with CDF F, and say we see Y=y when we open our envelope. If the distribution satisfies the constraints of the problem, then we have to have that, for all possible values of y: P(X > Y | Y = y) = 1/2 = P(X < Y | Y = y)

In practice, this is impossible. The reason why this is impossible is because P(X>Y | Y = y) = (1-F(y)). Since F(y) is a CDF, if we take the limit as y goes to infinity we have that (1-F(y)) = 1. So (1-F(y)) can't equal 1/2 regardless of y. That is, the probability that the other envelope contains the larger or smaller value is not independent of the value we observe in the envelope we open. And, more importantly, it can't be, such a distribution doesn't exist. You have to consider the sampling distribution to get a meaningful calculation.

</comment><comment><USER27> This sounds like a special case of the secretary problem [http://en.wikipedia.org/wiki/Secretary_problem](http://en.wikipedia.org/wiki/Secretary_problem), where instead of envelopes you swap secretaries and additionally you have a time constraint.

I find it very interesting how these statistical problems can be projected on our own life (swapping jobs, finding a better partner etc ..)

</comment><comment><USER28> The paradox is interesting to discover and discuss errors in reasoning, but we should agree from the start that any proof that one should always switch (or that any behaviour could increase the expected gain) is flawed.

This can be easily proven by:

1\. Taking at face value that choosing an envelope is random. 2\. Adding a second player that is given the other envelope.

In that scenario, any reasoning that leads one player to always switch would apply to the other player too. Both players would swap their envelopes. Clearly, the expected value for both cannot go up. Thus any reasoning that mandates switching /has/ to be wrong.

(If you have issue with adding another player, instead assume a parallel universe where the player chose the other envelope.)

The error in reasoning is usually to assume that the envelope contains X and then reasoning about X/2 and 2X. In reality, the envelopes always contain X and 2X. When you actually use the parallel universe version, clearly one has X and the otehr has 2X. Nobody has X/2 (nor 4X, nor any other expected value your theory may come up with).

</comment><comment><USER4> Many explanations here contain the same mistake: Some reasoning -->> seeing the money doesn't change anything and/or doesn't count as information.

If you arrive at this point you've made a mistake. Seeing the money is valuable information. The trick is not in ignoring it but making good use of it like this:

-use your best knowledge about person who puts money in the envelope, general human tendencies and how the world is to approximate distribution of money in the envelopes

-pick according to your personal utility of money assuming the distribution

What the the wrong reasoning in original puzzle suggests is to assume uniform distribution which is impossible but more importantly baseless. As other poster shows there are baseless distributions you could assume which leads you nowhere as well.

Just don't let it detract from the main point: there is new information but making use of it is not that easy.

</comment><comment><USER20> If it is possible to switch and gain on the average we could write a 10,000 or so iteration monte-carlo simulation of this switching algorithm to demonstrate the gains.

The envelope has either x or 2x. One envelope has amount z, but the agent doesn't know if z = x or z = 2x.

If it is true that not switching leads to an EV of 3x/2 but switching leads to a higher EV: then a single switch each round should come back with an average value of greater than 3x/2.

In the Montey Hall problem, we can code up such an agent to demonstrate that the naive intuition is wrong. However, in this case, regardless of how many times we switch, we still end up with an average value of 2x/3.

Therefore there is an error in the math that says that switching has a higher EV than staying.

<comment><USER3> Go ahead and run a few million simulations of the problem and graph the running mean over time. You might be surprised :)

Be sure to use exact arithmetic (eg [http://docs.python.org/2/library/fractions.html](http://docs.python.org/2/library/fractions.html))

<comment><USER20> [https://gist.github.com/tedtieken/6567112](https://gist.github.com/tedtieken/6567112)

Run the simulator and you'll see, the ev is 3x/2, regardless of switching behavior.

Feel free to modify if you think I've innacurrately conceptualizer the problem.

Also, please excuse some non pythonic names, I'm writing the code on an iPhone.

<comment><USER3> Sorry, I should have been more specific :S

You have assumed a fixed amount in each envelope. The article leaves the amount unspecified but implicitly assumes that there is no maximum amount that could be in the envelope. The problem only becomes interesting for certain distributions.

Try this code:

[https://gist.github.com/jamii/6567205](https://gist.github.com/jamii/6567205)

If you run it with the uniform distribution the running mean will eventually converge to 0. If you comment that out and uncomment the exponential distribution it is much more interesting :)

With the exponential distribution the expected value of switching does not even exist.

EDIT Doh, the original gist was totally wrong. That'll teach me to argue on the internet at 3am. The updated gist is correct.

<comment><USER20> Fixed amount vs variable amount is irrelevant, variable amount just requires higher n.

Either way, the gain from switching approaches zero as n approaches infinity.

<comment><USER3> It does make a difference but I messed up the code :S

If you try the updated gist you will find that the second distribution appears to converge for a while but always jumps away again. I've run it now for 20540000 rounds and its further away than it started.

</comment></comment></comment></comment></comment></comment><comment><USER29> Here's how I see it - the important unstated variable is how big a budget does the host of the game have? If you know that you're playing with a billionaire who loves this party trick, and is probably willing to give away a million dollars on this, then the $20 you found in you envelope is probably the smaller value. If your envelope contains $800,000 it's probably the bigger one.

The host has chosen a number randomly between 1-N, and they're not neccessarily going to tell you what N is. But in the long run you'll find that values closest to N/2 are more common than values further from N/2.

<comment><USER30> However, it says in the problem that you are offered the switch before you open the envelope. So, you can't know if your original choice was $20 or $80,000.

</comment></comment><comment><USER31> Since you don't know what is in either envelope surely both theoretically contain both x and 2x (al a Schrödinger's cat). Only upon opening an envelope will we know the amount; which one you take or how many times you swap make no difference.

Also swapping, in my understanding, doesn't increase your probability. In the first choice you 1 in 2 chance of gettng the higher amount. In the second choice (the ability to swap, which is fundamentally the same as choosing between the two envelopes) you have a 1 in 2 chance of getting the higher amount.

Correct me if I'm wrong.

</comment><comment><USER30> Correct me if I'm wrong, but here's my analysis.

There are two envelopes. One contains a value x, and the other has 2x. You pick an envelope at random. Let's call the envelope you choose z. The variable z is a random variable that has an equal chance of being x or 2x, so the expected value of z is 1.5x.

If you switch, there are two possibilities. In one case, you picked x, so switching gains you .5x over the expected value of 1.5x.

In the other case, you picked 2x, so switching loses you .5x from the expected value of 1.5x. So, the expected gain from switching should be zero.

</comment><comment><USER32> Just take both envelopes and run.

<comment><USER33> I agree. Plus, both have money. Win win.

</comment></comment><comment><USER34> [https://gist.github.com/CalvinCarmack/6567168](https://gist.github.com/CalvinCarmack/6567168)

        $1500308 500308/1000000 0.500308 never switched
        $1500301 500301/1000000 0.500301 always switched

</comment><comment><USER35> Interesting.

This reminds me of a bizarre version of the Monty Hall problem: [http://en.wikipedia.org/wiki/Monty_Hall_problem](http://en.wikipedia.org/wiki/Monty_Hall_problem)

</comment><comment><USER36> the flaw here is assuming that the value being higher on average in the envelope is a sufficient criteria to make swapping the best choice. this is not how probability works, if i have a million empty envelopes and one with a billion dollars in it, the average value of each envelope being £1000 doesn't mean anything in the face of the 1 in a million chance of actually picking the right one.

<comment><USER37> Actually, that's exactly how expected value works.

<comment><USER3> I think the point he is getting at is that expected value (or even expected utility) is not actually a good measure of human preferences.

Additionally, in this problem there are certain distributions over the amounts in the envelope for which the expected value is not defined, making it an interesting problem to study.

[http://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow](http://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow) is a good book exploring mathematical models for decision making.

</comment></comment></comment><comment><USER38> Just read it... Here is what I came up with. I start work at 9:00 am, before work I always go to my favorite coffee bar called "Muazam's Coffee" it's 10 minutes walk from my workplace. So I arrive there 8:30 am to have a morning mocha with breakfast while reading the newspaper. One day a man approached me right before the coffee bars entrance and asked me if he could experiment a problem on me, he promised I would get enough cash for a coffee.

Well, why not I thought. He presented me Two Envelopes and said "One contains twice as much as the other. You may pick one envelope and keep the money it contains. You pick at random, but before you open the envelope, you are offered the chance to take the other envelope instead."

Without giving it a single thought, I picked the one on left for me, and began to unfold it. He looked at me and said "don't you want to swap?!", I had just unfolded the envelope and pulled out $20 bucks, "too late for that I guess" I said. I could tell by the look of his face that he was disappointed, he unfolded the other envelope and pulled out $10 bucks. He left me without saying a word.

I went to the coffee bar and spent the money I've just gotten. I took my coffee and breakfast and sat down on one of the outside tables. I saw the Envelop man again, this time he had found another guy. He had two new envelopes and asked the guy the same question. But this guy took out a pen and paper and started doing some math. It was already 8:47, I had to go to work.

Next day, same routine. I go to the Muazam's Coffee bar, and I see Envelope and mathematician guy(let's call him Joe) sitting on one of the tables outside the bar. The guy is still trying to solve the problem. I was kinda impressed how much Joe had put effort into this, he already had four A4 full of notes that seemed like formulas.

This went for weeks, months, years.

Everyday, before work I would see them outside. The Envelope man having two envelopes by his side and Joe trying to solve the problem.

28 years had passed...

I was going to Muazam's Coffee bar as usual when I noticed there was ambulance outside. I went to see who they came for, for my surprise it was Joe. I asked the envelope man, what had happened, he told me Joe had a heart attack.

Joe was admitted in a hospital, he was in bad condition. The envelope man visited him and said "Joe, I am proud of you. We have spent so much time together and you still don't have the perfect formula to solve this problem. Maybe this problem can't be solved with math..".

Joe looked him in the eye and said "No, I won't give up. I will get this right and get the highest amount of cash". The envelope man said "You can have them both" and he unfolded both of the envelopes. One containing $1 and the other one $2. This is where Joe had another heart attack.</comment>


ICANN, Privacy and the internet - <AUTHOR> https://altslate.substack.com/p/one-two-three <comment><USER2> There is simply no privacy on the Internet. =) It's sad, but in the modern model of the Internet, where huge companies keep all information on their servers, it is impossible to talk about privacy.[https://utopia.fans/privacy/tiktok-collects-users-data- delet...](https://utopia.fans/privacy/tiktok-collects-users-data-delete-it- right-now/) All personal data is saved, sometimes hackers steal them, after that they go on sale to the darknet. To be honest, this makes you think that people should return the decentralized Internet as it was at the very beginning.</comment>


Max Weber Invented the Crisis of the Humanities - <AUTHOR> https://www.chronicle.com/interactives/20200206-MaxWeber <comment><USER2> I'd be interested in this if I didn't have to subscribe to the CoHE just to read the one article.

<comment> chao- Came here to post the same.. The "web" option isn't much assistance in this case either. Google preview showed a paragraph but led to the same paywall as before.

<comment><USER3> [http://archive.is/BRUXN](http://archive.is/BRUXN)

<comment><USER4> Thanks.

</comment></comment></comment></comment><comment><USER2> What a timely essay. It's hard to compare the current US to pre-Weimar Germany, but the same crises of economic value, secularism-vs-higher-purpose and excessive politicization of ordinary things discussed in this essay are major issues of discussion today.</comment>


Audit suggests Google favors a small number of major news outlets - <AUTHOR> https://www.cjr.org/tow_center/google-news-algorithm.php <comment><USER2> There's a bit of mixing of correlation and causation here.

Most reasonable people would consider Google to be an inferior search engine if it surfaced results from the Daily Caller, Kos or Breitbart above those of NPR, BBC and the WSJ.

Search engines therefore surface the more authoritative / linked to sources first. These smaller sites are trying to achieve through regulation a level of prominence that they can't get in the marketplace.

<comment><USER3> It seems though that they now surface "authoritative" sites over clear user intent in their search.

For instance, search for "has Infowars ever been correct" on Google and then Duck Duck Go. For Google, you will see results that have nothing to do with the clear intent of the query, such as Wikipedia and mainstream news articles trashing Infowars. For Duck, you will get several results that actually answer the question.

Since Google pioneered natural language processing for search queries, and these sorts of queries worked fine years ago, the only conclusion is that Google is actively burying certain results to manipulate the narrative. Why does their natural language search break all of a sudden when I want to find out something that is unpopular?

<comment><USER4> Doing that search gave me a Quora link as the first result, and then two links to infowars...which is an awful result.

<comment><USER3> I'm seeing 2 out of 10 links as Infowars, links #3 and #4. The top two are Quora. mediamatters and rational wiki are examples of other front page results.

They all directly address the query. Why exactly are they "awful"?

<comment><USER5> You seem in your answers to be prioritizing "direct answer to the question regardless of reliability" rather than "useful information about the question's subject." I suppose there's an argument to be made for that, but it's an argument that leads to prioritizing a search engine that will return the answer "orange pinstripe" to the question "what is the color of the sky" over a search engine that returns a scientific article about how sunlight behaves in atmosphere.

(In any case, this all seems a bit orthogonal to whether Google is right or wrong to prioritize widely-recognized news outlets over smaller ones when it surfaces searches for news articles, doesn't it?)

<comment><USER3> I don't believe it to be orthogonal. There is a deeper philosophical question that has barely been touched on what search engines should return in response to user queries, which also applies to surfacing news. In the past with Alta Vista, word matches were the heuristic and it was extremely obvious as to how to measure the quality of the result. Now you've got very abstract heuristics, such as intent of the user, trustworthiness of the source, whether the source is healthy and good for society, the correctness or honesty of the content, how much revenue will be made with particular set of results, etc. and the public is being left out of the discussion as to which heuristics are important to apply.

</comment></comment></comment></comment></comment></comment><comment> tj-teej Can someone explain why this is any different from the fact that almost every Train Station in the USA has the same couple newspapers available (NYT, WSJ, USA Today, etc.)?

If the problem is the monopoly on how people get their news, then shouldn't that be what is solved?

<comment><USER6> The train station is not algorithmically deciding what to show you. The station has brokered a business agreement with the newspapers in the newsstand. I am unaware of any deal Google has with CNN to make sure a percentage of their articles show up in search engine results.

<comment><USER7> What is stopping this from happening? What is stopping Google bombs?

What is preventing us from being smarter? From going out and educating ourselves? Thinking more deeply? Finding the truth? Introspecting?

Where do we get our thoughts from? Who guides our behavior?

There is a struggle going on right now to regain control of the human mind.

It used to be just a few major news/media outlets. Now it's the entire world.

Someone has to be in charge of the information we are given. Else chaos will consume us all.

No one is in charge any more. We have to fix that. People must be told what to think again. They are thinking too many different things to predict the future anymore and that is disconcerting to those in the old power structures.

<comment><USER8> what?

</comment></comment></comment></comment><comment><USER9> Good. The 21st-century internet has created a crisis of credibility. We've learned that "information wants to be free" and "The Net interprets censorship as damage and routes around it" applies just as well to lies as it does to factual information. We are in a storm of misinformation, and have been for many years.

I don't love the 500lb gorillas of mainstream media, but they have skin in the game if they get caught pushing falsehood. Look at men who've trainwrecked their careers over getting caught in a lie - Olbermann, Dan Rather, etc.

Can you imagine smaller no-name media sources facing any real consequences in that kind of situation? No.

We need credibility again. Until somebody finds a way to algorithmize credibility, I'm happy with saying "only the big boys are assumed to be credible by aggregators".

<comment><USER10> People who have a different worldview are probably just going to bury this, but in case it is useful information to anyone, I will write it anyway.

To make it easier for people to accept the premise, let's suppose that we are not concerned about the United States. Say we are worried about some other countries.

The issue is that when only a relatively small number of the largest media companies show up in the results, that makes it easier for state or corporate interests to control the information stream that citizens receive. If smaller, independent outlets were surfaced more often, it would mean that in order to push any particular narrative effectively, there would be that many more media outlets to control. It could make it much more difficult for a government or special interest to propagandize, since they would need to influence a very large number of independent outlets.

And again I realize that many people here may not believe that government propaganda still exists in American media. But I think that most can agree that it does happen in many other countries at least.

<comment><USER9> This assumes that an endless array of small media sources is more difficult to control than large ones. I'd say that's false. An endless array of small media sources can just as easily be all traced back to a single controller. That's actually the business model of Sinclair media, for example.

But because smaller media sources are forgettable and have fewer eyeballs checking them, there's no record when they do something unethical like the large companies.

Rathergate would not happen with some small, forgettable news source. The falsehood would be pushed and unquestioned because nobody wants to play whack- a-mole.

</comment></comment></comment><comment><USER11> > IN THE LAST WEEK OF APRIL, nearly 23 percent of all traffic to news sites > tracked by web analytics firm Parse.ly came from search engines. Google > alone accounts for nearly half of external referral traffic...

Is this surprisingly low to anyone else?

Depending on how you parse it, only 10% or 20% of news sites traffic comes from Google.

When I worked in comparison shopping, 80-90% of our inbound traffic came from Google, as we failed and failed again to cultivate any loyal, direct users.

<comment> jon-wood When comparison shopping all anyone really cares about is the price, and you're comparing like for like in most cases so Google works really well. News is something where I at least rarely find myself wanting just any old opinion on a story, so I'm more likely to go to a news source that I trust already for their take on that.

<comment><USER12> ? When comparison shopping all anyone really cares about is the price

I disagree - people care about delivery times, quality of goods (is it what I ordered), and in some cases, they are open to alternatives (I would like a cheap android phone, If you can give me a Nokia instead of a Huawei, I don't care).

Similarly for news, people may have some preferences in their browsing, but ultimately for breaking news it doesn't matter whether it comes from CNN or the India times if there's no opinion involved in it yet, and lets be honest, most short-form journalism has very little research/opinion

For long form/blog-like content I agree with you though.

</comment></comment></comment><comment><USER13> I certainly felt like Google does this for their 'Google News' product. I selectively blocked half a dozen sources I found to be excessively clickbaity and it really cut into the amount of news that was shown.

My guess would be that there are three things going on a least;

One is that news outlets are loath to let Google crawl them and post snippets, they argue that Google gets the 'benefit' without paying for the content. (this was used in a couple of lawsuits)

One is that Google may provide the advertising feed for news outlets that are shown, thus a double benefit to Google is that you go to their news aggregator, and when you pop off into a news story you continue to be served Google ads.

And finally, there is the polarization of people's news intake through a limited number of sources. Per my experiment I found after deleting the new sources I found offenses Google was unable to come up with additional "interesting" news sources, they just didn't crawl them or have access to them (or they don't exist I suppose). Regardless of the reason, the additional wasn't in their index to serve up.

<comment><USER14> Interesting you should say this, as I've found Google News to be "desperately local", to the point of excess. I'm all for regional news and mixing things up, but typically when I've traveled somewhere within the UK perhaps for a weekend, Ithen get inundated with totally inappropriate news from the area for ages, until I proactively block it!

Their algorithms can suggest amazing insight at times, but when you suddenly get stuff like "Local council changes bin days in Huddersfield" it's clear they have a long way to go to understand news relevance, especially when you were just passing through.

</comment></comment><comment><USER15> Before we get too deep in arguing about supposed favoritism by Google, let's back up and look at the types of search terms that were entered in this study. Just about all of them relate to the sorts of Washington-focused headline news that's primarily covered by "a small number of major news outlets."

Searching for "Rex Tillerson" is the perfect example. (Remember him?)

Typical news searchers are looking for a much wider swath of news than just the latest on the former U.S. Secretary of State. In my own experience, that includes local news (what were all those police cars doing last night?) and lots of sports news, business/tech/entertainment news, scientific news, etc. Those articles come from a much wider assortment of news outlets, and I think Google does a decent job of finding them.

There's a limit to how many news organizations can generate meaningful coverage about Rex Tillerson. I'm at peace with the idea that Google has found most of the ones that matter.

<comment><USER16> You'd be at peace if Google favored news and narratives that overlap with your world view. Hypothetically, how'd you feel if 65% of those articles were from Fox news and the greater Murdoch empire?

I wish they simply skipped narrative driven, opinion filth and stayed with factual reporting from AP, Reuters, NPR, ...

In the last 5 years, NYT, WaPo,.. have wholly embraced narrative driven opinions instead of sticking with plain news, because anger sells

<comment><USER17> 65% of articles in my Google News (not logged in) are from Fox News and/or Washington Examiner.

I think geography affects search results a lot. I'm in a blue city, but red state.

<comment><USER18> Are you positive about that? I just tried to VPN from a few different locations in the US (Texas, Georgia and Seattle) with a fresh private browsing session each, all of the headlines appear the same and doesn't change by region.

<comment><USER17> I'm not positive about that. All I can tell you is that I never see WSJ, but I often see Fox News, WaPo, Politico, and Washington Examiner. The rest of the sources are a mixed bag.

I've been hitting Google News without a logged-in session (but with my real IP address) multiple times a day for 5+ years.

I noticed the increase in Fox News as a source around the time conservatives were complaining about censorship a year or two ago. Interestingly, though, Fox News is rarely the source for a major political news story -- it's often their fluff pieces that hit my feed.

</comment></comment></comment></comment></comment><comment><USER19> Dear Google, I suggest that you embrace this kind of audit in service of transparency and credibility: Create an automated news.google.com/audit page so that readers can easily evaluate the sources and track how they evolve over time.

As a user I would also greatly appreciate the ability to toggle news sources on and off, and/or to weight them, and to easily share those filters.

<comment><USER20> >As a user I would also greatly appreciate the ability to toggle news sources on and off, and/or to weight them, and to easily share those filters.

IDK. Do we need more ways for users to create bubbles for themselves? Is the solution to avoid Google's preferred bubble just to let you build your own?

<comment><USER19> To mix metaphors, bubbles are essential sanity equipment when drinking from a fire hose.

<comment><USER20> Are you implying that to protect yourself - that you need to only see news sources that you think will already agree with your idea of how things are?

</comment></comment></comment></comment><comment><USER21> Major traditional news outlets are shown to be major news outlets online.

News at 11..

This study is just proving the Pareto principal and claiming its some big thing. Of course these outlets get most of the Google love, they're all linked to as sources by all the little news outlets, how is anyone surprised by this?

</comment><comment><USER22> Spoiler alert: so does everyone else (see subscription numbers).

There are a few niche and highly specialized publications which I view (GQ, Men's Health, etc.) but for major news and in depth reporting I'm more likely to frequent WaPo, NYT, Vox, etc. There is a huge drop off in quality when viewing work from the mid-tier news providers.

<comment><USER23> It's true, they're the only ones actually putting people on payroll to send them out to areas, do interviews and original investigation. Everybody else is just reaggreggating others from other news outlets, social media and public documents with their own analysis. But even the big mainstream companies suck at local reporting for their own towns. For things that are not national federal news you have to really get it from other people.

</comment></comment><comment><USER6> I'm not positive but I feel like this is getting mistakenly lumped into the issue of censorship by big tech companies?

I could be being naive but this seems like an echo chamber problem that faces websites attempting to surface content to you based on what you like or your network likes. This doesn't appear to me to be same issues of political-based censorship of certain people's accounts that we have been hearing a lot about over the last year.

</comment><comment><USER24> It's not difficult to understand:

Google shows users what other users like, and most of Google's current users favor a small number of major outlets.

I can appreciate that you may love RC Cola. But it shouldn't surprise you that most gas stations mostly stock Coke and Pepsi products.

If you think gas stations need to be more "fair" about which pop they sell, then I'd love to hear you explain how you think that should be done.

<comment><USER25> I think your logic is perfect. As long as you agree to treat Google as a publisher and not as a platform like it is currently being treated now.

Section 230 [1], which in theory was supposed to protect free speech, is actually doing the opposite and giving Google and other major media companies the power to censor free speech and decide what you can see or not.

I'm sure the gas station would be held liable if RC Cola made their customers sick and they kept selling it.

Google is not liable for anything they publish/display, but still have the power to choose what they publish/display.

You can't have your cake and eat it too.

[1] - [https://www.eff.org/issues/cda230](https://www.eff.org/issues/cda230)

<comment><USER26> You're thinking about Section 230 wrong. Censorship is a government only thing. But even if we accept your definition of censorship, in google's case, there is no censorship taking place. Because google always lets every user know about _every_ hit it found for their search at the very top of the first page of results. So if google knows about 1,690,000,000 instances of, say, "chocolate cake", it tells you that up front. Crucially, given enough time, google will always present to every user all 1,690,000,000 instances of which it is aware.

You are not required to present all of the information on your platform at once in order to be considered a platform. Indeed, it is generally accepted that platforms, by their very nature, are not able to do so.

<comment><USER25> I don't want to argue semantics, but even if we do, I would state that you are wrong. [1]

Censorship can come from anyone and anywhere. Especially when a private company like Google has more power that many governments in the world. Google could even be considered an un-elected government depending on how you look at it.

What they are doing by de-platforming people is censorship.

They offer no clue as to how their algorithm works or why some people get up on the list while others don't.

While they could theoretically show all 1,690,000,000 results, we know for a fact that just the first or second page at tops is what matters. So, in all practicality, the other results do not exist.

If the other results do not exist, then they are a publisher and should be treated as such since they pick the winners and losers.

If there's nothing wrong with what they are doing, what's the problem of calling then a publisher? Why not embrace it and take full responsibility?

[1] - [https://en.wikipedia.org/wiki/Censorship](https://en.wikipedia.org/wiki/Censorship)

<comment><USER27> You seem to be suggesting that by curating the content on their platform, Google is losing (or should be losing) their section 230 protections. This is simply wrong.

If I have a blog, and I ban someone from commenting, I don't lose my 230 protections. Even though I've 'de-platformed' someone from my blog. Google is no different than a blog, just scaled up.

<comment><USER25> @bilbo0s I'm sorry. Let me be more specific.

When I refer to deplatforming I'm talking about users and opinions being banned from the plaftorms (YouTube, Twitter, FB, etc).

When I'm referring to search engine results what I'm saying is that Google is promoting a few selected sites over others based on a undisclosed criteria. The undisclosed criteria here is what makes a world of difference.

By not making it clear they are picking winners and that should not happen.

<comment><USER27> On my blog I'm free to delete any comment I like, and I'm not beholden to the government to explain why I did it. That's simply the Constitution at work. How big, in your opinion, does a publisher need to be in order for their First Amendment protections against forced speech to be stripped away?

<comment><USER28> Right, because your blog is tiny and meaningless. When you control the eyeballs of a country it is a different story.

<comment><USER27> Can you direct me to the clause in the Constitution that limits free speech rights if you attract a big enough audience?

<comment><USER28> Anti monopoly laws, would be one. You can say whatever you want, you'll just have your platform split in smaller pieces to avoid harming the average individual.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER29> So the large news outlets have better SEO for their articles? I can't say that's very surprising.

</comment><comment><USER30> It seems like a popular response for this kind of this research is "Well of course - I expected this result" or to point out the murky causality going on when an intelligent technology is trained on the behavior of people using it.

I agree this murky causality is important to discuss and that indeed these results are consistent with other observed distributions of online content (i.e. Pareto/Zipf), but I think excessively focusing on these responses undermines the importance of audit research like this. Audit research is mainly descriptive by design (can't really conduct an experiment unless you're Google) and without the descriptive results from audit research we can't argue empirically about the these topics at all!

</comment><comment><USER31> Important caveat: the data analyzed is from November 2017, which is an eternity ago in the SEO world.

</comment><comment><USER32> This is an interesting thing and we can argue about it, but it's a distraction from the fact we have a de-facto 'single source' for a considerable amount of information.

Google fits many of the criteria for the definition of monopoly. Once you consider both systematic issues (i.e. Chrome, Android, default browser choices, behavioural things) and even brand and mass-market issues ... it's a monopoly.

The control of most information by a single source entity simply is not beneficial to the world, even financially.

Google could possibly be broken up, at least the distribution of search entry points should be a different business than search itself i.e. Android/Browser, Search different businesses.

Google's control of distribution (i.e. Chrome, Android) is much akin to Standard Oil's control of the railroad system, which was really how they developed their monopoly.

Also given the inherent power here, we may want to at least have some regulation on how information is processed. We regulate so many important things, we can require certain things for tech companies as long as they are smart regulations.

Edit: put differently we have, without realizing it, put an _enormous_ amount of trust and goodwill into Google - they are 'mostly' doing a good job with it, but there's not reason for us to believe that will continue indefinitely. Though I'm wary of the politicking and sometimes total lack of understanding of this stuff in Congress/Parliament ... at least theoretically we should be able to get past that.

<comment><USER33> The largest gain would probably come by splitting the Ad infrastructure out of Google, and forcing it to make their ads available for any search company on the same terms.

</comment></comment><comment><USER34> Arguably, the purpose of a search engine is to help people find sources that they don't already know about. Ideally, Google Search should present a diversity of results (the long-term trend of declining search quality, notwithstanding).

Google News, however, isn't a search engine, it's a link aggregator curated by a multi-armed bandit that seeks to maximize clicks and minimize political blowback (i.e. via censorship). Providing a diverse set of news sources is inconsistent with these constraints.

If it wasn't obvious, Google uses Search to identify topics that reach a large enough audience, and then they build specific properties around those topics curated by algorithms in order to create screen space in which to sell behavior modification products* (ads). While Google News, Maps, and Local are the obvious ones, even AMP fits this model with an attempt to monetize the "waste heat" of the lower click sites by appropriating their content.

* gfodor's observation that ads are behavior modification products: [https://news.ycombinator.com/item?id=19798178](https://news.ycombinator.com/item?id=19798178)

<comment><USER35>
        > Arguably, the purpose of a search engine is to help
        > people find sources that they don't already know about.

I certainly don't want that... I want a search engine to show me a set of results likely to be most relevant to my query and to my interests. Google does some limited randomization of results to prevent winner-begets-winner outcomes (based on my SERP result research) but they're purpose is certainly not to show people sources that they don't already know about. Most of these queries are basically "navigational" anyways: is someone searching for "trump tariff" looking for interesting results or are they looking to _navigate_ to a story about "trump tariffs"?

<comment><USER34> If people already know the answers they seek or where to find them, then they aren't using a search engine in the first place. At a fundamental level, the purpose of a search engine it to help people find sources they don't know about.

Is Google Search used as an atlas used to navigate the Internet? Almost certainly, but it's not fit for that use case. Google knows this and steers people towards their curated experiences - Maps, News, etc.

</comment></comment></comment><comment><USER36> It looks like the impressions follow an exponential (geometric? someone will surely correct me) distribution which is what you would expect if it was based on what people click on. If it was something different where niche outlets were featured more often than their "share", that would imply manipulation and would be a cause for concern.

<comment><USER37> Power law is the term you are probably looking for

</comment></comment><comment><USER38> _> As much as our results help better describe Google's curation of news, what our study decidedly cannot say is_ why _some sources dominate on Google._

 _> News source concentration on Google implies an unequal capture of attention and its benefits, including any advertising or potential subscription revenue that might result. If they are serious about supporting digital-first newsrooms, algorithmic news curators, including Google and others, might be more explicit in articulating the inherent design tradeoffs between the relevance desirable for individuals, the diversity desirable for society or democracy, and the fair competition desirable for news organizations._

So, because Breitbart doesn't get as much traffic as CNN, Google must be held accountable?

</comment><comment><USER39> Isn't this to be expected when you're using links as your primary/major ranking mechanism? The largest companies will have the most links, therefore they'll rank on top, be visible to users, and get more links. Sounds like the Matthew effect, in a way.

</comment><comment><USER40> Many years ago I asked google to provide a "reverse popularity" view for Google News via some customer feedback mechanism. Got back a noncommittal "interesting idea". I understand it would be hard technically and all the reasons why it doesn't exist. Still wish it did... are there any interesting solutions to this technically? A "show me stuff I would no normally want/seee" anti filter bubble tech?

</comment><comment><USER41> If you want a much more interesting read about the few brands that dominate google this article is an excellent breakdown.

[https://detailed.com/google-control/](https://detailed.com/google-control/)

</comment><comment><USER42> There is only one way to make sure that a platform is fair. Make it AGPL. That way there is an information symmetry and even if there is bias there is equal opportunity to exploit it.

</comment><comment><USER43> I'd like to see Google impose a "clickbait" penalty and derank clickbait articles, as long as they are imposing editorial control over content and not simply being a conduit.

</comment><comment><USER44> IIRC, the article from the HN post on AMP Stories the other day mentioned the Washington Post as a charter featured participant.

</comment><comment><USER45> WHY SOME NEWS SOURCES DOMINATE ON GOOGLE can be readily answered:

From the article: _"As much as our results help better describe Google's curation of news, what our study decidedly cannot say is why some sources dominate on Google. Perhaps some outlets have cracked the SEO code for Top Stories. Or there may be a number of other factors taken into account by Google's algorithm that end up prioritizing certain outlets over others. We just don't know unless Google is more transparent with the editorial design and goals of news curation in the Top Stories box."_

If you have the question, perhaps you could Google? I cheated on this search, because I recalled something about Google knowing where stories appeared first, and weighting that.

Google: how google ranks first to publish news original sources

Top result: "Under The Hood: Google News & Ranking Stories" by Danny Sullivan

Link: [https://searchengineland.com/google-news-ranking- stories-304...](https://searchengineland.com/google-news-ranking- stories-30424)

Summary:

> _"Google News tries to determine important story clusters by looking at what > individual stories are being featured across the board at various > publications. When those publications begin to feature new stories, that in > turn influences the creation of new story clusters that get prominent > placement at Google News."_

And then, the factor I was looking for after seeing the ranks in the "audit" ... who are likely sources of original reporting?

> _"So what causes an individual article to be the lead item in a particular > story cluster? Various factors are involved, Cohen said: 'The source ranking > involves many things. Is there original content? The timeliness. Coverage of > recent developments? The relevancy to the cluster at hand. In some cases, is > there local relevancy? Is there content from a local source with local > content?'"_

> _"Cohen also explained more about the balance between ranking the latest > content versus the originating content: 'Say you publish something and then > someone else sources you but adds no real new information. If they come > after you, you don't want to penalize the original source for being > first.'"_

And what is their reputation, aka NewsRank, not PageRank?

> _"There are source specific rankings, the equivalent of PageRank for a > particular site. We look at different third party metrics. What's the volume > of publication of original content in a given category? If you look at > Bloomberg and Reuters, they may have hundreds of original articles in > business. That's a pretty good indication of the quality of that source for > that category. Compared to sports, there's not that much original content > [and so they might not have as much authority for when ranking sports > stories]."_

</comment><comment><USER18> In machine learning we often use regularizers to reduce bias (amongst many other neat things that it can do), and I've been thinking if similar techniques could be applied to help us humans to reduce bias and strengthen our belief system against malicious information.

Some information contain objective facts (e.g. climates change, vaccination), thus citing reputable research is often enough.

Though many of the political and economical belief have little objective truth to them (e.g. government regulation, taxation plan), and it is useless to discuss bias, we can only hope to prevent from malicious information. Perhaps allowing both sides to show arguments can help reduce bias, but this technique is far from perfect.

</comment><comment><USER46> Presumably major outlets have good Pagerank scores?

</comment><comment><USER47> This is no surprise, the Trump-Russia news story did lots to make Google tighten down and reinforce the major news outlets.

A lot of this was documented when it happened when the concern of fake news was at its height, Google intentionally censored many outlets. This is one of the many articles that documented this happening: [https://www.truthdig.com/articles/the-silencing-of- dissent/](https://www.truthdig.com/articles/the-silencing-of-dissent/)

</comment><comment><USER48> Pareto suggested this before Audit did.

</comment><comment><USER49> It doesn't seem all that strange to me.

The NY Times, Washington Post and the BBC do original reporting. CNN and Fox News have no paywall, and they do something which passes for reporting.

Sometimes a regional paper has special coverage of an ongoing story, as the Seattle Times is doing with the Boeing 737 MAX scandal.

For each of those there are 100 blogs that are reloading real news sources over and over again and cut-and-pasting a story where the real reporting is "The New York Times reports..."

</comment><comment><USER50> The list is surprisingly similar to the green box on the media bias charts: [https://www.adfontesmedia.com](https://www.adfontesmedia.com) if you remove the sources with a hard paywall like the WSJ.

The only outliers are CNN and FOX, which are probably included simply because they are large, and would tend to cover more of the stories this survey searched for.

</comment><comment><USER51> people themselves also tend to favor a small number of major news outlets?

</comment><comment><USER52> CJR seems not compliant with GDPR. Do they think that the content should be delivered to all the world to read? The big newspapers have taken GDPR seriously as probably other regulations and are safe to be seen around the world or send you to a page were explains why you cannot see the content.

Why do so many small companies think that they have the right to sell their content worldwide without following local laws? (To be fair some big companies think the same)

"Columbia University and authorized third-party service providers collect information about users' activities and experiences while visiting a Columbia University website."

I do not agree to third parties having my data. Where I can opt-out? Where can I delete my data? Will I be notified if there is a data leak in less than 72 hours of the discovery?

</comment><comment><USER53> The author of this article and the Columbia Journalism Review should be ashamed of themselves. The article uses another study which as the article itself states didn't measure bias in news outlets but instead the self- reported political leanings of those who prefer the news outlets to classify news outlets as left or right leaning.

Google may indeed mostly provide links to a few select major news outlets but the article is trying to go beyond this by suggesting a left or right slant to the outlets chosen and therefore to Google using data which doesn't support this.

</comment><comment><USER54> There's an interesting case here of do you manipulate search algorithms for political reasons, or political alignment? The brand value of a lot of left wing brands is higher without question (and I imagine they have more links too), so do you show them more, because, hey, it's just an algorithm... Or do you have to start showing an equal number of left vs right publications?

Also, who says what's left or right? How left? How right?

<comment><USER55> > who says what's left or right? How left? How right?

It seems pretty pointless trying to adjudicate this. If not for any other reason that what is considered the centre changes over time (e.g. it moved to the left in the post-war years, and back to right in the Reagan/Thatcher years). Furthermore, these changes are arguably a good thing.

</comment></comment><comment><USER56> don't worry. Google's YouTube will happily send you down the right wing rabbit hole of white mail victimhood and climate change is a hoax after you watch a couple of non related videos.

</comment><comment><USER57> Headline should be: Audit suggests Google favors a small number of major _NEWS_ outlets

</comment><comment><USER58> Merchants of "Mainstream"

</comment><comment><USER59> That's a good thing. Too many outlets is chaos Too few is tiraney.

</comment><comment><USER60> A few months ago I was of the opinion that Google was falsely accused of bias and favoring (likely because I tend to agree with their results). However, it's getting easier than ever to test the bias for yourself.

I actively wanted to read some of the extreme right wing stuff a few weeks ago, so went searching for it. I've done this with left wing stuff before and it is very easy to find it via Google. (meta comment: I believe strongly that we do everyone a disservice by silencing people. It harms the rest of us by isolated us from anything not "mainstream" and it further pushes the silenced into radicalism and vindicates/bolsters their argument that they are mistreated. So, I try to put myself in uncomfortable territory to see what people are saying, even if it's just to get it from the horse's mouth than spun by somebody in the media or on twitter).

I literally could not find it on Google, but just moving over to Duck Duck Go I found what I was looking for right away.

You should try it yourself! Think about some polarizing political topic and plug it into Google and then do the exact same search in Duck Duck Go (and maybe Bing, I haven't tried that but I will). You'll be shocked at the difference.

<comment><USER61> Can you give some specific examples of the searches that did not work on Google but did on DDG?

<comment><USER60> Yes certainly. I just typed "abortion" into Google and DuckDuckGo and compared the results. Look at the news articles, and "coincidentally" planned parenthood are the top two links.

Google: [https://www.google.com/search?q=abortion&oq=abortion](https://www.google.com/search?q=abortion&oq=abortion)

DuckDuckGo: [https://duckduckgo.com/?q=abortion&t=h_&ia=web](https://duckduckgo.com/?q=abortion&t=h_&ia=web)

DuckDuckGo (news): [https://duckduckgo.com/?q=abortion&t=h_&iar=news&ia=news](https://duckduckgo.com/?q=abortion&t=h_&iar=news&ia=news)

</comment></comment></comment><comment><USER58> Fahrenheit 451 in real life. Instead of burning books they are suppressing links that don't favor their political inclinations. Case in point, google 'the gateway pundit' [https://www.google.com/search?q=the+gateway+pundit](https://www.google.com/search?q=the+gateway+pundit) and search for the same In DuckDuckGo [https://duckduckgo.com/?q=the+gateway+pundit](https://duckduckgo.com/?q=the+gateway+pundit) find out where they show the actual website.

<comment><USER62> Thanks for the report, that looks like a bug. I've passed it along.

<comment><USER63> I'd like to think it is a bug, but I wasn't born yesterday. You were caught. The behavior is consistent.

Another interesting search is American inventors. Out of the first 16 featured with photos, only one (Thomas Edison at #5) is not black. It mostly continues that theme for all 56 featured, not even getting to the Wright brothers. It's been that way for a while, as seen in this article from 2.5 years ago: [http://www.unz.com/isteve/great-moments-in-google- american-i...](http://www.unz.com/isteve/great-moments-in-google-american- inventors/)

Another interesting search is for the_donald. It is pretty clear that the subreddit should appear first, probably followed by the old.reddit.com version. Right now it is fourth, mixed in with a lot of far-left hate and conspiracies. The second link is a conspiracy claiming that /r/The_Donald is run by Russians!!! It was even worse last week, being totally missing from the search results -- but that was starting to become a news story, so it got "fixed" to be subtle.

You have a bit of a PR crisis forming here. The idea to abandon Google is spreading. Once lost, trust is hard to regain.

<comment><USER62> The issue with the "american inventors" query is a well known issue in information retrieval called "compound intrusion." The phrase "american inventors" occurs most often in web documents as a substring of "african american inventors" and unless you specifically exclude that from the results, it tends to dominate, producing the list of people you see there. If you search for "US inventors" you get a more reasonable list, because "US" isn't a substring of "african american."</comment></comment></comment></comment>


Large DDoS hitting CloudFlare - <AUTHOR>

Cloudflare confirms that a big DDoS is going on right now: https://twitter.com/CloudFlareSys<p>There are a lot of people out there talking about the DDoS as well, some of them are talking about 400+ Gbps: https://twitter.com/search?q=ddos <comment><USER2> CloudFlare, a product supposed to increase uptime, has been my product's only source of downtime for the past several months.

<comment><USER3> CloudFlare also caused huge performance issues on a side project which I didn't notice until recently. I was consistently getting 1.2s+ responses on a couple of pages, and 600ms on a completely static page.

So, not only did CloudFlare not help on the pages that were truly static, it was actually making everything worse across the board. Hitting the same site with my direct.* DNS cut from 1.2s+ to under 500ms.

CopperEgg and others also kept reporting that my site was down or otherwise super slow when on CloudFlare. I'm not sure if this is because of throttling or something else that might have been in place with the CF service, but either way I was often unable to reproduce the "down"/"more than 6s" that CopperEgg kept reporting.

In short, I think CloudFlare's services sound good in theory, but I'm not sure they have figure out all the issues with scaling and performance yet. So, I cut them out... and certainly not going to be paying any time soon.

<comment><USER4> I've been on the free account for a while now, and when I received a surge of traffic they responded to 460,000 of the 500,000-ish HTTP requests and served 48gb of the 50 that were requested over the course of about 12 hours. My server load sat at 0.01 the whole time and I could not have been happier. The concept is absolutely sound in theory and in practice.

You raise a great point about reliability however. The above example was a personal project so I don't really lose much if there's the odd down-time here and there. I'd love to recommend this for the enterprise-level clients where I work but the idea of a point of failure out of one's direct control is a bit alarming, and I am surprised and disappointed to see them succumb to a DDOS that affects what (I assume?) was their entire network - even paying customers.

I notice even now my homepage is taking seconds to load, when usually it responds instantly. My guess is that they're prioritizing access at this stage due to the DDOS, if it's still ongoing.

</comment></comment></comment><comment><USER5> This is why most CDNs avoid taking on controversial customers like 4chan, kaddafi.hk and other sites trafficking in illegal content. Due to their decision to accept anyone, Cloudflare is a constant target for DDOS attacks and their legitimate users occasionally share the brunt of those attacks.

<comment><USER6> 4chan "trafficking in illegal content"? I don't think that's a fair assessment. They specifically censor much less than most website operators but they aren't "trafficking" any more than highway underpasses "traffic" in illegal graffiti.

<comment><USER5> I was referring more to the numerous sites they host like kaddafi.hk which sells stolen credit cards online. 4chan has a recurring problem with child porn (which their moderators do their best to prevent), but I agree with your distinction regarding trafficking.

</comment></comment></comment><comment><USER7> The attack directed at CloudFlare has been mitigated at this time. Performance should be back at normal levels now. [https://twitter.com/CloudFlareSys/status/433017584670093312](https://twitter.com/CloudFlareSys/status/433017584670093312)

</comment><comment><USER8> I've had to move my blog away from EC2 because Cloudflare was killing it (it was a very simple wordpress install). Other more custom sites on other servers are doing fine though.

They manage so much stuff that any downtime they have will make a lot of people unhappy. It already happened once ([http://oneurl.me/cloudflare-broke- the-internet](http://oneurl.me/cloudflare-broke-the-internet))

</comment><comment><USER9> And here I was wondering why my internet (Germany) was very unstable an hour ago.

Looks like it's related: [https://twitter.com/CloudFlareSys/status/432997463562022912](https://twitter.com/CloudFlareSys/status/432997463562022912)

</comment><comment><USER10> Here at 1355 Market, the Twitter building, AboveNet is entirely down. They are claiming it's a main router issue. No ETOR.

</comment><comment><USER11> networksolutions.com is down... As well as a large merchandising site I'm working with that's DNS goes through network solutions...

</comment><comment><USER12> Is there an economic incentive for DDoS attacks? Or is it just some bored script kiddie with nothing better to do?

<comment><USER13> My theory is that some of them are proof-of-concept attacks: "We took down X for forty eight hours; give us $Y and we'll do the same to anyone you name."

</comment></comment><comment><USER14> We're seeing an increase in 'down' notifications in general here at NodePing.

</comment><comment><USER15> I would kill for a Null Route REST API at peering points/network upstream providers.

<comment><USER16> The reason the internet is still online is because people who would ask for this type of thing don't have access to it.

Routers shouldn't speak HTTP. People who don't know how to use blackhole communities have no business controlling them.

<comment><USER17> _People who don 't know how to use blackhole communities have no business controlling them._

What's a "blackhole community"? (I did Google, but found email messages from 2003 on page one. Surely there are better references?)

<comment><USER15> [https://www.nanog.org/meetings/nanog32/presentations/soricel...](https://www.nanog.org/meetings/nanog32/presentations/soricelli.pdf)

<comment><USER17> Thank you, commenters like you are why the HN community rocks.

<comment><USER15> You're welcome! Not a problem at all.

</comment></comment></comment></comment></comment></comment><comment><USER18> We would people try to do a ddos on them. A ddos is a temporary thing. Only time it is worth it do do a ddos is when there is some kind of event thing.

Like if some one wants to get a site down today so people can't vote.

And cloudflare does not really have anything like that. Their customers yes, them no.

<comment><USER19> > _We would people try to do a ddos on them._

The same reason anyone ever DDOSes anything: because they're assholes.

<comment><USER20> This is the most succinct and accurate assessment of anyone who DDOSSes anything.

</comment></comment></comment><comment><USER21> It's not me! :)</comment>


What Silicon Valley Can Learn From Zappos - <AUTHOR> http://scobleizer.com/2009/04/23/learn-from-zappos/

<comment><USER2> I love Zappos as much as the next guy, but I wonder if people feel like they're being forced to be happy.

If I was haxin mad codez, and someone walked in and it was expected that I'd get up and wave to them in a silly way, I'd probably flip them the bird. I wonder if you can decide not to partake in all the look-we're-so-happy- awesomeness. I love having fun at work as much as every other person, but some of this feels a little too much.

Maybe I should go take the damn tour.

<comment><USER3> No offense, but I think this is their strategy for that:

 _3\. Get rid of assholes. Zappos has a filtering system before, during, and after hiring to make sure they get rid of people who "don't fit the culture." That is the nice way of saying they get rid of assholes and they get rid of them quickly. They even pay candidates $2,000 after they go through training if they can admit they don't fit into the culture._

<comment><USER2> None taken. :-)

But you're saying that if you don't get up and wave to people with a stupid grin on your face when you're "in the zone" when writing code and being productive, that makes you an asshole at Zappos and hence fired? Nice.

<comment><USER4> No, they're saying people who don't fit into that culture, having only invested a few weeks of training, tend to take the money and run leaving only those who actually fit into that crazy happy culture.

</comment></comment></comment></comment><comment><USER5> Good article. But Scoble is a hypocrite. I haven't seen any company obsessed about their number of followers or cribbing about not being in Twitter's suggested friend list. But scoble does both. The benefit of being on the list is that you get more followers. Since techcrunch was on the list and scoble was not, he was mighty pissed and started throwing around accusations that techcrunch had paid twitter for the position on suggested friend list.

If as he says, no of followers doesn't matter, then why would not being on the list matter so much to him. Here is the thread on friendfeed where he accuses TechCrunch of paying twitter and makes a huge deal about number of followers

[http://friendfeed.com/e/c487ea34-ee9c-4794-9bc6-29d2cc1ceda1...](http://friendfeed.com/e/c487ea34-ee9c-4794-9bc6-29d2cc1ceda1/Prove- that-Techcrunch-did-not-pay-biz-10-000-to/)

<comment><USER6> I'm not sure what's worse: the guy complaining, or the guy complaining about the guy complaining.

Shouldn't this thread be about good corporate policy? I remember working for a government contractor and everyone seemed like a bunch of drones. I'm happy to hear one success story. It gives me ideas about how I want my own business to be someday :)

<comment><USER5> I had said in the beginning of my comment that it was a good article. Everything that zappos is doing is nice and it is evidently working for them. If the article was about strictly zappos I would have had no objections. I'm happy to hear success stories too.

What I'm not happy to hear is scoble's agenda. What I objected to was the mudslinging on O'Reilly, for promoting an event which focuses on number of followers and promoting 140 conference where he is a speaker. When scoble itself focuses more on number of followers than community and engage in trash talking O'Reilly (and TechCrunch) for the same, I see lack of credibility.

What scoble cleverly tried to say is "See, Zappos is doing great with twitter without focusing on followers. So come to 140 conference where we can teach you how". Riding on Zappos's success to promote his conference was distasteful IMHO.

</comment></comment></comment><comment><USER7> Its good to study companies like Zappos. They are very successful without being in Silicon Valley.

I hope to take a tour of Zappos in May, when I'm there for rails conf.

</comment><comment><USER8> there are so many stories like this that i've read about zappos... and i've enjoyed every one of them. the link to the video in the post was fascinating (learning about their order fulfillment tech).</comment>


He's Done Remastering Deep Space Nine to Maximum Quality - <AUTHOR> As previously discussed here:
https://news.ycombinator.com/item?id=19453745 https://news.ycombinator.com/item?id=19406060
Joel Hruska has wrung all he can from DS9 and has announced "I'm done":
https://www.extremetech.com/extreme/314653-remastering-deep-space-nine <comment><USER2> Ehhhhhhhhhhh. I'm done-ish. I'm done unless I find a reason not to be done, and since I've got 176 episodes of the show to convert, I figure I'll be poking awhile longer. And I want to build a benchmark. And somebody else may have some amazing idea that shows a still-better method of handling the footage. And I'm still exploring the idea of making some very modest color adjustments because DS9 is a very dark show.

I'm done. But I'm also done _. Because I have zero doubt that I 'll be working on some method to fix _some* problem with some specific episode before too long. I want an automated solution, but I always figured I'd have to do some per-episode optimizing at some point.</comment>


Ask HN: What does the term Hacker mean to you? - <AUTHOR>

My manager seems to think that a Hacker is someone who solely gains unauthorised access into a system and tries not to leave a trace. Causing some malice or ill effect. I think it's evolved much more to include developers and entrepreneurs. What are your views? Discus... <comment><USER2> Hacker in Silicon Valley type circles in recent years come to mean someone who moves fast, break stuff, and makes it work in some cases. In others, someone who is inventive and figures out how to get things to work. I'm sure there are a few others.

Some will proudly wear the badge and call themselves hackers because of the above. Facebook is huge on the term itself, even getting themselves the vanity address 1 Hacker Way.

Unfortunately, a huge chunk of the world does in fact view the word hacker very negatively like your manager and even in some reports I've come across, the authorities you don't want messing with you. For these reasons, I try to stay away from ever calling myself a hacker of any kind. It's a term. It can mean whatever you claim it to but it won't matter if others don't see it that way, it causes confusion, or at the end of the day means nothing to anyone else but you. After all, the only person that cares is the person claiming the title.

</comment><comment><USER3> Hacker for me is someone who knows what he is doing. Hacker is someone that from his solid knowledge at something (not necessarily "software engineering") can achieve his goal. His goal might be gaining an unauthorised access into a system or creating a piece of hardware that will make his life easier, or even having free drinks/snacks from a vending machine.

Having deep knowledge at something and being able to do what you want with it, makes you a hacker.

Note that sometimes there is a confusion with hackers and script kiddies. Script kiddies in short are those with little or no real knowledge use instructions and tools causing damage only.

Hope this helped.

</comment><comment><USER4> To me it's about knowing how a system (of any kind) works and exploring how that system can be used based on what you know.

The important thing is the effect/intent is completely independent of the definition. People inject their intentions into the definition, but on it's own, hacking is all about curiosity.

<comment><AUTHOR> Interesting reply. Thanks.

</comment></comment><comment><USER5> [https://medium.com/p/825de76cc7e9](https://medium.com/p/825de76cc7e9)</comment>


Chrome engineers make a site that showcases annoying browser API features - <AUTHOR> https://theannoyingsite.com <comment><USER2> Arghl! How do I get rid of that horrible techno music, after I closed all the pop-unders?

</comment><comment><USER3> This is exactly what it is</comment>


Tunneling Internet traffic over Facebook chat - <AUTHOR> https://github.com/matiasinsaurralde/facebook-tunnel

<comment><USER2> Hi, I started this project last year, as someone pointed out, we have free access to Facebook ("thanks" to the Internet.org campaign and TIGO, one of the biggest telcos here in Paraguay). It would be nice to port the client to Android (it seems that isn't possible to use tuntap directly) and implement some obfuscation and encryption stuff later. I'm open to any collaborations, I need to clean up some things and write a README and TODO list.

</comment><comment><USER3> An explanation in form of README.md would be nice. It might be a cool project, but without a proper documentation it will get lost.

</comment><comment><USER4> no README.md, no wiki page ???</comment>