How effective are NDAs? - <AUTHOR>

I've been asked to sign an NDA by a company who'd like to use my development services. They already gave me a run down of their entire company strategy post NDA. I'm curious how effective are NDAs anyway? I'm aware that they are only effective if the party chooses to pursue a case, but my intuition tells me that they're more of a preventative smoke screen than a viable document.<p>Thoughts? <comment><USER2> Depends on how powerful each party is.

As an individual or small company, the NDAs you give to others are totally meaningless. You generally don't have the desire/funds to pursue people who violate the contract, and it's often difficult to prove anything.

Large/smart companies will also realize that they have all the power, and they'll include upper limits to the damages they have to pay. One of my companies was recently, egregiously ripped off by one of the 3 largest insurance companies in the world, and they had capped their liability at $100k. Fortunately, there was easy-to-prove malice, so they will end up having to pay us some day.

If you're the one signing the NDA and the other party is large and powerful, you actually could be successfully sued.

So in short: NDAs are powerful when you have the resources to back them up with lawsuits. As a developer, I've signed NDAs and non-competes for many small-to-medium businesses, and I've never been tempted to violate them, so I can't attest to the validity of that situation.

<comment><AUTHOR> Fair enough, in my particular case it's complicated for example - before I even began speaking with this client, I already began coding an application in a particular niche space... it so happens that this client is also in the same space. However, my application is totally different, I just hope that there isn't an issue there because technically, I'm not stealing anything proprietary, and my app is obviously different. I just wonder, how that works.

<comment><USER2> It's very likely that they could pursue you legally for releasing your app in the same space after working on theirs. They might be ultimately destined to lose, but they'd pursue you based on the fact that you don't have the resources to fight them.

So by signing this NDA, you are very likely (in effect) saying that you won't release your app or discuss it. It depends on the exact wording, though.

If your app is important to you, you need do one or more of the following:

1\. Ask the company directly about the release of your app and get it in writing that your app is not their property and that you can release it without violating agreements with them.

2\. Talk to a lawyer

(I'm not a lawyer and am not giving you legal advice)

<comment><AUTHOR> Understood, however a couple points:

1\. He's based in a totally different country, I'm in Canada and he might as well be based in Japan (an example but close enough).

2\. The chances of him discovering the app are slim.

3\. The chances of him finding out the I created the app are even slimmer, and because the app isn't a direct competitor the chances of him looking up who founded the company isn't likely.

I think based on those 3 points, my odds are good. But I truly appreciate the dialog here because it got me thinking.</comment></comment></comment></comment>


The poor are better off when we build more housing for the rich - <AUTHOR> https://www.washingtonpost.com/news/wonk/wp/2016/02/12/the-poor-are-better-off-when-we-build-more-housing-for-the-rich/ <comment><USER2> My hope would be the supervisors for The Mission district as well as Chinatown in SF would read this and take notice. But... given their politics, even if they came to understand it, I don't think they'd come around on this, to the detriment of those they purport to represent.

The politics in SF are weird. Many politicians do superficially irrational things --they look good on the surface, but in effect they are detrimental. On the bright side, the mayors themselves have been rational about attracting business and bringing in development.

If you listen to KQED and their local reporters, it's like they'd like to have SF frozen in time in 1992. As if they'd rather it go down the way of Detroit rather than it become an economic center on the Pacific with the effects they bring ("gentrification" population moves, etc.) They want to eat their cake and have it too. And I understand it. It's very attractive. Let's have a great economy BUT let's have it so all other things stay the same, no disruption, no changes to neighborhoods, etc. Not even China can pull that off. Especially when SF proper is all but 49sq mi. But if we will it that way hard enough...

<comment><USER3> I do not think that San Francisco can in any way be compared to the decline of Detroit.

Poor civic planning, racism and the loss of jobs to non-union Southern plants all paid a part in Detroit's population dropping by two thirds over a fifty year span.

I read a great book over Christmas, Once in a great city by David Maraniss, which covered Detroit at its peak from 1962-1964. Wayne State actually pretty accurately predicted the city's future then but the report was ignored by those in a position to stem its decline.

<comment><USER2> You're right, it's not the same. Detroit is a remarkable exception. But, SF in the late 80's early 90's was not a place you might think would grow. People were leaving the city for other bay area cities. Labor intensive industries were leaving [bedding for one] SOMA, Tenderloin, Market were severely depressed areas. And companies can go --just like they left Detroit; just like they left SF in the late 80s early 90s.

</comment></comment></comment><comment><USER4> Funny thing about this theory: the luxury housing market in San Francisco is in decline, and yet, it's still a seller's market for middle- and down-market properties. [1][2]

This is not surprising to anyone who follows real estate. There's no such thing as "a market for housing" \-- there are several markets, each moving more-or-less independently, based on the net worth of the participants involved. A drop in the prices of units in the Millenium Tower doesn't have much of an impact on the rent of a pre-war hovel in the Mission.

The reason that the author doesn't understand this is because there's a latent correlation at play: the communities with the most high-end construction _tend to have the most construction overall_. But in a city like SF, where _all_ of the new construction would be high-end (if so allowed), all bets are off.

Observing that properties become cheaper as they age is also silly. There are _some_ properties that become cheaper as they age, but there aren't many of them, say, in Pac Heights.

[1] [http://wolfstreet.com/2015/11/08/san-franciscos-luxury- condo...](http://wolfstreet.com/2015/11/08/san-franciscos-luxury-condo-bubble- turns-into-condo-glut/) [2] [http://sfist.com/2015/11/09/at_high_end_sfs_housing_market_f...](http://sfist.com/2015/11/09/at_high_end_sfs_housing_market_fina.php)

</comment><comment><USER5> This is specifically related to the SF Bay Area and there is a link to the source which has a easy-to-use Excel sheet of the data, so feel free to make an awesome geo-mashup:

[http://www.lao.ca.gov/Publications/Report/3345#Technical_App...](http://www.lao.ca.gov/Publications/Report/3345#Technical_Appendix)

Also, the measure is not whether available housing is getting more expensive, but whether low-income households are being displaced.

<comment><USER6> They aren't being displaced. There's a good breakdown of the new construction that's happened in [http://rationalconspiracy.com/2015/05/25/new-york-times- make...](http://rationalconspiracy.com/2015/05/25/new-york-times-makes-up- facts-about-sf-housing/) ; the short version is, new units are replacing vacant lots and abandoned buildings.

<comment><USER7> I think you two are talking past each other because one of you is referring to "being displaced" in the sense of it happening because of new housing being built, and the other is referring to "being displaced" by the lack thereof.

</comment></comment></comment><comment><USER8> I think the headline is only true if the new housing for the rich actually increases the overall supply. Is that actually happening in SF?

<comment><USER6> No one is tearing down old units. People are trying to get new units built, mostly on vacant lots, and getting blocked by incumbent landlords who want to see prices go even higher.

<comment><USER8> Tearing down old units is not enough. The construction works need to actually increase the overall housing stock. If old tenement buildings disappear and are replaced by oversize luxury apartments the overall stock goes down and the situation for the poor is made worse.

<comment><USER9> That trend has been going on in SF for a long time. Dwellings are getting larger steadily over many decades, and the number of people who live in a dwelling has been getting steadily smaller.

<comment><USER10> Do you have any links that show this research?

<comment><USER9> The census tracks people per household. In 1940 this stood at 3.08. In 2010 it was 2.26. A "housing unit" built today simply doesn't hold as many people as it once did.

<comment><USER10> Isn't that the opposite of what you posted above? I thought you were claiming dwellings are getting bigger in SF.

My intuition is that they are getting smaller. The old houses are all subdivided. The new apartments are all small. The exception is some large lofts in SOMA but there are plenty of tiny lofts in China Beach, 5th and Mission, 8th and Mission, 10th and Market.

I'm not saying the average is smaller since I don't have any data only personal observation. It sounds like your extrapolating that 2.26 people are getting more space than 3.08 people were before?

<comment><USER9> Dwellings are getting bigger. All the new construction is larger than average for the given number of bedrooms. Dwellings are larger, and fewer people are living in each of them. Both of these factors are headwinds for the housing market.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> Extremely misleading headline.

The poor are better off when there isn't a housing shortage, because (surprise, surprise) the rich aren't the ones who are going to be forced out.

<comment><USER7> You say the headline is misleading, but you seem to be arguing the same point as it. Could you go into more detail about why you feel it's misleading?

<comment><USER12> Can't take a purportedly urban planning article seriously when it doesn't explicitly mention zoning and density.

"More housing" is a pretty fluff summary of the intricacies of building sustainable neighborhoods.

<comment><USER7> Do you take the original study seriously?

[http://www.lao.ca.gov/Publications/Report/3345](http://www.lao.ca.gov/Publications/Report/3345)

</comment></comment></comment></comment><comment><USER13> There is nothing in the article to justifying the headline. What the headline should have said is "more expensive housing keeps the prices of the expensive housing lower". While true, this has absolutely no reflection on the affordability, the latter being dependent on things like level of income and such. If the cheap housing is removed to be converted to expensive, there is less cheap housing, and it becomes less accessible, because more poor compete for it.

<comment><USER14> And if the total housing stock increases then that means that rich people won't be buying up the poor housing stock.

For every "Rich Person Condo" that is built in SOMA, there is one less "Up and Coming Apartment in The Mission" that gets taken up by a rich techie.

The rich are going to move to the city whether we like it or not. They have the resources, they can buy out whatever place they choose. The only choice we have to make is whether to build housing for them or to just let them displace existing residents.

</comment></comment><comment><USER15> Everyone (myself included) cries about house prices, yet I believe there are many many truly beautiful small cities and towns where you have great internet access, safety and amazingly affordable housing. People just needs to have the guts to move to these places and recruit friends and family and like minded folks to join them.

<comment><USER16> You seem to be forgetting that there are no _jobs_ in those places. Affordable housing isn't worth jack when the only job you can get there is working at Walmart, because there aren't any software development jobs there (for instance).

The other problem is that if you're a young professional, you may very well be single. Moving to a small city or town means you will not get a date, unless you want to date some overweight, uneducated woman who has 4 kids and talks about Jesus and guns all the time. If you're on this site, that probably isn't the right demographic for you. You'd have better luck moving to a foreign country; rural and urban America are _that_ different these days.

<comment><USER17> _unless you want to date some overweight, uneducated woman who has 4 kids and talks about Jesus and guns all the time_

I think if I wanted to pick a comment as the canonical example what is wrong with people on HN this would be the one.

<comment><USER18> His choice of words was harsh but I think he does have a fair point. The demographics of rural areas don't make for good dating opportunities for educated, secular professional young people.

<comment><USER19> Better or worse than San Francisco? (assuming a young male)

I think the last statistic I found was something like 140 single men in the 20s-30s age bracket for every 100 single women.

<comment><USER20> Competition is high, but at least there's a supply.

<comment><USER19> Assuming the women don't enter relationships with more than one guy at a time, this is very literally a case of no supply.

<comment><USER21> The advantages of same sex marriages.

<comment><USER16> Or, the advantage of polyamorous relationships.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER22> what about the rich foreigners buying up the new development for the rich to park their money

<comment><USER23> They don't rent what they bought? That seems dumb from investment perspective.

<comment><USER24> Some don't, and it can make sense. If you rent it out, then you have to be a landlord, which puts you on the hook for repairs, finding tenants, and collecting rent or evicting them. There's always the possibility of tenants trashing the unit to a large degree. Some of these problems become way worse when left untreated for a week, a month, or longer. I can imagine it making sense for foreign investors to leave units empty in these circumstances.

In a similar vein, it's not just foreign investors. It might also make sense to leave the grandparent's house empty while they are in the nursing home if you live further away. If the grandparents don't need to sell the house to pay for retirement, you'd probably come off cheaply enough to leave it empty, pay the property tax and minimal utilities, rather than risk having tenants.

<comment><USER23> > you have to be a landlord, which puts you on the hook for repairs, finding > tenants, and collecting rent or evicting them

I imagine that there are companies that do exactly that for you taking percentage of rent you'd get if you done all that by yourself. I also imagine there are insurances for landlords and/or investors. There's so much money in this when you approach this as investments (as opposed to not renting your grandfather house because it's too much of a bother to clean it out and think about) that I imagine no sane investor lets their property, they just bought to stand empty.

<comment><USER25> Property Management companies can be extremely inept. At the end of my last lease the property managers for my place were fired for letting the owners family home degrade so far. The property managers then tried to pass all the damages off to my roommates and I, but we had extensive documentation of the existing damages.

</comment></comment></comment></comment></comment><comment><USER26> A housing crisis is pretty much always caused by government policies. It would be perfectly possible to build a satellite town up to 50 miles outside a major agglomeration connected by a high-speed transport link, such as a highway or train. People would live up to an hour from their work places. Housing units costing between 15 000 - 20 000 should be affordable even for the very poor. It is important, of course, to make sure that there is enough local retail and entertainment, but that is just a question of not regulating it out of existence. Without heavy-handed government intervention, it would emerge spontaneously. None of this would be economically a problem. The reason why it does not happen, is government interventionism. Seriously, there does not exist a problem in the world that is not made worse by governments.

<comment><USER27> A two hour+ daily commute does not seem ideal.

<comment><USER28> It is if houses were much much cheaper.

Shit I had a 2 hr daily round commute INSIDE San francisco, taking bus from the ocean and then walking to office. Id be beside my self if there was cheap housing an hour commute away.

</comment></comment></comment><comment><USER29> Sounds like [https://en.wikipedia.org/wiki/Parable_of_the_broken_window](https://en.wikipedia.org/wiki/Parable_of_the_broken_window)

</comment><comment><USER30> Not a problem. Just move the poors to Flint and keep SF as low-density housing for uber-douches.

</comment><comment><USER31> Predictable drivel.

How about using international data?

</comment><comment><USER32> It's an absolute crisis. The government should response just as it would in the event of war time profiteering.

<comment><USER33> Or people could move, but sure it's just like war time profiteering.

<comment><USER34> If you're living paycheck-to-paycheck, or even close to it, no, you can't move.

Edit: I'm curious what people are disagreeing with here. Do you think that moving to a new city _doesn 't_ require expenditures?

<comment><USER7> Many landlords would love to be able to evict their tenants and replace them with new market-rate renters, in exchange for paying the old tenants' moving costs. Or even the moving costs plus a year's rent in the new location.

<comment><USER32> Not true in 99% of cases and no one said anything about paying below market rate rent except you...

</comment></comment></comment></comment></comment><comment><USER35> It's rough when the rich own numerous suites they rarely dwell in. They have to give some middle-class loner a basement apartment with discounted rent to meet occupant requirements. Yea, that'll help.

</comment><comment><USER36> You know, it makes sense to me that increasing supply should drive down demand and hence prices. But I frankly don't care.

The entire piece reads like an excuse to not do things for the poor. How about we actually do things FOR THE POOR instead of tossing them scraps from the tables of the rich?

<comment><USER37> Evaluating arguments on their merits seems nice, but mood affiliating feels so good.

</comment></comment><comment><USER38> The language in this is Orwellian. She talks about how new construction is needed and how people are stopping it. Then she points to examples where people are not stopping it, just not letting developers do whatever they want (the dangerous junk developers build near me is insane), then she goes back to saying we need more housing.

No working class housing groups are trying to stop new buildings. They're just asking for the same things groups like these have asked for almost a century.

Article translation: "Hack employee of the Washington Post Co. says wealthy parasite developers and trustafarian transplants should ignore everything long time, mostly working class black people want, they don't matter". Well why not, that's the same message we've been hearing in this election campaign all year.

<comment><USER7> _> No working class housing groups are trying to stop new buildings._

Ahem:

[http://www.savethemission.org/save_the_mission_district_yes_...](http://www.savethemission.org/save_the_mission_district_yes_on_prop_i_endorsements)

</comment></comment><comment><USER39> The data is nationwide. But California proves this NOT to be the case.

Sunnyvale/Mountain View have experienced 4 years of building new housing without limits, yet every year is more expensive. The new units (rental or for sale) are way higher than the average rent or price. (e.g. ALL the new housing being built now in the peninsula is priced at $1MM or more) I fail to see the point of pushing for new unregulated housing here besides saying in 30+ years the middle class will move to housing built for the rich now. Sure... In 30 years the middle class won't exist. (and in SV I give it 12-15 years)

<comment><USER40> It also neglects a huge factor in several hot markets, particularly New York: rich out-of-towners buying apartments as investments, and leaving them unoccupied. The owners aren't living there, and the buildings are typically reworking or replacing former residential units where people actually resided, so the net effect of that new construction is that the stock of housing available to house people who actually live in the city is _reduced_.

<comment><USER41> This problem is so easily fixed via the tax system I am surprised it has not been done already. Just tax high and rebate for residency.

<comment><USER42> Too naive, I'm afraid (as well as, in my opinion, misguided).

Just put yourself in the shoes of the investor. Government announces they're gonna make you pay through your nose if no one "lives" in your apartment.

Leaving aside the dubious morality of such penalization (it's a slippery slope!), is your reaction going to be "Oh noes, I am ruined! Better sell my apartment now."? Or can you think of half a dozen simple, practical ways around it?

I bet you have already.

<comment><USER41> It is very hard to get around residency because someone living at that address needs to pay income tax. You can even rebate back the occupancy tax via the income tax or sales tax systems.

If you don't want to do this you can do random surveys of the suspect property (say any property without a taxpayer) and if it is unoccupied more than a few times and not registered as unoccupied then a big fine results.

Finally, the owner does not need to sell, just have tenants. What you want to avoid is housing sitting empty or only being used as a holiday home a couple of weeks a year.

<comment><USER42> Right. Let's send a secret police on "random surveys" to "suspect properties", checking to see who lives in people's homes :-)

Did I mention "slippery slope"?

You don't have to be a diehard libertarian to find this whole concept somewhat creepy.

<comment><USER41> I hate to break it to you but there are dozen of authorities that can come around and inspect your property whenever they want - this slippery slope has long gone.

I have no problem with someone with the appropriate authority politely knocking on my door to see if I am home if I am claiming a large tax refund for living there. The IRS and its fellow ilk are far more intrusive - the IRS makes my life a pain as it is and I am not even a US taxpayer.

<comment><USER42> Where do you live that "dozen of authorities" can come around and inspect your property at will?

I see where you're coming from, but as is probably obvious, I don't subscribe to your "If you've got nothing to hide, you've got nothing to fear" world view on government surveillance. This seems to be the root of our disagreement here.

[https://en.wikipedia.org/wiki/Nothing_to_hide_argument#Argum...](https://en.wikipedia.org/wiki/Nothing_to_hide_argument#Arguments_for_and_against)</comment></comment></comment></comment></comment></comment></comment></comment>


US citizens warned they face arbitrary arrest in China - <AUTHOR> https://m.dw.com/en/us-citizens-warned-they-face-arbitrary-arrest-in-china/a-54144205 <comment><USER2> Note that US citizens have long been warned about stuff like this, especially exit bans. From the state department in 2018:

> Exercise increased caution in China due to the arbitrary enforcement of > local laws and special restrictions on dual U.S.-Chinese nationals.

> Chinese authorities have the broad ability to prohibit travelers from > leaving China (also known as 'exit bans'); exit bans have been imposed to > compel U.S. citizens to resolve business disputes, force settlement of court > orders, or facilitate government investigations. Individuals not involved in > legal proceedings or suspected of wrongdoing have also be subjected to > lengthy exit bans in order to compel their family members or colleagues to > cooperate with Chinese courts or investigators.

> U.S. citizens visiting or residing in China have been arbitrarily > interrogated or detained for reasons related to "state security." Security > personnel have detained and/or deported U.S. citizens for sending private > electronic messages critical of the Chinese government.

travel.state.gov archived from 2018: [https://web.archive.org/web/20180112180843/https://travel.st...](https://web.archive.org/web/20180112180843/https://travel.state.gov/content/travel/en/traveladvisories/traveladvisories/china- travel-advisory.html)

<comment><USER3> > Chinese authorities have the broad ability to prohibit travelers from > leaving China (also known as 'exit bans'); exit bans have been imposed to > compel U.S. citizens to resolve business disputes, force settlement of court > orders, or facilitate government investigations. Individuals not involved in > legal proceedings or suspected of wrongdoing have also be subjected to > lengthy exit bans in order to compel their family members or colleagues to > cooperate with Chinese courts or investigators.

AKA taking hostages.

<comment><USER4> I mean, is it really such a rare thing to do? I know that in Poland if you have any outstanding and overdue unpaid fines/alimonies/traffic tickets, you will be prevented from boarding a flight out of the country at any airport until those obligations are paid.

<comment><USER5> You're missing the point. People are being accused for crimes they (more than likely) didn't commit and denied basic human rights.

They face a 99% conviction rate in the Chinese courts.

[https://www.theglobeandmail.com/world/article-china- charges-...](https://www.theglobeandmail.com/world/article-china-charges- michael-kovrig-and-michael-spavor-with-spying/)

<comment><USER6> What do you think the conviction rates in western courts are? It's pretty close to 99% as well.

<comment><USER5> You literally just made that up.

[https://en.wikipedia.org/wiki/Conviction_rate](https://en.wikipedia.org/wiki/Conviction_rate)

<comment><USER7> No he didn't. The link itself shows both the lower 2018 percentages from the justice statistics bureau and also mentions the 2012 DOJ 93% rate. No clear explanation is given for the divergence over those years or how it fits together overall but he didn't lie about the source.

<comment><USER5> Re-read the part where they said "western courts".

Also, that 93% number is Federal only. As per the link, the state numbers range (roughly) from the low 60's to the low 80's.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> Not just arrested but tortured, as we've seen with the two Canadian Michaels and the British Simon Cheng.

Edit: British-employed not actually British

<comment><USER9> Aren't the Michaels just retaliation for Meng Wanzhou?

<comment><USER10> "just" retaliation... this makes their detention arbitrary. After Meng Wanzhou was arrested (and placed under house arrest in her multi-million dollar Vancouver mansion), the Chinese government arbitrarily picked two high-value Canadians (an entrepeneur and a Candian diplomat) and threw them into solitary confinement, interrogating them 24/7 in shifts, and giving ostensibly no access to lawyers/representation.

Oh, and by the way, you don't need to be a diplomat or entrepeneur to be picked up. They do it all the time [0].

You can read more about Spavor and Kovrig, the Canadians detained in solitary confinement in China for over a year, here: [https://www.theglobeandmail.com/politics/article-michael- kov...](https://www.theglobeandmail.com/politics/article-michael-kovrig- michael-spavor-canadian-detainees-china-explainer/)

[0] [https://torontosun.com/news/national/123-canadians- detained-...](https://torontosun.com/news/national/123-canadians-detained-in- chinese-jails-foreign-affairs)

<comment><USER11> It's difficult to determine whether it is arbitrary (in the legal sense) because the Chinese government does not communicate on these issues and the Canadian government will obviously not want to spill any beans.

What is plain to me is that a Canadian guy who can live in China at the border with North Korea, can run a business there, and personally knows Kim Jung Un is not just any "entrepreneur"...

Likewise we cannot know exactly what "diplomat" means for someone stationed abroad (including Afghanistan, and Beijing).

I'm not defending the Chinese government but these 2 guys were clearly not arbitrarily picked, it's a little disingenuous to claim that they were.

<comment><USER10> > I'm not defending the Chinese government

Strange, because this comment, as well as the vast majority of your comment history, expressly come to the defence of the CCP.

Being an entrepreneur nor a diplomat is not probable cause for an arrest, and the opaqueness of the CCP on all their criminal cases is by design for this very reason; it's a smoking gun, not a smoke screen.

You're right that these two weren't entirely arbitrarily picked, they chose two people high-profile targets as you normally would in a hostage situation.

</comment></comment></comment></comment></comment><comment><USER12> I've been to China a few times, most recently in late 2018. I traveled overland by train from Beijing to Xian and then to Shanghai. Everything as a non-chinese citizen was just harder than it was when I visited in 2014. Buying train tickets was a massive hassle without a Chinese ID. Paying for anything was difficult because many places no longer take cash and you can't really get a proper WeChat account as a foreigner.

The entire time I just couldn't shake the feeling that I was living in a dystopian future. Security around Tiananmen Square in Beijing was so overdone it would almost have been funny if it wasn't scary. Facial recognition scans on the metro system. In Shanghai they use facial recognition at crosswalks and then put up the name and picture of violators on a huge screen at the intersection (I can only assume they also deduct from their social credit score).

If this is the direction China wants to continue to go in, I no longer want anything to do with it. As long as the CCP is in power I won't be visiting again. And now that includes Hong Kong sadly.

<comment><USER13> Posted 10 mins ago and already downvoted. Do HN not have algos to stop this astroturfing behaviour? It's becoming very prevalent for anything that criticises the CCP.

<comment><USER14> [removed]

<comment><USER15> These isn't something vague, we're talking about a specific authoritarian government who's already been caught manipulating social media.

[https://about.fb.com/news/2019/08/removing-cib- china/](https://about.fb.com/news/2019/08/removing-cib-china/)

<comment><USER14> [removed]

<comment><USER15> We know manipulation is occurring, and to me that's an argument for greater transparency and openness so we can tell when it's happening. I don't agree with the "head in the sand" approach where manipulation is glossed over, ignored or rebranded under terms like "online reputation management".

That being said, the "anyone who disagrees with me is a shill" attitude is just as poisonous to online discussions as the shilling itself.

</comment></comment></comment></comment></comment></comment><comment><USER16> This is bad for Canadians recently, with many arrests and accusations of spying, all because of political retaliation over that rich lady

<comment><USER17> For Australia as well. They recently gave an Australian citizen a death penalty sentence in retaliation for the ongoing conflict between Australia and China.

At the rate things are going, in another five years nobody from a liberal nation will be able to safely visit Xi's new hermit kingdom.

It's a shame. I always thought it would be nice to visit Hong Kong.

<comment><USER18> > Karm Gilespie, had been sentenced to death for attempting to smuggle 7.5kg > of methamphetamine through a Chinese airport in December 2013.

The death penalty threshold is 50 gram for methamphetamine in China, in absence of other 'good behavior' such as collaboration, etc. If any Chinese citizen attempted to smuggle this much drugs into China, it's going to be the same fate, except maybe faster.

Drug dealers in China are executed for much less. To paint that as 'retaliation for ongoing conflict' is laughable.

<comment><USER19> So the police can conveniently plant an amount of drugs that fits in their hands quickly into your pocket and you get the death penalty?

Sounds like a super convenient way to kill off politically inconvenient people.

<comment><USER16> One interesting part is that, when the authorities lose the people's trust, people can't rely on what they say to be accurate and you end up second guessing everything they say or do. It's actually an important Chinese (Confucian/daoist-sometimes) idea to emphasize the importance of legitimacy and that the working of society is relies on it

</comment></comment></comment></comment></comment><comment><USER20> I've visited China nearly every year for the past 6 years and never had any sort of issues. I thought all the security personnel I encountered at the airport, immigration, subway stations etc in China were very professional and respectful. In contrast to one of the last times I was returning from China to the U.S. and witnessed a highly unprofessional U.S. CBP officer who was yelling and berating incoming international visitors because they couldn't understand his English, which was infuriating. All this to say that if you want to visit China recreationally (outside of pandemics), you'll almost certainly be fine and treated well.

*Edit: I'm just offering my personal experience here for others to consider amongst other sources of information.

<comment><USER10> I'm not sure how you can make that assurance.

Fellow Canadians have had to watch as an increasing number of Canadians are arbitrarily detained and held in torturous conditions.

These Canadians range from high-profile and wealthy[0] to low-profile [1,2], and the conditions in which they are detained are an affront to human rights.

[0] [https://www.theglobeandmail.com/politics/article-michael- kov...](https://www.theglobeandmail.com/politics/article-michael-kovrig- michael-spavor-canadian-detainees-china-explainer/)

[1] [https://www.theglobeandmail.com/world/article-canadian- sente...](https://www.theglobeandmail.com/world/article-canadian-sentenced-to- eight-years-in-jail-by-china-renounces/)

[2] [https://torontosun.com/news/national/123-canadians- detained-...](https://torontosun.com/news/national/123-canadians-detained-in- chinese-jails-foreign-affairs)

</comment></comment><comment><USER21> I get the impression that we're now in an undeclared Cold War with China.

<comment><USER22> How does a country declare a Cold War, anyways?

<comment><USER23> Serious answer: its national media starts saying there is one.

</comment></comment></comment><comment><USER24> Use this to negotiate hazard pay with your company when traveling to China. There's money to be made by leveraging the difference between perceived and actual risk.

<comment><USER25> > There's money to be made by leveraging the difference between perceived and > actual risk.

You think the risk is not that big?

<comment><USER26> How big is "not that big"? What do you think it is?

</comment></comment></comment><comment><USER27> Deleting this comment - I will email dang.

<comment><USER28> I've also noticed a large influx of new accounts posting low quality, politically charged comments in the past few months. You can find plenty of those in this thread.

I think its pretty likely that we're headed for an extinction type event for upvote based "anonymous" message boards like HN, reddit ... or at least a significant decline in their usefulness

<comment><USER27> What's a good alternative architecture that promotes quality content/comments and prevents regression short of manual compilation, moderation and curation?

Edit: Jotting down some ideas:

1) Occlude/Delude/Fog-up the votes for newly created accounts. Mature accounts get to see gray posts.

2) Consesus amongst mature accounts has higher weight over newly created accounts.

3) Penalty for being flagged is higher for mature accounts as with power comes responsibility of good behavior.

4) Karma should have more meaning that just a tally of points. Upvoting costs Karma (-1 from balance). So, people are more careful and have to strongly agree to upvote. Probably some caveats and downsides here.

5) Buffer out the oscillations of upvotes/downvotes. Sort of like a mass- spring-damper system.

6) Hire moderators that are vetted and publicly funded by HN members.

7) Verified accounts with work email or some other means. These accounts would have the highest weight in anything they do.

<comment><USER29> I think you essentially need something pseudonymous and reputation based. Something where getting banned has a reasonably high cost in terms of re- acquiring reputation required to post.

</comment></comment></comment></comment><comment><USER14> [removed]

<comment><USER30> This is a very uncharitable and pedantic reading of it. If there were a law to "kill political dissidents of party x" then enforcement of that law would technically be enforcement of the law, but it would certainly not be for the purposes of maintaining law and order in the manner that the term is commonly understood to mean.

<comment><USER14> [removed]

<comment><USER30> Law and Order the euphemism is not the same thing as Law and Order as it's commonly understood, even if the euphimistic usage appears more often in modern politics than the common usage. It's unfortunate that euphemisms dillute meaning (and it's probably unfortunate that they must exist at all), but that doesn't mean the actual meaning no longer exists.

<comment><USER14> [removed]

</comment></comment></comment></comment></comment><comment><USER31> The current administration has been bullying China incessantly. I'm not going to say if the reasons were justified or not because it's complicated, but we shouldn't be surprised by China's response. It's not like we're actively trying to improve relations either.

<comment><USER8> China's state-sponsored IP theft has been called the largest transfer of wealth in history. I'm honestly curious what the best diplomatic strategy is to combat it, other than declaring war.

<comment><USER32> I doubt this claim very much. How was such a calculation made?

<comment><USER8> It was in the book "Dawn of the Code War".</comment></comment></comment></comment>


Colleges That Offer Best Return on Investment - <AUTHOR> http://blogs.wsj.com/economics/2010/06/30/top-20-colleges-that-offer-best-return-on-investment/

<comment><USER2> The article's author overstates the meaning of these results. (1) It doesn't take into account financial aid (e.g., the average actual price paid by students), and merely reflects the sticker price. This likely means that the top schools, which are quite generous with their financial aid, are being undervalued... BUT (2) On the other hand, it has no ability to disentangle correlation from causation. Viz, perhaps those who got into those colleges would have done quite well even had they decided not to apply there. This "study" has no ability to answer that question, and therefore has no real ability to evaluate ROI - the main thing they claim to have evaluated.

Don't get me wrong, these numbers are fascinating, but the claims made in the article about what these numbers demonstrate are completely fallacious.</comment>


Gusto Went for Women Engineers:  How It Worked Out - <AUTHOR> https://www.forbes.com/sites/susanadams/2018/02/13/how-tech-unicorn-gusto-hired-women-engineers/#164295b36949 <comment><USER2> How can they title the article "How It Worked Out" with zero discussion of subsequent performance? This push for diversity is usually justified by a statement that diverse teams automatically produce superior products. Wouldn't they want to at least mention how it's going now that they have improved the team so much?

<comment><USER3> I've noticed a trend in headlines toward devaluing the word "how". "How X did Y" is really an article that simply states "X did Y", with none of the details you'd expect for a "how" article. It's frustrating because it makes it harder to find the legitimately useful "how" articles.

</comment></comment><comment><USER4> This seems realistic and fair: a goal tied to the percentage of women in CS and a clear cap on duration from the outset.

</comment><comment><USER5> Isn't this just treating the symptoms to some extent? Sure there is a gender bias in hiring and salaries for women. But isn't the big elephant in the room the socio-economic and behavior bias created at a young age at homes and schools where gender roles are assigned very early?

Women didn't just drop out of the Software tech force in the 80's. What about all those personal computers marketed and targetted towards young nerds(ugmmmm boys)?

Software needs to be taught at grade level in an interesting way for people to catch on early. Natural biases and inclinations are just random noise unless we as people perpetuate them.

<comment><USER6> That's why they set their goal based on the proportion of female computer science graduates. If all you hire is CompSci grads, then your hiring proportion matching the graduate proportion is equality, no?

The gender disparity with interest in tech obviously starts from a much younger age, but that's a separate issue to address.

> But isn't the big elephant in the room the socio-economic and behavior bias > created at a young age at homes and schools where gender roles are assigned > very early?

So yes!

</comment></comment><comment><USER7> When a company like Gusto invests in and reaches equal gender representation, does it make it harder for smaller firms to do the same, or does it help more women enter the engineering workforce?

</comment><comment><USER8> If talented women are being overlooked just because they are women, doesn't it make sense to seek them out and give them an opportunity to reach their potential? If I look at a talent pool and see of those who haven't been hired yet are mediocre men as the best of the men have been hired and enormously talented women who haven't been hired because of sexism in the industry, I'm going to hire the women based on talent.

<comment><USER9> The problem is that the situation you just described is a fiction. Most engineering divisions hire on talent, irrespective of gender, so the talented woman would have been hired over the mediocre man by the system which pays no attention to gender at all.

<comment><USER10> Perhaps so. But perhaps the talented woman also experienced a hostile or otherwise undesirable work environment and leaves after a short period of time. Or leaves the industry altogether.

<comment><USER8> A Baby Bell utilities company lost a court case in the 60s which requires them to hire women in the field. Their argument was that women can't handle the heavy tools. After losing they purchased aluminum ladders to replace the heavy wooden ladders and wrenches with longer handles so they had more leverage being easier to use. A reporter interviewed some of the men about the changes and asked what they thought. One man replied, "Everything is so much easier now. I wish we did this a long time ago."

Edit: I heard this story in the 80s when I was a kid. It may be real or not, but the point is the same. I'm still looking for the FWD: FWD: FWD: but I've never heard it again since the mid 80s.

<comment><USER11> > A Baby Bell utilities company lost a court case in the 60s

No it didn't; the Baby Bells were created by the 1984 breakup of AT&T.

EDIT: As a sibling post provides details, the error was calling it a "Baby Bell"; the essence of the story, at least the lawsuit itself, is true, but Southern Bell was a pre-breakup Bell Operating Company (an AT&T subsidiary), not a post-breakup Baby Bell. [Errors like pointing to a Baby Bell 20ish years before they existed are _often_ , but as this incident shows not _always_ , signs of folk mythology.]

</comment></comment></comment></comment></comment><comment><USER12> I have said before that we need the equivalent of a Rooney Rule for tech. I don't know if this constitutes that.

Paying attention to how things are worded is a good first step. It fits with my experience of using less exclusionary language instead of intentionally inclusionary language. But the article fails to paint a compelling portrait of a company that genuinely has it figured out. It is not possible for me to determine if this is a defect in the writing or an indicator that they are running an experiment and don't actually know what they are doing with some kind of confidence (in the statistical sense).

[https://en.m.wikipedia.org/wiki/Rooney_Rule](https://en.m.wikipedia.org/wiki/Rooney_Rule)

</comment><comment><USER13> It's one thing to hire. Let's check back in three years and see how many have stuck around.

<comment><USER14> Anectodally, women may be less likely to switch jobs if they're satisfied with their work environment.

<comment><USER15> ...isn't that pretty obvious (and universal)?

<comment><USER16> Not necessarily. For example I am happy in my current job (it is, without a doubt, the best job I have ever had) but don't get paid anywhere near what I would like to so am actively looking for new work. If rdlecler1's anecdote is true a woman would be more likely to stay and only start looking to leave once the job starts making her unhappy.

</comment></comment></comment></comment><comment><USER17> > Initial steps included writing job descriptions that avoided masculine > phrases like "Ninja rock star coder."

When my fellow male engineers see that in a job ad, they avoid it like the plague.

> Now that 17 of Gusto's 70 engineers are female.

After a focused campaign to hiring women, Gusto's results are nowhere near 50/50\. And they had to go out of their way to hire women and focus resources to attracting women.

The male engineers at Gusto were no doubt happy to see diversity, especially since Gusto did not drop their standards just because an applicant is female.

<comment><USER18> Not so much "masculine" as juvenile.

<comment><USER19> Both those terms are often accurate and frankly they're correlated.

</comment></comment></comment><comment><USER20> I (male) had a online coding challenge with a woman engineer from Gusto. I could literally feel her disinterest through the phone. Now I understand why.

</comment><comment><USER21> [My previous comment confused Zenefits with Zenpayroll, and i've removed it to avoid further confusing the issue]

<comment><USER22> You're referring to Zenefits. Gusto was originally ZenPayroll. The Article mentions it "At the outset Gusto even had a similar name, ZenPayroll, which it changed in 2015 when it started offering a more complete selection of employee-tracking software."

<comment><USER21> thank you for correcting me!

</comment></comment></comment><comment><USER23> > Gusto's women-only recruiting effort lasted six months. It stopped, Kim > says, because "we exceeded our goals." In 2015 Gusto was trying to hit 18% > women engineers, the proportion majoring in computer science as > undergraduates, according to the National Center for Education Statistics, > and it reached 21%.

This is a bit disappointing. It reminds me of Ruth Bader Ginsburg's answer to "how many women on the Supreme Court would be enough?"

"When there are nine."

Men have been in a position of disproportionate power for so long it's unimaginable to be on the other end of the spectrum, but shooting for 18% representation of women will only uphold the current gender iniquities.

They should have aimed higher.

<comment><USER9> If they hired more than the proportion with which they were graduating or even available at, that would not make any sense whatsoever, and might not even be numerically possible...

<comment><USER23> They state in the article that they exceeded it.</comment></comment></comment>


It's Time to Switch to a Privacy Browser - <AUTHOR> https://www.wired.com/story/privacy-browsers-duckduckgo-ghostery-brave/ <comment><USER2> This is sad. This twisted misapprehension of privacy is worse than useless, bit it's so ubiquitous that many people are coming up believing that this rubbish is what online privacy actually means.

Privacy is not about preventing tracking, hiding your identity, disguising your location, or any of the other useless bullshit that these products are all about.

Privacy is about building and curating your online identity to contain the information you want to disclose, and to not contain the information you want to keep private. Maintaining your privacy is about exercising restraint when communicating in forums, when filling out profile information, when posting pictures, or otherwise communicating online.

It's about disconnecting activity from your curated identity as appropriate, but it's not about disappearing entirely; that way leaves an information vacuum for others to fill in ways you may not agree with.

Most of all, it's not about distrust of corporate entities or other concentration of power. It's about learning how to interact with them in ways that benefit you and avoid harm. You don't need to hide from all-powerful watchful eyes, you just need to be reasonably mindful of everyone else who is there online with you, and avoid painting a picture of yourself that you may later regret.

<comment><USER3> This argument presupposes that users are informed about what is being tracked, what that data can be used for and the user consents. None of these are true. Hell I bet most web devs embedding the tracking script la even understand the full implications of them. To make it worse, The big players are constantly adopting darker and darker patterns to inject more and more tracking.

</comment></comment><comment><USER4> Does anyone know why iOS is the only major platform where the Tor browser isn't available? Is it an app store restriction?

<comment><USER5> All web browsers on iOS are required to use WebKit. I think they're actually restricted to a webview and don't really have any control over the underlying WebKit

<comment><USER6> "Onion Browser" seems to be able to do it...?

[https://apps.apple.com/us/app/onion- browser/id519296448](https://apps.apple.com/us/app/onion-browser/id519296448)

(i'm not implying they don't use WebKit, but i meant they provide a browser that uses Tor.)

<comment><USER7> Which is not at all the same thing as providing "the Tor browser".</comment></comment></comment></comment>


Ask HN: How do I flag a(n) HN post for later reading without upvoting? - <AUTHOR>

Reddit separates the act of voting from saving a submission. Is there a way to do that on HN?
Frequently, I want to flag a submission for later reading, but that doesn't necessarily mean I want to endorse it.
I could always add to my burgeoning browser bookmark pile, of course.
Thanks! <comment><USER2> Instapaper and Pocket have 'read later' browser buttons. That keeps is somewhat separate from your bookmarks and still in sync across multiple browsers/devices.

<comment><USER3> Yes, it shows a "save to pocket" link, for instance.

</comment></comment><comment><AUTHOR> I was hoping to keep the saved content within my HN account, but these options are great food for thought.

Thank you!

</comment><comment><USER4> browser bookmarker folder for HN posts?</comment>


Colorbox by Lyft Design - <AUTHOR> https://www.colorbox.io/ <comment><USER2> Discussed yesterday (different URL):

[https://news.ycombinator.com/item?id=19800718](https://news.ycombinator.com/item?id=19800718) (58 points, 9 comments)

</comment><comment><USER3> Cute, but people need to stop using HSL and similar color models for this kind of thing. It is horrifically bad for it, designed by software researchers in the 1970s who had very slow computers and as far as I can tell little background in color vision, color reproduction, visual art, or graphic design, and then passed down through the generations with very little thought along the way.

A human being who is not already an expert should essentially never be looking at numerical values of "R", "G", "B" or these trivially derived quantities of "H", "S", "L", "V", etc. and trying to assign meaning to those numbers. It just results in confusion.

A visual color picker interface (with no numbers involved) is pretty much the only justifiable context where HSL or HSV ever makes sense. And arguably not there either.

[https://en.wikipedia.org/wiki/HSL_and_HSV#Disadvantages](https://en.wikipedia.org/wiki/HSL_and_HSV#Disadvantages)

<comment><USER4> _people need to stop using HSL and similar color models for this kind of thing_

What is your suggested alternative?

<comment><USER3> Munsell, NCS, OSA-UCS, CIELAB, CIECAM02, CAM16, ICtCp, Jzazbz, ....

Pretty much anything designed by color experts from the past 100 years would be better.

Here's an implementation of CAM16 I was making today, [https://observablehq.com/@jrus/cam16](https://observablehq.com/@jrus/cam16)

</comment></comment></comment><comment><USER5> Okay this is pretty neat. I tried to create an palette that doesn't work well together but from what i have tried it seems almost impossible. I'm honestly amazed! Great work.

</comment><comment><USER6> An interesting read regarding gradients and color palettes is the concept of perceptual uniformity.

1\. [http://www.hsluv.org/](http://www.hsluv.org/)

2\. [https://programmingdesignsystems.com/color/perceptually- unif...](https://programmingdesignsystems.com/color/perceptually-uniform- color-spaces/)

</comment><comment><USER7> I am not a designer but can this thing help you pick colours for your UI if you give it start and end colours and number of steps (ie how many colours you want) ?

<comment><USER3> If you want to pick colors for your UI, you should first design in grayscale, then add color (while preserving the perceived lightness using a reasonable color model like CIELAB or CIECAM02 or ...).

This tool is likely to create confusion and yield poor results in the hands of someone who isn't already an expert (either through studying human vision or long practice as a visual artist), because the numerical values sound important and technical, but in the HSL color model are actually rather arbitrary and unrelated to human vision.

<comment><USER7> Thanks for answering.

> This tool is likely to create confusion and yield poor results in the hands > of someone who isn't already an expert (either through studying human vision > or long practice as a visual artist)

Where can I study human vision. Could you point me to some books/ resources ?

<comment><USER3> [https://www.handprint.com/LS/CVS/color.html](https://www.handprint.com/LS/CVS/color.html)</comment></comment></comment></comment>


Estimating the chances of something that hasn't happened yet - <AUTHOR> https://www.johndcook.com/blog/2010/03/30/statistical-rule-of-three/ <comment><USER3> This reminds me of a different "rule of 3": If you want to compare two things (e.g., "is my new code faster than my old code"), a very simple approach is to measure each three times. If the all three measurements of X are smaller than all three measurements of Y, you have X < Y with 95% confidence.

This works because the probability of the ordering XXXYYY happening by random chance is 1/(6 choose 3) = 1/20 = 5%. It's quite a weak approach -- you can get more sensitivity if you know something about the measurements (e.g., that errors are normally distributed) -- but for a quick-and-dirty verification of "this should be a big win" I find that it's very convenient.

<comment><USER4> Man, I wish I understood frequentist statistics to know if your reasoning makes sense. Bayesianly, if O = "the XXXYYY ordering", and F = "algo B is faster than algo" A, then

        P(F|O) = P(O|F) * P(F) / P(O)

then... what? There isn't even a clear P(O|F) likelihood without making assumptions about the process' noise. If the measurement is very noisy compared to the gain, then XXXYYY is just dumb luck, and doesn't tell you anything. If there is no noise at all, then just getting XY is enough to make a decision.

<comment><USER5> The OP is making some short cuts. 1/(6 choose 3) means that they start with the assumption that every possible ordering is equally likely. If that is the case, then the odds that XXXYYY pops out is 5%. What happens if we change our assumption? Let's assume that XXXYYY is _more_ likely. This would imply that algo B is faster than algo A. If we assume that any of the other (or all of the other) combinations are more likely, then this _reduces_ the odds that XXXYYY would pop out.

This means that either we had it right (algo B is faster than algo A), or the result we saw was at least as unlikely as we predicted. That's what it means to have a "confidence interval".

I'll leave the Bayesian version to someone else because I don't really trust myself to do it.

<comment><USER6> > _they start with the assumption that every possible ordering is equally > likely_

If the times in each run are independent, this assumption is (for _some_ distributions) the weakest form of "X is not faster than Y."

For a distribution where this is not the case: assume

\- X always runs in 999 seconds, and

\- Y runs in 1000 seconds in 99% of runs and in 0 seconds the other 1% of the time.

Then XXXYYY is a very likely ordering (~97% chance), though Y runs faster than X "on average". (Not in median or mode though.)

For a more concrete example: say you have two sorting algorithms, one that is a little slower most of the time, but worst case O(n log n), and another that is usually a bit faster but can be O(n^2) on pathological input.

</comment></comment></comment></comment><comment><USER7> If the author is reading this: When someone provides you a pro-bono translation, by all means credit them, but _do not_ let them host it on their own site.

Frequently, they are siphoning your PageRank. They will eventually replace the translation with monetized content of their choice. A good translation takes work! If you got a translation for free, why should you believe it's good, or that it has no ulterior motive?

A firm called "WebHostingGeeks" used to do this all the time. They would offer free translations of blog posts, into languages the author probably didn't speak (and they didn't either, they were just using machine translation). They'd ask authors to link to the translation on their site, and over time they would add their SEO links to the translation.

I first noticed this when WebHostingGeeks offered me a Romanian translation of ConceptNet documentation, my roommate spoke Romanian, and he said "maybe I'm not used to reading technical documentation in Romanian but I think this is nonsense".

<comment><USER8> Yeah this is weird. Also, if OP doesn't speak Italian themselves, how can they attest to the quality of the translation?

<comment><USER9> I have a couple L2's that I mostly speak and read, virtually no writing - <USER2> grammar is poor and my vocabulary is limited. Because of this I would feel uncomfortable/embarrassed to translate any tech stuff I wrote to it. However, I could read someone else's translation and know if it's way off the mark. Also, I have plenty of friends that speak the language natively, so I could ask them to review. If they say it's good, I'd vouch for the translation because I trust them (even if I couldn't read it at all).

<comment><USER8> Oh ok cool :)

</comment></comment></comment></comment><comment><USER10> What the author glosses over somewhat is the method of sampling. If you read the first 20 pages, find no typos, and use this rule to arrive at 15%, that could be way off. He's assuming the risk of typos are evenly distributed when there's a lot of reasons it may not be. For example, the first half of the book could've been more heavily proof-read than the latter half. It's not out of the question that editors get lazier the farther they get into the book.

If you were to _randomly read 20 pages in a book_ and find no typos, 15% probability makes more sense.

It's understandable to not mention this in a short blog post about the rule of three, but never forget that when you're interpreting statistics...how you build your sample matters.

<comment><USER11> Doesn't the book example need to factor in the book length? If you read 20 pages, and it's 20 pages long, you should be 100% confident, but if you read 20 pages and it's 5000 pages long, your confidence should be near 0.

<comment><USER10> Not in this case, because the probably being estimated is "the chance a single page could contain a typo" and not "the chance the rest of the book could contain a typo."

To do the latter, you'd need to know the length of the book, yes.

</comment></comment></comment><comment><USER12> I've seen a very similar problem to this referred to as "black swan events"[1]. The whole point is you can't actually compute it. You see the period, it's there. What he's doing here isn't science, it's a guess.

As others have pointed it, it's rather rare that events happen with perfectly distributed probability (probably the opposite). For instance, the chance of being in an accident is much higher if an accident just occurred right next to you. It's almost way more likely to get sick, if others are already sick. In fact, although I don't have any statistics to back me up, I'd guess that most events happen in clusters (including spelling errors, or when you test for perfect pitch in children, when you go to the music class).

This is essentially a guess, and it's better to say you don't know than guess wildly.

[1] [https://en.wikipedia.org/wiki/Black_swan_theory](https://en.wikipedia.org/wiki/Black_swan_theory)

<comment><USER13> >The whole point is you can't actually compute it.

You can't compute the probability. But you can compute an __upper bound __on the probability with a reasonable confidence, which is what the author is doing here.

This is standard statistics and might be useful in some cases. It does assume that the events are independent, but this is a pretty standard assumption that you have to check whether it applies in your case, or at least approximately applies in your case.

>What he's doing here isn't science, it's a guess.

Which is a pretty big thing in this subfield mathematics called statistics.

</comment></comment><comment><USER14> So basically the '3' comes entirely from the choice of a 95% confidence interval. If you want a 99% confidence interval it's instead the 'rule of 4.6', which doesn't roll off the tongue as well.

<comment><USER15> You could do the 'rule of 5' though and have a confidence of 99.3%, which is pretty close to 99.

<comment><USER16> If you're truly estimating, though, wouldn't you use a 50% confidence interval — which gives you the rule of 1 (chances of the thing being true are 1/n — if you've seen 20 pages without a typo, chances are 1/20 that there's a typo on a page, with 50% confidence)?

<comment><USER17> So, for a 50% confidence interval, you could look at the first word -- if it is a typo, boom, done. Otherwise, flip a coin.

This stinks. Estimating is about approximating an answer with low information -- it's about efficiency of using data, not _only_ doing better than guessing.

<comment><USER16> Sorry, I meant once you've examined n trials, what is your 50% confidence interval about the odds for a single trial? I think it would be 1/n.

</comment></comment></comment></comment></comment><comment><USER18> Intuitively, this seems like an obverse of the "optimal stopping problem." [https://en.wikipedia.org/wiki/Optimal_stopping](https://en.wikipedia.org/wiki/Optimal_stopping) or the subset of the Odds Algorithm ([https://en.wikipedia.org/wiki/Odds_algorithm](https://en.wikipedia.org/wiki/Odds_algorithm)) where optimal stopping point to find the highest quality item in a sample is essentially N * 1/e = 0.368...

Handwavily, this resembles the rule of three, where we could probably say instead, "the rule of 1/e," because both of these appear to be artifacts of the same relationship and same type of problem.

</comment><comment><USER19> So, according to this rule, if I wait for 10 minutes for a bus to come and none does, and then I wait for another 10 minutes for an alien invasion and none happens, the two have the same upper bound on their probability?

Or are we going to start talking about priors, on buses and alien invasions, in which case the rule of three is not really useful? If I want to know how likely a specific book is to have typos, can't I just go look for statistics on typoes in books, and won't that give me a better estimate than a "rule" that will give the same results no matter what it is that it's trying to model?

<comment><USER20> This is the scenario where you know _nothing else_ other than you waited 10 minutes for this event and it didn't happen.

Sure, if you have more data, then you can get much tighter bounds for your estimate.

<comment><USER19> You always have more data -background knowledge- unless you've only existed in those last 10 minutes.

And if something has really never happened before, like my alien invasion example, what have we learned by applying the rule of three?

Honestly- perform the experiment yourself. Wait for X time, then calculate 3/X. Do you now have an upper bound on the probability that an alien invasion will happen?

<comment><USER20> Yes, I do have a reasonable (95% confidence, as per article) upper bound on the probability - as I haven't noticed an alien invasion during my lifetime, it seems reasonable to conclude that noticeable alien invasions are very rare events and happen less frequently than once every decade. Possibly _much_ less frequently, possibly many orders of magnitude less frequently, possibly never, but that's how upper bounds work.

<comment><USER19> An upper bound of "maybe, who knows" is not useful or informative enough to make a whole "rule" about it.

</comment></comment></comment></comment></comment><comment><USER21> The rule of three requires quite a lot of assumptions about the nature of the phenomena.

Or as Taleb says it:

> Consider a turkey that is fed every day. Every single feeding will firm up > the bird's belief that it is the general rule of life to be fed every day by > friendly members of the human race "looking out for its best interests," as > a politician would say.

> On the afternoon of the Wednesday before Thanksgiving, something unexpected > will happen to the turkey. It will incur a revision of belief.

<comment><USER20> It certainly does not require unwarranted assumptions, the proposed approach is consistent with the well-known turkey scenario.

From the experience of feeding a turkey can infer that being slaughtered is a rare event, and that the likelihood it happening exactly tomorrow (without having access to a calendar) is not necessarily 0, but is below a certain rate - and it's definitely not likely to happen three times in the next week. Surviving for 180 days is reasonable justification to assume that, on average, turkeys get slaughtered less frequently than every 60 days, i.e. that the likelihood of Thanksgiving suddenly arriving tomorrow isn't 50% but rather something below 2%.

<comment><USER22> Completely wrong. Continued survival of given that gives no information at all of survival rates and its distribution. This is because variability of data input is extremely low so mutual information between each of the days is vanishingly small.

This is why a good experimental design will observe a measurement with some expected variability.

An even better trick is the sleeping beauty problem. To solve it you need external information.

</comment></comment></comment><comment><USER23> I wouldn't use a probability density of typo's per page, but instead the probability that a word is spelled wrong.

Then it's like drawing marbles from a vase containing an unknown proportion of blue and red marbles.

I would use the formula (M+1)/(N+2) where N is the number of words, and M is the number of mistakes. Note that for a large corpus (M+1)/(N+2) approaches -> M/N, so we recover the frequentist probability.

Also note the author (John D Cook) correctly expresses intuitive doubt that 0 typos / 20 pages can not be construed as certainty of no mistakes. Similarily, seeing a mistake in every word in a subset does not guarantee that all words in the book will have a typo. Let's look at the modified formula (M+1)/(N+2) in these cases: if we observe no typos (M=0) in 1000 words, it estimates the typo probability as (0+1)/(1000+2)=1/1002 != 0, so we can't rule out mistakes. Similarily if all words were typos (M=N) for the first 1000 words we get 1001/1002 != 1, so we can't be sure every future word is a typo.

Check out Norman D Megill's paper on "Estimating Bernoulli trial probability from a small sample" where the formula (M+1)/(N+2) is derived, on page 3 it appears:

[https://arxiv.org/abs/1105.1486](https://arxiv.org/abs/1105.1486)

</comment><comment><USER24> > If the sight of math makes you squeamish, you might want to stop reading > now.

Sigh…another article normalizing the concept that math is something it's OK to be uncomfortable about…

</comment><comment><USER25> I'd be interested to know how to estimate things that are very rare or don't have a normal distribution.

For instance, let's say I have a bold plan to protect us from meteor strikes that will cost $100B. How would a person make a decision about whether that's a good trade-off or not? And how would a mathematician help them make that decision?

How would it change for more complex cases, like a shield to prevent nuclear ICBM warfare which has never happened, but we all are worried about?

<comment><USER13> Simple answer, this is a rough upper bound. The more information about your problem you have, the better you will be able to model the probability distribution and the better priors you have on it happening.

</comment></comment><comment><USER26> In the example given, the author says that the odds of a given page having a typo is _less than 3 /20_. Sure, but if we don't want a range, but an _exact number_? That sounds like a more interesting challenge to me.

Formal statement:

\- You have observed N events, with 0 occurrences of X

\- Someone wants to make a bet with you about the likelihood of X happening

\- Once you've quoted a number, your counter-party then has the option of making an even bet about whether the actual likelihood is greater than or less than your prediction

Eg: If you predict 3/N using a 95% confidence interval, then 95% of the time, the actual likelihood will be less than 3/N. Your counterparty will then win the bet 95% of the time, simply by predicting it to be lower.

Your ideal strategy would be to quote a likelihood which is over/under 50% of the time, not 95%.

Ie, you want to pick E such that 50% of the time, it matches the observation you've made (no occurrences), and 50% of the time it does not.

E^N = 0.5

N log E = log 0.5

log E == log 0.5 / N

E = 0.5^(1/N)

For the example given, that comes out to 0.966. Ie, there's a 96.6% chance of no typos in a given page. Across 20 pages, this comes out to 0.966^20 => 50% chance of no typos. If your goal is to quote the single best estimate which can hold up well in a betting market, I believe this would be the ideal strategy

<comment><USER27> [https://en.m.wikipedia.org/wiki/Rule_of_succession](https://en.m.wikipedia.org/wiki/Rule_of_succession)

[https://en.m.wikipedia.org/wiki/Sunrise_problem](https://en.m.wikipedia.org/wiki/Sunrise_problem)

<comment><USER22> This presumes conditional independency - violations of which are common.

Instead, you get to estimate the dependency between each observation as in advanced variants of Bayes chain rule. Ultimately, some place of the estimator will contain an assumption giving only bounded optimality.

</comment></comment></comment><comment><USER22> Unmentioned assumption of normal distributed errors is pretty evil.

This is exactly this approach is worthless for rare events which by definition have very skewed distributions. In that case, probably is overestimated a lot.

On the other hand, I'd the is a rare but systematic error, the error probably will likely be grossly underestimated.

Thought experiment: suppose you're writing a long string of digits that consists of 1 followed by a large number of zeroes (say 99 for simplicity) followed by (say 10000) uniformly distributed digits.

Your writing system has an issue that changes half of 5 digits into 6. What probability of error will be estimated by this dumb method after 100th digit?

Correct bayesian approach updates the prior based on input variability keeping the error estimates high when input has low variability etc. (This can be with variance or another method.) The even better method tries to estimate the shape of input distribution.

In other words, your result would be a difference of likelihood ratio of both input and output prior distributions (estimated to date - since no errors the ratio would be 1) minus likelihood ratio of posterior distributions.

</comment><comment><USER28> This reminds me of an 1999 article in the New Yorker by Tim Ferris called "How to Predict Anything" [https://www.newyorker.com/magazine/1999/07/12/how-to- predict...](https://www.newyorker.com/magazine/1999/07/12/how-to-predict- everything)

> Princeton physicist J. Richard Gott III has an all-purpose method for > estimating how long things will last. In particular, he has estimated that, > with 95% confidence, humans are going to be around at least fifty-one > hundred years, but less than 7.8 million years. Gott calls his procedure the > Copernican method, a reference to Copernicus' observation that there is > nothing special about the place of the earth in the universe. Not being > special plays a key role in Gott's method.

</comment><comment><USER29> This is cool.

The given example of typos on page kind of highlights the fact that more sophisticated math involving a prior might give better results in some cases.

The beta(1, N+1) prior is an assumption that you start with the a priori knowledge that a typo rate of 1% and a typo rate of 99% are equally as likely as each other.

Most people would assume that books don't have typos on most pages and a 99% typo rate is unlikely.

However as your sample gets bigger the prior matters less and less so this rule is still useful.

Just know that it is reasonable to bias the results a bit according to your prior when N is small.

</comment><comment><USER30> If you're wondering where does the formula (1-p)^n come from, it's a number often used in gambling (if I throw a die 7 times, what are the chances of getting at least a 3). The probability of an event having probability p happening after n trials is 1-(1-p)^n, and he's using the inverse of that.

[https://en.m.wikipedia.org/wiki/Binomial_distribution](https://en.m.wikipedia.org/wiki/Binomial_distribution)

</comment><comment><USER31> This is a useful rule of thumb.

Lets try stretching it: Humans havent destroyed the world in their 300,000 years of existence, so probability of them destroying the planet in future is less than 0.00001 (1 in 100,000). I feel like that might be an underestimate.

Reminds me of the Doomsday Argument

[https://en.wikipedia.org/wiki/Doomsday_argument](https://en.wikipedia.org/wiki/Doomsday_argument)

</comment><comment><USER32> I'm not very good at maths, so I didn't understand the whole post. However, does the size of the whole population affect the "3/n" thing? For example, if I've read 200 pages of a 201 page book and not discovered a typo the chances are 3/200 if I've understand the post correctly. If the book has 20000 pages, is the probability still 3/200?

<comment><USER33> N is the sample size that you've sampled already in your observation.

If you've sampled all the pages, then we are talking about certainty which this wouldn't apply.

<comment><USER32> In my example, there's either 1 page or 19800 pages left. But the post suggests the probability of a typo is the same in both cases.

<comment><USER34> It's the probability of finding a typo on a page, not the remainder of the book.

<comment><USER32> Ah now I see the difference, thank you.

</comment></comment></comment></comment></comment><comment><USER35> Although interesting, the article doesn't relate to predicting things that haven't happened yet, just things that aren't known yet.

When predicting things that haven't happened yet, a publicized "certain" prediction will inevitably influence the actual probability in an unpredictable way.

</comment><comment><USER36> This post is of course talking about the difference between MLE and MAP estimation. Consider the converse case: you flip a coin once and observe it is heads. Do you then conclude that the probability of heads is 100%? No, because even though you have data supporting that claim, you also have a strong _prior_ belief in what your probability of heads should roughly be. This is encoded in the beta distribution, as mentioned in the article

</comment><comment><USER37> This is very similar to the sunrise problem. Laplace said the probability of the sun rising tomorrow is (k+1)/(k+2), where k is the number of days we know the sun has risen consecutively, if we always saw it rise, and if we don't have any other information.

[https://en.m.wikipedia.org/wiki/Sunrise_problem](https://en.m.wikipedia.org/wiki/Sunrise_problem)

</comment><comment><USER38> I don't know the first thing about schools of thought in statistics (frequentist? Bayesian?) but something feels fishy about extrapolating a probability based on sample size (or number of trials) alone. 20 pages, no typos, <15% chance of flawless spelling? What if the manuscript is 2×10⁴¹ pages long?

Would someone care to explain why the math says it shouldn't matter (for reasonably small values of p)?

<comment><USER13> ><15% chance of flawless spelling?

per page

</comment></comment><comment><USER39> Clearly this is a bad estimating rule for some processes. The one that popped into my mind is the probability of earthquakes (guess where I live...). This would have the probability of an earthquake declining with each quake-free year that passes, where in fact the USGS would say the opposite.

<comment><USER40> There's something that feels different about that. You're talking about the rate of some event occurring (like a Poisson process), and measuring how many occurrences are in an interval (a year). What are the 6 samples you are collecting? 6 years of numbers of earthquakes in a year?

Edit: sorry I thought you were replying to another comment.

</comment></comment><comment><USER41> I feel like I'm having a math stroke.

The posterior probability of p being less than 3/N for Beta(1, N+1) should be integral(Beta(1, N+1), 0, 3/N), right?

That trends toward zero, so I must be wrong, but I can't for the life of me remember why.

EDIT: Ah! I was accidentally using Beta instead of the PDF for Beta.

</comment><comment><USER42> This rule of 3 may be a good example for reading books and finding typos. It is no where as good for estimating the chances of "something that hasn't happened". It's so random using this rule & claiming the result as an estimate.

</comment><comment><USER43> There's a trivial corollary, which I remember from my first physics class. Never base anything on less than three measurements. Or maybe it wasn't even physics, but rather carpentry. The old "Measure twice, cut once." rule is iffy.

<comment><USER34> Three data points is not the same thing as three measurements of the same object.

<comment><USER43> There's no question that three measurements of some property of an object are three data points. I vaguely recall that my first physics lab experiment was measuring something with a ruler.

</comment></comment></comment><comment><USER44> This feels related, and is interesting:

[https://en.wikipedia.org/wiki/Sunrise_problem](https://en.wikipedia.org/wiki/Sunrise_problem)

(estimating the probability that the sun will rise tomorrow)

</comment><comment><USER45> related question: probability that the sun will rise again tomorrow

[https://en.m.wikipedia.org/wiki/Sunrise_problem](https://en.m.wikipedia.org/wiki/Sunrise_problem)

A practical use of a Stilton to this is included in reddits "best" sorting of comments, leaving from room for doubt with low sample sizes of votes.

[https://redditblog.com/2009/10/15/reddits-new-comment- sortin...](https://redditblog.com/2009/10/15/reddits-new-comment-sorting- system/)

</comment><comment><USER46> Mildly interesting trivia: there's a Chinese idiom stating "compare 3 shops for a good deal" (貨比三家不吃虧), I guess that makes sense from a statistical standpoint. :)

</comment><comment><USER47> I was just on a flight, hanging with a guy who's PhD was this topic. Josh is that you?? What are the odds.

</comment><comment><USER48> Interesting: the frequentist derivation is using the logarithm, while the Bayesian one, the exponent.

<comment><USER49> But note that log and exp are inverses of each other, and it's applied to opposite sides of the equation. In particular, this:

        1 - exp(-3) ≈ 0.95

Can be rewritten as:

        -3 ≈ log(1 - 0.95)

</comment></comment><comment><USER50> (1-p)^n is one my favorite things. happy to see it get some attention.

</comment><comment><USER51> Murphy's law "Anything that can go wrong will go wrong"

</comment><comment><USER52> This is from 2010.

</comment><comment><USER53> Oh, numeric astrology and probability tantras.

Future is not predictable by definition. It is just an abstract concept, a projection of the mind. Any modeling, however close to reality it might seem to be, is disconnected from it, like a movie or a cartoon. Following complex probabilistic inferences based on sophisticated models is like to act in life guided by movies or tantric literature (unless you are Goldman Sachs, of course).

For a fully observable, discrete, fully deterministic models, such as dice or a deck of cards probability could only say how likely a certain outcome might be, but not (and never) what exactly the next outcome would be.

Estimation of anything about non-fully-observable, partially-deterministic environments is a fucking numeric astrology with cosplay of being a math genius.

No matter what kind of math you pile up - equations from thermodynamics, gaussian distributions or what not it is still disconnected from reality stories, like the ones in tantras.

</comment><comment><USER54> i think you get more bang for your buck if you try to understand the mechanics that generate successes and failures. assuming a flat prior is crazy in almost every real world case. in otherwords, i think effort is probably better spent understanding the problem rather than understanding how to make the most of ignorance.</comment>


Yahoo, religion and the web - <AUTHOR> http://sahay.co/post/27542376285/yahoo-religion-and-the-web

<comment><USER2> Using monotheism and polytheism as a metaphor here was a poor choice. I really get tired of the myth that the polytheistic cultures of antiquity were much more religiously tolerant than the later monotheistic cultures.

This is a myth caused by an appeal to common sense that is not supported by the evidence. Let's take Roman society as an example. They may have had plenty of gods to worship, but there was an official state religion with compulsory worship, and they were hardly tolerant of deviation (see the ancient Christians for examples).

<comment><USER3> Actually, romans were usually tolerant of other religions. Yes, some religions were persecuted, but that was usually due to political reasons. Many (most?) christians were persecuted not for not following official religion but for creating unrest and speaking against Rome.

<comment><USER2> By far the most extreme christian persecution was by Diocletian. His hatred for christians began when fortune tellers, and priests of the Oracle of Apollo told him they couldn't predict the future because of the influence of Christians--definitely a religious rather than political reason.

>creating unrest and speaking against Rome

Those were the charges leveled against them (many times falsely) primarily by their fellow pagan citizens who were motivated by religious intolerance not political concern.

Most of the time the state wouldn't have persecuted the Christians had not the people demanded it.

The Romans also had a history of banning religious cults before the rise of Christianity.

</comment></comment></comment><comment><USER4> Setting up a huge collection of straw men (all geeks are fanboyz), and then beating your own chest at not being one of them. Seriously?

</comment><comment><USER5> While I think the monotheists are the most outspoken (and easier to get behind), I don't know whether the overall point is true. Continuing the purely anecdotal style of the OP: I know a lot of people that might fall into the monotheist camp at first glance, but once you start talking, it really isn't true. In my circles, I can quickly name the following configurations:

* The all-out Mac/IPhone user that is constantly eying for B2G and some Linux desktops to finally hit the point where he wants to switch. * IPhone users complaining that Windows Phone hasn't taken off. * Windows users using OS X at work and being pleased by both...

I could go on and on. Once you start talking to all of those people, they will happily admit many advantages and flaws of their platforms. The problem is that once you picked one, people automatically assume that you dislike the other. You won't talk much about the nice little things of a platform that you don't use on a daily basis.

</comment><comment><USER6> I couldn't agree more. I remember talking to a senior exec (acquired as part of a startup buyout) at a previous job and he stressed that there's always room under the big contenders (in areas they can't or don't care about) in any market until you begin to chip away at their market share and become one yourself. In an ideal world, I guess. Good post.

I really do want to be a Yahoo! fanboy too. Wish they would improve Flickr before I migrate to 500px.

</comment><comment><USER7> I think it's easy not to be a fanboy. But it's hard not to be a hater, especially when it comes to the likes of MS (or what it used to be) and the modern-day Apple.

</comment><comment><USER8> I read through this entire article and came to the conclusion: this can be summed up into a tweet.

</comment><comment><USER9> What is the point of this article? People should see the good in all things, not just things they're predisposed to like? Well, okay, but I that's fairly obvious. There's a reason the term 'fanboy' is never used in a positive way.

</comment><comment><USER10> Let's be honest. There's no such thing as a Yahoo fanboy.</comment>


Struquine: A Useful Lisp Trick - <AUTHOR> http://dorophone.blogspot.com/2008/08/struquine-useful-lisp-trick.html <comment><USER2> Here is an arc version of the struquine macro:

      (mac strq (name . slots)
         (withs (bname
                 (coerce (string name "#") 'sym)
                 creator `((def ,(coerce (string name "?") 'sym) (thing) (is (car thing) ',bname))
                           (def ,bname ,slots (cons ',bname (list ,@slots))))
                 accessors nil)
                (on arg slots
                    (= accessors (cons `(def ,(coerce (string name "-" arg) 'sym) (thing) (thing ,(+ index 1))) accessors)))
                (cons 'do (join accessors creator))))

I had to use "#" instead of "!" since that is used for special function access in arc, but the effect of this code is pretty much what was written about in the blog post. It creates a constructor for the struquine with name#, accessors for each arg of the form name-arg, and "type" checker of the form name?. For example:

      arc> (strq point x y)
      #<procedure: point#>
      arc> (= mypoint (point# 1 2))
      (point# 1 2)
      arc> (point? mypoint)
      t
      arc> (point-x mypoint)
      1
      arc> (point-y mypoint)
      2

</comment><comment><USER3> As I use other languages more and more, I begin to realize just how powerful lisp's simple "everything is a list" paradigm is.

For example, one homework was to create an object oriented forum. The thing was so complex because I was supposed to differentiate between forum, topic, thread, and post. However, if I made a post a cons cell, then it all collapsed into a branching list where everything is a post, which also gave me easy branching threads.</comment>