Saudi Arabia Agrees to Let Women Drive - <AUTHOR> https://www.nytimes.com/2017/09/26/world/middleeast/saudi-arabia-women-drive.html <comment><USER3> "Low oil prices have limited the government jobs that many Saudis have long relied on, and the kingdom is trying to push more citizens, including women, into gainful employment. But some working Saudi women say hiring private drivers to get them to and from work eats up much of their pay, diminishing the incentive to work."

I'm guessing that this is the biggest driver - Saudi Arabia just cannot afford to not allow this anymore.

<comment><USER4> The dual income household had an unbelievable impact on Western and Asian economies, buoying incredible growth.

How the KSA handles the coming social liberalization will be interesting, Chinese women still grapple with social conservative attitudes regarding working women (ie: leftover women). It is inevitable that the people of the KSA will encounter similar difficulties.

<comment><USER5> Actually, the "leftover women" phrase refers to the relationship / marital status of some women, not their job status. Many of these women (including people I know personally) have reclaimed the term as a compliment -- there's a pun where they substitute a word that roughly means god-like or badass.

My mom and both of my grandmothers lived in a world where the dual income household is the norm, and has been the norm since the 50's, but since the 80's and 90's, there has been a retrograde shift because for a while, being a stay-at-home-mom became a status symbol. However, all of my cousins, male or female, are part of dual-income households. (They all live in Shanghai.) But that's a much longer rant and off-topic to the current discussion.

But +1 to your initial point, re: effect of dual income households on the economy.

<comment><USER6> Knowing more than a handful of 20s-30s ladies in SH through my Toastmasters club, although there is selection bias, they are some of the most capable, driven, hard working, talented young people I have ever met. And I am almost 50 so have seen a bit over the years. If they were unleashed by the system and cultural expectations, norms, and restrictions, there is nothing they could not accomplish.

</comment></comment></comment></comment><comment><USER7> ::slow clap::

It is absolutely appalling that this type of backwardness is still a reality in the 21st century.

The faster we can transition to clean, renewable energy, the better for everyone in the world.

<comment><USER8> My favorite example is the South Carolina law that was finally rescinded a few years back, where bars had to serve their mixed alcoholic drinks with liquor poured from single-serving 'airline' style bottles. A colossal waste of time and resources, but the kicker was the arguments against rescinding the law-- some folks arguing that DUI rates would go through the roof and The Children (TM) would all die, and others arguing that bartenders would rip off patrons with 'short pours'.

All without a single thought to stop and consider the example of the other 49 states, not to mention other countries...

<comment><USER9> Laws related to firearms are often similar.

The current example is the effort to make suppressors more available. Those opposed are already pushing tales of dramatically increasing crime rates and poaching instead of acknowledging that even suppressed firearms are still very loud or that many other OECD countries (which have more strict regulation) allow or even encourage the use of suppressors.

The sad part about using fear to support or oppose change is that you rarely end up with solutions that actually solve the underlying societal problem via a sensible compromise.

<comment><USER10> Why do some countries encourage the use of suppressors?

<comment><USER11> Think about it.

The threshold for potential instant noise-induced hearing loss is 120 dB SPL at the ear. A rifle can exceed 170 dB SPL at a distance of one meter[1]. When you're shooting a rifle, it's going to be cradled firmly against the shoulder -- a lot closer than 1 meter. Even the lowly .22lr fired from a handgun can exceed 160 dB SPL[2]. A suppressor can knock that down about 30 dB. 130 dB is still a lot, but it's out of the ' _will_ -immedately-and-permanently-damage- your-hearing' range at least, and makes further hearing protection (earmuffs, earplugs) that much more effective.

tl;dr - A suppressor is safety equipment.

1\. [https://en.wikipedia.org/wiki/Sound_pressure#Examples_of_sou...](https://en.wikipedia.org/wiki/Sound_pressure#Examples_of_sound_pressure) 2\. [https://en.wikipedia.org/wiki/Suppressor#Effectiveness](https://en.wikipedia.org/wiki/Suppressor#Effectiveness)

<comment><USER12> Maybe it should be a thing for the gun range then. Require people to attach them there and require that they leave them there.

In general, if people are shooting outside in my neighborhood, I'd rather be woken up. That's not the type of thing that should go unnoticed. And I live in the middle of San Franciso where I've watched people get shot and killed.

<comment><USER13> As someone living in a rural area, near a nature preserve, I'd prefer the opposite.

I'm tired of being woken up way too early in the morning by gunshots. Waterfowl season started last weekend, so it was shotguns going off a mile away. Rifle season for deer starts soon (it's bowhunting season now), and that will bring its own set of loud noises from even closer.

I'm all for suppressors, if only so I can sleep past 5AM without having to keep my windows closed.

</comment></comment></comment></comment></comment></comment></comment><comment><USER14> This is a REALLY big deal. I was born and raised there. Glad to see some progress.

<comment><USER15> is it the first time or, like lebannon or iran, there was a time when women could drive before ?

<comment><USER16> First time in Saudi. Women were never banned from driving in Lebanon, not sure about Iran, but I don't think so

<comment><USER15> Sorry I didn't mean driving per se, but in the 70s women had more freedom and it regressed since.

<comment><USER17> That's not exactly true. People tout those photographs of "liberated" women in Iran in the 70's but that was a tiny slice of a female urban lifestyle restricted to a small number of elite women. That was not representative of the lives of the overwhelming majority of Iranian women.

<comment><USER18> Iran was still far more liberal in general before the cultural revolution, no? Women don't have to be in nice clothes in urban areas to not be repressed by clerical rule backed by state rule... There may have been societal pressure but it's significantly different when it's backed by government security services and religious police.

<comment><USER19> Yes, the Shah of Iran pushed a massive modernisation program [aka liberal agenda] to westernise Iran. The pushback from conservative elements was one of the driving forces of the revolution.

The interference from western powers to prop up the Shah did not help matters.

At least this was the impression I got from my time in Iran and visiting the tourist sites.

<comment><USER20> Actually, and this is forgotten by most non Iranians, the revolution against the shah was NOT Islamic.

It was a coalition of factions that took him down including socialists and communists. The ayatollah and his followers seized the country within a few years though and turned it into an Islamic republic.

<comment><USER19> I'm aware of that. Everyone from socialists to islamists banded together to overthrow the shah, and the from this chaos the ayatollah and his followers seized the country.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER21> The use of "Agrees" in the headline reveals societal bias. "Decides" would have been better.

<comment><USER22> Is there something wrong with societal bias? Especially towards ideals like personal liberty?

<comment><USER23> The word "bias" probably should have tipped you off

<comment><USER24> Or not? One can be biased toward life, liberty, the pursuit of happiness, or any number of generally-agreeable things.

I'm biased against death, and toward life. I don't generally feel a need to disclose that, since it's a widely-shared bias.

</comment></comment></comment></comment><comment><USER25> I'm totally in support of this. Strangely, the second thought after "wow" that I had was: I wonder what the backlash will be. I think my big, pessimistic lesson from the Obama years was that moves that I see as progress will probably be met with an equal and opposite antithesis.

</comment><comment><USER26> But do they have to be accompanied on that drive by a male relative?

<comment><USER27> It's pretty messed up how this mirrors similar questions that come up when discussing autonomous vehicles- While we in the West try to come to grips with the agency of computer systems, other places are still trying to come to grips with agency for adult humans.

<comment><USER28> We in the West are still struggling to come to grips with agency for adult humans.

<comment><USER27> Sorry if I didn't couch my argument with enough disclaimers to satisfy every variant of moral relativism- Yes, both the US and Saudi Arabia struggle with human rights, in some fashion.

</comment></comment></comment></comment><comment><USER29> Serious question: how safe is it to drive with a burqa?

<comment><USER30> Germany has recently banned burqas while driving[1], but not because they were considered unsafe. Instead, the ban forbids having anything that makes you impossible to identify in camera.

So my guess is that they are not really unsafe, or they would have been banned earlier for that specific reason.

[1] [http://www.dw.com/en/german-bundesrat-approves-burqa-ban- for...](http://www.dw.com/en/german-bundesrat-approves-burqa-ban-for-drivers- beefs-up-road-race-sanctions/a-40642060)

</comment></comment><comment><USER31> I wonder to what extent private families/society at large will accept women driving?

</comment><comment> anon-123 Posting as anon because I don't want to be condemned, honestly, you can't marginalize half of your population and be successful and you don't need a degree in econ to understand that. It's simple math.

<comment><USER32> But a degree in history would show you that yes, indeed you can marginalize half your population and be quite successful for a very long time.

<comment><USER33> When money bubbles out of the ground or grows on trees, yes, yes you can.

<comment><USER32> The window of time in which a majority of the members of a society have been franchised in the history of human societies is fleetingly short, totaling the last 50-100 years.

</comment></comment></comment></comment><comment><USER34> How is this helpful if they cannot be outside alone without a male relative?

<comment><USER35> It is helpful because women can be outside alone without male relatives. You realize that most women go around with personal drivers rather than male relatives.

<comment><USER36> Isn't the male relative's guardianship considered to pass to the 'servant' that accompanies them?

</comment></comment></comment><comment><USER37> Would they still need permission of the male in their household to drive?

</comment><comment><USER38> When do they stop executing gays?

<comment><USER39> When climate change makes the Middle East too hot to be habitable by humans ([http://money.cnn.com/2015/10/27/news/climate-change- middle-e...](http://money.cnn.com/2015/10/27/news/climate-change-middle- east/index.html)).

</comment></comment><comment><USER40> Anyway that doesn't mean they can actually drive in real life, i am sure there will be more strict regulations about this issue beside the masculine religious community opposition.

</comment><comment><USER41> I had to just check, yep it is the 21st century.

<comment><USER42> The 21st century is not as homogeneous as you think. Not every country will have the same cultural timeline as the USA, and it's odd to me that comments like yours are so prevalent. The world is bigger than america. Comments like this just seem incredibly self-centered, and a bit culturally supremacist.

<comment><USER43> Why is the OP automatically deemed an American for expressing cynicism? Its amazing that you yourself see no hypocrisy in making such discriminatory remarks based on what you perceive another's nationality to be.

The 21st century might not be homogenous but please name me one other place on the planet where women can't drive, where women can't open a banking account[1], or where women can't enter a Starbucks[2] or where there are public beheadings on Fridays[3]

Actually name one other place that even comes close to that level of chauvinism and barbarism. That cynicism is not without merit.

[1] [http://www.independent.co.uk/news/world/middle-east/five- thi...](http://www.independent.co.uk/news/world/middle-east/five-things-that- saudi-arabian-women-still-cannot-do-a6765666.html)

[2] [https://en.wikipedia.org/wiki/Deera_Square](https://en.wikipedia.org/wiki/Deera_Square)

[3] [https://www.eater.com/2016/2/1/10885722/saudi-arabia- starbuc...](https://www.eater.com/2016/2/1/10885722/saudi-arabia-starbucks- women-banned)

<comment><USER42> To your first point, I could/should have replaced "America" with "western culture", but it originally made the sentence less clear. I noticed the hypocrisy =)

To your second point, I was not talking about the specifics of Saudia Arabia, but rather the common idom of going "what centry is this" with respect to judging other cultures. People forget that western history is distinct from world history. And it feels a bit supremacist too - "Western culture is at X point, so everyone should be at X point"

</comment></comment></comment></comment><comment><USER44> Don't know whether to be happy this happened or sad that it took so long!

</comment><comment><USER45> Welcome to the twentieth century, Saudi Arabia.

</comment><comment><USER46> Yeah, someone needs to buy their oil.

</comment><comment><USER47> Just today I had someone casually mention that he thought women should not be allowed to vote. This is the Netherlands, the person is Dutch, not muslim or some other culture that I might not be used to, we're master's students... nothing out of the ordinary. And the third person in the conversation apparently agreed, calling it bad to prohibit it in principle but "seeing the effects of allowing women to vote" he agreed.

What the hell. Don't let women vote because they tend to vote "left" and "emotionally" (allegedly). What if the women of this planet decide that male votes are shit, what then? Are we just going to have to fight it out? (I didn't really get a response to that.)

I just don't have words for this, it's so alien. Am I weird for rejecting the notion out of hand?

Edit: two downvotes, please explain? I am honestly interested in opinions; I just found it _very_ weird to hear someone say this.

<comment><USER48> Yeah you're right that is weird especially that both agreed, are you sure they weren't trolling you? But I do agree with voting irrationally being a problem in democracy. I just don't think it's only women that do that.

I'd love it if, in order to vote, you had to name one policy position that the party you are voting for holds. I know we want everyone to vote so we can say participation is high but I'd rather only those interested enough to know what's going on voted.

<comment><USER49> There's already a self-selecting mechanism to eliminate most voters. It's called partisanship. Those people who vote for their favorite party no matter what cancel out each other's vote so they don't count. That leaves only the people who're making a decision each election that matter.

<comment><USER50> This is a good joke, but please don't believe it's actually true.

In application it's based on the false assumption that there are an equal amount of partisans on both sides.

<comment><USER49> Considering that neither major party consistently dominates in the US, I have a feeling they do cancel out. That, or the deciding-each-election voters apply pressure against any long-lasting incumbent.

<comment><USER51> They cancel out because the parties move towards the political positions that cause them to approach 50%.

I had this realization after meeting a political strategist through a mutual friend and having a conversation with him about what he does, and then I felt stupid for not having realized it before. His job is, at a high level, to find areas where polling indicates that his party's support in a particular election is a little under 50%, and then find ways to get a few more people to vote. If they reach 50% + epsilon, he's succeeded. If they reach significantly over 50%, that effort is wasted; you win an election equally well with 51% or 90% of the vote, so as long as you have 50% + a comfortable margin of error, you might as well spend your time trying to win other districts. And then, of course, the other party is trying to do same thing.

So a roughly 50/50 split in votes in a district, or in senators in Congress, or whatever, isn't indicative of an equal number of the voters being partisans of each party. You might have 10% partisans of one and 40% partisans of the other, which would cause the first party to run a centrist campaign/candidate to attract a lot of undecided voters and the second party to be willing to run a much less centrist/candidate.

Essentially, as long as you have two political parties, you're basically never going to lose the partisans (they certainly won't vote for the other party, and an extremist third party will always sound like a spoiler), so once you've settled on two major parties, they'll end up getting about 50% of the vote, even as their actual political positions move.

One way to avoid this problem would be to move to a non-first-past-the-post voting system, so a centrist third party (or one with orthogonal ideas) can successfully beat them both by attracting the non-partisans. Or, at the congressional level, move to a parliamentary system that requires forming coalition governments, which would enable something similar. But the US has neither of these.

<comment><USER52> Not sure if you're familiar with Hotelling's law, but you've basically described it here. I always like the beach shop analogy: if two vendors are selling beach chairs along a strip of sand, the optimum location is right next to each other in the middle of the strip. That way each vendor gets 50% of the customers.

</comment></comment></comment></comment></comment></comment></comment><comment><USER53> > One cleric claimed — with no evidence — that driving harmed women's ovaries.

Wait. What?

<comment><USER54> It would be surprising if someone hadn't made such a ridiculous claim. But in an article on gay marriage in the US, would the NY Times really have felt the need to observe that "one preacher even claimed - with no evidence - <USER2> homosexuality caused earthquakes"?

<comment><USER55> It depends on how much political influence that specific preacher wielded. In Saudi, clerics wield a LOT of political influence. They're not just religious advisers, they get to set and interpret law based on their understanding of the Quran and the Hadiths.

<comment><USER56> No, mainly depends on how much outrage can be mined out of it. Influence is largely the byproduct of the press attention, at least in the scales we have in the US, and the press wants to have people engaged (clicks, views, ads) and also preferably show their political opponents are crazy evil idiots. Emphasizing outrageously stupid claims by some obscure personality serves both goals - people get outraged, share the links, ad impressions follow, discussion follows, etc. etc.

<comment><USER55> >Influence is largely the byproduct of the press attention

That's not how it works with clerics in Saudi. They have official standing with the government, have their own police force with the authority to enforce morality laws (the mutaween), and even get to decide who the next king will be.

</comment></comment></comment></comment></comment><comment><USER57> In 2017 ...

<comment><USER58> Actually, 2018. June of next year.

</comment></comment><comment><USER59> Simultaneously they are banning all non-autonomous cars.</comment>


Arc: Virus-Like Protein Is Involved in Memory and Cognition - <AUTHOR> https://reliawire.com/virus-like-protein-cognition/ <comment><USER2> (Previous HN discussion here: [https://news.ycombinator.com/item?id=16139798](https://news.ycombinator.com/item?id=16139798) )

So this 'Arc' protein has been in our genomes for quite some time. One of the two papers published simultaneously last month demonstrated their results on development and cognition in flies. [2,3]

The protein is the one that acts like the 'patches' on a soccer ball, where the inside is 'sticky' to RNA. And you can assemble a viral-like capsid [1] around that RNA, and seemingly transfer that RNA to another cell. A single RNA can produce a number of proteins over an extended period of time, while the transfer of proteins (for instance, in a lipid vesicle) is a much faster action, but much less long lasting or impactful (in terms of number of proteins in a location over time).

One very curious idea is that this is part of an as-yet undescribed mechanism for the delivery of nucleic acids into human cells. This process (of delivery) is also the missing piece to making most gene therapies useful. And in those terms, this particular discovery might be about as useful in our long-term bio-capabilities as the discovery of CRISPR's capabilities. The one really distinguishing feature is that this 'virus-like' protein is, literally, already humanized. Your immune system won't attack it - which is one of the primary concerns with using existing retroviruses or lentiviruses in a therapeutic setting.

If you want to play with the human Arc sequence and build your own delivery mechanisms, go for it:

[https://serotiny.bio/notes/proteins/arc/](https://serotiny.bio/notes/proteins/arc/)

This is also yet another way in which particular organisms have 'captured', and even completely subsumed other organisms for particular effect (see mitochondria). There are likely many more buried in our historical genomic archive.

[1] [https://en.wikipedia.org/wiki/Capsid](https://en.wikipedia.org/wiki/Capsid)

[2] [http://www.cell.com/cell/fulltext/S0092-8674(17)31502-7](http://www.cell.com/cell/fulltext/S0092-8674\(17\)31502-7)

[3] [http://www.cell.com/cell/fulltext/S0092-8674(17)31504-0](http://www.cell.com/cell/fulltext/S0092-8674\(17\)31504-0)

</comment><comment><USER3> Is this related at all to octopuses rewriting their RNA to pass inter-cellular information?

[https://www.washingtonpost.com/news/speaking-of- science/wp/2...](https://www.washingtonpost.com/news/speaking-of- science/wp/2017/04/06/octopuses-and-squids-can-rewrite-their-rna-is-that-why- theyre-so-smart/?utm_term=.f02cb7bcc452)

<comment><USER2> Only in that RNA, as an intermediate between the relatively immutable DNA- based source code, and the running protein programs acts as a nice intermediate with which to compress, alter, and otherwise regulate how the code actually runs. RNA has traditionally been tricky to study - it's very fragile. And so our current studies are constantly revealing really cool new capabilities (CRISPR is also an RNA-interacting protein).

In the octopus' case, the RNA is being altered post-compilation, in ways that are somehow critical to their neural development. And that kind of process is unusual, and seems to be one that the octopus really depends on for its 'smarts' \- it's not just an evolutionary side-project.

While in the OP's case, the protein likely comes from an HIV-like infection that we've repurposed to help transmit compressed, ready-to-run code without having to transmit the large and short-lived binaries (proteins).

</comment></comment><comment><USER4> Reminds me of this result: [http://newsroom.cumc.columbia.edu/blog/2015/07/02/long- term-...](http://newsroom.cumc.columbia.edu/blog/2015/07/02/long-term- memories-and-prions/)

Kandel himself gives some wonderful lectures on this, easily found on YouTube.

It leads me to wonder how many other structures out there we have already described and unwittingly utilise, patterns re-discovered for purposes at another level of abstraction.

</comment><comment><USER5> Always intrigued by memory RNA [https://wikipedia.org/wiki/Memory_RNA](https://wikipedia.org/wiki/Memory_RNA) since reading Larry Niven's _Rammer_ , but the role here is probably more like a neurotransmitter. It's curious that it seems to transport RNA, but (for the memory RNA hypothesis) why would you transport memories between cells?</comment>


Spotify seems to crash for all iOS users on launch - <AUTHOR> https://linustechtips.com/main/topic/1220284-spotify-not-working-on-ios/?tab=comments#comment-13806431 <comment><USER2> Facebook SDK strikes again: [https://news.ycombinator.com/item?id=23790089](https://news.ycombinator.com/item?id=23790089)

<comment><AUTHOR> Thanks

</comment></comment><comment><AUTHOR> Workaround: enable airplane mode and disable WiFi</comment>


INDIA – Reliance Jio Announce Free Calls Forever and Data for INR 50/GB - <AUTHOR> http://jiofans.com/discussion/38/reliance-jio-all-the-prepaid-and-postpaid-plans-and-tariffs-per-day-per-month-plans <comment><USER2>
      As a welcome offer all the new Reliance Jio Subscribers will
      get access to Unlimited Data, Unlimited SMS and Calls upto Dec
      2016 and From Jan 2017 you can choose any of the plan from
      above sheet.

Wonder why don't they post the conditions too? In a country where people change their cellphone numbers faster than their handkerchiefs, who would prevent them to use the services until end of 2016 (for free) and move on. There must be more to it.

<comment><AUTHOR> Reliance has made it clear that from Jan 2017, the minimum need to pay is INR 149 per month to keep the connection Active. Everyone else is costly than them and the data speed sucks for others, as of today I don't see any reason than people won't stick to Jio for sometime.

</comment></comment><comment><USER3> Rs 50/ GB.. Trustworthy data counters please!</comment>


The next big 'micro' thing? - <AUTHOR>

i'm positive about seeing more products emerging that micro-x (or altleast claim to be) a micro version of some existing x.
Carrying on the idea of "less is more", seem to have paid off hansomely for lot of innovative avtars of existing as well as new products. The upperhand you get when it comes to using phones as a platform is also very encouraging. It must be easier to engineer for the phones, when your actual data /content is itself small is'nt it.
So hence this discussion to think/rant on what you think could/coudnt be the next big 'micro' thing 8)
here's a list of a few i could think of that already exist or could become a reality pretty soon...
@micro-blogging
twitter, tumblr, ?
@micro-contacts/ids microformats have already shown the potential of this , ?
@micro-email don't ask me why ,but i'm surprised i hav'nt see this already :D
@micro-html ? perhaps publishing for small dimensions like phones/google IG/netvibes/pageflakes? makes sense doesnt it...
@micro-photos , micro-videos perhaps an only thumbnail version of a flickr/youtube/etc (again could be adopted for avtars directory or nice drag'n drop playlist editor for videos) . another thing that would also make sense for phones/devices.
@micro-podcasting starting with Qik, ?
@micro-playlists i think this is an awesome opportunity. be it a playlist of url's , video's or pictures ,being able to add a to-play while on the phone, and see your favourite micro-info . this would ofcourse need to have a micro-ecosystem of all kinds of data (ie. many of the micro's listed here) ,and being able to see each of them in the same dimensions. be it a biz card - or news or a blog post or video streaming
@micro-novels already seen in japan via the use of twitter(that too published via phones!)
@micro-calendars perhaps on the lines of 'google agenda' version?
@micro- ????
Keep Clicking, Bhasker V Kode <comment><USER2> micro-payments: TipJoy <http://news.ycombinator.com/item?id=113122>

</comment><comment><USER3> @micro-to-do

micro-html = widgets?

</comment><comment><USER4> micro-email is known as SMS

</comment><comment><USER5> how 'bout micro-social-networking?</comment>


Show HN: TwLng.com – Tweet with more than 140 characters - <AUTHOR>

TwLng.com is a side project that lets users tweet with more than 140 characters.
I wanted to experiment with Google's PHP AppEngine runtime and am using the awesome PHP-GDS [1] library as an API to the Google DataStore.
Feedback and ideas for how to take this further would be appreciated :)
[1] https://github.com/tomwalder/php-gds <comment><AUTHOR> Clickable: [http://twlng.com](http://twlng.com)

</comment><comment><USER2> Ability to embed links will be useful for marketers

</comment><comment><USER3> Example link at bottom is broken

<comment><AUTHOR> Thanks now fixed.</comment></comment>


Including Illegal Activity in the U.S. National Economic Accounts [pdf] - <AUTHOR> https://www.bea.gov/system/files/papers/WP2019-4_4.pdf <comment><USER2> I've often wondered how this should be dealt with in countries (specifically thinking of Eastern Europe) where large potions of the economy are off the books. I've assumed it would significantly skew GDP / GDP per capita figures if such values were included, and wondered how it would affect general quality of life indicators in places where even engineers often work in the unreported economy.

<comment><USER3> Adding grey zone activity to GDP numbers a great trick to increase the available govt debt ceiling without compromising the debt/GDP % numbers. I'm wondering if the US will do the same, given how large the deficit already is.

<comment><USER4> Neither party in the US actually care about the debt or deficit.

If we add illicit activity into the GDP it will be so that whoever is President can claim more GDP growth.

</comment></comment></comment><comment><USER5> The chart of illegal drug expenditures as a percentage of average person consumption is wild. People did a lot of drugs in the 1970s and 1980. "In total, this paper calculates that illegal drugs accounted for 5 percent of total PCE in 1980. This high expenditure share is consistent with contemporaneous reports. For example, a Washington Post article estimated that Americans bought $35 billion of cocaine (O'Toole 1981)." That figure has fallen to under 1% today. Maybe the war on drugs worked.

<comment><USER6> Going from "we don't use as much coke as in the 80's" to "the war on drugs worked" is a bit like going from "we have less teen pregnancy now than in the 90s" to "abstinence worked". It's not only ignoring a very large number of alternate factors, but it's assuming the latter is a successful strategy, when all of the evidence points to the contrary.

<comment><USER7> Interesting you bring up abstinence because kids are having way less sex nowadays than in the past.

<comment><USER8> Most definitely not because of abstinence education

</comment></comment></comment></comment><comment><USER9> Adjacent to this, has anyone ever seen statistics/estimates on income percentiles of illegal/underreported income? I've always wondered about this as a huge portion of jobs is based off tips or cash transactions (servers, contractors, maids, landscapers) and I can't imagine all these people report 100% of their income.

What percent of of the US population would you estimate doesn't report at least 10 thousand dollars a year?

<comment><USER10> The standard deduction removes this as a concern for most Americans.

<comment><USER9> I'm a really naive when it comes to tax law and a lot of economics statistics - but I'm having trouble seeing this.

If I am a waiter who made 35,000 in reported income and 10,000 in unreported income I made 45,000. I can take the standard deduction on 35,000 and in all statistics used by think tanks/reporters/politicians I will weight in the category of those who made ~35,000. Am I missing something?

<comment><USER11> No, your parent poster is completely wrong. You'll pay more in taxes reporting 45k with the standard deduction than if you report 35k with the standard deduction. The only scenario in which the standard deduction "fixes" this issue is if it reduces your taxable income to the point where you do not owe income tax regardless of if you report that extra income or not.

</comment></comment></comment></comment><comment><USER12> When Italy started including this stuff in their economic report, the joke was that the change happened to make their economic numbers look better...

<comment><USER13> How could it not? Supposing illegal activity exists, including it will increase GDP. GDP per capita could decrease if you suddenly count people you'd missed, I suppose, but I doubt that that'd be significant.

</comment></comment><comment><USER14> Here's similar information from the Australian Bureau of Statistics

The Non-Observed Economy and Australia's GDP, 2012 - Measuring Illegal Production

[https://www.abs.gov.au/ausstats/abs@.nsf/Products/5204.0.55....](https://www.abs.gov.au/ausstats/abs@.nsf/Products/5204.0.55.008~2012~Main+Features~Measuring+Illegal+Production?OpenDocument)

</comment><comment><USER15> It would be interesting to see the proportion of legal to illegal economic activity over time.

<comment><USER16> For the types of illegal activity measures here, the paper provides that in the figure measuring revision to nominal GDP. Basically, it peaks in the 70s around 4% of GDP and declines to around 1.67% today.

<comment><USER15> Thanks!

</comment></comment></comment><comment><USER17> The BEA is an interesting little bureau. They have to resist a lot of pressure both from politicians and law enforcement.

</comment><comment><USER18> The most amusing part is stagflation disappears when coke is included.

</comment><comment><USER19> I almost always tip in cash, and I frequent the same places so the service employees learn who I am. And they love my cash tips! You will get better service if you leave a $5 bill in cash rather than credit. Every bit helps.

<comment><USER20> So you think people who receive part of their pay in cash should be taxed less than people who don't?

<comment><USER21> Presumably they think that everyone should be taxed less, but that this approximation is the best they can do.</comment></comment></comment>


Banning Sites That Don't Respect Adblock - <AUTHOR> Certain sites that are linked to here are very difficult to navigate for those of us with adblockers installed. Such a site would include forbes. I propose sites that break adblockers, or are generally horrible to use because of advertisements be banned from hackernews. <comment><USER2> Please don't try to prevent _me_ from using things because you don't like them.

Sure, some sites limit who can view them based on visitor preferences. Analogy: Some restaurants have a "no shirt, no service" policy. It would be ridiculous to impose a blanket ban for all people on such restaurants just because you refuse to wear a shirt.

If you want to pick a more sensible fight, try going after links to paywalled sites (whose content is blocked through their decision rather than mine). However, bear in mind HN policy already has those covered: "It's ok to post stories from sites with paywalls that have workarounds."

<comment><USER3> Nobody here is able to prevent you from using Forbes or other similar sites. So, your analogy is way off base and that's why it sounds ridiculous to you.

It would be more accurate to compare this to "removing a restaurant from a single list of good restaurants because their food has a high chance of making you sick or making your computer download some malware".

<comment><USER4> _Nobody here is able to prevent you from using Forbes or other similar sites. So, your analogy is way off base and that 's why it sounds ridiculous to you._

This is actually a very good piece of advice that holds true in all sorts of conversations. If you attempt to argue the other side of an argument, and find it completely ludicrous? You don't understand the other side. Instead of arguing, ask questions.

I only wish that I could follow my own advice!

</comment></comment></comment><comment><USER5> Serious question: what are these publishers to do?

I look at a site like Reddit: no tracking, no flashy ads, no sound, significant investment in their own custom ad serving software to try and make things more targeted and relevant, clearly marked ads, etc.

They are doing _everything_ that our community continually says that we would like more publishers to do and they are severely struggling despite being one of the most popular sites on the entire Internet.

If you have even a semi technical audience for your site, you're today looking at a rapidly growing 30% of your audience using an ad-blocker.

Except in very few cases, most people aren't willing to pay for content and more paywalls (the non-advertising business model) further fracture the open web.

I feel kind of scared for where we're going.

<comment><USER6> Reddit does a shitload of tracking

<comment><USER7> Here's the reddit CEO bragging about it

[https://youtu.be/6PCnZqrJE24?t=499](https://youtu.be/6PCnZqrJE24?t=499)

</comment></comment></comment><comment><USER8> Maybe we should create a community maintained list of blocker unfriendly sites, and serve a public blacklist compatible with existing software.

Forbes and others are blocked on my home DNS for hostility and crippling of features. If a company can demonstrate how it protects its visitors from its advertisers and respects my rights, I used to exempt them from my ad blocker. Now I'm sick of seeing rude messages targeting me as an ad blocker user telling me what a horrible person and a thief I am, so I block all ads, and I've discontinued use of nearly all ad supported software. Bottom line: It's not our job to make their business models work, they need to adapt and quit whining.

<comment><USER9> Have you tried Brave? Whenever I have to view a website with a particularly onerous anti-adblock system, I use Brave. It blocks the ads and loads the pages fine.

</comment></comment><comment><USER10> So far, I think Guardian has the best approach towards ad-blocking. There's a non-intrusive banner message at the bottom of the screen, which says that they know you are using adblock and asks if you would like to support them via a paid subscription.

<comment><USER11> That's a step in the right direction. The problem I have with signing up for a subscription is two fold. First is that I don't feel like I traffic particular sites often enough to justify the price they ask. Second is that I don't want to give up the personally identifiable information they ask for (and sell off).

I'd much rather there be an anonymous mechanism for making one-time payments or tips. Maybe a metadata header that indicates a BTC address that can be picked up by a plug-in of the user's choosing?

<comment><USER12> I'd love to be able to anonymously say "Yeah! This article was sooo good, it deserves $10!" or "That article was crap. It doesn't deserve anything". But that has to be done _after_ I've read it. In the current ad model, articles get paid based on how much they are shared, not how much people like the content.

EDIT: It's kind of like tip-sharing. If I have a good waiter, I don't want to the crappy waiter to have his share of the tip.

</comment></comment></comment><comment><USER9> I see this as a double-edged sword. I think people should have the right to block ads, but sites should also have the right to block your access to their content. It's the same no-shirt no service rules for businesses. Correct me if I'm wrong here, but no one has a legal right to gain access to the data on a website. If that website chooses to restrict access that's their problem. I and others will just go somewhere else.

<comment><USER13> Your point is a good one. However, it doesn't imply that HN must allow these sites to be posted.

I don't have an opinion on whether HN should make such a change, but just wanted to connect your statement to the question at hand.

<comment><USER9> A fair point. I think HN should let these sites be published because it gives us all the knowledge that something exists. We can then freely make the choice of how we wish to interact with that site. Taking the position that something should be not be shared, disseminated, or indexed, simply because of the bad behavior by that link, brings to mind the delisting of material from search engines that may be offensive to others. Put the articles and websites out there and let people choose for themselves.

</comment></comment></comment><comment><USER14> What we need is a little plugin that puts a * or some sort of symbol next to a link to a site that blocks content for adblock users.

<comment><USER15> I recently wrote a Greasemonkey script to do this - it changes the link text background to yellow and disables the click.

If anyone is interested, the code is posted here: [http://scottlilly.com/greasemonkey-block-links-to- annoying-w...](http://scottlilly.com/greasemonkey-block-links-to-annoying- websites/).

</comment></comment><comment><USER16> I do agree in certain extent. Forbes is really bad regarding adblockers.

I don't mind being reminded that they use ads to generate revenue to deliver news for free, but doing it in a way where they actually loose readers because of that is nonsense.

Give me a popup asking me to turn adblocker off, not banning me from the content.

In the information age where the same news gets presented on numerous sites it's a bit shooting oneself in the foot denying access to the content because of adblockers.

People on hacker news should try to avoid posting content from sites like forbes, but we shouldn't force them to do so. Because then hacker news wouldn't be better then forbes.

</comment><comment><USER17> This is the reason why I haven't visited Wired or Forbes in months. And it's not that I couldn't figure out a way past their ad-block detectors. The idea that I must disable what is inherently a _security_ feature just so they can satisfy their marketing goals is antithetic to an open web.

I noticed Wired no longer blocks uBlock Origin (or more humorously, uBlock Origin has now began filtering out anti-adblockers) but I continue to see no reason to frequent their site.

<comment><USER18> Wired was suggesting I needed to whitelist their site but I had disabled and removed ghostery and adblock some time ago. WTF. I haven't been back since.

</comment></comment><comment><USER19> I subscribe to the NYT and yet still get asked to be whitelisted. So a subscription model will not generate enough revenue for them without ads.

It's like the cable tv model - we pay to watch more ads.

<comment><USER20> You're still served ads with a paid NYT subscription?

edit: well, I had been thinking about a subscription. I suppose not anymore.

<comment><USER21> Yeah, I brought that up when I called to cancel and they said "well the ads help keep the cost of your subscription low!". But that's not the point! I pay to remove ads, I don't watch ads to keep the price low!

<comment><USER22> That's what you _think_ you're buying, but that's not what they're selling.

</comment></comment></comment></comment><comment><USER23> I run with NoScript because I don't like the crap that a lot of these sites do. But if a site is broken for me and I'm not that interested in chasing it down, well, the Back button is a tiny mouse movement away.

The mods already have enough to do without having to curate a blacklist of sites for lazy users.

</comment><comment><USER24> Hacker News' purpose isn't to promote discussion from explicitly free or noncommercial sources, or from sources which enforce a particular moral framework. Even though paywalled sites and sites that block adblock are annoying (although many provide workarounds,) banning them outright would deprive the site of potentially worthwhile content, and the _content_ is what's important, not the commercial model behind the site hosting it.

If you're not willing to RTFA, fair enough, just don't comment in threads about TFA. That would improve Hacker News more than blocking those sites altogether.

</comment><comment><USER25> Personally I haven't clicked on a single Forbes link in months after being forced to unblock them to view content that should be available to the public with no catch. I never go back to sites that implement that system.

</comment><comment><USER26> I agree with you. The worst part is that most of the links in front page are from these big média players that have paywalls and adblockers blockers. A strange shift in HN profile.

</comment><comment><USER27> Linking to paywalled content should also be prohibited. Those links should be treated as ads themselves. The submitter is posting spam to HN.

<comment><USER20> >The submitter is posting spam to HN.

Except in the case where it is the submitter's own paywalled content, how is that spam? The submitter isn't benefitting from the paywall.

</comment></comment><comment><USER28> Get a better ad blocker or try find the content they've spent time and money creating elsewhere.

</comment><comment><USER29> Because how dare a company try to make money on a service they provide to us for free!

<comment><AUTHOR> It is the right of a consumer to not patronize businesses with that particular monetization model

<comment><USER30> No one argues that. It's much more dubious whether the consumer has the right to patronize a business in explicit disregard of the terms that business sets. If you don't want to see ads on Forbes, you can not go to Forbes. I'm not so sure you're entitled to just read Forbes without the ads.

I use an ad-blocker too. I'm just not so sure I'm defending some principled public good in doing so.

<comment><USER12> And that's kind of what OP is doing. He's saying "lets not even go to those sites". In a weird way, it's more ethical to just not share them, than to share them with ad-block users.</comment></comment></comment></comment>


This Is What the Earth Would Look Like Without Its Oceans - <AUTHOR> http://www.huffingtonpost.com/entry/this-is-what-earth-would-look-like-without-its-oceans_55ca1e11e4b0923c12be2b76?utm_hp_ref=canada&ir=Canada&section=canada&adsSiteOverride=ca&kvcommref=mostpopular

<comment><USER2> Altitudes are hugely scaled, btw. If it were a ball with radius of 10cm, then difference of ~30km would be: 10cm / 6300km * 30km == .0476cm (i.e. 0.5mm, almost invisible on display scale). Real highest and lowest marks are at ~9km above and ~10km below water line, iirc.

Earth is completely round with and without water.

</comment><comment><USER3> I would not call it beautiful. Color choices are appalling. False relief is way overdone. Arbitrary circles splotched everywhere cannot be real data, but statistical? Needs a designer to completely redo it.</comment>


Tinder and Bumble Are Seriously at War - <AUTHOR> https://www.nytimes.com/2018/04/04/style/tinder-bumble-lawsuit-explainer.html <comment><USER2> A war Bumble will most definitely lose since the app is a ghost town. Just google recent comment trends.</comment>


Palantir in Talks with Germany, France for Virus-Fighting Tool - <AUTHOR> https://www.bloomberg.com/news/articles/2020-04-01/palantir-in-talks-with-germany-france-for-virus-fighting-tool <comment><USER2> As a person that had the opportunity to work with them and their products, seeing a post like this makes me angry and sad. I personally find the company extremely incompetent, that hires young postgrade students without any prior experience to solve big problems for the biggest companies around the world. How it ends up is technical depts and gigantic costs. Not to mention few shady things that seems normal for them - full access to their customers business data and source code. Epidemics should not be a time for business opportunities. It's not like they have any closed patented solution that will suddenly solve any problems... It's definitely not a equivalent of openai in data engineering world. They are just experts in marketing bs.

<comment><USER3> So wait.

Are they the big bad all-present panopticon, or are they incompetent, inexperienced muppets?

<comment><USER4> If a platform involves tracking huge amounts of personal data and then making decisions off of that data, incompetence makes it more dangerous, not less.

LAPD has been using Palantir to track people in poor and working class neighborhoods in LA - [https://www.bloomberg.com/features/2018-palantir-peter- thiel...](https://www.bloomberg.com/features/2018-palantir-peter-thiel/)

>The LAPD uses Palantir's Gotham product for Operation Laser, a program to identify and deter people likely to commit crimes. Information from rap sheets, parole reports, police interviews, and other sources is fed into the system to generate a list of people the department defines as chronic offenders, says Craig Uchida, whose consulting firm, Justice & Security Strategies Inc., designed the Laser system. The list is distributed to patrolmen, with orders to monitor and stop the pre-crime suspects as often as possible, using excuses such as jaywalking or fix-it tickets. At each contact, officers fill out a field interview card with names, addresses, vehicles, physical descriptions, any neighborhood intelligence the person offers, and the officer's own observations on the subject.

>The platform is supplemented with what sociologist Sarah Brayne calls the secondary surveillance network: the web of who is related to, friends with, or sleeping with whom. One woman in the system, for example, who wasn't suspected of committing any crime, was identified as having multiple boyfriends within the same network of associates, says Brayne, who spent two and a half years embedded with the LAPD while researching her dissertation on big-data policing at Princeton University and who's now an associate professor at the University of Texas at Austin. "Anybody who logs into the system can see all these intimate ties," she says. To widen the scope of possible connections, she adds, the LAPD has also explored purchasing private data, including social media, foreclosure, and toll road information, camera feeds from hospitals, parking lots, and universities, and delivery information from Papa John's International Inc. and Pizza Hut LLC.

The LAPD have been careful not to cite information from this software in court documents so that its constitutionality cannot be challenged.

<comment><USER5> > stop the pre-crime suspects as often as possible, using excuses such as > jaywalking or fix-it tickets

So organized targeted harassment of innocent people.

<comment><USER6> Yeah this is some dystopian stuff. Guarantee that people on here will (rightly) criticize non-Western countries for doing stuff like this, but will ardently defend Palantir doing similar stuff because they're "innovative" or a "tech company".

</comment></comment></comment></comment></comment><comment><USER7> Worked for them as a Product SWE for 2 years. For the record, I would not recommend it if you have good alternatives as a SWE.

That being said, I want to give a more nuanced comment because, in my opinion, crisis require some pragmatism which I find to be surprisingly lacking from the usually reasonable HN crowd.

I understand why Palantir create this epidermic reaction and in most cases I would agree that they are justified. There is an issue of sovereignty, countries should aim be self-sufficient in critical areas like Defense, Healthcare, Counter-terrorism. It is a shame that it is not the case. Many Palantir employees are motivated to help their country but concluded that the most straight-forward path, if they did not want to spend many years in a bureaucracy hell, was to join a startup/software style company, that would respect their value-added as young/bright engineers and make some of them (the majority in commercial works on boring stuff) able to work on those topics.

Yes, compared to many cool tech companies, there is nothing great about their tech. It's a glorified Spark wrapper with a pretty UI and terrible UX. Nothing with significant tech depth is built-in house. However, THAT is compared to cool/good tech companies. Compared to a lot of what big corp and bureaucratic governments can readily provide TODAY, it's a significant improvement and it is ready, TODAY. It is also more secure compared to what an in-house gov team could build, as in, they actually encrypt things and there is no obvious security flaws. Now is the time to use the tool that can reduce the death tools without compromising security, tomorrow is the time to ask our politics to invest in their own technology sector and promote young/bright engineers to build state-of-the-art data platform that does not involve Palantir.

Why Palantir is doing such action is not to gather data (they do not), or to earn money (it's likely pro bono) but to establish a relationship with governments, a trojan horse if you will, so that they can sell their services to more departments. The pragmatic action would be to accept the help now in crisis time and leverage their tools and somewhat competent Forward Deployed Engineer workforce, but decline their expansion when we are past the crisis.

<comment><USER8> Also, their big principle is human insight over algorithms and machine learning. This was derived from experience in Paypal, where human in the loop approaches worked much better for fraud detection than algorithmic approaches.

</comment></comment><comment><USER9> In some positive light, 3-4 years ago Google or Facebook would've jumped on and published a contact tracing app without thinking twice. I wonder what's going on inside in terms of discussions. Some positive side effect of the tech slash to step back and think twice of side effects or maybe they've learn to avoid possible bad PR.

<comment><USER10> How is that positive? They have great engineers, great infrastructure and a lot of money. It would be great if they could help.

<comment><USER11> When everybody acts as if they're already infected, and so is everyone else, how is this helping?

<comment><USER10> Everybody is not acting like that. I meant help in any way that would benefit everyone.

<comment><USER11> > Everybody is not acting like that.

The vast majority does.

> I meant help in any way that would benefit everyone.

And I'm asking how that would help. So far, every single proponent of "surveillance because covid" I asked this avoided the question.

It's also pretty incredible how you split hairs about "everybody" in the first sentence, and then talk about something that will help "everyone" in the second, without even saying how.

<comment><USER10> > The vast majority does.

How can you say that? The Netherlands still has people going to cafes. The UK is talking about banning outside exercise. Even if you can point to a figure saying people are moving less and adhering to their government's guidelines I don't think it follows that you can say the majority of people are acting like they are infected.

As for the second part, by definition, if those companies actions benefit everyone.. it would help. I didn't say and I don't know how they would do that.

Since this conversation started, it looks like Google has used their great engineers, great infrastructure and lots of money to publish movement reports. This might feed into Government decisions and hopefully will be a benefit for "everyone". Check it out here: [https://www.google.com/covid19/mobility/](https://www.google.com/covid19/mobility/)

<comment><USER11> > How can you say that? The Netherlands still has people going to cafes.

Because vast majority isn't, certainly not in Germany. I can say that because I have been experiencing it for _weeks_ now.

> Even if you can point to a figure saying people are moving less and adhering > to their government's guidelines I don't think it follows that you can say > the majority of people are acting like they are infected.

Those guidelines to after entail acting like people oneself might be infected. That's why people are to keep distance from anybody, never ever touch their face while out and about, and wash their hands thoroughly when they come home (and I might add, also before they leave home).

> As for the second part, by definition, if those companies actions benefit > everyone.. it would help.

I ask how surveillance help, and you say "if it helps, it helps by definition". That is hardly an answer.

Let's say 10% simply don't take it seriously. If they don't keep distance from anyone out of the 90%, those people will know. In Bavaria, someone already went to jail for 8 days because of exactly that, for not paying heed despite repeated warnings.

There is no surveillance necessary for that because -- by definition -- the only way for those 10% to "secretly mingle" would be with other people of those 10%. Movement tracking also doesn't tell us anything about if people cough right at other people or into their elbow, or if they wash their hands when coming home. Mere distance itself isn't saying all that much.

As for the Google thing: super general, anonymized "where do people move around how much" is something else entirely than the location data of all indiviuals.

> _What data is included in the calculationdepends on user settings, > connectivity, andwhether it meets our privacy threshold. If theprivacy > threshold isn't met (when somewhereisn't busy enough to ensure anonymity) > wedon't show a change for the day._

That isn't the kind of surveillance we are talking about, like the kind of German justice ministry previously rejected, for example.

Information that harms nobody, violates no rights, is 100% anonymous, doesn't even _have_ to be useful for me to have no problem with the collection of.

Palantir on the other hand "declined to comment", so you give me something completely different from Google. If what Palantir have in mind is so benign and useful, why don't they proudly explain it? It's not like CoV19 could overhear that and adjust, like criminals or terrorists might.

And from a PR perspective, declining to comment is the ONE predictably bad move here. If they want to help, if they have something helpful to bring to the table, why aren't they open about it? With something like this, they should have had their story straight before even anyone asked them, before they even finalized their offers to those governments. This is nowhere good enough. That isn't "the second part", that is the main issue.

</comment></comment></comment></comment></comment></comment></comment><comment><USER12> Please don't let this company anywhere near the healthcare system.

<comment><USER13> Second to that - that company feels so sketchy.

<comment><USER14> They are sketchy: [https://duckduckgo.com/?q=palantir+cambridge+analytica](https://duckduckgo.com/?q=palantir+cambridge+analytica)

</comment></comment></comment><comment><USER15> I wonder who responds positively to Palantir recruiters?

Is the money so good that it is easy to suspend your morals?

[https://en.wikipedia.org/wiki/Palantir_Technologies#Controve...](https://en.wikipedia.org/wiki/Palantir_Technologies#Controversies)

<comment><USER16> Outside of the minority in the Twittersphere bubble most engineers are apolitical. Palantir offers the opportunity to work on interesting software solving interesting problems and that gets them very far.

Defense contractors building bombs and the NSA don't really struggle to hire engineers, either. Assuming everyone has your same belief system is a critical mistake.

<comment><USER17> Are you sure about that last point? I remember reading somewhere that U.S. government agencies were having a hard time hiring tech talent, as they did not pay as well as Silicon Valley nor had the same growth opportunities. Moreover, they had issues with the patriotic argument, as many millenials don't share the same ardent patriotism that previous generations had. It's been a while since I read that article, so things might have changed and I might be wrong.

<comment><USER16> My thinking is the former problems -- pay, growth, perks -- are more damaging to their hiring prospects than the actual work the US gov is doing.

</comment></comment></comment></comment><comment><USER18> I actually really get the general reaction here to Palantir. What I don't get is why the FOSS community hasn't just done the right thing and produce a competitor yet. It's not like link analysis [1] with shared backends isn't an interesting problem and big business producing billions in revenue for various companies every year.

\- Palantir $6B valuation

\- IBM Analyst Notebook - [https://www.ibm.com/security/resources/demos/i2-analysts- not...](https://www.ibm.com/security/resources/demos/i2-analysts-notebook- demo/)

\- Maltego - made by a company called Paterva with something like 200 employees popular in cyber circles

Where can somebody go and just apt-get gnu-links?

1 - [https://en.wikipedia.org/wiki/Link_analysis](https://en.wikipedia.org/wiki/Link_analysis)

<comment><USER19> > Where can somebody go and just apt-get gnu-links?

[https://gephi.org/users/install/](https://gephi.org/users/install/)

But it was last updated in 2017 - [https://github.com/gephi/gephi/wiki/Releases](https://github.com/gephi/gephi/wiki/Releases)

<comment><USER18> Gephi (and other graph tools) aren't quite the same thing.

The front-end user interface, and the workflows are different. It's a little closer to how maybe yED works with dragging and dropping of entities, connections, groups, but also a robust meta-data system attached to all of those things + some tools that make bringing in new information from free-text (or wherever) simple. Then all that stuff should be stored in some kind of centralized semantic database that any user using that system can search and find stuff somebody else already put in there. Maybe a document search as well so you can see what raw documents have been ETL'd.

Then take various attributes off of some of the entities and relationships (like dates, or geos, or IPs) and builds various ways of looking at, filtering, and search that stuff quickly.

</comment></comment></comment><comment><USER20> Little known fact is that peter thiel used palantir analytics to help Donald trump understand where to campaign.

<comment><USER21> Do you have a source?

<comment><USER20> There's no source, it would never be made public.

</comment></comment></comment><comment><USER22> I had a recruiter from them reach out to me and ask if I was interested in moving back to Germany for a 'hot' work position. I guess that's why.

</comment><comment><USER23> Goodbye-Civil-Liberties virus.

</comment><comment><USER24> It's a great opportunity for Palantir to do some corona-washing and get their hooks into EU Govt.

On the plus side, Palantir has a reputation for delivering results. Govt needs to do location and contact tracking on a mass scale, integrated with the test regime, and it needs rapid analysis of this data.

Negatives include what they will do for governments post the immediate Corona crisis. I'm expecting forcible deportation on a mass scale to deal with the long tail of hard to manage infected populations, and general xenophobia.

</comment><comment><USER25> That's the last company I want to see involved here.

</comment><comment> Sol- I sure hope governments will be wise enough to not cooperate with Palantir, which is the embodiment of the worst excesses of surveillance capitalism. Managing the spread of the virus is a critical issue and if some surveillance of the population is required, it necessitates a lot of trust and transparency on part of the government. Working with Palantir would undermine that from the start.

In principle, I'd have been open voluntarily to installing such a privacy- preserving bluetooth contact tracing app (provided it's open source and auditable). But knowing that Palantir may or may not be involved in analyzing that data, I'll probably pass.

<comment><USER26> >> ... such a privacy-preserving bluetooth contact tracing app (provided it's open source and auditable

Hey, whip that for the the world, if you don't mind?

Queue the inevitable deluge of "it's not that big of a project, more than 5 Devs would be a shocking waste, etc." comments...

<comment><USER27> Fraunhofer is already field-testing an app based on that concept

</comment></comment></comment><comment> ur-whale [https://archive.is/NHJLn](https://archive.is/NHJLn)

</comment><comment><USER28> FYI, they are already working for a while with the police in some German states (Bundeslaender) [1]. So this is not as big news, as it might seem to some.

[1] Source: Someone who works for them on that.

</comment><comment><USER29> How is a stupid Java app with clueless consultants going to help against covid? Or is it Thiel selling yet more snake oil?

<comment><USER30> Thiel thinks he is the first to discover that government contracts pay exorbitantly for software that under delivers.

<comment><USER31> Inviting a lot of unwanted attention too, given his reputation. This is going to be interesting.

</comment></comment></comment><comment><USER32> no. just no.

</comment><comment><USER33> Palantir sales pitch:

We can provide you a tracking solution, and after the pandemic is over, you can continue to use the product to track and spy on all your citizens. What a perfect way to incorporate our superior technology into your Gov't, all under the auspices of a public emergency !

</comment><comment><USER34> This is fantastic news - if you're gambling on governments failing to contain this virus in a prompt manner.

By the time Palantir ships something that works well, we'll probably be half a year past having a vaccine.

</comment><comment><USER35> It doesn't affect me (because I don't trust my government enough with this data to use that voluntary service, use of any such things tends to expand more and more and deleted data hasn't been deleted), but when I first read that Palantir is one of the companies being talked to I had to laugh.

</comment><comment><USER36> I strongly oppose the existence of companies like this and the sorts of mass surveillance activities that the NSA, GCHQ etc engage in. Having these programs makes core processes in our society like democracy and trial by jury uncertain.

But given we have let them in, I am also amazed how little we seem to demand of them. The NSA should easily be able to track everyone an infected person has been near. That includes people who have been to places shortly after the infected person was there.

So why aren't they doing that? The marginal cost to our society is nothing and this epidemic is already worse than all the terrorism ever in most western nations. These organisations are not just dangerous, they're dangerous and so far totally useless.</comment>


The Current State of E-Commerce Search - <AUTHOR> http://www.smashingmagazine.com/2014/08/18/the-current-state-of-e-commerce-search/

<comment><AUTHOR> This is great. Hats off to the Author, and to the Design and User-Experience Community for putting this kind of information together. As a Developer though, I'm left with a bit of anxiety after reading this.

I think that part of the reason the top 50 E-Commerce sites haven't implemented these features is due to the fact that their impact may not have been demonstrated enough, and articles like this will certainly help address that and bring more attention towards solving them.

The other part of the reason is that these behaviors are difficult to build (IMHO). I admittedly haven't done much research on how to implement complex search features like these, but I bet that the amount of information regarding best practice or established patterns for these search types is lacking. Without that information, the skill level required to design them is relatively high.

I'd love to implement these features in my apps, I feel the frustration of the users. But what I'd love to see now is an article or blog post in response to this that pairs each issue or desired behavior with a pattern or solution.

Making the problem visible is taken care of now, but the development community needs to come up with and/or share the solutions.

<comment><USER2> I would love trying to fix these search problems. The first step would be to make sure you're using decent search software, like SOLR, rather than something constructed ad hoc from the database. That's easy. The next step would be exploring the options the search software provides and coming up with a schema that makes the most of its capabilities. I think a lot of sites go off track at this point, because they don't take the time to customize the search and they let the potential go to waste.

As a consumer, I've all but written off site search engines, and I use Google or bing to search their sites when I really want to find something.</comment></comment>


Ask HN: What's your favorite exit strategy from a webapp - <AUTHOR>

I'm implementing the exit strategy for a web app so a user can export all his data from the service and move on.
I've seen some apps provide a way to do this via an API, a XML file.. even a dump from the DB.
If you were to export your data from a web app what would you prefer?
note: the app is an enterprise-level CRM solution. <comment><USER2> Why not all the above?

You can hack up a quick library (or find one online) that will take the output of a query and output in the selected format.

Not sure what framework/language you are using but, something like this can be very easily done using PHPs inbuilt functions (ie: get from db and store in array..then output array as XML or JSON etc...).

</comment><comment><USER3> Open Source as many parts as possible and provide private data to your customers. I think the data format is less important than good documentation of how to use it again.

<comment><AUTHOR> Thanks for your reply. Haven't thought about providing documentation about that, just adding an option to do it. I don't mind to write tutorials on how to export the client's data to competing services...</comment></comment>


Show HN: Introducing Harp – A static web server with built in preprocessing - <AUTHOR> http://sintaxi.com/introducing-harp <comment><USER2> Nginx is 11 years old and faced similar skepticism from many people in 2002 who made statements about how feature-packed, stable and popular Apache was.

I've been using Harp for about 3 months and really like the sweet spot they have found between a bucket of static files and a full-blown web app stack. It's a very new paradigm and you probably need to try it to fully appreciate it.

<comment><USER3> We're offering another flavor in this same "value add static hosting" paradigm with BitBalloon.

[https://www.bitballoon.com](https://www.bitballoon.com)

So far we don't deal with the authoring abstraction layer. Our focus is on solving deployment by automating performance optimization like asset bundling, caching & CDN integration.

The big vision is to provide static hosting for the programmable web. We make HTML forms work and provide an API for sending the input wherever you want. This OAuth2 based API also handles deploying sites, injecting code snippets, and updating individual files.

The real moonshot here is developing a standard similar to JSON Patch for updating HTML via PATCH requests. If Harp, or anyone reading this is interested in this idea, we'd love to collaborate on developing a spec.

</comment></comment><comment><USER4> One interesting LESS feature I found when we were evaluating it for letting some users use it for customization: it can evaluate arbitrary Javascript and open and embed files. Just in case you thought it was some pure transformative shorthand CSS syntax.

Of course, letting users upload arbitrary CSS served from your domain is an XSS-level risk, but let's say you let each user have his own domain and let them upload .less files.

Or you have a say, node.js APP that lets a user enter LESS and spit out the -- escaped -- CSS.

Now they can do:

        div.hack {
          background-image: data-uri("/etc/passwd");
        }

When compiled with lessc 1.4.2, this embeds the contents of that file as base64.

Also, with 1.3.x."@import" lets you open any filename and insert it, but it's required to be valid .less. I suppose you could try to @import /dev/random or urandom although I don't know if there's any practical attack surface for decreasing entropy. I don't think I could offhand find a valid file.

However in the theorethical web page evaluating .less, the error output will contain the contents of the file @imported.

Oh, and the evaluation of Javascript... I couldn't make it do much useful, as it seems there's some kind of limited JS environment that code executes in. The worst I could do was:

        div {
          background: `process.kill(-1, "SIGKIL")`;
        }

The "fs" object does not seem to exist there, but I don't know enough about node.js/V8 environment to see whether there's equivalent of jailbreaks.

So, uh, don't compile arbitrary LESS files and send users their output, and LESS files someone sends you or checks into a project can kill your processes (at least).

<comment><USER5> In 2013, does it make sense to put each user in a docker/lxc container? I don't know the resource overhead, but I'd hope/expect that it would be within tolerances.

Then you can have rich features like includes without worrying about file- based info leak. (cpu-timing attacks are perhaps another kettle of fish).

</comment></comment><comment><USER6> "Build on request" is a good idea. We do something similar in our stack: tup[0] as the (insanely fast) build system, and developer VMs run mitmproxy[1] on port 8080 which runs tup before every request.

I prefer this style of a setup to coupling them together a-la Harp. If the webserver doesn't work out, I can swap it out. Same with the build system. Tying them together makes me antsy. And for good reason I think: since 2004, we have switched from Apache -> Lighttpd -> Nginx, and have switched from custom shell scripts -> Make -> Tup (with some fabricate.py, and with calls out to other build systems like Leiningen)

[0] [http://gittup.org/tup/](http://gittup.org/tup/)

[1] [http://mitmproxy.org/](http://mitmproxy.org/)

<comment><USER7> As a sidenote, I appreciate the sense of humor that the tup author has:

"In a typical build system, the dependency arrows go down. Although this is the way they would naturally go due to gravity, it is unfortunately also where the enemy's gate is. ... In tup, the arrows go up. This is obviously true because it rhymes."

"<diagram of tup vs make> See the difference? The arrows go up."

[http://gittup.org/tup/tup_vs_mordor.html](http://gittup.org/tup/tup_vs_mordor.html)

<comment><USER8> It is not a reference to Ender's Game and "the enemy is always down"?

</comment></comment></comment><comment><USER9> Another nitpick about the source code:

Let the code breath a little -- no whitespace around control structures is a bit jarring:

      if(error){
        ...
      }else{
        ...
      }

Instead:

      if (error) {
        ...
      } else {
        ...
      }

Maybe adopt this Node style guide: [http://nodeguide.com/style.html](http://nodeguide.com/style.html)

<comment><USER10> > Do not extend the prototypes of any objects, especially native ones. There > is a special place in hell waiting for you if you don't obey this rule.

A strong statement whose motivation escapes me; it's not as though you can't trivially get the source of the extension in your REPL of choice.

<comment><USER11> Naming conflicts aside (what if your definition of "empty" doesn't match some other part of the code?), consider the `Array.prototype.empty` example. You extend `Array`, then some other bit of code _anywhere else_ in the environment decides to do the following:

      var pets = ["dog", "cat", "rabbit", "turtle", "owl", "alligator"];
      for(var i in pets) {
        console.log(pets[i]);
      }

And what do you get?

      dog
      cat
      rabbit
      turtle
      owl
      alligator
      function () {
        return !this.length;
      }

Oops! And here you could argue that the other coder should've known to use hasOwnProperty or forEach or whatever, but, all that aside, extending native object prototypes is an easy way to break code all over the environment.

<comment><USER12> This actually can be done safely with Object.defineProperty, using non- enumerable properties.

<comment><USER13> That doesn't work in IE8 (for non-DOM objects).

IE8 is the last version of IE for WinXP.

<comment><USER10> If you have to support IE 8, you probably shouldn't extend prototypes without serious thought, especially in a legacy codebase, because the interpreter support isn't there to do it right.

On the other hand, Windows XP is barely six months shy of EOL, and IE 8 currently has something like 4% share. Now would seem to be an excellent time to start dropping support for IE 8. Such support is never going to gain you more users, and many other browsers run on Windows XP.

"Don't use Object.defineProperty because a dying browser on a dying platform doesn't support it" strikes me as a rather weak argument. I mean, there are still people out there using IE 6, God help them. Why not shoot as low as we can?

<comment><USER13> The company I work for makes software for banks.

Neither WinXP nor IE8 are dead there, not by a long shot.

</comment></comment></comment></comment></comment></comment></comment><comment><USER14> Reminds me of Middleman ([http://middlemanapp.com/](http://middlemanapp.com/)) and of my own [https://github.com/stdbrouw/draughtsman](https://github.com/stdbrouw/draughtsman), which has sadly languished. I've found it indispensable for prototyping, not just because of the precompilation but also because it'll search for metadata on the filesystem that you can use to flesh out those prototypes with real data.

<comment><USER15> Also reminds me somewhat of Mixture ([http://mixture.io/](http://mixture.io/)). Not open-source, but aiming to solve similar problems.

</comment></comment><comment><USER10> That Rails vs. Angular graph would sure be useful in arguing for a move away from Rails, except that it's unsourced and unitless on the Y axis. I'd love to know how it was generated and from what data, so I can make one that represents the same information in a way that's suitable for my purposes.

<comment><USER16> Not really. Those data points are independent. A lot of people use Angular and Rails together. Rails still provides a nice controller and model layer for an API.

<comment><USER10> I find it undesirably weighty for such purposes, and potentially limited in future prospects by the politics of its developer community. Decoupling the view layer from the backend makes replacement of the latter, with something offering an identical API but better prospects, easier to contemplate.

</comment></comment></comment><comment><USER17> This doesn't seem to be on the right level of abstraction - why replacing the entire web server, adding dynamic translating (compilation) as a main feature?

<comment><AUTHOR> I look at it this way, we basically have two types of web servers, static and full-stack web frameworks. What if all you want is a static web server but you need a way for your files to share a common layout? If you don't have any server side compilation a full-stack framework is overkill. We could use a static site generator such as Jekyll and then put behind a static web server but now we have ugly urls, no 404 pages, and way more dependencies than needed.

The simple fact is a static web site on its own is not good enough but its 95% there. Using a full-stack framework is overkill, and needing an Static Site Generator just seems counter productive when considering simplicity is one of the biggest upsides to shipping static assets.

There is no good reason for static web servers to do a little extra for you like smarter redirects, clean urls, layouts/partials. This makes sense to me considering the direction we are headed.

<comment><USER18> Why shouldn't I call Harp a static site generator? I understand it's more than that, but a SSG is one use case.

<comment><AUTHOR> Harp does work great as a Static Site Generator but the reason why we avoid that term is that there have been several cases of people using Harp as a SSG by running the compile step each time they save a file. This is how they expected it to work (likely because of their experience with Jekyll). It has even gone so far as someone requesting a way to "preview" the compiled output not realizing that not only can harp serve a preview, it removes the need to generate the compiled output all together.

It really comes down to the mental modal of how people look at the tool and how they expect it to behave. Avoiding the term SSG has helped with that.

</comment></comment></comment></comment><comment><USER19> _" The obvous question with building front-end applications is where does the state live? Fortunately there are many services ..."_ Does any body know what is a great solution for simple Contact Forms?

<comment><USER20> A service like that was recently featured on the front page: [http://www.squaresend.com/](http://www.squaresend.com/). Several similar solutions and products were mentioned in the discussion at [https://news.ycombinator.com/item?id=6468113](https://news.ycombinator.com/item?id=6468113).

</comment></comment><comment><USER21> I like the idea, but I don't understand the problem they're trying to solve. For example I have an app using node as the backend behind nginx and ember as the front end.

Right now I'm using grunt to precompile my app to plan old HTML / JS /CSS but there are modules for node that can also compile and cache for me, Either way, I don't have to mess with something that works (NGINX)

<comment><AUTHOR> Configuring Grunt to do this every time you want to ship a web app seems like a path to insanity to me. Also, NGINX doesn't by default give you clean urls and 404 fallbacks. Web frameworks do this for you and IMHO It's time static servers do as well.

Regenerating all the assets every time there is a change just feels sloppy to me. Also, I think it is asking a lot from novice developers to setup these configuration files and the knowledge one needs to have to do it effectively is a needless burden on someone who specializes on front-end development.

No disrespect to NGINX. Its too busy powering the internet to be concerned with these problems :)

</comment></comment><comment><USER22> I currently have Grunt + the RequireJS Optimizer building ~5 js files from 20+ AMD modules. So, with harp I would move that build script away from my app and into Harp? Sounds very interesting, but does this separate the application from the build script?

Less/Sass seems easy, but I'd like to see more regarding JS build/min/concatenation scripts in Harp. Or is this not its goal?

</comment><comment><USER23> Will Harp support adding plugins?

For now, I use Wintersmith. It supports Jade and Markdown out of the box, and you can extend it by writing plugins.

I wrote one to help me "load" dependencies, without having to use <script> tags. I use Devon Govett's importer[1] for that.

[1] [https://github.com/devongovett/importer](https://github.com/devongovett/importer)

</comment><comment><USER24> This is really neat.

The most horrible thing about trying to build out Node.js applications these days is wishing for the Rails asset pipeline.

[http://harpjs.com/docs/environment/lib](http://harpjs.com/docs/environment/lib) looks like it should make it very possible to tie this up next to rendr and get the best of all worlds.

<comment><USER25> [https://github.com/adunkman/connect- assets](https://github.com/adunkman/connect-assets) seems pretty close?

</comment></comment><comment><USER26> Harp server (direct link: [http://harpjs.com](http://harpjs.com)) seems to be powering [https://harp.io/](https://harp.io/) (paid service), didn't notice it mentioned in the article.

<comment><AUTHOR> Shhhh, there is more to come ;)

We use Harp to power our development platform that has Dropbox integration. It is currently in public Beta and I'll definitely be talking about it in the future.

There is no mention of it in my article because we didn't want our business to get in the way of the open source initiative. We believe in Tim O'Reilly's guidance "Create More Value Than You Capture".

</comment></comment><comment><USER27> So this is like Apache's mod_* but for modern things. Sounds neat. One of the things I like most about the Play Framework is that it autocompiles LESS, CoffeeScript and Scala on refresh. Extending that concept to more languages is a good thing.

<comment><USER28> Rails has been doing this for years, I'm sure other framework do it too.

</comment></comment><comment><USER29> How easy is it to build nginx modules to accomplish similar preprocessing?

<comment><AUTHOR> We are drafting a spec so this will hopefully happen one day (still a work in progress).

[http://spec.harp.io/](http://spec.harp.io/)

I think the big challenge will be with the languages themselves. LESS, Stylus, CoffeeScript, and Jade are all implemented in JavaScript. Finding C implementations would be tough. Though SASS has a C implementation.

</comment></comment><comment><USER30> Does Harp have livereload support? Kill all the refresh buttons (for dev)! That would be my only must have for using a different dev server than grunt based for JS app dev

<comment><USER31> Live reload is in the roadmap, stay tuned!

</comment></comment><comment><USER32> For all skeptics, this can actually make lot sense with the current service- oriented web.

Actually, this is where I tried to go with Punch([http://laktek.github.com/punch](http://laktek.github.com/punch)) too. Vast majority of the content in web can essentially be static, it's just the minor inconveniences of plugging data sources, updating and managing them is what should be addressed.

</comment><comment><USER33> I have been using simple Node HTTP server to fulfill my small need[[http://stackoverflow.com/a/13635318](http://stackoverflow.com/a/13635318)]. And of course, Node's event-driven architecture will help Harp. But, I think Nginx or Apache is quite featured web server you are trying to replace.

</comment><comment><USER34> I am using docpad ([http://docpad.org/](http://docpad.org/)) and I want to try mimosa ([http://mimosa.io/](http://mimosa.io/)). Does Harp compete with this frameworks? I would appreciate any comments on this because I might consider Harp as another option.

</comment><comment><USER35> This seems like a less mature and more opinionated edition of DocPad? - [http://docpad.org](http://docpad.org)

I find it hard to understand why one would use this over DocPad... Can someone fill me in, I'm curious to know what the interest is about in case I'm missing something!

</comment><comment><USER36> The idea is cool but I would never serve my static sites with NodeJS. Don't ask why. Benchmark it.

<comment><AUTHOR> NodeJS is plenty fast enough. When the dust settles we will provide benchmarks alongside NGINX. We're not pulling the wool over anyones eyes, I don't expect Harp to reach NGINX speeds anytime soon (or perhaps ever). That is not the focus of this server. Faster web server than NGINX is not what developers need. They need something that is easier to use, install, doesn't require any configuration, and more importantly is a suitable development tool on its own. Our focus are on the needs of front-end developers not Server operators.

We are publishing a spec for how Harp compatible servers should behave and we expect to see other implementations in the future.

<comment><USER36> I benchmarked static servers in NodeJS, using couple of different libraries such as "st" etc.

They're 7 times slower than a static site server that I wrote in Go.

Forget comparing with Nginx.

<comment><USER37> How big is the difference when you put them all behind varnish?

</comment></comment></comment></comment><comment><USER38> This looks like it would be useful for a simple AngularJS app of mine where we use LESS, CoffeeScript, and Haml. We had written a simple Rack server to compile assets on demand. It'd be nice to get rid of the config.ru file and just use Harp as a drop-in replacement.

Does Harp handle Haml?

</comment><comment><USER39> This is the best thing ever if you're pulling data from flat JSON files, and I would prefer it to any static server simply because I would rather not worry about asset pipelining when working on a static, simple website.

</comment><comment><USER40> Nitpick: on the harpjs.com site, under the "The beloved Layout/Partial paradigm" header, you have "orginized" instead of "organized".

Looks like an interesting project, though!

<comment><AUTHOR> fixed. Thanks :)

<comment><USER41> And: warrents -> warrants

</comment></comment></comment><comment><USER42> I don't think letting node.js handling static files is the very best idea.

<comment><USER43> I ask genuinely: why?

</comment></comment><comment><USER44> Any benchmarks available?

</comment><comment><USER45> Stopped reading after this spelling mistake on the first line of text:

"We already have extreamly reliable"

<comment><USER46> Why would you stop reading? Are spelling errors that outrageous to you?

<comment><USER47> unhandled exception, maybe :P</comment></comment></comment>


Meet Facebook's new number two - <AUTHOR> http://money.cnn.com/2008/04/11/technology/facebook_sandberg.fortune/?postversion=2008041116

<comment><USER2> Sandberg was a good part of the "business" side of Google's rise to being the most important company on the web. If Facebook has also managed to hire someone who can come up with an AdWords-style technology that makes all of those eyeballs staring at Facebook worth more than a few pennies each, Facebook might eventually be worth some of the hype.

I've never met Sheryl but I know a lot of the folks that worked directly under her at Google, and they were universally in awe...so, she's probably just the right kind of grownup Zuckerberg needs to have around.

</comment><comment><USER3> The problem with the social network is that it's more personal than search. Search, while personal, can be made innocuous by keeping annonymous information.

The holy grail of the social graph is to find out who the viral supernodes are. By that measure it's no surprise they went straight for the purchase history. If you can pinpoint the people who most influence other people you've got the potential for a very powerful statistic which can be fed into a viral marketing campaign.

</comment><comment><USER4> I can't trust any article that says Microsoft's investment gave Facebook a valuation of $15 billion. It wasn't a pure investment: an ad deal was part of it.</comment>


A Puzzle of Clever Connections Nears a Happy End - <AUTHOR> https://www.quantamagazine.org/a-puzzle-of-clever-connections-nears-a-happy-end-20170530/ <comment><USER2> As soon as Ronald Graham turned up it wasn't surprisinng that this was related to Ramsey Theory, even as a non-mathematician. Graham's Number is such a wonderfully large thing that introduced me to tetration and up-arrow notation. [https://en.wikipedia.org/wiki/Graham%27s_number](https://en.wikipedia.org/wiki/Graham%27s_number)

</comment><comment><USER3> I like the terms "cup" and "cap". They correspond to the LaTeX shortcuts \cup and \cap for set union (∪) and set intersection (∩).</comment>


September 26th is Petrov Day - <AUTHOR> http://lesswrong.com/lw/jq/926_is_petrov_day/

<comment><USER2> My philosophy at work toward my leadership has always been "You're paying me to be right, not to agree with you."

It's just an application of the golden rule, in my case. I want to work with smart people who are confident enough to disagree with me.

Still, it's a much easier to position to take when your job isn't the release of nuclear weapons. Bravo to Mr. Petrov and his substantial cohones.

<comment><USER3> I think politicians think in _priorities_ and not _right or wrong._

</comment></comment><comment><USER4> Spasibo tovarisch!</comment>


Elastic Network Adapter – High Performance Network Interface for EC2 - <AUTHOR> https://aws.amazon.com/blogs/aws/elastic-network-adapter-high-performance-network-interface-for-amazon-ec2/ <comment><USER2> Does anyone know what type of hardware they are using to accomplish this? Is it just your standard Intel X520/540 10GbE adapter, or is there some secret hardware they have access to?

FWIW, saturating line rate 10GbE has been problematic at times in the past due to challenges like single-threaded drivers, which could only run on 1 core at a time, even if there are 128 logical cores in a system (like the X1 instances). There are also challenges offloading VXLAN encapsulation to hardware at 10GbE line rate, but it seems like AWS has figured that out.

<comment><USER3> Since it offers 20Gbps to the VM, i'd guess it's a 25GbE NIC, since those and accompanying switches have just come out.

VXLAN in the NIC (and the switch) at full line-rate offload has been a thing for at least 18-24 months.

Most Modern NICs also support all of the features listed here, like checksum offload, Receive steering, multi-queue, etc. Check out Mellanox CX4/5, Intel X710, QLogic, and Netronome if you want fancy stuff. They aren't even that expensive anymore.

Their Linux driver has support for every current and future standard: 1/2.5/5/10/25/40/50/100/200/400G, nice future proofing.

They also have a DPDK driver: [http://dpdk.org/browse/dpdk/tree/drivers/net/ena/base/ena_co...](http://dpdk.org/browse/dpdk/tree/drivers/net/ena/base/ena_com.c)

<comment><USER2> Thanks, but my guess is that it's actually a bonded 2x 10Gbps link, which would also give you 20gbps, and cost a lot less per port.

<comment><USER3> With current per port costs and economics, that doesn't make much sense. The $/Gbps/Watt improves with every spin of the chip, and every increase in serdes bandwidth.

10GbE (64x10G) - ~$4,500, $70/port

10GbE (32x40G) - ~$6,000, $50/port

25GbE (72x25G) - ~$7,000, $100/port (Seen as low as $5k)

25GbE (32x100G) - ~$9,000, $75/port (Seen as low as $7k)

2x 10GbE = $100

1x 25GbE = $75

Not to mention the extra cabling costs, and complexity.

</comment></comment></comment></comment><comment><USER4> Huh -- why is Amazon introducing ENA as a paravirtual virtual device? They already provide Xen netdevs to VMs.

The Xen PV device would be able to use the (more efficient) xen bus/event channels for interrupts vs. ENA (MSI-x, by looking at their driver on the Linux netdev mailing list).

The Xen PV device already also supports multiple queues for xmit, see netback_init() in netback.c.

2\. The ENA LLQ interface (pushing short packets via CPU write-combiner writes) only makes sense when you are writing to a physical PCI-e/Hypertransport/QPI device.

I suspect that this ENA NIC is at least partly implemented in silicon/firmware rather than dom0 s/w.

Perhaps something like Azure's smart NICs? (SIGCOMM 2015 keynote)

</comment><comment><USER5> 20gpbs is fast enough for a raw 4K video stream at 60 frames per second in ABGR format (e.g. what you'd get out of a video card). That's slick.

</comment><comment><USER6> Tl;dr 20gbps virtual Nic

<comment><USER2> Oversimplified - it's also a multi-threaded driver with multiple queues. This should hopefully allow you to fully saturate the 20gbps available to you.</comment></comment>


On "Quantum Supremacy" – IBM - <AUTHOR> https://www.ibm.com/blogs/research/2019/10/on-quantum-supremacy/ <comment><USER2> Don't blow out someone else's flame to make yours burn brighter?</comment>