South Korea is stuck with Internet Explorer for online shopping - <AUTHOR> http://www.washingtonpost.com/world/asia_pacific/due-to-security-law-south-korea-is-stuck-with-internet-explorer-for-online-shopping/2013/11/03/ffd2528a-3eff-11e3-b028-de922d7a3f47_story.html <comment><USER2> I've lived in Korea for quite a length of time and my wife is Korean...and I concur that it's a nightmare. No Korean financial institution will let you log into their interfaces on anything other than IE. Often the homepage would be completely broken in anything other than IE anyway. This is true for most sites, though those were easily avoidable. Banks and the like, not so much.

To perform financial transactions online in Korea, you would need a plethora of software (often one from each party you would deal with) that revolved around security certificates that were issued by the banks that would store a hard copy of the certificate locally on your computer. Often it didn't work at all, not even getting into the security implications of the system. Bank hacking is so common in Korea, it's really disturbing. There is absolutely no accountability where the attempts at security do exist.

Also, you need to use your Citizen Number (basically a Social Security Number) to register for ANY service in Korea, even common websites. So everything you do can be traced via that single number. For foreigners, registering for common sites is usually impossible because our alien numbers are stored wherever normal citizen numbers are, so unless the site has a separate process for foreigners, you'd be out of luck. It's quite a mess. I can't say enough bad things.

On the bright side, start ups like Vingle in Seoul are doing a lot of tip the scales for the younger generation by only supporting modern browser versions (IE8+, not the most modern, but definitely a step up from IE6, which has a huge market share still, too), but it's a slow change.

<comment><USER3> > Also, you need to use your Citizen Number (basically a Social Security > Number) to register for ANY service in Korea, even common websites.

Since 2011, websites cannot ask or store resident registration numbers (that's the official name) for non-financial purposes. Sadly, it happened after a major incident which exposed RRNs of more than 70% of Koreans. [1] It is a common estimate that every Korean person have his/her RRN hacked at least twice due to frequent incidents.

The Korean government endorses i-PIN nowadays, which is basically... uh... redundant aliases to the unique RRN. This is obviously stupid, you can hack i-PIN instead of RRN and you have the same credential. Well, at least i-PIN is random. (RRN had very low entropy, and even shallow information about the target may limit possible RRNs to only hundreds.)

[1] [https://en.wikipedia.org/wiki/Resident_registration_number#O...](https://en.wikipedia.org/wiki/Resident_registration_number#Online_use) for the 2011 incident.

<comment><USER4> That's really f'd up.

Israel has an "ID number" system, which you use when interfacing with e.g. health providers, or when applying to an academic institute. However, it is not assumed to be secret, and any action that would require positive identification will have it done with a physical government issued ID (national ID card, national driver's license, or passport) - knowing the number is not enough.

The system is far from perfect - there is a lot of information leakage, but identity theft requires forgery of physical artifacts, and more often than not - appearance in person - so it is not as common as e.g. in the US.

</comment></comment></comment><comment><USER5> It is strange that I think of this after reading this, but this seems to be the dystopian future scenario that freaked out the DOJ due to Microsoft bundling the IE browser with Windows. It is strange that this situation actually came about, but due to reasons of law, not due to Microsoft's own market clout.

<comment><USER6> It isn't strange, most monopolies are born out of bad government policies, not the market.

<comment><USER7> > It isn't strange, most monopolies are born out of bad government policies, > not the market.

Citation needed. There are government mandated monopolies but you can also achieve a monopoly using purely free market tools and simply lock up all supply and/or distribution via aggressive investments.

You seem to imply government-induced monopolies > free market monopolies, but provide no evidence or even an argument that this is indeed true. You just state it as a universally recognized fact, when that is obviously not the case.

<comment><USER8> > Citation needed.

Any real economist.

Inevitably, if any industry is profitable, competition will arise. Government regulation (often through corruption) is the only mechanism which ensures a lack of competition. This can be observed in any industry throughout the last 100 years.

You're right that companies can lock up the vast majority of a market by out- competing others, but this generally doesn't last very long (typically a decade or less).

<comment><USER9> > Any real economist.

Any real economist can cite many monoplies that arise through market forces alone.

> Inevitably, if any industry is profitable, competition will arise.

This is not true. There are many different definitions of "profit", and there are many cases where an industry that makes a real business profit will not support a competitor that makes an economic profit. The obvious example is markets with very large economies of scale [1].

> Government regulation (often through corruption) is the only mechanism which > ensures a lack of competition. This can be observed in any industry > throughout the last 100 years.

I'm interpreting this to mean "only through government intervention can we ensure a lack of competition". Again, this is not true. Any party that can make a massive capital investment (which already vastly reduces the pool of competition) can enter a space and make any future entrance by a competitor completely unprofitable.

To see an example of this, lets go a century back in time to the day of Standard Oil. Standard Oil was notorious for leveraging its massive capital advantage to destroy its competitors. It would enter a new market and lower its prices (leveraging its massive war chest). Once its competitors left business, it would raise prices again to screw over consumers. After many people caught wind of this blatant market manipulation, it turned to deceptive practices and things like tying agreements [2].

What government policy led to the dominant monopoly of Standard Oil?

> You're right that companies can lock up the vast majority of a market by > out-competing others, but this generally doesn't last very long (typically a > decade or less).

Standard Oil was supreme for over thirty years. If only we could invent a time machine, to hear the gales of laughter from the businessmen of the day at the notion that Rockefeller became the king of oil by "out-competing" others.

He became dominant through backroom deals and anti-competitive practices. Government intervention was what finally ended the Standard Oil monopoly. It was also likely the only thing (barring the death of Rockefeller or some kind of market shift) that ever would.

1\. [http://en.wikipedia.org/wiki/Economies_of_scale](http://en.wikipedia.org/wiki/Economies_of_scale) 2\. [http://en.wikipedia.org/wiki/Standard_oil#Monopoly_charges_a...](http://en.wikipedia.org/wiki/Standard_oil#Monopoly_charges_and_anti- trust_legislation)

<comment><USER8> > He became dominant through backroom deals and anti-competitive practices.

Backroom deals AKA corruption. Corruption and regulation are two sides of the same coin. Had the market been perfectly competitive (ie. state governments not succumbing to corruption) the monopoly likely would not have formed and lasted.

Your only example merely proves my point.

Try to find a monopoly that has arisen in an open market, free from government 'intervention' (either regulation OR corruption).

<comment><USER9> It's clear from your response that you didn't read the cited link.

> Backroom deals AKA corruption. Corruption and regulation are two sides of > the same coin.

You seem to labor under some obtuse notion that the state governments were the subject of the backroom deals. Let me dispel that for you, by quoting from the article cited that you apparently couldn't be bothered to read:

> In a seminal deal, in 1868, the Lake Shore Railroad, a part of the New York > Central, gave Rockefeller's firm a going rate of one cent a gallon or forty- > two cents a barrel, an effective 71 percent discount from its listed rates > in return for a promise to ship at least 60 carloads of oil daily and to > handle the loading and unloading on its own

> Rebates, preferences, and other discriminatory practices in favor of the > combination by railroad companies; restraint and monopolization by control > of pipe lines, and unfair practices against competing pipe lines; contracts > with competitors in restraint of trade; unfair methods of competition, such > as local price cutting at the points where necessary to suppress > competition; [and] espionage of the business of competitors, the operation > of bogus independent companies, and payment of rebates on oil, with the like > intent.

> The general result of the investigation has been to disclose the existence > of numerous and flagrant discriminations by the railroads in behalf of the > Standard Oil Co. and its affiliated corporations. With comparatively few > exceptions, mainly of other large concerns in California, the Standard has > been the sole beneficiary of such discriminations. In almost every section > of the country that company has been found to enjoy some unfair advantages > over its competitors, and some of these discriminations affect enormous > areas.

> Almost everywhere the rates from the shipping points used exclusively, or > almost exclusively, by the Standard are relatively lower than the rates from > the shipping points of its competitors. Rates have been made low to let the > Standard into markets, or they have been made high to keep its competitors > out of markets. Trifling differences in distances are made an excuse for > large differences in rates favorable to the Standard Oil Co., while large > differences in distances are ignored where they are against the Standard. > Sometimes connecting roads prorate on oil—that is, make through rates which > are lower than the combination of local rates; sometimes they refuse to > prorate; but in either case the result of their policy is to favor the > Standard Oil Co. Different methods are used in different places and under > different conditions, but the net result is that from Maine to California > the general arrangement of open rates on petroleum oil is such as to give > the Standard an unreasonable advantage over its competitors

> The evidence is, in fact, absolutely conclusive that the Standard Oil Co. > charges altogether excessive prices where it meets no competition, and > particularly where there is little likelihood of competitors entering the > field, and that, on the other hand, where competition is active, it > frequently cuts prices to a point which leaves even the Standard little or > no profit, and which more often leaves no profit to the competitor, whose > costs are ordinarily somewhat higher

Note that the word "government" appears nowhere in any of these allegations. All of these backroom deals existed with other market participants.

If you redefine "corruption" to mean "not involving the government whatsoever" then your points are indeed true, but you are then proving the exact opposite of your initial assertion that "only governments can create monopolies".

Anyway, it's not clear if you're a troll or ignorant at this point, and I doubt that further effort to dispel your quaint notions will be worth my invested time.

<comment><USER8> Well first of all, I did glance at the article, and now concede I may have been mistaken. However, there have been charges of government corruption vis à vis Standard Oil.

[http://www.pagetutor.com/standard/chapter13_part1.html](http://www.pagetutor.com/standard/chapter13_part1.html)

The whole text. [http://www.pagetutor.com/standard/toc.html](http://www.pagetutor.com/standard/toc.html)

It does seem as though they (mostly) legitimately competed, and given the historic oil price throughout their reign (which fell drastically, [http://www.pagetutor.com/standard/chapter16_part1.html](http://www.pagetutor.com/standard/chapter16_part1.html)) it doesn't seem as though they exercised monopoly power (at least not nation- wide).

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> I covered this on my blog in 2007: [http://kanai.net/weblog/archive/2007/01/26/00h53m55s](http://kanai.net/weblog/archive/2007/01/26/00h53m55s)

My blog post was heavily covered in Boing Boing, Slashdot, Salon, etc. at the time.

<comment><USER11> So nobody else is allowed to write about it ever again?

<comment><USER10> Where did I say that? My point is that in the 7 years since 2007, effectively nothing has changed.

<comment><USER11> "My blog post was heavily covered in Boing Boing, Slashdot, Salon, etc. at the time."

Ugly boasting.

<comment><USER12> Or effective means of making a point.

</comment></comment></comment></comment></comment><comment><USER13> I lived in South Korea for a while and this "security plugin" you have to install for IE is an absolute nightmare. Some sites even ask you to install a separate application to be able to perform transactions, only through IE of course...

Truth be told, I never actually managed to buy anything online when I was there. It's like everything was designed to keep me from buying. What didn't help was that I was on Visitor status, meaning you don't get your national ID, which is required on a vast number of SK sites. Without ID, you just become some kind of virtual hobo.

It's a shame considering the amazing infrastructure there is over there. Anyone who's ever visited SK websites will tell you how poorly put together they are, both technically and visually. I've rarely seen such disparity between the underlying infrastructure and its use anywhere else.

</comment><comment><USER14> An example of a premature optimization and what not following standards can do.

Some more background [http://en.wikipedia.org/wiki/SEED](http://en.wikipedia.org/wiki/SEED) and [http://kanai.net/weblog/archive/2007/01/26/00h53m55s](http://kanai.net/weblog/archive/2007/01/26/00h53m55s)

There was some hope last presidential election cycle that this would become a topic for the new administration to tackle [1], but that candidate (Ahn Cheol- soo) lost the election and it seems to have fallen off the table for now.

1 - [http://www.theregister.co.uk/2012/11/14/ahn_lab_internet_exp...](http://www.theregister.co.uk/2012/11/14/ahn_lab_internet_explorer_seed_replace_korea/)

</comment><comment><USER15> Typical of most MS bashing articles - this article is troll-bait at best and false at worst. Internet Explorer is not mandated by law.

What happened was, the US had banned export of 128bit encryption software. The Korean government said screw that and created browser plugins - for BOTH Netscape and IE - to use 128bit encryption for online transactions. Netscape died and IE remained. I guess their implementation is proprietary enough that nobody else has managed to implement it on other browsers.

<comment><USER16> The 128-bit export restriction ended 13 years ago, so the use of the home- grown encryption standard has been a choice made by the Korean government and corporations. Yes, the export restrictions were stupid, but there has been plenty of time to switch over.

NPAPI is supported in most browsers, so the SEED implementation had to have been pretty tied to Netscape Navigator/Communicator implementation as it existed pre-Firefox

If the government implementations are the de facto standards and the Korean government has only maintained an ActiveX version for the last 10 years, I can't see how you can interpret it as anything other than an IE mandate.

<comment><USER15> >If the government implementations are the de facto standards and the Korean government has only maintained an ActiveX version for the last 10 years, I can't see how you can interpret it as anything other than an IE mandate.

Because the government is not mandating IE? The _public_ overwhelmingly used IE over Netscape which caused the government to go "okay since nobody is using netscape we're going to only maintain one plugin". I think its a pretty important distinction to make. Anyway, we disagree. No biggie :)

<comment> guard-of-terra Having to choose between dead Netscape and alive IE is a fake choice. May I have my third choice please?

<comment><USER15> What they should really do is open source their plugin or at the very least their spec.

</comment></comment></comment></comment></comment><comment><USER17> I'm really surprised that in the last 14 years noone wrote a compatibility layer. It's definitely possible (see the comment about tablets) and embedding crazy stuff from windows is nothing new in linux world (wine, ndiswrapper).

So what's the actual barrier to doing that? (also, why doesn't FF solve the issue since [https://bugzilla.mozilla.org/show_bug.cgi?id=478839](https://bugzilla.mozilla.org/show_bug.cgi?id=478839) was fixed? - it looks like guys from KISA are actually cooperating to implement the needed ciphers)

<comment><USER10> Mozilla added SEED support to Gecko many years ago. Nothing has been done since then (on the Korean side) to implement support beyond that.

<comment><USER17> But what exactly is needed? What is stopping someone from writing a FF extension that will capture all activex object tags and replace them with something doing the same operations in FF's chrome?

Is the auth protocol completely unknown? Is the activex control obfuscated more than is possible to reverse-engineer?

<comment><USER18> ActiveX is designed to hook directly into the Windows OS. That's what makes it so dangerous, but useful in this case, since you can add a cert to the trusted store.

Firefox is designed to be secure and sand-boxed, especially its plugin architecture.

<comment><USER19> You don't need to replicate all of the functionality that activex has. You just need to emulate the behaviour from the banks point of view of whatever their particular activex does. Maybe the ActiveX rewrites a bunch of files with admin privileges "for security", and then negotiates some sort of key exchange... in that case just write JS that _says_ it did the shit that requires admin privileges, then negotiates the key exchange).

So the question is, if you are willing to ignore their activex and run your own custom JS instead, could these websites be made to work?

<comment><USER20> A major online bookstore (www.aladdin.co.kr) actually tried that this year. They teamed up with another startup company (Paygate) and allowed users to use credit card with no plugins, on any browser. The few people who tried that loved it.

And guess what happened?

Major credit card companies pulled out one by one, because they "cannot ensure" that a page without Active-X is secure enough. Of course nobody's pulling any strings, no government officials are receiving unknown gifts, and nothing can be ever proved. So, there. You work for months to provide users with modern browsing experience, and those banking powers-that-be just pull the plug.

The whole system is corrupt beyond imagination.

Citation (sorry, in Korean): [http://www.hankyung.com/news/app/newsview.php?aid=2013091203...](http://www.hankyung.com/news/app/newsview.php?aid=201309120399g) [http://www.leejeonghwan.com/media/archives/002331.html](http://www.leejeonghwan.com/media/archives/002331.html)

<comment><USER17> That's a solution from the provider's side though. I would have thought that it's much easier to handle it from the client side really... just pretend you did whatever verification was necessary and return the expected result.

Noone should be able to shut it down, because it's on the client side, rather than the service provider, so the retailer shouldn't be blamed.

<comment><USER20> No no no, it doesn't work that way. These banking websites don't expose a well-defined API that you can emulate. Instead they force you to install a bunch of ActiveX plugins (usually with administrative privilege) and you have to just trust that they won't, say, read the whole content of your hard disk and stream it to a third-party site.

A few years ago (when I was still in Korea), it was usually impossible to open two different bank's pages at the same time: I'd assume that's still the case now. As far as I know, the reason is that both banks will force you to install "anti-hacking" plugins, which hooks directly into your Windows Kernel and makes sure nobody _else_ snoops on what you type. Yes, these webpages try to establish a direct connection between your keyboard and the website, completely bypassing every layer. Now imagine the fun when _two_ such plugins try to run at the same time.

And of course without these plugins you can't use the site. Hell, sometimes these sites spontaneously break just because you're accessing it from the US, because nobody had thought to test them from a client with ping time > 200ms.

Now try emulating that in client. (I don't know if I should laugh or weep.)

EDIT: Besides, if you seriously try to make a platform that can emulate these "security" plugins, sooner or later you will be arrested for making tools to circumvent security measures, and people will be reminded that they should never install anything from "suspicious" sites. (But of course install everything from banking sites.) As a bonus, some news media will claim you were paid by North Korea, and many will believe that.

<comment><USER21> Thanks for the great explanations. It actually makes some sense that Koreans are getting hacked all the time, since the binary rootkits they're required to install have probably not seen the scrutiny that more open solutions have. Secure systems used in the rest of the world get hacked because of weak entropy or through timing attacks: fairly esoteric stuff. It wouldn't surprise me if these ActiveX blobs have basic algorithmic errors. Do they crash quite a bit?

<comment><USER20> Sure, they crash (or do something funny) a lot of time, but that's the problem with these ActiveX controls. If you buy anything from a Korean site you end up with an unknown number of plugins downloaded from everywhere, so how do we know if it's a bug in some plugin, a plugin already infected with virus, or just some malware you downloaded accidentally? (The last one, because you have to hit "OK" all the time: actually recent versions of Windows is being quite reasonable and wants to warn you when you install binaries from random websites, but this is exactly what Koreans are asked to do every day, so Korean bank websites contain these helpful pages showing how to lower Windows' "security level" and override the warnings.)

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER22> This is why governments shouldn't be passing laws to 'protect' citizens online - they'd manifestly bad at understanding change and worse at designing legal systems that can cope with it.

<comment><USER23> You're half right: governments should prefer to dictate outcomes rather than mechanisms. Telling banks how to code their databases is a losing game but making them liable for losses due to weak security works quite effectively.

</comment></comment><comment><USER24> Some additional details that the article doesn't mention:

1\. Technically, the law doesn't require that you use Internet Explorer. The law merely requires that you use a bunch of technologies, ranging from 128-bit encryption to government-issued client certificates to government-mandated antivirus to (craziest of all) an anti-keylogger utility. Conveniently, the spec was written with Windows & IE in mind, so it's very difficult to write alternative implementations for other platforms.

2\. This is not a matter of being stuck with older versions of IE like many corporate intranets in the West. In fact, most banks in Korea work perfectly well in IE11 as long as you don't try to use the Modern UI (Metro) version. Because this is not so much about IE as it is about the WIN32 environment.

3\. The proliferation of phones and tablets has motivated banks and payment gateways to write iOS and Android implementations of the spec. This was the first time anybody tried to implement the spec outside of Windows & IE. But once you have one alternative implementation, it's much easier to port it to other platforms like Mac, Linux, and FF/Chrome on Windows. This is happening slowly.

4\. Despite the appearance of these alternative implementations, the spec itself is still very problematic. For example, the antivirus and anti- keylogger requirements cannot be met unless the programs in question have root privileges on your device. It feels insane when you browse to a bank's home page in Linux and it tells you to download a bunch of apps and execute them as root. And of course those apps are only designed for specific versions of specific Linux distributions, so they break as soon as a new Ubuntu release comes out. No thanks! Even in Windows, the Firefox & Chrome plugins are not packaged as proper extensions, but as standalone programs that integrate loosely with the browser like Flash and Java, Because you can't meet the spec within the confines of a browser's sandbox.

5\. Okay so why not just run Windows in a VM? Actually that's exactly what I do. But it's not a perfect solution. Some of the Korean "security" apps have begun to detect when the user is in a VM, and refuse to work in a VM. There is no technical reason for this policy, they just don't like people getting around the rules. My bank refuses to whitelist my VM as a trusted device. I've encountered at least one government agency that won't offer online services to a VM. The last time I bought a bus ticket online, the e-ticket wouldn't print because the printer port was virtualized and therefore could be used to produce duplicates or whatever.

6\. Even mobile apps, which the article mentions, are very pesky about their environment. The app for my bank won't run on my phone because it's rooted and therefore can't be trusted. Fuck that shit. This affects everyone who uses CyanogenMod. (What's even more ridiculous is that the same bank _requires_ root on my PC.)

7\. _Therefore, porting the spec to non-IE platforms and /or writing compatibility layers is not the answer. The spec needs to be fixed, period. No website should have the right to demand the use of any software other than a standards-compliant web browser. No website should require root, or even want to know anything about the environment (virtualized or not, rooted or not) in which it is being visited, except what the browser exposes to it by default._

8\. Of course this isn't going to happen any time soon, because removing even one of the requirements on the current spec will be seen as a decrease of security, and nobody wants to take the blame the next time 10 million people get their account information stolen. Wait a second, every Korean citizen has had his or her personal information stolen multiple times in the last several years anyway. All the banks and merchants have desensitized users to the point that anytime any website ask them to install some app and run it as Administrator, they do. All the security theater of the last 14 years has done is to decrease the security of the entire country. It has also hurt the rest of the Web. Because it's so much more convenient to write a Windows Forms app than to write a website that works in both IE6 and IE11, lots of interactive and media-heavy websites in Korea (especially gaming and file-sharing websites) have become mere landing pages where you download the actual app. After all, the banks are doing it, so why shouldn't everyone else do the same?

9\. One move in the right direction is that since this September, every large (over ~$3000) online transaction requires two-factor authentication. They've been handing out one-time password generators like candy lately. The ubiquity of mobile phones also means that you can even choose to use three-factor authentication (login + one-time password + SMS token) for certain types of transactions. Hopefully this will eliminate the justification for the anti- keylogger utility, since the passwords and SMS tokens can't be reused anyway.

[Edit] 10. Another positive development is that the Korean government has finally begun to pay attention to accessibility on the Internet. At the moment, among Korean web developers, accessibility is an even hotter topic than standards compliance, because lack of accessibility can get you into nasty lawsuits and hefty fines. Everyone's busy adding "alt" attributes to <img> tags. But hopefully, in the long term, focusing on accessibility will also bring people to care about standards compliance.

</comment><comment><USER25> Well that's a stupid law. Requiring people to use a specific product. That's never a good idea. Now if you don't mind I need to go renew my car insurance and sign up for health care.

<comment><USER26> But you can choose your insurance companies and plans. This would be like the government mandating that you have to use Aetna for health insurance.

<comment><USER27> You can choose plans that adhere to the government standards. That is a lot different than you can choose your plan, _period_.

I'm still confused about why the government needed to create any sort of web site since they are just marketing private insurance plans (which meet the ACA standards) and these companies have been and continue to market their insurance plans on their own web sites. And hasn't anyone in the federal government heard about 'independent insurance agents'? I guess not since they had to dream up a new job title of 'navigator'.

I do support _some_ of the policy goals of ACA, but it seems like almost any other implementation would have been better than the convoluted-rube- goldbergish mechanisms created by the ACA.

</comment></comment></comment><comment><USER12> Boot Camp is a secret weapon? It costs $70? I'm confused.

Could they not get a free VM and then download an image from modern.ie?

<comment><USER3> Many native "security" solutions required by websites, e.g. anti-keyloggers and antiviruses, would block the VM. They also tend to trigger weird problems in the VM, e.g. some of them makes the entire desktop fail to respond to any events. (What?)

<comment><USER12> Out of curiosity, how would the security applications know that it was being run on a VM? Why would they care? That suggests they believe no one has a legitimate reason to run Windows in a VM.

<comment><USER28> On Detection:

There are a few things that leak presence of a virtual machine to running code (even in userspace).

There are the easy signs, e.g. that the virtualbox guest kernel-modules or utility programs for clipboard exchange are running. Or that your harddisk controller has a PCI ID identifying it as a Oracle Virtualbox AHCI device.

But on a hardware-virtualized machine there are also numerous inconsistencies to be observed which are not influencing the correct execution of "normal" code, but leak the existance of the hypervisor to the guest OS (and sometimes even to non-privileged user-code in the guest OS).

Google for "Red Pill VM detection", unfortunately currently invisiblethings.org (the site of the author) seems to be unavailable.

</comment></comment></comment></comment><comment><USER29> This is state-controlled economics (corporatism) at its usual finest.

</comment><comment><USER30> "But the back-and-forth was technologically complicated, and it came with a catch: It required a piece of additional software, or "plugin," known as ActiveX — which is also made by Microsoft and worked in tandem only with Internet Explorer."

That phrasing made me cringe and shows the lack of technical understanding of the author of this article. ActiveX is a technology, not a piece of software or a plugin in itself.

</comment><comment><USER31> $70 for bootcamp? For a company as technologically advanced as South Korea, can't they just use refit or refind? Heck even a VM would do fine.

<comment><USER24> Last time I tried to access a South Korean government agency's website using a Windows guest in VirtualBox, it recognized that I was using a VM and refused to allow me to proceed. Same thing happened when I bought a bus ticket online and tried to print the e-ticket. Apparently everyone thinks VMs are only good for malicious activity.

</comment></comment><comment><USER32> Whole the source of problem is Korean majority is conservative. They don't want to change anything. And want to force their belief of right. And that belief in financial industry is current Korean online banking system. Korean financial industry practically has no freedom to choose some security solution, so even foreign banks - such as CitiBank - use that stupid system.

And the conservativeness of the Korean majority elected conservative major party, and the party - of course - has no will to change it at all. And actually they enforces old rules to keep their existing benefits.

So Korea has no hope to change this before replace major party. In last president election, there was a candidate promised fixing this issue, but finally defeated to candidate from conservative party.

And they need to wait all the old McCarthyists - who are main supporters of conservative party - disappears.

</comment><comment><USER33> Isn't this because the US wouldn't export crypto software that supported keys > 40 bits to Korea in the 90s, forcing them to develop their own stronger algorithms?

Anyway, it's so bad that my (technophobe) wife refuses to shop online anymore because she is forced to use IE (and vastly prefers to use Chrome)

</comment><comment><USER34> I like to think of South Korea as a nation state, gripped by such fear-of-the- other, that they'll agree to any irrational suggestions their military advisors might make.

So then, 10 or 20 years ago, when the NSA needed a secret laboratory to experiment in, where they could blunder away at trial and error, perhaps a huge wind tunnel to test the aerodynamics of this bird, well South Korea sounds pretty good. Let's see if the pentagon can get them to agree to a few absurd pre-requisites and static global variables, while we bootstrap this absolutely enourmous program we're shoe-horning into place.

</comment><comment><USER35> An enormous amount of South Koreans play Starcraft and MOBAs (e.g. League of Legends). Those are Windows games.

However, as I've learned from doing tech support, I find hardcore gamers are more computer illiterate. They know enough about computers to turn them on and play their game, but because they play for so many hours they don't do anything else on the computer.

Basically you get a nation of computer illiterate users, who use Windows because they don't know any better. Most probably don't even know what Firefox or Chrome are.

<comment><USER36> I play (the old) Starcraft quite well on Linux.

</comment></comment><comment><USER37> Is it easier to buy something out of a catalog over the phone? If that is still an option, then one way around this law is to use a web site to set up an order (fill your shopping cart, shipping details, etc). Then call the vendor to finalize the order. Of course, this would add to the cost of online purchases, since you'd have to pay for the person taking the call, but still it might be a work around.

</comment><comment><USER38> Non-paginated version: [http://www.washingtonpost.com/world/asia_pacific/due- to-secu...](http://www.washingtonpost.com/world/asia_pacific/due-to-security- law-south-korea-is-stuck-with-internet-explorer-for-online- shopping/2013/11/03/ffd2528a-3eff-11e3-b028-de922d7a3f47_print.html)

</comment><comment><USER39> You'd think someone would sort out a way to setup the certs via some other method. I think this is a great example of why legislators shouldn't be allowed to make laws about things they don't understand, like the internet.

</comment><comment><USER40> Sounds like someone needs to educate Koreans on using VirtualBox and Modern.ie images... both are free.

[http://www.modern.ie/en-us](http://www.modern.ie/en-us)

</comment><comment><USER41> So are most of the Indian IT services companies. A friend of mine is still using windows vista with IE7 cus some moron decides the outdated company policies.

</comment><comment><USER42> Surely this impacts Apple's sales in Korea. Amazed they haven't invested more in making a Mac technical solution for this.

<comment><USER3> Mac users did have successfully made one major bank to support Mac in 2004, by pledging about 16 billion KRW (~15 million USD at that time). The situation hasn't improved since then though.

</comment></comment><comment><USER43> This is exactly what I was saying about [made-for-prison] software being unleashed on the public writ large:

> But those with Apple computers — for which IE isn't available — have it > harder. Some go to Internet cafes. Some rely on their office desktops. Some > dash into hotel business centers. Some hold on to their old computers and > boot them up when it's time to make purchases. Still others depend on a > secret weapon called Boot Camp, a software program that allows a Mac to run > Windows.

Your bar code is your laptop.

</comment><comment><USER44> Who cares. Developers need to pull legacy browser support sooner. It's pretty clear that certain people and organizations will wait until they absolutely have to change before they will make the effort. Once it becomes inconvenient enough they'll do whatever needs to get done. The effort can be justified in business terms.

IE8, for example, is going to be around for a while. However, if enough developers stop supporting it now, the conversation will begin. Otherwise, it's gonna be 2020 and ie8 will still have significant market share.

<comment><USER10> If you lived in S. Korea and had to be limited to using IE on Windows with ActiveX controls for each site you wanted to transact with, you'd care. Did you read the piece? The current system is the law.

<comment><USER44> This is not a new story. It has been discussed for the last several years.

Like I said, certain organizations will change when they absolutely have to. In South Korea's case, it's going to be costly so they are probably going to be the last.

<comment><USER10> Indeed. I was the first person to report this situation in English in 2007, so I know this topic well.</comment></comment></comment></comment>


Andrew Yang's ideas on automation and mass unemployment are not based in reality - <AUTHOR> https://slate.com/business/2019/10/andrew-yang-automation-unemployment-freedom-dividend.html <comment><USER2> _In a 2018 paper, UC–Berkeley sociologist Steve Viscelli suggested that in the most likely scenario, long-haul truckers, who tend to make middle-class wages, will be replaced by poorly paid drivers tasked with steering autonomous vehicles through tricky city streets, which onboard navigation systems handle less well than highways. That's not all jobs disappearing. It's jobs changing._

I'm not sure if the author is claiming that the job displacement will be 1:1. Does anyone honestly think it would be 1:1? I don't think so. And if more jobs are disappearing than are being created, there is a net job loss; also the author agrees that the new jobs would be lower pay. Whether jobs will be created in entirely and unpredictably new industries is besides the point. Yang's point is that everyday people are inadequately prepared to make _that transition_ to new jobs. He frequently cites data that the US federal government sucks at job retraining.

IF various tech companies are successful in their endeavors (and yes, that's a big if), it will be true that a lot of people will need to transition to something new. And given recent demos by various tech companies for voice AI tech (yeah, Google's really was impressive when it came out, and it will probably get better until it's commercialized one day), self-driving tech (I'm a skeptic for the short-term, but it will probably happen in the long-term), and etc, I think the subject is ignored at the working population's peril. Never mind self-checkout, which is already a reality and steadily gaining ground.

<comment><USER3> Yeah, honestly I don't really understand the article's point.

Is it that say McDonalds switching to electronic kiosks to order your burger is going to create as MANY tech jobs as it took away? Because that doesn't pass the smell test.

<comment><USER4> There were a slew of anti-yang articles that came out from the leftmost media sources immediately after the debate, owing to the fact that Yang made Elizabeth Warren look unprepared in her answer about job loss.

Andrew Yang, snake oil salesman: [https://www.nationalreview.com/2019/10/andrew-yang-tech- entr...](https://www.nationalreview.com/2019/10/andrew-yang-tech-entrepreneur- fears-automation-self-driving-cars/)

Is Yang doing more harm than good to the case for universal income?: [https://www.politico.com/magazine/story/2019/10/16/andrew- ya...](https://www.politico.com/magazine/story/2019/10/16/andrew-yang- universal-basic-income-229847)

Andrew Yang is Full of it: [https://slate.com/business/2019/10/andrew-yang- automation-un...](https://slate.com/business/2019/10/andrew-yang-automation- unemployment-freedom-dividend.html)

There was one more I can't find. It's good to be generating hit pieces; it means someone views you as a threat.

<comment><USER5> National Review a leftmost media source?

<comment><USER6> Or Politico, or Slate. When I hear "leftmost" I'm thinking like, Jacobin or Chapo

</comment></comment></comment></comment></comment><comment><USER7> I'm not a Yang supporter but this article is total garbage. Slate has been very disappointing lately.

The key takeaway seems to be while Yang says automation has displaced jobs there's a lack of academic consensus on the weight placed between trade policy and technology improvements; but everyone agrees both have had a significant effect.

Ok... so we shouldn't be coming up with a plan for how to address economic changes due to technology based on "it's historic effect has possibly been outweighed by other factors" even though we all agree it will continue to have an effect in the future?

Also just want to point out that anyone using the reported federal unemployment rate as a measure of overall employment health has already lost all credibility to write on the topic.

<comment><USER8> Agreed, there are some good points in the piece, but the headline and concluding paragraph are not supported by the body.

<comment><USER9> Curious as to what you identified as a good point? I didn't really see any to be honest

<comment><USER8> I think Yang has been overstating automation's impact, in part because that's necessary for political messaging. Sounds like the Ball State study has some issues.

</comment></comment></comment></comment><comment><USER10> "[...] Long-haul truckers, who tend to make middle-class wages, will be replaced by poorly paid drivers tasked with steering autonomous vehicles through tricky city streets, which onboard navigation systems handle less well than highways. That's not all jobs disappearing. It's jobs changing."

Uhm, that's halfway decent careers turning into far fewer and shittier jobs. That's jobs disappearing. And it ignores that eventually someone will automate even that part away, it'll just take a bit longer.

The article just seems to just be a series of quibbles that don't show Yang to be wrong in any meaningful sense.

<comment><USER11> That's BS about being capable of navigating Interstate highways autonomously.

Whoever does the research needs go actually go along an Over The Road truck driver and actually see what challenges technology face.

Different highways with crumpling road markers. Severe weather. Unpredictable human drivers. Fueling. Construction sites and their differences between displaying markers.

I do not see autonomous technology taking over long haul truckers anytime soon, especially if the state and federal government doesn't maintain the highways and keeps cutting budget.

The few scenarios where the present technology can handle is a local route that has greatly maintained roads and stable weather. Even in this case, it's foolish to think human drivers will not be as careless as before. In fact, I predict they will think its so safe that they will cut the truck off ever closer, thus creating more disasters.

Source: I'm a trucker and know the challenges computer vision and its sensors could possibly face.

<comment><USER12> There are many routes which are possible to automate today. Those will become human-free first, leaving the humans to compete for the remaining 95%.

Then another 5% of routes will become computer navigable, and the remaining truckers will compete for the remaining jobs.

Then between automation and improved maintenance due to the higher regulatory fees afforded by automation, more roads will become auto-navigable by dint of maintenance and standardised marking.

There will be a pincer movement, roads coming up to standard, standards converging on what automated trucks need, and automation becoming better at navigating any road.

By 2030 there will be far fewer humans driving anything.

<comment><USER13> What? 10 years?

</comment></comment></comment></comment><comment><USER14> Since I like Yang and have Googled him before, Google thinks I should know about every article written about him and keeps prompting me about them as they come up.

This one, like so many lately, all have been hammering me with this idea that Yang is predicting a dystopian horrible future.

On the contrary, I think Yang is fundamentally _optimistic_ about the future and technology. He predicts that we will lose a lot of our current jobs and that we need to put in place a government safety net in the form of UBI, but I don't think _he_ considers it a dystopia.

I'm frustrated by all the editorializing in the headlines that I'm seeing. If all you saw were all these headlines of late, you'd think Yang were some kind of "sky is falling" candidate. But actually I like him because I feel like he has a quite hopeful vision of the country and our future.

<comment> dao- Agreed. If we're honest about it, labor sucks for most people. You have to get up early, spend the day doing something you don't really care about, and get home too burnt out to enjoy even the rest of your day. It's absurd to think that liberating people from that labor regime through technology and a new social contract must be dystopian.

</comment></comment><comment><USER15> I read the whole thing. It's not so much that Yang is wrong but that the automation-is-taking-all-the-jobs mantra is incomplete and that outsourcing to places like China is a substantial part of it. From what I can tell from Yang- like arguments outsourcing could be seen as a form of automation. The general idea is that companies are incentivized to reduce labor costs so whether it's outsourcing or automation the end result is the same. Presumably if we restrict outsourcing then just more investment into automation will take place. It's inevitable.

<comment><USER16> The most charitable way of looking at this is that yang wasn't wrong, but he wasn't right either. He made explicit claims like "this isn't a rules problem" in response to trade policy; actually, there's a lot of evidence that there _is_ a trade policy problem.

</comment></comment><comment><USER17> I stopped reading after the author mentioned that Yang thinks that Ubi will solve issues. Have you actually read into some of the things he said, Ying explicitly says he does not think it will solve everything. Rather it gives people a bit of leeway in the short-term to focus on other things. With the hope that it will lead to more entrepreneurial things, but knows more likely it will go towards random things in each individual's life

<comment><USER18> I think the best summary of Yang's view of UBI comes from his own FAQ [1]:

> We are experiencing the greatest economic and technological shift in human > history, and our institutions can't keep up. Without the Freedom Dividend, > we will see opportunities shrink as more and more work gets performed by > software, AI, and robots. Markets don't work well when people don't have any > money to spend. The Freedom Dividend is a vital step to helping society > transform through the greatest automation wave in human history.

[1] [https://www.yang2020.com/what-is-freedom-dividend- faq/](https://www.yang2020.com/what-is-freedom-dividend-faq/)

</comment></comment><comment><USER18> > over the very long term, automation probably has played a role in limiting > manufacturing's share of employment in the U.S. Factories really have become > more advanced and efficient since, say, the 1970s.

Can anyone at Slate read a chart? US employment in manufacturing as a percentage of non-farm labor dropped from 32% post-WW2 to less than 10% today [1]. This change didn't start "sometime in the 1970s", but has been a consistent downward trend post-WW2. The low hanging fruit of automation (from mechanical feedback mechanisms to programmable logic controllers) are all pre-1970s inventions.

And so what if different jobs are created for those that are lost? The article itself admits those jobs will either be lower paying or require more advanced education. Job re-training is mostly a myth [2]. No matter what, an entire generation of unemployed will suffer. I grew up in Detroit. I know what happens when the jobs base shifts.

[1] [https://fredblog.stlouisfed.org/2014/04/the-decline-of- manuf...](https://fredblog.stlouisfed.org/2014/04/the-decline-of- manufacturing/)

[2] [https://www.nytimes.com/2017/02/23/magazine/retraining- jobs-...](https://www.nytimes.com/2017/02/23/magazine/retraining-jobs- unemployment.html)

<comment> dao- > And so what if different jobs are created for those that are lost? The > article itself admits those jobs will either be lower paying or require more > advanced education

And even this will only last so long. With ever more advanced robotics and AI, there will be fewer and fewer possible lower paying jobs as well as advanced jobs that average Joe could even be educated for. Humans have finite muscle power and brain capacity. Is it not obvious that as technology becomes more potent, we'll reach a point where it just doesn't make sense anymore to make the masses spend their days working bullshit jobs?

</comment></comment><comment><USER19> > Following the debate, a "fact check" by the AP claimed that Yang was > right... "Economists mostly blame [manufacturing] job losses on automation > and robots, not trade deals," it stated. But this was incorrect. No such > consensus exists

So they claim AP and Yang are incorrect but also state there is no consensus. This piece seems more like it is trying to justify the warren gaffe in the last debate by muddying the waters. The truth is be it automation or outsourcing our economy is changing and we need to be prepared for it.

<comment><USER20> I would like to know which mainstream economists blame trade deals. Ideally stuff that was published more than 5 years ago.

</comment></comment><comment><USER21> Yang is totally wrong about automation. The main cause of the shift of manufacturing from US to China is trade policy and China's monetary policy.

If automation was the problem, then US would be price competitive on goods with China. Somehow US automation is only good enough to eliminate jobs in the US, but not good enough to compete with cheap labor in China.

<comment><USER22> Whatever automation you can do in the US you can also do in China but with lower regulation. Whatever labor costs left over are also cheaper. What I mean is that automation is not an inherent advantage of the US.

<comment><USER23> Absolutely. In fact industrial automation components (robots, servos, linear bearings, etc) have been in huge demand from China. Suppliers often sell the same components there for half the price as they charge in the US for the same quality and quantities. Domestic Chinese companies are becoming major suppliers of a lot of factory automation equipment like presses, laser welders, coil winders, etc.

There's no reason to think the US has an advantage in automation. In fact, it's much easier as an automation engineer to design automation processes for the factory that's around the corner than to do so for a factory in another time zone that 90% of your team has never seen in person.

</comment></comment></comment><comment><USER24> I generally favor UBI because it's basically wealth transfer from the wealthy to the less wealthy and I support those policies.

UBI seems like a poor response to the shifting nature of work. Yang himself implies that automation will "come for" some groups before others. I.e. subsequent subsets of the population will lose 100% of their income. Increasing the income of everyone seems like a poor response. If we want to support workers who worked or trained for a job that has been automated away why don't we...do that?

If we want to overhaul our society into a place that no longer forces people to work to live I'm also in favor of that! It's just different.

<comment><USER25> Because UBI is the only welfare system that can't be gamed. How do you identify "workers who worked or trained for a job that has been automated away"? Non-universal welfare wastes a lot of resources on conflict between potential welfare recipients and the state, and it inevitably mis-categorizes people, which causes suffering either directly when the "deserving" are denied it, or via resentment when the "undeserving" are granted it.

<comment><USER24> >Non-universal welfare wastes a lot of resources

It seems like you're saying that UBI is more efficient we don't need spend resources on targeting it. I 100% agree for the _general case_ of wealth transfer. I think it's very weak for addressing specific job losses.

So, let's say we have 100 people and 10 of them lost their jobs to automation but we don't know which. One option is the "Yang plan" of $12 a year ($1 a month), so we have a budget of $1200[0]. That spends $1200 giving $120 of assistance to the people we want to help. This plan is 10% efficient _if the point is to help those who have lost their jobs_. In fact, with a budget of $1200, we can spend up to $108 per person identifying if they lost their job due to automation and have the same efficiency (10%). That is a really high ceiling on the cost of conflict.

I'm very skeptical that in any real world situation the cost of identifying worthy recipients is greater than the "loss" of giving everyone else money. Again, only if the goal is _helping workers who lost their jobs._ Hell, even if you're wrong 50% of the time and give 15 people $12, you would need your method of identification to cost $60 (5x the benefit) for it to be 10% efficient!

Again, I favor UBI and I favor transforming society to function differently, but that's not helping workers who lost jobs.

[0] These are ofc not the real numbers Yang is suggesting

<comment><USER26> I always assumed part or perhaps most of the funding would come from flattening the tax system.

</comment></comment></comment></comment><comment><USER27> China or not, most of the jobs won't come back, they were eliminated by automation. The bottom line of the so called UBI is the redistribution of wealth. It's not only for individuals, but regions too. I somewhat image that the money you give to individuals is able to encourage them to move to regions where there are less jobs, but lower living cost. meanwhile, the newly moved in individuals are able to revive the region, creating new local business and jobs...etc.

<comment><USER28> Notably, jobs in China are and will be automated away just the same as in the US and everywhere else. Technology and science are universal, you can't just put a border in and think you're insulated. Apart from why one would even _try_ to obstruct progress?

</comment></comment><comment><USER29> I don't think Yang is claiming the _only_ thing causing job loss is automation. It's just one major factor, which will continue to rise into the future.

I also don't think he believes it will put everyone in those industries out of a job. I can see how the sound bites from the debate sound that way, but unfortunately he probably has to use the worse-case scenario type examples.

</comment><comment><USER30> Use your eyes. Have you walked into a Supermarket? Or better, when's the last time you saw MORE retail store hiring MORE help and not actually cutting (because Amazon's choking them off). Better, ATM machines are now charging $5.00 per transaction fees because cash is no longer needed. Automation is a bad word for all this -- it should be called simply "job replacement" \-- and it comes in all forms of technology than can help us in many ways, but also has ramifications like job loss. People need to use their eyes. He's not wrong. He's not full of it.

</comment><comment><USER31> That's a strangely angry article and seems to miss the bigger point.

America changed its trade situation with China, which either made the US better off or worse off. (I'm guessing the economic consensus for is better off).

Basically the same applies to automation.

No individual factory worker or doctor or police officer made that happen, so the benefits and/or costs should be spread around equally. UBI is one tool for doing that. People who lose their job and need to retrain automatically get cushioned, which lets your society be more ruthless about creative destruction.

<comment><USER28> > _either made the US better off or worse off._

It's very hard (meaning impossible) to make such an assessment for the US as a whole. Some groups net benefited from globalization and trade liberalization, others net lost, for many it was a wash.

Remarkably, this is one of the predictions of mainstream economists (MSEs) that is empirically true. What they should have stressed more, therefore, is the distributional aspect of trade liberalization: that the gains from trade must be distributed to compensate the losers. The gains are so large that this could have been done and still leave something over for the natural beneficiaries of free trade.

What Yang is saying, and I TOTALLY agree on this point, is that automation or progress in AI is like that only ten times worse. We all know that tech is full of natural monopolies, while the economic gains are HUGE because they are aggregated over so many people/transactions/etc. and marginal costs are low.

Therefore we should think about reorienting our economies (talking globally here, not just US) to ensure that we don't see the same that happened with trade now with automation and AI. Namely huge segments of the population left behind "because market" and all the gains accruing to a handful of data oligarchs.

</comment></comment><comment><USER32> I develop key performance indicator software for my company's manufacturing plants. It's basically a real time analytics dashboard that also trends data over time. One of the results of this software has been a better understanding of the efficiencies (or really the lack thereof) and bottlenecks in our processes. Identifying and understanding these KPIs has allowed our pilot plant to cut significant labor costs without losing any productivity (no one was fired, the positions in question weren't refilled due to attrition, which is always high in a manufacturing job).

My point being, it isn't just robots and automation that reduce the labor workforce. Better data and analytics are also having an effect, a trend I don't see slowing down. As the market continues to shrink for the blue collar workforce, something will definitely need to happen so as to not disenfranchise large groups of people. Whether a freedom dividend is the answer or not is up for debate. I think it will help more than it will harm. There's always going to be someone who finds a way to game the system (the perceived welfare queens). For many, though, an extra $1000 a month would literally be life-changing.

</comment><comment><USER33> At the risk of continuing the non-conversation here, I feel like it's important to consider countervailing long-term trends. I was listening to a fascinating episode of the Macro Musings podcast with Alex Tabarrok of George Mason and he brought up a very thought provoking point: the largest single group of workers was the Baby Boom generation, the next largest is (broadly speaking) their children, the Millenials. The number of Americans in the workforce is currently at its peak for about the next hundred years simply by number of people in their prime working years. This means that unless automation vastly outstrips the decline in the number of prime age workers, this problem won't directly be nearly as bad as people are saying. This is further reinforced by the demographic trends among the changing generations of workers -- Millenials aren't going into trucking or welding or other not- currently-automated jobs at nearly the rate their parents did.

The obvious followup to that whole point is "what are the comparative effect sizes of all of these various causal phenomena?" I don't happen to have the answer to that, but I feel like it's worth pointing to at least some mitigating factors.

</comment><comment><AUTHOR> The article's most compelling point for me is the graph it links to showing all U.S. manufacturing jobs from 1939 to 2009.

[https://fred.stlouisfed.org/series/MANEMP](https://fred.stlouisfed.org/series/MANEMP)

There's a big, sudden drop in 2000.

It also cites a paper that says that the number of _factories_ in the U.S. has been dropping (between 2000 and 2014)...

<comment><USER34> >There's a big, sudden drop in 2000.

actually it is in 2001. Also in 2008. The years of big busts.

<comment><AUTHOR> Yeah, but the jobs never come back after those drops. (It's almost entirely a one-way trend over the last 19 years...)

There's a big three-year drop that starts in 2001. Then it goes from that steep drop to a slowly-declining drop -- before that next massive "2008" drop (which lasts through December of 2009).

So it seems plausible that the loss of jobs was caused by specific events. (Whereas if it were the creep of automation, it seems like it'd be happening more gradually, factory by factory....)

I guess you can argue that the economic hardship of a bust forced companies into sudden and widespread cost-cutting measures like replacing humans with robots. But another study cited in the article argues against that theory. (The article acknowledges that automation has taken some jobs, but that it's a much smaller fraction, and that latching onto that as the single explanation makes us miss the other factors -- as well as possible solutions.)

</comment></comment></comment><comment><USER35> The real threat to jobs is business process automation. The ideal is to change business processes to be like self serve gas stations where in the usual case the customer does all the labor required and the individual transaction never requires an employee's attention. Better sensors and actuators, embedded technology, and artificial intelligence to handle all the corner cases of the business process will allow 100% flow through for many other business processes. This will result in a significant decrease in administrative, back- office, and low level managerial jobs.

Trucking is just a nice example that is easy to explain to people. But the overall trend is much broader and more complex involving 100s of thousands of IT workers incrementally removing costs.

</comment><comment><USER36> > Meanwhile, the overall job market is fine...

In constant (inflation-adjusted) dollars, according to the US government's Bureau of Labor statistics, current earnings for production, nonsupervisory employees on private non farm payrolls is $23.65, in constant inflation- adjusted 1983 dollars for September 2019.

In February 1973 that was $23.33 an hour. Almost a half century of economic growth, in a good economy, and workers are making only 32 cents more an hour than they did 46 years ago.

If pundits were describing wages in the Soviet Union, this would be called 46 years of stagnation and so forth. Whereas we have someone from the Bill Gates funded corporation Slate telling us "the overall job market is fine" and other niceties.

<comment><USER13> Wondering aloud if there are any studies that figure out how much these workers should be earning proportional to the productivity increases in the last 40 years?

What is the "cut" workers can expect from their output charted through the last century - this number must exist

</comment></comment><comment><USER37> Even in China labor is getting more and more expensive. So sure there was some loss of manufacturing jobs to China in the 2000s but now even China is losing it to automation and outsourcing as labor rates go up.

</comment><comment><USER38> This complaint about robots and AI taking all our jobs is hundreds of years old. Still hasn't happened. US is at record levels of employment. Humans are terrible at predicting the future. Let's just wait and see and react when it actually happens. My guess... there will be plenty of jobs for humans for thousands of years to come.

Ever since the industrial revolution jobs have gone by the wayside. We no longer have wagon wheel makers, or people cutting pins from long spools of wire. If you just look at the jobs going away, you're missing the other side of the picture, the jobs being created.

Jobs are essentially just things that humans do to improve our lives. If you imagine that you were infinitely rich, what kinds of things might you want someone to do that isn't getting done? I've got tons of housework and yardwork, construction work, programming ideas, hell my brain is overflowing with stuff that will never get done. Too many jobs. All those things are jobs. Most of them cannot be automated anytime soon.

<comment><USER2> _If you just look at the jobs going away, you 're missing the other side of the picture, the jobs being created._

I don't think Yang is missing the other side of the picture. He's saying that society sucks at getting displaced people to the other side in a humane manner.

<comment><USER38> I don't disagree with that.

</comment></comment></comment><comment><USER39> Just more of a smear against Yang. The establishment is afraid.

Also the tint of anti Asian American racism - similar to SV when the hiring team or management group tries to talk themselves into finding reasons to not like a qualified candidate that does not fit the mold of the obediant Asian worker drone. Usually done insidiously to cast doubt on a person's character etc.

</comment><comment><USER40> The other part of the equation is that retraining are a complete failure. The idea you can teach a truck driver or a coal miner to code is a fallacy.

Retaining programs have a 0-15% success rate. It's basically useless. What happens is that middle aged people get their jobs automated away and they are left with nothing. Look at what happened after manufacturing jobs left the Midwest. This is the bigger problem is that large swathes of the population will have no viable jobs after automation.

</comment><comment><USER41> I've done some consulting at quite a few bigcorps in the Houston area. The amount of outsourcing of IT generalist work that I've seen is downright scary. Regardless of political affiliation, this should be front and center for both parties. It is the go to move for executives that want to spin stagnant growth into profitability. I generally disagree with tariffs. However outsourced work should be subject to a 100% tariff so that it's disincentivized.

</comment><comment><USER42> Don't know how quickly it is coming but I think ubi is the right direction for civilization. People are fighting tooth and nail for meaningless bullshit jobs that low key function as a form of government assistance but waste your entire adult life in the process.

</comment><comment><USER43> I could understand if they questioned universal basic income or whether Yang's 1k/m will do any good, but plainly denying what is happening before your very eyes indicates a hack or a hit job. Also, focusing on manufacturing jobs alone is simply misleading.

</comment><comment><USER44> If the need for low to moderate skilled labor is going away, why are we still importing so many workers in those skill ranges?

</comment><comment><USER45> So the article is probably correct that trade has been a bigger issue than automation. Well let's assume that's true, to be charitable to the mentally challenged Slate authors.

So what happens after you put the screws to trade? That certainly isn't going to decrease automation as an alternative. What happened from 2000-2010 doesn't really matter, they should save their breath.

</comment><comment><USER46> So I guess this means that Stephen Hawking was wrong as well?

</comment><comment><USER47> Andrew Yang is the Trump of tech. And his brand of politics is more a dangerous threat to democracy than Trump's is. His messaging and policies are just like Trump's in that they're dumbed down, optimized for social media likes and subtly enforce a world view where extreme income inequality is inevitable so let's make the best of it. AI/tech isn't going to kill the middle class by itself. This destruction will be assisted by technocracy propagandists like Yang.

</comment><comment><USER12> Obligatory CGP Grey "Humans Need Not Apply" spruiking: [https://youtu.be/7Pq-S557XQU](https://youtu.be/7Pq-S557XQU)

Deals with the issue of why current types of automation are different to industrial automation an why we won't be creating new "machine fixer" jobs when the new machines take our old jobs.

</comment><comment><USER48> "You're creating a slippery slide transfer straight into socialism"

This seems to beg the question: "Would Amazon be less successful if it paid a living wage to those who labor in its warehouses and those who work to manufacture the products it sells?"

<comment><USER13> 1\. Does Amazon get away with paying warehouse workers less than others?

2\. Are you presuming no one else will pay them a living wage either so they MUST work at Amazon?

3\. Are Amazon warehouse workers prevented from getting living wage jobs elsewhere?

<comment><USER48> Those are stupid excuses posing as questions, but I'll play along and answer them...

1\. Yes 2\. No & Yes 3\. Yes

</comment></comment></comment><comment><USER49> _And therein lies the real problem with Yang's outlook. It's not just unrealistic. It's lazy._

And this is why Yang is the most prescient of any of the candidates. He looked at the problem, broke it down into its basic components, extrapolated what was going to happen, and found the most straightforward answer (UBI) from the large search space of possible solutions. You know, like a computer programmer would.

So I fundamentally disagree with the article for the following reasons:

* If I had a budget of a few million dollars and a small team of geeks like me, I could write a (non-general) AI myself in 2-10 years that would pass the Turing test in any specific area, a bit Like IBM's Watson. And I'm nobody. The reason this won't happen is that I don't have the social proof (I haven't won the internet lottery or proceeded through the graduate level accolades or submitted papers so anyone would listen to me) so I'll likely spend the rest of my life making rent instead of making any meaningful contribution to society. And there are a million software developers around the world just like me. And a million architects who will never design a building. And a million biologists who will never cure a disease. This is the great tragedy of the commons of our time, that creates the underemployment that contributes to the wealth inequality that is the one thing holding AI back. You know, the very thing that Yang is trying to solve with UBI.

* I agree that shortsighted trade policies started by Reagan in the 80s with his "I'm from the government and am here to help" deregulation anti-pattern, exacerbated by Clinton with NAFTA, and beaten like a dead horse with George W Bush's offshoring/tax cut for the wealthy blue pill orgy, is the second biggest reason that we may have passed the point of no return on preserving (at least American) jobs. But half the US population won't concede this point or think outside the box for empathetic solutions, so I don't predict that this will be fixed before the singularity. It's a moot point now.

* I predict that the tech industry will go the wrong direction with AI, in roughly this order: industry invests heavily in cybersecurity and profit-driven artificial intelligence research to better exploit users by leveraging their shopping addictions, social/mobile bubble pops as independent developers are undermined further by receiving an ever-smaller slice of the pie, a global recession rivaling the dotcom crash and housing bubble pop hits which halts progress in social causes similarly to what happened during the lost decade of the 2000s, the US doubles down on trying every wrong thing before being forced to do the right thing (further tax cuts for the wealthy, privatization of remaining government services, no UBI because there is no money for that, further regression into service industry feudalism), sometime between 2025 and 2030 the world comes out of recession but now all money passes through an online gatekeeper (whatever follows Amazon) as the world transitions from sovereign nations to having its monetary policy determined by corporations, Stockholm syndrome far exceeding the Trump phenomenom grips the world in luddite-inspired rejection of magic technologies like AI, climate change falls from the forefront of concerns as simple human incursion into every last wild place on Earth creates an instability in civilization due to having so many insurrection cells crossing national boundaries that civic discourse gives way to political infighting and canabalistic government policy, a third world war or dystopian new world order or singularity hit all at about the same time between 2040 and 2050 so that only the rich and powerful have oracle-level AI technology and live in gated communities while the world burns, the masses find that they still need food in VR and then rise up and something happens which unfortunately can't really be predicted past this point.

I really hope that I'm wrong. But the last 20 years have been so bizarre compared to what I thought was going to happen while watching the Matrix in 1999 that I'd argue that we're already living in the early stages of dystopia. So I think we have a year or two left before the next planned recession takes the choice away from us. If even Hacker News is skeptical of the things that Yang is suggesting, then we are well and truly hosed.

<comment><USER13> > If I had a budget of a few million dollars

But 1k a month would make this happen?

If 1k UBI is causing this tragedy of the commons then you owe it to yourself to fix this, earn more, move to LCOL area and start curing diseases

<comment><USER50> $1000 a month 12 months a year 10 people team 10 years.

That's 1.2 million.

</comment></comment></comment><comment><USER51> Sounds like this is a paid article for an agenda. Yang sees where automation is going and as far as I know he's the only candidate talks about automation and coming up with solutions.</comment>


COST in the land of databases - <AUTHOR> https://github.com/frankmcsherry/blog/blob/master/posts/2017-09-23.md <comment><USER2> Tangentially, graph platforms have _long_ had the issue that performance claims are astonishingly unreliable and come with so many caveats and qualifications (documented or not) that the claims don't generalize usefully in practice. One of the reasons I got out of the graph database business was the pervasive lack of rigor and repeatability when it comes to measuring scalability and performance across platforms; there is a cost asymmetry to properly addressing the volume of questionable performance measurements made in the graph database world.

In fairness, graph workload performance, compared to other types of workloads, is unusually sensitive to a diverse set of details that are rarely captured adequately by researchers. As often as not, you are actually measuring things unrelated to the graph algorithm itself. Measuring graph databases properly would require a level of specification for how to measure such things that hasn't happened yet. And many people like it that way, it makes it relatively easy to contrive details that make your particular algorithm or implementation look good despite the reality that serious weaknesses are pervasive across graph database platforms. A generally, objectively strong graph database platform doesn't exist AFAIK -- they all kind of suck in unique ways.

<comment><USER3> A graph is a mathematical object. It has well defined properties, such as 'degree', 'radius', etc.

Just as in e.g. relational domain you can propose well defined operations (such as join two relations based on a common foreign key) and devise benchmarks for these operations, it should be possible to do the same precise thing for graph databases.

For a graph database, you would want (a) ingress, (b) comps, and (c) recovery benchmarks. The (b) comps per above can be based on a set of common patterns of queries applied to a parametric graph spec.

> In fairness, graph workload performance, compared to other types of > workloads, is unusually sensitive to a diverse set of details that are > rarely captured adequately by researchers.

That reads: we don't understand graphs.

</comment></comment><comment><USER4> That was a good read, but I think the author's conclusion is a little bit too punchy (although it did make it a fun read).

Figuring out how to scale-out non-trivial problems is difficult. Many of these scale-out solutions are going to be worse than local processing, and many won't add meaningful value at any scale. However, there's real value in approaching difficult problems, and there's real value in "the fastest way to do X in a cluster" even if it's not "the fastest way to do X". That's because scale-out is a real requirement to do a lot of interesting things, and because if we can find better ways to scale out we can start to compete with single- machine solutions (and "custom cluster" and "supercomputer" solutions) then we have made real progress towards something that's faster at all scales. It's a valuable research goal.

Now, it would be great if the database community (and the systems research community at large) was more explicit about that. They should set the standard that authors need to clearly differentiate between "improved on result X under constraints Y" and "moved the state-of-the-art forward under all constraints".

It's also good to read systems research skeptically, and think about whether you really need to pay the cost of scale-out. For companies who's output isn't primarily "research", it makes a lot of sense to constantly look for "the cheapest way to do X", not "the cheapest way to do X in a cluster".

<comment><USER5> Note that the author himself is working on a framework for distributed differential dataflow computations (dataflow computations in which it is very cheap to make small updates to the inputs and see the differences in the outputs), and he has plans to apply the same metrics (comparing to a single- threaded implementation) to his own work once he gets a little further:

[https://github.com/frankmcsherry/blog/blob/master/posts/2017...](https://github.com/frankmcsherry/blog/blob/master/posts/2017-03-28.md)

I think that answering the "fastest way to do X in a cluster" question is only particularly interesting if X is something that you need to do in a cluster. But clearly they are testing X that is capable of being done on a mid-range laptop. There's a real question of whether the X that they are testing will actually scale to larger datasets than the laptop (or a decently beefy single node) could handle; it's entirely possible that the cluster solution will blow up at that point. And of course, if the single-threaded implementation is faster, then you could presumably just do "X in a cluster" by doing "X on a single node" and using the rest of the nodes to mine your CPU bound cryptocurrency of choice.

<comment><USER6> > And of course, if the single-threaded implementation is faster, then you > could presumably just do "X in a cluster" by doing "X on a single node" and > using the rest of the nodes to mine your CPU bound cryptocurrency of choice.

As the author hi,self points out, amusingly.

 _These numbers are for sure more interesting. SEED does better than any number I can actually achieve on my laptop. That demonstrates non-triviality, in that their system isn 't just a laptop computing the answer and 156 cores mining bitcoin._

I really enjoyed the snark in this, and the series of blog posts he links in the beginning. I tend to be nice to a fault myself unless someone has shown themselves to not be acting in kind (and even then I confirm), but I admit I take quite a bit of voyeuristic pleasure in people being assholes in smart and interesting ways. Vicariously living and all that, etc...

</comment></comment></comment><comment><USER7> Haven't read the article yet, but these slides (from HOTOS 2015) are beautiful: [PDF] [https://www.usenix.org/sites/default/files/conference/protec...](https://www.usenix.org/sites/default/files/conference/protected- files/hotos15_slides_mcsherry.pdf)

That article is [https://www.usenix.org/conference/hotos15/workshop- program/p...](https://www.usenix.org/conference/hotos15/workshop- program/presentation/mcsherry)

</comment><comment><USER8> For anyone who is just passing by, the article attempts to verify some claims from recent VLDB and SIGMOD papers that claim decent* performance on graph algorithms in RDBMs. It's worth reading not only for the technical merits of the discussion, but also because the style of the article is ...feisty.

*for a given definition of decent.

<comment><USER7> There is one fundamental step you should always take when reading systems papers with benchmarks: _Always look for what the authors are trying to hide._

It's (a) always there, and (b) always hilarious. From operating systems comparisons that need "we didn't actually slow down the processor physically" comparisons to look good to these "we look great, compared to that guy over there licking the floor."

</comment></comment><comment><USER9> I love this quote by Paul Barham:

"You can have a second computer once you've shown you know how to use the first one."

<comment><USER10> Is there actually a source for this? The COST paper uses it, but I still haven't found what it's quoting.

</comment></comment><comment><USER11> He has some great points about SIGMOD/VLDB reviewing - they emphasize far too much on "algorithmic novelty", instead of impactful simple systems. However, in terms of graph processing, I too think in many instances, single machine processing is too limiting.

In general, I think people would be more convinced if there were real cases in industry where engineers switched from distributing graph algos to single- machine algos because they found performance benefits were better. In all my years, I haven't heard of a case like this (it's always been the other way around).

</comment><comment><USER12> The author should spend some time delivering solutions to "real problems". It's easy to be snarky while writing a blogpost with toy datasets but it's difficult to actually deliver production solution in an organization. The real COST is often not the time consumed but costs that arise from maintaining heterogeneous architecture and remaining legally compliant.

A database or hadoop cluster is not just an execution engine, but rather a managable, reproducible execution engine. Sure you can code up a smarter algorithm on your personal i9 workstation. However having something available as part of DB or Hadoop ecosystem enables reproducible execution while taking care of compliance (e.g. privacy), security, access control etc.

Eventually we will have a containerized extensions to DB query execution flow that will allow us to have best of both worlds but for 99% practicing data scientists downloading a Gigabyte sized subset to their own workstation is not a viable option.

The real opportunity lies in ability to add arbitrary compute/memory capacity to DB query execution flows.

<comment><USER13> That's absolutely true, but I think the author's argument is likely to be that the popularity of Spark and Hadoop in the first place is misguided, and in a world where people were paying attention to performance instead of novelty, we'd have something like Spark and something like Hadoop that was optimized for a single computer, not for scaling for the fun of scaling.

<comment><USER12>
       scaling for the fun of scaling.

I think your arguments are misguided, for every compute bound task where Hadoop/Spark undeperform, 1000 other ETL type tasks where hadoop is indispensable. As a result any organization running a large hadoop cluster will already have underused compute capacity for free and a maintainance staff which already taking care of the cluster. Thus from an organizations perspective the time difference is not material especially for batch jobs, this is the reason why Presto and Spark have been so successful. They enabled underutilized hadoop clusters to be used for ML and data science while delivering reasonable performance for Zero cost.

<comment><USER14> > _"... Staff already taking care of_ the _cluster."_

"The" cluster, singular ...

This is the catch. If you need security and compliance, you can't today derive the benefits of re-use by other teams.

Given today's distros, and assuming your threat model needs to account for insider threat, you need a different cluster for each data ownership grouping and data sensitivity level. For four teams with three levels of data, you'd need twelve completely independent clusters.

Unless, as noted above, you've done a ton of in-house multi-tenancy work to provide full stack security and compliance assurances and audits.

<comment><USER12> However you may split clusters by role/access. You will have I/O bound ETL/Interactive tasks as primary use case. Thus the investment in security will be constant whether you reuse the same infrastructure for compute/memory bound ML tasks or not.

</comment></comment></comment></comment></comment><comment><USER15> Not sure if it's related, and AWS Athena is not a Database in a strict sense (or in any sense at all since it's not a transactional DB), but it seems to offer both scale and lower cost if it fits your requirements (read only, DW like traffic)? Anybody has experience using it in prod as a DW replacement?

<comment><USER8> I would argue this question is not really related, but we use PrestoDB (the technology AWS Athena is built using), and it is particularly great for when data becomes too large to fit in a "regular" database. PrestoDB and Apache Impala are the two major open-source surprisingly-fast-distributed-SQL-query- planning-engines. If you pair either of them with a columnar data storage format like Parquet or ORC, you can make analytical workloads work on inconveniently-sized data. AWS Athena abstracts the cluster management for you and runs PrestoDB on spare EC2 instances under the hood.

<comment><USER15> As mentioned in the other comment, I was actually only thinking about use cases like Athenta + csv_on_s3. and that's the really "low cost" approach I was referring to for analytical data.

</comment></comment></comment><comment><USER16> I am overloaded with acronyms. For crying out loud can we please just say what we mean instead of sending everyone down an acronym wormhole three whitepapers deep?

<comment><USER7> From this post:

" _Our first paper is Scalable Distributed Subgraph Enumeration or "SEED". We will have a future post about rules for picking acronyms._"

I suspect the author would agree.

</comment></comment><comment><USER17> Is this guy for real?

He is limited by not having an "expensive macbook with 4 real cores" ?? He gave up on a test because it "paged out the 15GB of data".

Of course at these scales a cluster is x5 slower.

Can someone please explain - am i missing something here ?

<comment><USER18> > Of course at these scales a cluster is x5 slower.

So this complaint should be levelled at the papers and the academic communities- why is it acceptable to only provide empirical evidence for these distributed algorithms at data scales where they cannot out perform simpler implementations? Why not require empirical performance measurement on truly large scale examples for these kind of algorithms?

<comment><USER19> Of course academics don't have access to truly big data, so this amounts to saying that they can't publish anything. That's fine with me but I don't think the academic CS establishment will agree.

<comment><USER7> " _the academic CS establishment_ "

Strictly speaking, I think that term attributes more organization to the academic CS establishment than my experience supports.

The point here is that, if they're trying to establish that Smart Technique 2 is better than Smart Technique 1, they need to first establish that Stupid Technique Q doesn't work at all. Which they haven't.

If they don't have access to "truly big data" to do so, well, most of the people who want to use the Smart Techniques probably don't either.</comment></comment></comment></comment>


Exploring and Dynamically Patching Django/Python Using GDB - <AUTHOR> https://stripe.com/blog/exploring-python-using-gdb

<comment><USER2> Couldn't you do almost the exact same thing using pdb, without all the C API / register stuff?

Maybe I misunderstood the post, but it sounds like you're just stopping right before the request returns, reloading the module, generating a new response, and allowing execution to continue.

Why is GDB required for any of this?

<comment><AUTHOR> You're right! You could do all of this with pdb, but only if you have enough foresight to run the app under pdb to begin with (which I definitely never do).

Using GDB, you don't have to change the app or remember to run it in a particular way.

<comment><USER2> Right, I see the use in this now. How difficult would it be to wrap this up in a reusable script that you could run directly, instead of having to manually work through the debugger prompt?

<comment><AUTHOR> Hmm, it'd probably be doable. In particular, you can use the "commands" command to script what happens when you hit a breakpoint (a friend talked about this in a Ksplice GDB blog post: [https://blogs.oracle.com/ksplice/entry/8_gdb_tricks_you_shou...](https://blogs.oracle.com/ksplice/entry/8_gdb_tricks_you_should#command))

Normally, the "finish" command will interrupt any script you're in the middle of executing, so we have to do a bit of an ugly hack to make sure our script keeps running:

b PyEval_EvalFrameEx if strcmp(PyString_AsString(f->f_code->co_name), "handle_uncaught_exception") == 0

commands

disable

frame 3

python

gdb.execute('finish')

gdb.execute('shell git stash pop')

gdb.execute('call PyImport_ReloadModule(PyImport_AddModule("monospace.views"))')

gdb.execute('call PyImport_ReloadModule(PyImport_AddModule("monospace.urls"))')

gdb.execute('set $self = PyDict_GetItemString(f->f_locals, "self")')

gdb.execute('set $request = PyDict_GetItemString(f->f_locals, "request")')

gdb.execute('set $get_response = PyObject_GetAttrString($self, "get_response")')

gdb.execute('set $args = Py_BuildValue("(O)", $request)')

gdb.execute('set $rax PyObject_Call($get_response, $args, 0)')

gdb.execute('enable')

gdb.execute('c')

end

c

end

(Bah. Edited because I couldn't get monospace text working)

You could put those commands into a file and run "gdb -p <my django process> -x <my commands file>"

Of course, instead of shelling out to git stash pop, you'd probably want to pause so you could update the code. And you may need to reload more modules than just monospace.views and monospace.urls, depending on the change.

<comment><USER3> _> Edited because I couldn't get monospace text working_

If you start each line with a few spaces, HN will format it as source code:

        b PyEval_EvalFrameEx if strcmp(PyString_AsString(f->f_code->co_name), "handle_uncaught_exception") == 0
        commands
        disable
        frame 3
        python
        gdb.execute('finish')
        gdb.execute('shell git stash pop')
        gdb.execute('call PyImport_ReloadModule(PyImport_AddModule("monospace.views"))')
        gdb.execute('call PyImport_ReloadModule(PyImport_AddModule("monospace.urls"))')
        gdb.execute('set $self = PyDict_GetItemString(f->f_locals, "self")')
        gdb.execute('set $request = PyDict_GetItemString(f->f_locals, "request")')
        gdb.execute('set $get_response = PyObject_GetAttrString($self, "get_response")')
        gdb.execute('set $args = Py_BuildValue("(O)", $request)')
        gdb.execute('set $rax PyObject_Call($get_response, $args, 0)')
        gdb.execute('enable')
        gdb.execute('c')
        end
        c
        end

</comment></comment></comment></comment></comment><comment><USER4> I've always missed this feature from Seaside/Smalltalk. (Not to mention having a proper language-level debugger at your disposal.)

</comment><comment><USER5> nice use of cpython knowledge. very black magic, but very usefull in time.</comment>


Show HN: Using Social Network Analysis to predict your event's guest list - <AUTHOR> http://whodidiforget.com/

<comment><AUTHOR> Given that my work as a PhD student focuses on social groups, I thought I'd launch a little website to showcase the applications of what I do. I'd appreciate any feedback you might have.

For those interested in the gory little details, all the analysis is done client side, in Javascript (as a matter of fact, the whole app is just a static website hosted on github pages). If some are interested, I might blog about the different aspects of the implementation.

<comment><USER2> Please do blog about the different aspects of implementation. Or at least post the link to the Github repo. This seems very interesting, and I'd love to hear more about how you did it.

<comment><AUTHOR> Unfortunately, the github repo is private because I can't open source the whole algorithm right now. But I'll definitely do a write up in the next few days.

</comment></comment></comment><comment><USER3> > _I'm not looking at messages/interactions/networks/existing groups/friends > list. Just the underlying graph of friendships._

Other interactions may give you better signals may increase the accuracy of your system. Cool app!

<comment><AUTHOR> Of course, that's be ideal, but it would take longer to compute the weights. The idea here was really to show that just by looking at friendships it's possible to find accurate results.

<comment><USER4> There are a lot of apps in Facebook that does weight the friendships and generated images. Maybe one of them have their algorithm opened and you may use it to improve the accuracy.

<comment><AUTHOR> I'll rephrase that: adding a weight to an edge is not hard, I actually having several heuristics on hand to do that. It's just really time consuming the first time you do it, because you have to load a lot of data in order to be accurate. Moreover it requires additional permissions, which I'd rather avoid.</comment></comment></comment></comment>