What The Chinese Tech Industry Is Like - <AUTHOR> http://www.businessinsider.com/what-the-chinese-tech-industry-is-like-2014-1 <comment><USER2> I would love to see the down sides. It feels like he's on his honeymoon stage with the country.

I know of a few downsides but the current article make it seems like a tech paradise.

* pollutions

* the # of people graduating college doesn't imply that the quality of education in China is the same as USA. I've heard stories and my ex went to China to study abroad and cheating is rampant. It's mostly rote learning.

* the stock and IPO he've mentioned make it seems like China stock market is awesome. I've played around with China stocks when China started to trade. Let me tell you, there were tons of pump and dump and companies lying on their balance sheet. An example was JRJC, CAST, CCME, etc.. It's hard, very hard market.

<comment><USER3> All of the amazing websites and apps created here are just copies of other apps, localized for China. I don't see this as anything to be too proud off, creating the apps isn't hard, and china has a huge number of people so you have that going for you.

Show us were China is innovating, that is where the real story is.

<comment><USER4> Roll back in the 80ies, you will see the same comments about Japan doing no innovation on cars, electronics, etc.

I work in China tech sector, young Chinese are innovating furiously, but it will be more obvious when there will be one or two worldwide success. I would bet it'll come from hardware and probably shanzhai movement.

</comment></comment></comment><comment><USER5> All those numbers are very misleading. China has shit load of college grads because 90% of the universities are just diploma mills that you put in 4 years and you get your degree, there is almost no way to fail out. Most of those graduates cant even find jobs because they know nothing when they graduate.

The chinese stock market is a joke, there is little regulation, insider trading and pump and dump schemes are rampant.

All those apps have lots of user but many of them are bots. Companies will use bots to self promote and repost their shit. I have a weibo and have maybe 4 or 5 posts over the last year and have around a 1,000 follower almost all are bots who just repost everything i have posted. They are used to boost other people follower count to make them seem more popular and gain more legit followers, people call them zombies, they will follow random people just to make themselves seem more legit and not get deleted by the system.

The biggest hurdle in china is regulation. The Didi app got shutdown last year because they were allowing people to add tips to their request for taxi, i.e. you would say i need to go to x location i will give you 5 dollars extra if you are here in 5 minutes. This was declared illegal as taxi prices are set by the government and tipping the driver is considered an illegal bribe. They had to remove this feature before they were allowed to continue operation but it took weeks of them negotiating with the government before they could go back online. Apps like Momo are filled with hookers soliciting guys and other illicit activity.

Google play is actually not banned in china. It is just not the default app store for android phones and many providers have disabled the ability to install the play store on their phones. But if you have a compatible phone it works just fine. There may be so money being paid by other apps stores like baidu to make sure the google play store is hard to use in china.

This article is look at things with rose colored glasses. Things are not that great for startups in china. In fact if you dont have a large bank roll or good government connection chances are your chinese startup will be shutdown by some form of government intervention since the internet is so heavily regulated almost everything you do on the internet required special permission from the government

<comment><USER6> Again, you just told one side of the story.

It is true that quality of large number of universities in China is poor, but they still provide millions of Chinese teens every year the opportunities to know more about the world outside their native towns or villages. They learn a lot from Internet and classmates even nothing valuable taught in classroom. They will be more confident and be important part in make everything in China better and better.

And it is also true that China invested more and more in education. The average quality is not that good, and the efficiency is low, but there still are big positive impact on China society in general. Before the big leap in higher education, only very few teenagers can enroll the universities. The overall education level is much lower.

As for bots and zombies, it is true that the numbers especially of Weibo are a bit overstated. But the real number is still big, very big. After all China has the biggest population. The smart phones sold in China has exceeded USA last year. Most of the users will install some apps like Wechat, QQ, Weibo and Alipay, etc. So you can calculate the real numbers. The order of magnitude is the same (in hundreds of millions).

Regulation is a problem, but the government does not rule everything. You can find your ways to survive and grow. Nearly all Internet giants including Big Three (we called BAT=Baidu,Tencent, and Alibaba) in China grew from nothing, without a large bank roll or good government connection at least at the beginning.

</comment></comment><comment><USER7> With those huge user bases for these Chinese sites how come we don't get much feedback on the web about there scaling issues and challenges?

What platforms are these massive sites built on? as they seem to dwarf the US darlings we regularly discuss on HN.

Where can I find the Chinese dev and admin forums and read via google translate?

Thx

<comment><USER8> A lot of innovation seems to come out of China. I know that, at least in the world of Clojure, the fastest webserver is http-kit, which Shen Feng spent a lot of time developing:

[http://shenfeng.me/async-clojure-http-client.html](http://shenfeng.me/async- clojure-http-client.html)

[http://http-kit.org/600k-concurrent-connection-http-kit.html](http://http- kit.org/600k-concurrent-connection-http-kit.html)

[https://github.com/ptaoussanis/clojure-web-server- benchmarks](https://github.com/ptaoussanis/clojure-web-server-benchmarks)

</comment></comment><comment><USER9> Are most of those numbers that surprising considering China's population? To me, it's not very interesting that a country with a huge population has a huge amount of graduates, etc.

<comment><USER10> I think the interesting thing is how little awareness of all of it there is in the US. It's easily explained (language barrier) but many people probably think that Twitter is the most active micro-blogging service in the world, that PayPal is the only really huge payments service, or that Amazon and eBay are the major players in ecommerce. For any service you can think of in the US, there's one in China that might be even bigger.

<comment><USER11> > but many people probably think that Twitter is the most active micro- > blogging service in the world

It is. Weibo doesn't do well outside of China, it is not international like Twitter is.

> that PayPal is the only really huge payments service

From the perspective of a non-China resident, that is definitely true.

> Amazon and eBay are the major players in ecommerce

Ditto. Though Taobao is trying to become a major international player.

> For any service you can think of in the US, there's one in China that might > be even bigger.

Except the services in the US are mostly international. The services in China are almost always limited to just China. The fact that UnionPay is the largest ATM network in the world (by users...almost all in mainland China) doesn't mean diddly when I visit the Netherlands or India where my ATM card is USELESS.

China has decided they can lock out international internet services (via the GFW) and build their own isolated network. Internally, it is huge, but the isolation cuts both ways.

<comment><USER6> GFW did do something but can not explain everything. China Internet companies have their own advantages: local leaders can make agile decisions to local market need, employees are more motivated, the management structure is flatter... For example, ecommerce was not regulated much here, but Amazon and eBay were beat heavily by local competitors.

<comment><USER11> This is quite ridiculous. If the GFW didn't block Facebook or Twitter, they would have market shares in the mainland similar to those that they have in Hong Kong and Taiwan.

Chinese Internet companies are still quite Confucian in their organization with meaningless titles and lots of seniority plays; they are hardly flatter than the typical American internet company. Many of them still rely on freaking time cards for their tech workers, for crying out loud!

Amazon does well in China; we order things from them all the time (even rice).

<comment><USER6> >If the GFW didn't block Facebook or Twitter, they would have market shares in the mainland similar to those that they have in Hong Kong and Taiwan.

They would have some market shares but not very much, just like Amazon.com is a small player and one of few foreign survivors in China although I also like its service. In the past, MSN messenger, Yahoo, Google, eBay, MySpace, to name a few, all failed in competition with local China players. That is not by chance.

As for management, the biggest problem of foreign companies is that decisions especially critical ones are always made in USA, which is normally slow and ignorant. And founders and top guys in China companies(still very young at their 20s or 30s) like Pony Ma at Tencent and Robin Li at Baidu can directly attention the specifics of key products or services even at pixel level.

Who will win?

<comment><USER11> > They would have some market shares but not very much, just like Amazon.com > is a small player and one of few foreign survivors in China although I also > like its service. In the past, MSN messenger, Yahoo, Google, eBay, MySpace, > to name a few, all failed in competition with local China players.

Then why doesn't the CCP just unblock twitter, facebook, g+, and so on? I have a hard time to believe its for reasons of porn (easy to find in the GFW) or subversion (also easy to find in the GFW). And how do you explain Facebook's success in HK and Taiwan? People outside of China just being afraid of Chinese services? That could make sense...

> As for management, the biggest problem of foreign companies is that > decisions especially critical ones are always made in USA, which is normally > slow and ignorant.

The biggest problem is that we can't play very well at all in the market. First, foreign companies have to follow all the rules very strictly: local companies have the ability to be "flexible" until (and if!) the government cracks down; they also don't have to deal with the Foreign Corrupt Practices Act (FCPA). The lawyers of most foreign companies also won't go near ambiguous China privacy and secrecy laws with a 100 ft. pole, meaning anything that deals with data or communication inside the GFW absolutely requires a local partner (say TomTom).

I have friends who worked or are working for local companies, and they tell stories of extreme shadiness. These are not even small no-name companies, but companies you've probably heard of before; I won't name names since the foreign community that works in these companies is very small. They work very hard, but the market is cut throat and they do what they need to do to survive.

<comment><USER6> >Then why doesn't the CCP just unblock twitter, facebook, g+, and so on?

It seems that you believe China government helped local players succeed. That is not true in general. The private local Internet companies are not the government's child. They are mostly controlled by foreign investors.

<comment><USER11> The CCP likes companies they can control, whether there are foreign investors or not. This is especially important since the rule of law isn't so developed in China and often directives go through non-legal channels, which Google couldn't deal with while Baidu has no problem with it. Both of these companies are listed on the Nasdaq, but one is controlled by Americans and the other by Chinese.

<comment><USER6> If CCP wanted to control the Internet, it should help Renmin, Xinhua, Wangfujing or China Mobile to be market leaders, not Baidu, Sina, Taobao or Tencent.

Actually except Baidu vs. Google, you can not think out other examples the government made big impact. The other competitions in Internet between Yahoo vs. Sina, eBay/Amazon vs. Taobao/JD, MSN messenger vs. QQ, are all good counter-examples.

Foreign companies can not adapt to needs from local users. That is the key. The market is the king to decide the fates of Internet companies and products in China generally. Others are important but not critical.

<comment><USER11> > Foreign companies can not adapt to needs from local users. That is the key. > The market is the king to decide the fates of Internet companies and > products in China generally. Others are important but not critical.

And yet, all those companies do very well in HK, Singapore, Taiwan; aren't they Chinese also? Why is the mainland market so different from other smaller Chinese markets? They also do well in Indonesia, Thailand, Nepal, India...while the Chinese companies...they are just going to be stuck being local.

<comment><USER6> I do not know a lot about the things outside China. But Japan is another example that many foreign Internet companies struggle. Maybe we could add Russia. These markets have something common.

HK and Taiwan are too small market to have strong enough local team.

<comment><USER11> Google and Facebook still do very well in Japan. Yahoo does also, obviously. Russia is no where as near as closed as China is, and many international companies do well there. China is quite unique.

Supposedly, Chinese internet companies should do well in HK/Taiwan, these markets are definitely open to them, but since they don't have to compete with international brands in the mainland, they are not competitive enough to operate very well on the outside. Besides, any service run inside China is taboo outside of China (and Americans think the NSA is bad...).

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> Barra used a graph showing that China produces more college degrees to draw the conclusion that China is "more educated" than the United States.

It'd be helpful to see college degrees conferred per capita before drawing this conclusion, as a country with 4.3x the population will obviously produce more degrees.

EDIT: In fact, it looks like China has about 8 million degrees conferred at last count, and the US about 3 million. Given the population multiplier, China seems to be far behind the US per capita.

<comment><USER13> The average quality of Chinese higher education is also lower. The top schools/students are of course good, but the long tail of subpar education is longer than it is in the US. Source: I've lived in China.

That's not to say it will always be that way, though.

<comment><USER11> Even PKU and Qinghua aren't that great; its the quality of the students coming in that saves them.

</comment></comment></comment><comment><USER14> Do those slides all add up? Incomes are rising ... except the world probably can't support a billion chinese at 1st world income levels. There are many billionaires ... because their gini curve is shot to hell. They have a higher percentage of graduates ... but most of their degrees are of a low standard. Looking at the company valuations on top of that and I start to get worried that this is a giant bubble that's going to take the rest of the world down with it when it bursts. I hope you can prove me wrong China!

<comment><USER10> What is it about the world that you think can't support that? It's new territory, and it may "disrupt" the US (to some extent) over the coming decades, but China has been bootstrapping itself out of farming for quite a while already. It's mainly the internet industry that lagged the US by a few years, but has caught up or surpassed it now.

<comment><USER15> The world already can't support the existing first world lifestyles we have in the long term. We are doing a tiny bit to move toward sustainability, but if you tack on a few hundred million more people living that way, you run out of essential resources fast.

</comment></comment></comment><comment><USER16> To those who are interested in startups in China:

I personally know some people who worked on start ups in China. From their experience, I see no hope for start ups without connection with government. I will show you some facts and some real examples here.

Here are some basic facts:

Chinese government controls Internet content _tightly_. Note that it is not the same level as NSA's surveillance. NSA's surveillance is bad. But it won't, or rarely, (directly) change/delete the content. The Chinese government, on the other hand, can even shut down a whole datacenter without telling the administrator with no reason --- By "no reason", I mean absolutely no explanation.

I had a blog and I know the service provider personally (It was a website hosting blogs like myspace). My blog was a high ranked one on their website. I posted some content discussing related to democracy/free speech in China and the whole website was down for a day. My friend received a call from network management department after they cut the network. They forced my friend to delete my blog entirely (the whole blog including posts for years.)

Chinese website was required to control any content on their website, otherwise, like the case I mentioned above, their website might be shutdown before telling you the reason. This is especially true for small startups.

Here is another example: during the early age of twitter (around 2008.), there were several startups in China trying to make a copycat of twitter in China. We call it micro-blogs and there are companies like hainei, jiwai, etc. working on twitter-like services. As we all know, content on such websites generated very fast. Much faster than traditional blogs. This makes the filtering work extremely hard --- those websites has to identify contents which are "politically incorrect" and delete them before any government officer found them (And yes, we DO have government employees who are dedicated to found those contents and report it to their higher authorities.) Such filtering work is impossible for a startup with tens of employees -- They used some filtering algorithms but people can always use other words to bypass the filter. Such micro-blogs (or twitter-like services) grows so fast with too much "unwanted" content, that finally, by mid-May 2009, they were all shut down by the government for half a year. Imagine a startup running without a live service for half a year, you know how hard it would be.

During this time, large companies with government connections like sina, tencent, 163, ect, prepared their own micro-blog services and finally put it on line. Such companies has large teams who dedicated themselves to filter content on their websites. Nowadays, the micro-blogs are only run by those large companies and none of the startups survived.

I told these stories to those who do not familiar with Chinese Internet Policy and intent to do some startups in China. My advise is: do not run a content- based startups, otherwise you can hardly afford the cost of filtering the users' contents.

EDIT 1: changed some spellings and expressions.

<comment><USER7> So I think the world understands that doing business in china requires some special attention.

Despite all of that they are doing business in China. Serious dollars are being made and value being delivered to millions of users. We should want in on that or to at least have some understanding of how its being done in comparison to US/EU.

<comment><USER11> You assume the Chinese market is free, which it is not. The barriers are high, and the government just doesn't want you getting in on the business.

<comment><USER17> The last 30 years of Chinese economic growth (fuelled by foreign investment) disagrees with you.

<comment><USER11> The Chinese are totally ok with taking our foreign money (or HK, Taiwan), they just don't want us to get rich off of it (at least very easily). The economy is still very protectionist and you have to be very careful when investing here; many industries are just completely off limits and even those that are left open are often hobbled by protectionist ownership rules (e.g. 51% must be owned by a well connected Chinese company).

</comment></comment></comment></comment></comment><comment><USER18> This just sounds like pure propaganda.

<comment><USER10> Why? Do you not believe the numbers? Do you not believe that the services mentioned exist? Do you not believe that the services work well?

<comment><USER19> Well, take--for example--the number of degrees awarded: do those students know anything of actual value? Are they able to solve problems, or did they just get a piece of paper?

</comment></comment></comment><comment><USER20> Any good startup accelerators in China?

<comment><USER21> Cyberport and Science park in Hong Kong are both quite good.

[http://www.cyberport.hk/en](http://www.cyberport.hk/en)

[http://www.hkstp.org/en-US/Homepage.aspx](http://www.hkstp.org/en- US/Homepage.aspx)

</comment></comment><comment><USER22> By the way, the interviewer was Loic Le Meur of LeWeb

</comment><comment><USER23> Is it just me or HN users are shitting their pants after knowing that China is bigger than US on some aspects

<comment><USER24> I think it's that much of his evidence doesn't really prove anything. Example: "Weibo is like Twitter, except bigger. Barra only ever had 6,000 Twitter followers. In two months, he has 200,000 Weibo followers." The second sentence does not prove the first sentence.

<comment><USER25> Weibo has about 60m daily, twitter has around 100m daily users

Source: random & recent google result titles</comment></comment></comment>


Customer Buying Hierarchy - <AUTHOR> http://travid.info/blog/customer-buying-hierarchy/ Free creates its own problems that stand in the way of innovation. <comment><AUTHOR> I heard about the concept from Clayton Christensen's Innovator's Dilemma. [http://www.amazon.com/Innovators-Dilemma-Revolutionary- Busin...](http://www.amazon.com/Innovators-Dilemma-Revolutionary-Business- Essentials/dp/0060521996) The book talks about the buying hierarchy in the context of disruptive innovations. It's highly recommended.

</comment><comment><USER2> Googling for "Customer Buying Hierarchy" shows no other references to this, other than this blog entry & related videos. Can you provide some other pointers to where others might use this concept?</comment>


Should I use infiniband on my Hadoop cluster? - <AUTHOR>

All my cool friends say infiniband is a must, but I think that it's too expensive. Who is right, why should I believe them? <comment><USER2> Isn't the point of Hadoop to use the network as little as possible?

<comment><AUTHOR> Workloads have to use the network to do the following :

1\. Shift data onto HDFS (obviously), this is rare though 2\. Write results after reduce (or you can't read them) when you are doing ETL this can be significant; you are sorting out an image for a production system and this is non-trivial abouts of data (100's GB at least) 3\. Shuffle; this is the worry (for me) MR jobs can shuffle a lot of data around pre reduce.</comment></comment>


Database Design: Relation Predicates and "Identical Relations" - <AUTHOR> http://www.dbdebunk.com/2016/09/database-design-relation-predicates-and.html <comment><USER2> Wow, this was hard to read. Not because it was very complex, just because of the incomplete sentences and unessescary use of abbreviations.

In the other post [1], linked from this one, he nit-picks because the question uses the terms "tables" and "fields" instead of "relations" and "attributes".

I'm sure the author knows more about relational modelling than me, but I'm not sure I would enjoy asking him for advice.

1: [http://www.dbdebunk.com/2012/09/NormalOrthogDBDesign_3.html](http://www.dbdebunk.com/2012/09/NormalOrthogDBDesign_3.html)

<comment><AUTHOR> > Wow, this was hard to read.

Well maybe the reader had something to do with it.

<comment><USER3> Are you trying to communicate, or be right on the internet? If you're trying to communicate, readability matters.

</comment></comment></comment><comment><USER4> Article is poorly written, long winded, nonsense. I am happy for Relation names to distinguish entities. He is basically arguing that this violates some sacred principle and that all meaning should be available from column names, so that relations names can be ignored. I don't think that even Chris Date who is a zealous RM fanatic would argue this - his books are actually quite good much more logical than Fabian Pascall. I'm also sure that Joe Celko would not agree with Fabian on this.

</comment><comment><USER5> > If two relations have identical designs, the difference can only be encoded > in the relation (and possibly attribute) _names_ , in violation of the IP > [information principle]. As such, it is inaccessible to the DBMS, which > cannot rely on the RPs [relation predicates] to accept/reject tuples as > correct/incorrect. Users--with little, or no help from the DBMS--must ensure > that the correct tuples are inserted in the proper relation. Moreover, > because information is not represented explicitly (as data values), > relational operations lose it: if you UNION the two relations, you get users > that either viewed or downloaded items, or both.

The author appears to be suggesting that the name of a relation has nothing to do with the meaning of a tuple in it. I've read dozens of books and papers on database design and I have never come across anyone ever suggesting any such thing.

He also seems to be saying:

(1) The name of a relation is inaccessible to the DBMS.

(2) You should be able to throw tuples at a DBMS and it should just figure out where they go.

(3) If there are tables A and B in your database and you can't reconstitute A and B separately from (A UNION B), then there's something wrong with your database design.

<comment><USER6> > The author appears to be suggesting that the name of a relation has nothing > to do with the meaning of a tuple in it.

The name of a relation has to be a good mnemonic for human use, but ultimately, the real meaning of a relation is what the DBMS can enforce about its tuples, just like the meaning of a type is what the type system can enforce about its inhabitants. You can't prove much about how humans will actually use a relation (or a type), since that depends on their whims, but you can prove useful things about the _limits_ of what humans can do with a relation (or type), even in principle. And the purpose of a good database schema (or typeful design) is to make nonsensical things impossible, even in principle.

<comment><USER7> >And the purpose of a good database schema (or typeful design) is to make nonsensical things impossible, even in principle.

Can you elaborate on what is the nonsensical thing possible with two tables of the same type?

To me the (A UNION B) example makes very much sense exactly in the case that you want all x s.t. (x is in A) OR (x is in B). With differently named id columns you might have to write this as (excuse my pseudo code):

      (RENAME(A, a_id TO a_or_b_id) UNION RENAME(B, b_id TO a_or_b_id))

which is indeed more explicit. Having written that I might see the rub here. In the article the column "item_id" is kind of a "view_or_download_id" and which it is is only specified the namespace (i.e. the name of the table ITEMS or DOWNLOADS the tuple belongs to). In an OO world this would be the difference between:

      class Person { public string Name { get; set; } }
      ...
      var employees = new List<Person>(...);
      var managers = new List<Person>(...);

and the version where employee and manager types are incompatible:

      class Person { public string Name { get; set; } }
      class Employee : Person {}
      class Manager : Person {}
      ...
      var employees = new List<Employee>(...);
      var managers = new List<Manager>(...);

The latter would indeed prevent e.g. employees.Concat(managers) on accident. However, in many programming scenarios the former would be preferred for the increased code reusability. Is this not an objective in database design (with e.g. stored procedures that could be applied to more than one set of tables)?

<comment><USER6> > Can you elaborate on what is the nonsensical thing possible with two tables > of the same type?

There's nothing intrinsically wrong with it a priori. But see below.

> To me the (A UNION B) example makes very much sense exactly in the case that > you want all x s.t. (x is in A) OR (x is in B).

To me, the very idea of making a collection whose elements have different logical types is completely nonsensical. I consider it a weakness of SQL that I can form the relation:

        select customer_id as id, customer_name as name
        from customer
        union
        select employee_id as id, employee_name as name
        from employee

The attributes customer_id and employee_id should have _different abstract types_ , so that they can't be accidentally conflated with one another, even if their internal representation is the same (say, an autoincremental int). Then, if you want to make a derived relation containing both customers and employers, you should be forced to use a sum type (à la Haskell's Either) as the primary key.

> In an OO world this would be the difference between: (snippet)

Object-orientation is simply the wrong paradigm for building data-oriented applications, at least if you care about automatically enforced data integrity and declarative data manipulation (which you should): [https://news.ycombinator.com/item?id=11861520](https://news.ycombinator.com/item?id=11861520)

</comment></comment></comment></comment><comment><USER8> Hey, I can talk like that too.

      A mental construct has occurred to my consciousness
      subsequent to the exposure of said mind to myriad
      graphological artifacts (GE's) conjecting observations
      that are encapsulated in a word structure (WS) of such
      syllabic cornucopia that the reliable transmission of
      said observation to the recipients takes on a high
      degree of uncertainty.

Now it's your fault if you don't understand it.

<comment><AUTHOR> I wouldn't expose your inability to understand if I were you.

<comment><USER8> Why is that?

</comment></comment></comment><comment><USER9> I would say the biggest decision point here is how either entity might evolve. If these are truly different domain entities then one may end up with different attributes than the other in the future, meaning these should definitely be different tables. They just happen to have the same set of fields at this point in time.

</comment><comment><USER10> Wow, for a site that claims to be "database fundamentals made accessible", that's a fairly inaccessible, overwritten piece.

I think the fundamental argument it is making has some value; if 2 or more tables [0] are structurally different _only_ in the name of the table, then the facts in those tables are almost certainly related instances of some general common supertype, such that there should be one table, with an additional field disambiguating the subtype (and which probably a foreign key into a new table specifying the valid subtypes, which have a 1:1 relationship to the tables in the "bad" schema.)

OTOH, in lots of practical applications no application would ever care about that "ideal" base table, and all interaction would be through views that exactly correspond to the tables in the "bad" design. Outside of a DB serving as an ideal, application-independent store (which is an important role, and even in single-app DBs designing for this can have advantages in dealing with growth and change and unexpected future uses), the _value_ of this ideal transformation may be minimal.

[0] base relations, if you prefer.

</comment><comment><USER11> Fabian Pascal tends to write as if the point of database work were to construct perfect ontologies. Perhaps we would be better off if we did that, perhaps not. In this case, perhaps it would be a cleaner design to have a single table with ID, USER_ID, FIRST_VIEWED, FIRST_DOWNLOADED. But would I lose sleep over this? No.

<comment><USER6> > Fabian Pascal tends to write as if the point of database work were to > construct perfect ontologies.

Wait, is it not?

</comment></comment><comment><USER12> I found this approach deeply troubling because it moves us away form semantic design towards logic design... which always runs into problems when the database itself becomes semantic content. The write up was rather confusing.

The solution seems straightforward. A single table that captures the meaning expressed by the separate VIEWS and DOWNLOADS tables. eg. USERACTION (USER_ID, ITEM_ID, ACTIONTYPE) where ACTIONTYPE is a value like V for view and D for download. Of course, that solution is hard to see because it's a synthesis of meanings occurring at different levels and not the product of predicate logic.

<comment><AUTHOR> Database design IS logic. That's the point of the RDM: to formalize and symbolize semantics such that the DBMS can enforce integrity on and manipulate data, such that logical and semantic correctness is guaranteed. Leave that to users in apps at your peril. We used to do this before the RDM and the whole shabang collapsed. And we're still doing it because practitioners know nothing beyond SQL and coding.

<comment><USER10> > We used to do this before the RDM and the whole shabang collapsed.

No, it didn't, and relational-theory-purists aren't going to sell their ideas to practitioners in the real world by pretending that it did. The RDM certainly _offers_ all kinds of abstract benefits, which practitioners often do not fully understand or leverage, and there is a very real problem when the not fully leveraging is due to not fully understanding (rather than weighing practical costs and benefits in the particular use case.)

OTOH, the reason that things built on the relational model took off in practice wasn't that non-relational systems had reached a point of catastrophic logical failure that led to their rejection, but because the relational model had a convenient mapping to implementations that were convenient and efficient in the technology of the day (particularly, hard disk storage), combined with _some_ of the structural improvements over other approaches being particularly attractive for important application domains.

> And we're still doing it because practitioners know nothing beyond SQL and > coding.

Yeah, look, we're probably never going to have a time when most practitioners are deep theoreticians rather than expert tool users, and if you want to sell practitioners on deeper consideration of the underlying theoretical models, you're going to need to make explanations of the practical benefits _much_ more accessible than you have in the source article _or_ your comments in this thread (and you're going to need to be a _lot_ less personally abusive.)

<comment><AUTHOR> Right, there is never time to do it right and lots of time to do it over.

I do not think that your reading of history of the field is anywhere close to reality. I do suggest that you read as carefully my comments as I write them: I did not say practitioners ought to be theoreticians, I said they should no engage in a field founded on logic without ANY intro to logic. Big difference.

In fact, the initial mapping to implementation--direct image SQL implementations--was not in the relational spirit at all and is in large part responsible to logical-physical confusion and confusion of tables with relations. And to call those initial implementations efficient in the technology of the day is from another planet. IBM would not budge implementing the RDM until Oracle forced it. It just so happened that even the limited relational fidelity of SQL proved superior to the rigidity, complexity and lack of soundness of hierarchic and network technologies. Have you ever seen IMS or Codasyl code?

Listen, have done nothing but exactly making the practical implications of the theory for the last 40 years. I suggest you read my stuff and tell me exactly what is wrong with it. The problem is lack of fundamental education which has been replaced by tool training. Practitioners are not even aware that there is something beyond experience with tools that they need to know.

So pls.

<comment><USER10> > Right, there is never time to do it right and lots of time to do it over.

Purists like to snear when they say this, but in point of fact its often true: its often more efficient to do things _good enough for now_ and fix the things that turn out to need to fixing later (because the real pace of change often means the things that are broken-in-theory, but good-in-enough-in-practice are going to need completely replaced because of requirements changes before they become problematic in practice.)

But its true, OTOH, that lack of knowledge of relational theory, the anomalies that it identifies that are tied to improper data models, and the practical impact of these can lead to poor analysis of the tradeoffs, and that the common cargo-cult rules of thumb (say, for degree of normalization to pursue) that are frequently used in practice are poor substitutes for deep understanding of the relational model and the problem of concern, and analysis of the real risks in the system under design of taking shortcuts.

> I do suggest that you read as carefully my comments as I write them

One of the biggest problems with your comments is that they don't appear to be written carefully -- particularly, if you hope to influence the practitioners that you treat with such condescension, you have failed to put due care into consideration of your approach to the audience, which is a central element of any communication.

> I did not say practitioners ought to be theoreticians, I said they should no > engage in a field founded on logic without ANY intro to logic.

You seem to also, however, keep suggesting that either a lack of deep familiarity with relational theory or a disagreement with your interpretation of how that theory ought to shape practice are equivalent to (or can only be a result of) a complete lack of grounding in logic. Whether you are actually _conflating_ these things or just engaging in particularly obnoxious condescension and personal abuse when you do this is less than clear, but neither is helpful or useful.

> In fact, the initial mapping to implementation--direct image SQL > implementations--was not in the relational spirit at all and is in large > part responsible to logical-physical confusion and confusion of tables with > relations.

I'm not sure what you mean by the "relational spirit". SQL's design was clearly shaped by the relational data model, though SQL itself (in its current form as well as its early forms) is certainly not _ideal_ from a relational perspective, even before considering the whole NULL controversy.

> And to call those initial implementations efficient in the technology of the > day is from another planet.

The initial implementations weren't what became popular though; the implementations that _were_ efficient were key to driving popularity.

> It just so happened that even the limited relational fidelity of SQL proved > superior to the rigidity, complexity and lack of soundness of hierarchic and > network technologies.

Sure. I just think that its easy for theory-purists to overstate the degree to which the "lack of soundness" was the problem driving adoption, rather than "rigidity and complexity". Insofar as the _linguistic_ and _expressive_ features of SQL and the relational model proved attractive, simplicity and flexibility were particularly important, and while the capacity for soundness _is_ an important improvement, its one that's been underused since day one. Its simply not the case, as you seem to present, that we've "fallen" to a state where that is ignored from some rosier days when that aspect of RDBMS capacity was strongly embraced and effectively and rigorously used by practitioners.

> Have you ever seen IMS or Codasyl code?

I've even had to write (well, modify) some IMS code, far more recently than I'd prefer to have.

> The problem is lack of fundamental education which has been replaced by tool > training.

I don't think that's true. I think that the _number_ of people with "fundamental education" in the field is probably greater than ever before. Sure, the number of people with tool training has increased faster, but that's not tool training replacing fundamental education, its just that with _any_ technology, the first generation of users will _all_ (or, at least, disproportionately) be versed in the underlying principles because they are also the builders of the technology, but over time that's going to fade as, even with more people educated in the principles, people who are just pragmatic users of the technology with a more limited focus are going to grow at a faster rate.

> Practitioners are not even aware that there is something beyond experience > with tools that they need to know.

Most practitioners I've encountered seem to be aware that relational theory _exists_. Sure, lots of them aren't well versed in it or what light it has to shine on their craft, but abstruse descriptions without clearly explained pragmatic benefits aren't an effective way to correct that and motivate them to dig more into theory, and neither is condescension and abuse.

<comment><AUTHOR> > Purists like to snear when they say this, but in point of fact its often > true: its often more efficient to do things good enough for now and fix the > things that turn out to need to fixing later (because the real pace of > change often means the things that are broken-in-theory, but good-in-enough- > in-practice are going to need completely replaced because of requirements > changes before they become problematic in practice.)

Do you have stats that prove your point, or is it based on the fact that this is how it is usually done because that is what is possible given the poor level of education and knowledge in the industry? If you're not part of that, don't underestimate its size. I spent 40+ years demonstrating the ignorance and its consequences.

By the way what does "impure theory" mean?

> But its true, OTOH, that lack of knowledge of relational theory, the > anomalies that it identifies that are tied to improper data models, and the > practical impact of these can lead to poor analysis of the tradeoffs, and > that the common cargo-cult rules of thumb (say, for degree of normalization > to pursue) that are frequently used in practice are poor substitutes for > deep understanding of the relational model and the problem of concern, and > analysis of the real risks in the system under design of taking shortcuts.

> Glad we agree on something. My claim is that there's more to that than you > seem to think.

> One of the biggest problems with your comments is that they don't appear to > be written carefully a-- particularly, if you hope to influence the > practitioners that you treat with such condescension, you have failed to put > due care into consideration of your approach to the audience, which is a > central element of any communication.

The title of one of my books is "for the THINKING practitioner". He is the one I try to influence and aren't that many. It's not entirely their fault--it's how the industry and business in general operates. I can detect very easily the difference between a thinker who is uninformed and a non-thinker and I treat them differently. It's just that there's many more of the latter than the former.

> You seem to also, however, keep suggesting that either a lack of deep > familiarity with relational theory or a disagreement with your > interpretation of how that theory ought to shape practice are equivalent to > (or can only be a result of) a complete lack of grounding in logic. Whether > you are actually conflating these things or just engaging in particularly > obnoxious condescension and personal abuse when you do this is less than > clear, but neither is helpful or useful.

So according to you it's not possible to detect the difference between a poor argument due to ignorance of logic and one grounded in logic? Again, I have spent 4 decades doing this and pls permit to believe that I discern quite readily who should be treated with respect and who not. You're entitled to disagree.

> I'm not sure what you mean by the "relational spirit". SQL's design was > clearly shaped by the relational data model, though SQL itself (in its > current form as well as its early forms) is certainly not ideal from a > relational perspective, even before considering the whole NULL controversy.

The authors of SQL did not have a good grasp of the RDM, which is why SQL cannot be considered truly relational. It violates too many rel. principles. This according to Codd and Date who were both at IBM when SQL was developed. The specific spirit I was referring to is physical independence, a core objective of the RDM which a direct image implementation is not in the spirit of.

> The initial implementations weren't what became popular though; the > implementations that were efficient were key to driving popularity.

Yes, but many of the reasons for which SQL was slow to make efficient was its poor relational fidelity. I've written a few articles on that subject.

> Sure. I just think that its easy for theory-purists to overstate the degree > to which the "lack of soundness" was the problem driving adoption, rather > than "rigidity and complexity". Insofar as the linguistic and expressive > features of SQL and the relational model proved attractive, simplicity and > flexibility were particularly important, and while the capacity for > soundness is an important improvement, its one that's been underused since > day one. Its simply not the case, as you seem to present, that we've > "fallen" to a state where that is ignored from some rosier days when that > aspect of RDBMS capacity was strongly embraced and effectively and > rigorously used by practitioners.

I did not say soundness drove the adoption. In fact, there is no way this can happen given that a vast majority of practitioners have no clue of how a formal foundation for db mgmt is different and superior to non-formal ones. This is not different than the notion that data science is science. That's precisely why I keep stressing that substituting training for education killed the capacity to appreciate the difference.

> I've even had to write (well, modify) some IMS code, far more recently than > I'd prefer to have.

My sympathy.

> I don't think that's true. I think that the number of people with > "fundamental education" in the field is probably greater than ever before. > Sure, the number of people with tool training has increased faster, but > that's not tool training replacing fundamental education, its just that with > any technology, the first generation of users will all (or, at least, > disproportionately) be versed in the underlying principles because they are > also the builders of the technology, but over time that's going to fade as, > even with more people educated in the principles, people who are just > pragmatic users of the technology with a more limited focus are going to > grow at a faster rate.

Well, having spent so many years documenting the deterioration of education, I believe my evidence than your perceptions.

> Most practitioners I've encountered seem to be aware that relational theory > exists.

Very well put. That's about the gist of it. And whatever little they know about it is wrong.

> Sure, lots of them aren't well versed in it or what light it has to shine on > their craft, but abstruse descriptions without clearly explained pragmatic > benefits aren't an effective way to correct that and motivate them to dig > more into theory, and neither is condescension and abuse.

They should not be motivated to do it on their own. There should be a basic level of education required to be inducted into the profession by faculty that are themselves proficient in the material and not industry hires that are teaching coding and tools because that's what univs teach now to be "relevant".

<comment><USER13> I hear you on the education thing. Intuitively, the "two tables with same structure" approach is redundant, but it's helpful to have the math/logic to unambiguously define what "redundant" is.

Having 2 tuple types with 3 values seems harmless. This explodes quickly when done with many columns or many tables. Having some kind of foreign key to act as a "discriminator" scales much better to constrain the number of value types (# of tables x # of columns) that must exist.

But I only say this because of having to deal with a legacy DB at work that violates the Hell out of this :-)

... which brings us full circle to the tragedy of a lack of education.

</comment></comment></comment></comment></comment></comment></comment><comment><USER14> "If two relations have identical designs, the difference can only be encoded in the relation (and possibly attribute) names,"

True, though the (and possibly attribute names) part seems dubious to say the least. Different attribute names makes the designs non-identical, no ?

in violation of the IP [information principle].

This interpretation of the IP is outright absurd.

One, the "I" in "IP" was obviously intended to cover only ever the [end-]user's own business information. The "information" that is being "hidden" under this absurd interpretation is the mapping that applies from relation names to intended interpretation. That information is never part of the [end-]users "genuine business information". How could it ? The mapping in question arises only when the models are being developed. As long as no computers are involved, no information models and no mapping, but the "genuine business information" stays the same.

Two, the relation names _are_ present as a value of an attribute in a tuple in a relation. In the catalog that documents the structure of the database that will contain the [end-]users "genuine business information".

As such, it is inaccessible to the DBMS, which cannot rely on the RPs [relation predicates] to accept/reject tuples as correct/incorrect.

Not sure what is intended here. Is it claimed that because "table names are inaccessible to the DBMS", it is impossible for the DBMS to enforce declared constraints ? Ouch. The SQL REFERENCES clause ought to suffice to counter that.

Users--with little, or no help from the DBMS--must ensure that the correct tuples are inserted in the proper relation.

So they must know the mapping from relation names to intended interpretation. That is not an insurmountable problem. Hundreds of thousands of developers have already been doing that [or something extremely similar] since before databases even existed.

Moreover, because information is not represented explicitly (as data values), relational operations lose it: if you UNION the two relations, you get users that either viewed or downloaded items, or both.

Then don't union the two together. (It is alas left unclear whether the usage of the term UNION here refers to invocations of that relational operator on the two relations in the original design (in which case the "loss of information" is (a) intentional and (b) not really loss of information because the original relations have not magically disappeared by computing the union), or in the sense of blindly mergeing the two tables together at the schema level without adding an indication of view vs. download. In which case it's just a stupid design mistake even my cat probably wouldn't commit.)

<comment><AUTHOR> The RDM is an attempt to maximize database management by the DBMS. What is left only in the mind of users--particularly the kind of users we witness here --is nothing but trouble. Reliance on sheer user discipline--for no good reason, for the only reason that SQL is not truly relational and people don't know, understand and appreciate the theoretical foundations of the RDM is what is really absurd here.

That's the reason the IP insists that ALL information is represented EXPLICITLY and in EXACTLY one way, to maximize soundness, power and simplicity. Whoever does not understand this will get nothing better than SQL.</comment></comment>


Apple should shrink its finance arm before it goes bananas - <AUTHOR> https://www.economist.com/news/business/21730631-worlds-biggest-firm-has-financial-arm-half-size-goldman-sachs-apple-should-shrink <comment><USER2> People continue to say Apple makes huge profits. But in reality they have a world wide market share of 20%, with Gross Margin at ~40%, Net Margin ~20%.

Both Gross and Net Margin figures have been very consistent over the years, And increasingly many profits are from the financial arms, and other services.

One way to look at it, is that despite iPhone has been getting more expensive over the past years, Apple are actually earning less % on each unit.

In some ways, this is worrying.

<comment><USER3> They still take the vast majority of profits from three enormous markets, smartphones, tablets, and PCs. Long way from trouble.

<comment><USER2> Sorry I should have word it more carefully, Not that I think it is worry for Apple, but and overall general Economy.

The only reason why Apple is earning 85-95% of profits of smartphone industry per year is because all other manufacturer are posting loses. When everyone decide not to make a dime like Amazon, ( or they cant ), the ones who do will obviously emerge as winner from a Profits POV.

What I find worrying is

1\. Apple is obviously working in diseconomy of scale, i,e they aren't paying less per component unit with their massive scale, they are likely paying more then everyone else.

2\. It is very clear there is an inflation in component ( Or everything else in Asset ) happening. From NAND, Wireless, Memory, Fabs, Screen, everything is getting more expensive, which we wont quite used to because we expect the same Screen, memory or CPU with similar characteristic or performance will cost less 3 - 4 years down the road, but this has not happen.

Both point relates to interest rate, the way those company were sustainable because they are making some profits in a likely stable or monopoly environment, and getting very cheap cash flow.

That is what i am worry about. I wish someone could tell me what happens when too much liquidity is injected into the system..... but it seems history hasn't written a page on this yet.

<comment><USER3> Pretty much all your assumptions are wrong. There is no evidence 1 & 2 are true, in fact the opposite. Apple sells more single models of phones and tablets than virtually anyone. Almost every phone they sold the last year was a 7 or a 7+.

And the reason for their profit dominance isn't other companies losses, it's they are competing against commodity makers with compelling unique products and value. Beyond iOS and iTunes is their custom SOCs, which are years ahead of their competitors.

</comment></comment></comment></comment><comment><USER4> people like to through notional value of derivatives because those numbers look huge but are generally meaningless "the notional value of JPMorgan's derivative trading could be as high as $70 trillion" makes for a eye catching statement. Apple most likely is holding derivatives to hedge currency risks and their exposure has very little to do with notional value.

<comment><USER5> > In theory losses on derivatives would be offset by gains in the value of > Apple's underlying business.

That is, in fact, exactly the point of them. If the derivatives are hedging currency exposure, then a move in exchange rates should lead to a change in their value equal and opposite to the change in value of the overseas assets. The whole idea is that they neutralise the effect of changes in exchange rates.

</comment></comment><comment><USER6> What should the corporate tax rate, currently at 35%, drop to, for Apple to repatriate foreign cash?

It seems like a no brainier to drop the rate when the alternative is financial skullduggery by the Apple capitals of the world. All while the local mom & pops pay the 35% and bear the financial burden of the nation

<comment><USER7> > It seems like a no brainier to drop the rate when the alternative is > financial skullduggery by the Apple capitals of the world.

Absolutely not. Just because an entity will try to work around your well- intentioned and globally standard laws does not mean you should give up on those laws.

In addition, it's quite an elitist and anti-democratic statement -- Apple is by far the world's most profitable public company, to give them a 'tax break' is to add icing onto their cake.

Apple is doing what it thinks is best -- taking on debt with the assurance of foreign assets, no matter if those assets are readily available or not.

The current US president has tossed around the idea of taxation amnesty for corporate asset repatriation -- if I was Apple, I'd lobby for that while continuing on their current practice.

Lastly, there's no real fear Apple won't be able to borrow at a handsome rate. Interest rates are still incredibly low (historically) and the Fed doesn't raise rates by full percentage points, they do it slowly and incrementally.

<comment><USER6> Disagree. Apple is one among thousands of companies avoiding tax, while Mom- and-pop companies pay exorbitant rates. No one's working around laws - they're following it diligently while maintaining their fiduciary duty to investors.

US corporate tax rates are among the highest in the world - even Canada - having a reputation for being socialist - has a lower rate. You're forcing companies to pay 0% tax in the US if they can get away paying only 5% in Ireland. And Ireland is benefiting disproportionately from American innovation - because of onerous tax rates

<comment><USER8> The US is the only country to apply a _global_ income tax to its non-resident nationals. It should apply a global tax to its businesses as well.

<comment><USER3> That would drive US corporations to reincorporate overseas and cost the US most of its corporate tax revenue.

<comment><USER9> You install tariffs on the products and services of those corporations attempting to subvert the taxing authority, utilizing payment networks if necessary to capture the tariff.

<comment><USER10> So person A sells an iPhone to person B in China. The phone was manufactured there and never left the country. Can you tell me the reasoning that our federal government owns 35% of that sale?

There's a reason other (more "progressive") countries don't tax overseas business. It's stupid and anticompetitive.

<comment><USER9> The same reason I owe the government taxes when a non US company pays me directly when I work outside of the US. That's what I owe the US government for their services I receive as a US citizen regardless of where my business activity took place.

You think it's stupid and anti competitive. I think that it's a race to the bottom otherwise and fully support the current model (although 35% is a tad too high; 20-25% is more reasonable IMHO).

<comment><USER11> There is only one other country that agrees with you: Eritria. Almost all other countries tax only residents.

<comment><USER9> To each country their own model. This works for us. Probably helps to be one of the largest economies in the world. You get to make your own rules at that scale.

<comment><USER3> Except it doesn't work for us. It makes the US a lousy place to invest and drives capital overseas. The US economy is huge because of most of its first 200 years, it's been stagnant every since.

<comment><USER9> And yet, investment in the US continues unabated.

<comment><USER3> "investment" will continue as long as we run large deficits, if you count funding our rapidly growing federal debt as "investment". It's the "benefit" of our trade deficits.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> Apple has big volume of cash. What exactly is the downside in investing it in riskier assets?

<comment><USER13> I think the assumption of the author is that the folks who run Apple will be seduced by profits generated by the financial arm and start emphasizing that more. Which if anyone knows anyone who works at Apple is ridiculous in the extreme. Alternatively, they take on more balance sheet risk and then the whole company is jeopardized by a financial volatility event, e.g., a big jump in interest rates.

Tim Cook and others do seem to be playing a game of chicken with congress. How long can the US division continue to borrow to find the dividend while those enormous piles of cash remain offshore?

<comment><USER14> How long? At least as long as the pile of cash overseas grows faster than the debt.

</comment></comment></comment><comment><USER15> To what extent can Apple leverage internal research to make these investments? They have sales data and supplier numbers for their products... this could be useful not-public information, no?

<comment><USER16> Is it really fair to use such non-public information? How far can you go till it becomes insider trading. I'm aware that some investors have been looking at parking lot volume via satellite imaging in order to predict the performance of retail companies. Is that fair? Would it be insider trading if Amazon trades stocks of companies which sell through Amazon? Amazon would then ACTUALLY KNOW their sales numbers. Perhaps this whole insider trading thing is a broken idea to begin with. What if I figured out the sales numbers for a cell phone manufacturer by analyzing MAC addresses using a network of wifi sniffers, sales numbers for car manufactures by analyzing traffic camera data using ML. Used security camera footage from stores that I own to figure out what kind of hand bags people caried and then shorted Prada when I knew fewer people were bringing Prada bags into the store...

<comment><USER17> Funny enough that Google is exactly in that position. Google owns a company called double click who does adword bidding optimization in between other things. As I work in the business, one thing that is clear is that the only way to do that efficiently is to keep track of conversion data. This puts Google in a highly unfair position since they manage the auction and own the data which drives the auction. That said, this is not even close to what Google knows about businesses in general...

<comment><USER16> But the really interesting thing is not what Google can do with the ad exchange. But what they can do with that ad buy data in order to reconstruct the sales numbers for a company and unfairly play the derivitives markets around it's earnings call. When I say unfairly, I mean that ONLY google is privy to such knowlege. If there is any reason to think that insider trading is wrong, then Google would be similarly in the wrong here.</comment></comment></comment></comment>


SandDance – Dynamic Visual Data Exploration - <AUTHOR> https://www.sanddance.ms/ <comment><USER2> Great, I can't wait to spend the next month trying to bring BI people back down to reality.

There's a persistent myth that now since we have "big data" that data analysis needs a magic tool to magically tell us what data means. This will only fan the flames.

</comment><comment><USER3> Is this closed source?</comment>


Witness: Police shot Portland slaying suspect without warning or arrest attempt - <AUTHOR> https://www.washingtonpost.com/nation/2020/09/10/reinoehl-portland-antifa-killing-police/ <comment><USER2> I'm going to hold off until we get more actual verified details from this to form any opinion other than that I feel bad for anyone named Dinguss.</comment>


How to use the Gig Economy to your advantage - <AUTHOR> https://42hire.com/how-to-use-the-gig-economy-to-your-advantage-9c04574587a2?source=linkShare-d19eba0196e0-1476543493 <comment><USER2> > Drive for Lyft but haven't received that high of a wage in recent weeks? > Drive for Uber. Uber not paying that high either? Drive for Fasten. Fasten's > wages low? Deliver for Postmates. And on and on.

What happens when the market brings down the average wage and it's impossible to find a decently-paid gig? When we're all freelancers, it's going to be really tough to have any kind of collective bargaining power. Yes, flexibility is great, but the precarity that comes along with the gig economy far outweighs that.

<comment><AUTHOR> We'll have to see what happens. I would guess if there are opportunities for companies to make money than more and more will enter the market increasing competition over workers which may keep wages up or even increase them.

There are definitely some downsides to the gig economy but the purpose of the article was to shine some light on the existing opportunities for workers to help leverage themselves if they are in it.

</comment></comment><comment><USER3> I'm amused to see that this focuses on the arbitrage opportunities afforded to the individual contractors, rather than the (far more significant) advantage that the gig economy offers to employers who can use it to acquire a rapidly scalable workforce while avoiding those pesky laws about workplace conditions and employee rights.

<comment><AUTHOR> There are definitely advantages for the employers but I wanted to bring attention to the fact it can also be a win-win situation for both parties involved.</comment></comment>


Security Disclosure Policy Best Practices - <AUTHOR> https://titanous.com/posts/security-disclosure-policy-best-practices

<comment><USER2> The CEO of apple/oracle/mozilla/acme should not get a copy of every message sent to security@.</comment>


The Cost of a Hot Economy in California: A Severe Housing Crisis - <AUTHOR> https://www.nytimes.com/2017/07/17/us/california-housing-crisis.html <comment><USER2> I always see news articles about companies building campuses and headquarters out there. Can anyone tell me why Google/Facebook/apple can't just build a residential tower for 10,000 people next to their building? It seems they get approval for everything else they build out there.

The only time I went out there it was very suburban, with plenty of room for more buildings. Seems odd that the businesses with all the money paying all the taxes can't get a city council to approve giant apartment complexes right next to their headquarters.

Anyone have the answer? Must be the only place in America where corporations don't get their way, unless they don't actually want housing for their workers.

<comment><USER3> Prop 13 is a major component. Property tax grows at below inflation, whereas commercial tax grows at the pace of the local economy.

So, if you zone housing, eventually the cost of servicing that housing exceeds the tax revenues from it, and there is little you can do to avoid that process. If you zone businesses, you get sales tax and similar revenue from the workers. Financially, it's best to let your neighboring municipalities build homes for your workers.

There is also a backlash from locals who see higher density housing as the core cause attributed to most local issues: traffic, crime, overtaxed city services, etc. This is mostly an incorrect perception, but it's a persistent one.

These combine to prevent zoning for housing.

<comment><USER2> The property tax seems like a likelier answer. NIMBY-ism always strikes me as a weak answer. At least in the midwest, if one city won't build it the city next door will. Seems odd (and in my view, unlikely) that every single city bands together and can resits the market forces of the most powerful companies on the face of the planet.

I'd have thought a municipality would have cracked by now and let everyone build huge residential buildings and used that population growth to fund everything else. Seems every weird no municipality isn't building houses like crazy.

</comment></comment></comment><comment><USER4> 1\. Commute

2\. Cost of living

3\. Income potential

Pick two. Like everyone else, everyone wants everything. There's no place on Earth that has a short commute, low cost of living and amazing income potential. Compromise is the name of the game. Public transportation can help, but even still you'll have a commute. The only solace there is that you can do other things while you commute.

It'll be very interesting if commutes become on average 1.5+ hours each way and employers count time spent working while commuting.

<comment><USER5> I don't see any theoretical reason to only choose two, and reject it. We can choose three. Building more housing to lower the cost of living probably has very little to do with the bay area's tech income potential.

<comment><USER4> New York, the only example I can think of, has plentiful housing and it's still more expensive (yes, all boroughs) than most of the country. Will more housing help? Sure. The question _really_ is whether or not increasing housing supply will or will not increase demand more than the newly available supply.

Why not just move to another area? If there's something uniquely good about the Bay Area perhaps it's that same thing that makes the incumbents not want to let more people in. It's kind of a geopolitical problem.

<comment><USER6> Look at Seattle, it has built a lot more than SF in response to the demand and as a result, the price of housing has not gone up nearly as much.

So yes, increasing supply does lower price.

<comment><USER7> Haven't prices in Seattle been going up at a faster rate then SF during the past 5 years?

<comment><USER6> [http://www.seattletimes.com/business/will-seattle-really- bec...](http://www.seattletimes.com/business/will-seattle-really-become-the- next-san-francisco/)

[https://goo.gl/images/xYPfR8](https://goo.gl/images/xYPfR8)

</comment></comment></comment></comment></comment></comment><comment><USER8> What's the alternative to San Diego or San Francisco if you are in Pharma? Boston? You really need to live in a hub city, nowadays everyone changes employers every two years.

Wouldn't it be nice of we could go back to the old times, and there was a choice between Philly, the New Jersey pharma cluster, Kodak in Rochester, Solvay in Syracuse, Dow in Midland, MI, Upjohn in Kalamazoo and many more?

<comment><USER9> >nowadays everyone changes employers every two years

They don't but, yes, there's a lot of clustering for various industries and industry sub-segments. I expect Kendall Square in Cambridge is bigger than most other pharma centers these days though.

</comment></comment><comment><USER10> I am amazed at the scale of the RV housing in industrial areas of Oakland. I walked several miles yesterday in southeast Oakland and saw up to five RVs on a block, obviously housing. As the article says, this is not a question of a few very poor people having trouble finding housing in few very expensive coastal areas. The Bay Area town I live in has a median household income of $100K. The payment on a 3-bedroom condo or 2-bedroom home is now just about $100K as well. And my area is a solid 1.5 hours from San Francisco.

<comment><USER11> This is one of those things the municipalities need to stop doing, promptly. If this becomes an established part of the housing market, it will acquire a constituency and it will be much harder to stop later. Just look at what a political mess decades of winking at illegal immigration from Mexico caused.

</comment></comment><comment><USER12> This is going to be a massive thread I predict. There's a big world out there. If you can't afford California, or you don't want a 2 hour commute with a $180,000 salary, then DO NOT LIVE THERE. It's that simple. Are you skilled, and you can't afford or don't want to afford to live in California? Give up the damn weather and move somewhere else. I don't even care if it's because the laws cause unaffordable housing, just don't live there. Besides, if you drop the housing prices so much that anybody can afford to live in California, how great do you think it's going to be there? The issue with a place like California is that there are almost always going to be more people that want to live there, than there is space. If you build more/better/affordable housing, more people will move there, then what?

<comment><USER9> California has been popular for various reasons for a long time. But it's really since the recovery from dot-bomb that things have gotten out of control.

Zoning etc. is part of the issue. But by all accounts, the (not so great) public transit--especially in SF itself--is pretty near capacity as, obviously are the highways. Can this capacity be increased over the timeframe of a couple decades? Probably. But there's nothing in the 5-10 year horizon that's going to make a [EDIT: not so big] difference other than a combination of people choosing to not live there, incremental housing construction, incremental public transit improvements, and incremental increases in congestion.

<comment><USER13> I'm hopeful that autonomous buses make this extra capacity possible. The main problems are unpredictable/long waits and overcrowding; the proposals I've seen involve smaller (<20 passenger) buses that run much more frequently and can fit more people into standing-room space because automated drivers have better planned acceleration and longer braking. It's so much cheaper and easier to add hundreds of small buses to the peninsula's streets than to build rail or subways.

Another crazy low-footprint idea: connecting our city of hills with an aerial gondola network. [https://www.fastcodesign.com/1671214/a-mass-transit- proposal...](https://www.fastcodesign.com/1671214/a-mass-transit-proposal-to- connect-a-city-using-aerial-gondolas)

<comment><USER9> Honestly? Sounds like magical thinking about how self-driving will make things possible that aren't with a relatively low-paid human driver today. And probably wouldn't be available for a few decades anyway. Small buses are available in a lot of countries today.

<comment><USER13> Unionized MUNI drivers make three times the SF minimum wage [0] before overtime so they're not "low-paid" by any means, but yes if money were unlimited we could achieve a smaller-but-more-numerous bus strategy... the ride will just be really jerky and require more seats at lower capacity, in addition to costing a prohibitive amount of money.

And "a few decades" seems extremely conservative when we're already testing driverless buses in San Francisco today [1]

[0] [https://missionlocal.org/2014/06/is-32-an-hour-too-much- for-...](https://missionlocal.org/2014/06/is-32-an-hour-too-much-for-muni- drivers/)

[1] [http://www.reuters.com/article/us-california-autonomous- bus-...](http://www.reuters.com/article/us-california-autonomous-bus- idUSKBN16E08I)

</comment></comment></comment></comment></comment><comment><USER14> While I generally support the legislation discussed in this article, I wish the YIMBY people would stop talking about this in such simplistic terms. In some parts of California (e.g. San Francisco's eastern, urbanized third), no amount of building will _ever_ satisfy the need, because all you can build on expensive land is "luxury" housing, and when those units don't sell/rent, developers will just stop building, because the unit economics don't work out.

Some places definitely need to build more housing -- see, for example, the _western_ 2/3rds of San Francisco, or the cutesy little suburban enclaves of Silicon Valley -- but this won't help until transit allows people to commute from those neighborhoods to their jobs. So we should also be saying "YIMBY" to transit infrastructure and reducing/eliminating the broken tax incentives that are helping to create the problem in the first place (like the ill-conceived Mid-Market tax incentives, which have replaced SROs with tech offices, thus making the homelessness problem _worse_.)

<comment><USER5> I am a YIMBY and we consistently say all those things. I also reject the notion that the eastern district is "built out" or that developers will stop when things don't "pencil out".

What pencils or doesn't is a direct consequence of the regulation and tax burden we place on construction, and there's probably an order of magnitude of levers we've yet to pull.

<comment><USER14> _I also reject the notion that the eastern district is "built out" or that developers will stop when things don't "pencil out"._

Reject what you like, but you're wrong. Investors don't finance real-estate projects where the per-square-foot returns don't justify the expense.

The expense, in the case of eastern SF, is >80% land and materials. It's been documented, extensively, by groups like SPUR. Taxes and regulation are less than 20% of the cost of construction in San Francisco. It's simply _not_ the limiting factor [1]

[1] [http://www.spur.org/publications/urbanist- article/2014-02-11...](http://www.spur.org/publications/urbanist- article/2014-02-11/real-costs-building-housing)

(You'll note that this link shows $48,000 of a $469,000 unit -- or about 10% of the cost -- is attributable to permits and regulatory fees. The cost of land and labor and materials amounts for >75% of the cost.

When you read people complaining about "regulatory" costs in SF, it's really a political dog-whistle to the BMR housing subsidy, which developers _hate_. Most self-professed "YIMBYs" I talk to do not know this.)

<comment><USER5> Sure, that's fine. The current level of what investors have demonstrated they're willing to build is by far below the norm for even SOMA. There is still a very significant amount of surface parking or 1-3 story developments in SOMA that could potentially become a very large amount of housing.

<comment><USER14> _" The current level of what investors have demonstrated they're willing to build is by far below the norm for even SOMA. There is still a very significant amount of surface parking or 1-3 story developments in SOMA that could potentially become a very large amount of housing."_

That's like saying "there's a very significant amount of land in America that could become a very large amount of housing.".

Sure it could. Except, someone owns the land and wants you to _pay for it_ (trouble, that), or it's being used for something else that has a higher value, or it _just isn 't practical_ to build there for whatever reason. Or maybe the owner doesn't _want_ to rebuild. In any case, be more specific: I _live_ in SOMA, and I don't know of many vacant lots around here that aren't already slated for development. These sorts of stories are usually apocryphal.

This is another part of the "YIMBY" narrative that I dislike. You don't get to point at every building you think could be taller and say "I think that building could be taller, therefore, SF isn't building enough." That's not how it works.

</comment></comment></comment></comment></comment><comment><USER15> Here's a question: why don't more companies locate in the East Bay? There's a bigtime 'follow the herd' mentality when it comes to where VCs are allowing companies to make their headquarters. You don't need to have an office in Mountain View or Palo Alto. Many of your employees will live in places like Fremont, Pleasanton and Walnut Creek. Why not start a company there instead?

</comment><comment><USER16> I would love to see a state pass an amendment removing all density restrictions on residential zoning and making all commercial zoning mixed use.</comment>


Transfer Learning Will Radically Change Machine Learning for Engineers - <AUTHOR> https://medium.com/modeldepot/transfer-learning-will-radically-change-machine-learning-for-engineers-78732b2bb415 <comment><USER2> For those of you who want to play with transfer learning, Keras has a nice collection of models for computer vision available here: [https://keras.io/applications](https://keras.io/applications)

Shameless plug: And if you want to try them out in your browser, I made a website that allows you to try pre-trained models: [http://pretrained.ml](http://pretrained.ml)

<comment><USER3> change angular src? {{model.name}} {{model.description}}

</comment></comment><comment><USER4> That's a PR piece that has very little depth to it.

<comment><USER3> it is a PR piece that rightfully talks about the "most underrated trick in deep learning": transfer learning.

<comment><USER5> Oh come on - surely ever since Microsoft published that paper about roman alphabet recognition being improved by using chinese trained networks everyone has been using it as a default</comment></comment></comment>


NetWare 3.12 server taken down after a decade and a half of duty - <AUTHOR> http://arstechnica.com/information-technology/2013/03/epic-uptime-achievement-can-you-beat-16-years/ <comment><USER2> Previously submitted twice with no comments each time (but I think it is a cool article as well):

[https://news.ycombinator.com/item?id=7385399](https://news.ycombinator.com/item?id=7385399) [https://news.ycombinator.com/item?id=5462445](https://news.ycombinator.com/item?id=5462445)</comment>