Open-source companies gather to gripe: Cloud giants sell our code as a service - <AUTHOR> https://www.theregister.co.uk/2019/09/20/open_source_companies_cloud/ <comment><USER3> I remember in the 90s IBM started packaging and selling Apache web server. Some of my colleagues thought this was bad. But I went to ApacheCon and they were totally cool with it. There was a good talk, I think by Jim Jajielski, explaining that the purpose of the Apache license is to allow this.

I don't get these articles because it seems like "duh, that's the point." Either the press is stupid, or the companies are acting in bad faith, or the founders didn't understand OSS, or something because cloud giants are doing nothing wrong. It's permissible under the license of the named software.

It's like complaining that Apple and Google sell phones with Unix stuff in there without licensing it. There's no requirement to pay and no expectation.

Separately weird is that these cloud giants make tons of contributions to open source.

<comment><USER4> > Separately weird is that these cloud giants make tons of contributions to > open source

In some cases a cloud provider makes millions of dollars from a particular open source project and gives nothing back to it. This is the behavior that concerns many. It is also easy to see this as against the spirit of the GPL license, even if projects should switch to AGPL if that is really the case.

<comment><USER3> Maybe not to a particular project but I don't know any "cloud giants" without big OSS contribution footprints- Amazon, Microsoft, Google all have lots of open source projects and contribs. I don't care to measure OSS karma but these aren't people always taking and never giving.

I don't think this is against the spirit of the GPL license and that's why there is AGPL. The companies complaining here aren't GPL anyway. Although I think the opinion that this is against GPL is wrong, I kind of understand it. But it's loco that it's against MIT/BSD/ASL.

<comment><USER5> The whole problem is that Amazon contributes very little, if at all, and frequently creates closed source forks of projects

<comment><USER3> Amazon contributes a ton [0]. Contribs in most major projects and 1600+ GitHub projects.

Technically their fork of ES is more open than the software they forked.

I'm not sure what is appropriate to consider "very little" but Amazon is a direct contributor and member of the Linux Foundation.

What's an acceptable level of participation?

[0] [https://aws.amazon.com/opensource/](https://aws.amazon.com/opensource/)

</comment></comment></comment></comment></comment><comment><USER4> This is a horrible article. I don't think the reporter was at the conference. I was on Thursday. There was one talk with this complaint (speaker mentioned in the article) and everything else was just discussing open source business in pragmatic ways without lamenting cloud provider behavior.

<comment><USER6> I was also there on Thursday and agree. The talks were actually observing that cloud giants do alter the business landscape for open-source, but there are always innovations and possibilities. There were one or two talks on alternative licensing schemes, but the most interesting talks to me were alternative virtuous cycles that are available for supporting open-source while allowing any company to benefit.

</comment></comment><comment><USER7> Look, this all comes down to a simple thing: If you do Open Source you get Open Source - with all the up- and downsides this may mean for you. Upsides include that you may get community contributions (though that's not a given) and that many people will like your software more because they prefer Open Source. Downsides include that you can't force any user of your software to pay you.

What you can't have is all the upsides without the downsides. Sorry.

</comment><comment><USER8> In the 80s, Bill Gates and others figured out that hardware was a commodity and most of the value is actually in the software.

It's possible we're seeing a reversal of that trend; like we're seeing the trend of a distributed/decentralized Internet increasingly look like a single big computer run by Cloudflare, Google and a few others; Minitel 2.0.

<comment><USER9> Right now you have a bunch of companies in the top 10 that see software as a commodity they either give away for free or keep entirely to themselves and do not sell directly: Apple, Alphabet, Facebook and to a degree Amazon (AWS) and even Microsoft.

Apple makes money on Hardware, followed by services (incl the App Store). The software that people actually run they give away (mostly), like macOS, iOS, productivity apps bundled with their OSes, XCode, etc.

Alphabet makes a living mostly off of services (mostly ad brokerage, some cloud stuff too) and gives away a lot of software (such as Android, the Play* apps and other google apps), but rarely sells "old school"/non-subscription software licenses.

Facebook? They do not sell software either, they sell services (ad brokerage). They give away a lot of software, be it their apps you can install (Facebook, Messenger, WhatApp, Instagram) or their open source low-level stuff.

Amazon AWS sells services. The actual software they run is an implementation detail. Even core Amazon moved a lot from selling actual goods as a retailer to being a "marketplace" service platform for other retailers. And with Prime Video and Music, they carved out their place in the subscriptions space too. They also sell some hardware too (Echo, Fire, Kindle), tho mostly not to make a ton of money directly off of it but bolster their other products.

Microsoft wants in on the services thing too, gave away a ton of Win10 licenses to make sure the network effect isn't compromised, and started entering the services/subscriptions market more aggressively with things like Xbox Live (the actual console bordering on being a loss leader) and Office 365, and to a degree the Microsoft Store, tho not particularly successful so far.

Even Oracle desperately wants in on the services/cloud thing for a long time.

Netflix is subscriptions. The software again is an implementation detail.

Samsung sells hardware and to a degree services and subscriptions, but gives away their software.

Adobe is all about subscriptions too.

Red Hat? Subscriptions and services.

Their new parent IBM? Subscriptions, services and hardware.

There is hardly any major player left that actually sells you software like Microsoft did. Those who sell software usually sell you subscriptions not the software itself.

</comment></comment><comment><USER10> That is totally absurd argument, those cloud or industry giants are not taking any extra privileges, they have explicit permission to use those software. In fact, they are the major adapters who boosts development and contribution to eco-system. They actively motivate their employees to improve the software and publicly contribute who are mostly the major contributors.

Big companies are not bound by these software, they can re-invent their own wheel if needed. They are the one who opensources major new technology without thinking about the profit for example. Anguler.js, React, Hiphop, Mysql etc.

Edit: If you really feel uncomfortable, you can release your software with GNU AGPL, That would be tit-for-tat in your way.

</comment><comment><USER11> I don't understand where this sentiment is coming from lately? All of a sudden there are people who seem shocked that corporations are profiting off of Open Source software, even though nobody in the community has ever had a problem with it.

<comment><USER12> I would argue that in many cases, they aren't even making money off of open source _directly_. Like, people aren't paying Amazon for MongoDB, they're paying someone for the infrastructure necessary to run MongoDB.

<comment><USER13> They need many millions of lines of open source just to serve a html page... from kernel to Firefox and every bit of hardware between. Take away the open source and they need billions in proprietary software licenses. Take away the open source and Facebook didn't get written with PHP and MySQL. Take away the open source and Amazon has to license Oracle and MSSQL and Windows instead of profiting off MongoDB and PostgreSQL.

Amazon are still actively migrating _away_ from Oracle because it "doesn't scale", what they mean is proprietary software bills per CPU core or server so it scales proportionately to Amazon's own profits. This is just one way FAANG leverages open source to _save_ hundreds of millions of dollars a year in their own operations. Google forked MySQL for their advertising empire instead of licensing MSSQL or Oracle.

I do not believe you could even count the full extent they are dependent on open source without ripping it all out and watching them crumble. What's it worth to FB when every job candidate is highly proficient in React? Without open source culture driving self-education, without open source community in place to embrace React, React needs a 2 - 3 month training course for all newly-hired engineers!

<comment><USER11> This is true. Amazon needs tons of code just to get to the point of being able to even run Apache or nginx at all, and all that code written by contributors for free. So is your argument that all of those developers, or, at least the Open Source projects should be compensated by Amazon for that effort?

If so, then that suggests that Apache and nginx owe something to Linux, GNU, etc... as well. Sure, either can be run on alternative platforms but, let's be honest; they're usually run on Linux. One might even argue that neither Apache or nginx would exist had Linus not made Linux Open Source.

You bring up React. What's it worth to Facebook for all those engineers proficient in React? Well, React was created by developers who work/worked for Facebook so it seems to me the question is really, what's it worth all of us who might use React? Facebook sponsored it and has easily spent hundreds of thousands of dollars on engineer salaries, equipment for them, office space, etc... By your logic, all the developers not working for Facebook who use React should compensate Facebook for their generosity. See how absurd this starts to become?

This is all a false dichotomy. To suggest these companies owe the open source community is wrong. No open source developer I've ever met (and I've been in this community for over 20 years) has EVER said they expected some kind of compensation. That's not what Open Source is about.

<comment><USER13> I'm not saying they owe open source. I'm saying they leverage an absolute ton of open source, they are deeply dependent on it. All their communication, infrastructure, software and services and processes are dependent upon open source.

If supporting open source with money is paying it forward, I think React & other endeavors could be called paying it sideways. FAANG get legal tender fit for any purpose, hundreds of billions of dollars in revenue, billions saved not licensing proprietary software, and what proportionately little code they give back is not legal tender for any purpose.

But again I'm not saying they owe open source. I'm saying they could fund it many times over and couldn't exist without it. One day they will pay it forward just by paying fair taxes on their massive fortunes. Today they don't even do that much.

</comment></comment></comment></comment></comment><comment><USER14> To give an idea of the balanced nature of the article, I've pulled a few select quotes: "the peril of cloud-provider parasitism", "open-source code cobblers – Confluent, Elastic, MongoDB, Neo4J, and Redis Labs", "ward off predatory cloud titans".

<comment><USER15> "Biting the hand that feeds IT" indeed.

The style is typical of The Register, which doesn't claim any sort of neutrality.

</comment></comment><comment><USER13> I think you can look at FAANG almost as the commercial arm of open source, it's not necessarily bad that they are the best at monetizing open source and increasingly specialized in doing so. It's only bad when they are greedy and keep all of that money, dodge taxes globally and strip benefits from all the employees they can.

This is like the "billionaires can be philanthropists" problem... they overwhelmingly prefer not to so it is vastly less efficient to rely on their philanthropy individually vs a tax collected from all of them.

FAANG have the funds and the expertise monetizing open source to carry the entire open source community many times over, but they will hit a combined trillion dollars in savings before that happens. They will dodge hundreds of billions more in taxes before fueling the open source fire that furthers technology and feeds them recruits, ideas and software.

I've heard it argued that they do pay it forward through their own open source contributions. I think this is more accurately called paying it sideways if anything. They get legal tender fit for any purpose, and what proportionately little code they give back is not legal tender for any purpose.

</comment><comment><USER16> Its very good this topic is coming up. Fortune 100 companies pay a lot of money for these services, a classic example is Azure Search, which is using ElasticSearch. I wonder how the ElasticSearch community or the founder is benefited. I am curious to know. Also AWS sells a dozen of open source project as services.

<comment><USER17> There's a really simple solution to the problem of people you don't like using stuff you give them: don't give it to them.

Use AGPL, or add a viral exclusion for any company with a $10B market cap or whatever you want.

</comment></comment><comment><USER18> I remember, not so few server software devs like that of mongo were categorically against projects switching to AGPL few years ago because of fears of this preventing them selling their stuff in SaaS format.

Well, now their software is being turned into SaaS, except not by them, but by Amazon

</comment><comment><USER19> I don't see a problem with the current model..

If open source projects change their licensing to force paying for business use and the cloud giants doesn't want to pay, the cloud giant could afford forking the project and make their own version.

This could lead to the original project being unmaintained and the new fork becoming the standard, if that happens and given enough time, the cloud giant can make software patents on the changes they've committed and change the license of that code/features. Making it even harder to create something similar or even something new with the same features.

<comment><USER20> > I don't see a problem with the current model..

The question is why you are doing OpenSource. If you are a VC funded startup doing OpenSource for marketing (engaging a community ofnusers, producing an environment of tools around your product and sometimes providing a patch) there is an issue if revenue goes to AWS.

If you do OpenSource since you actually need the tool and/or it's not your core business there is little issue if others are using it to provide a service. They might even make your software more robust and stable and better for you. As long as your goals are not misaligned. If they are misaligned they form away and it's like they were never there for you ...

</comment></comment><comment><USER21> Sometimes I want to pay someone else to handle everything (Upgrades, Security patches, Bugs etc.) While no one stopping me from buying a VM and doing it myself. So what's the problem?

</comment><comment><USER22> Would it be possible to have licensea based on usage. I believe the image (photography) sites do this, and prices reflect it.

That said, perhaps the industry needs an ethical framework similar to tithing? While I understand these software publishers are upset, no one involved has done anything wrong. Big companies taking advantage of free software can't be a surprise.

</comment><comment><USER23> You give away your software without too many constraints, and you should expect this.

</comment><comment><USER24> I've taken a cut at a license that attempts to address monetization while preserving what I see as the underlying strengths of Open Source - <USER2> the risks of collaboration and adoption:

[https://github.com/db4j/PUPL](https://github.com/db4j/PUPL)

It requires paying for usage above a level, but is symmetric and permissive and makes future costs predictable, ie prevents the copyright owner from pulling the rug out from underneath you after you've adopted it. The software that I intended this license for never gained traction (so completely that no- one ever told me "i'd try it out if it was open source") so it's untested.

But if a popular software used this license, would you be willing to use it or contribute to it ?

<comment><USER25> The gist of that license seems to be:

\- you can use this software for free at home for personal use

\- if you use it at work, you can try it out for free but you have to pay to really use it

\- we welcome your code submissions, which will hopefully grow our license revenue

I feel like there are already licenses out there that hit these major points. If I ran across software that used this license, I would recommend they use one of the other popular licenses or just dual-license.

In general, I want licenses to be written and vetted by lawyers.

<comment><USER26> > In general, I want licenses to be written and vetted by lawyers.

I want them to be so standardized that I don't have to read them. If the license doesn't match something I already know, I'd be reluctant to include the code in any of my projects.

<comment><USER24> yes, i totally agree - as much as i like the terms of my license, if i came across an interesting software that used the license, i probably wouldn't try it out just because the cost of understanding any new license is too high.

i've more or less given up on the software that I intended the license for, but i mention license here in the hopes that one of these open-core companies will create, vet and popularize a license with similar assurances

</comment></comment></comment></comment><comment><USER27> I believe in Free Software, but I also believe if software is being used to turn a profit, there should be a cost to that.

<comment><USER28> I have seen many OSS projects that enforce exactly this. One example that comes to mind, HighCharts.js — free until you make any money.

[https://shop.highsoft.com/highcharts/](https://shop.highsoft.com/highcharts/)

<comment><USER29> It's not "free software" or "open source". It's proprietary software that's source available. If that's what you want to write or use, great.

</comment></comment></comment><comment><USER30> STOP IT!

Open Core is freaking not Open Source.

The article states many companies have moved AWAY from open source to "source available".

I will proudly die on this hill, to defend true Open Source, we give away our database tech used by 8M users (across non-profits, community ops, and for- profit startups) in production for free ([https://github.com/amark/gun](https://github.com/amark/gun)).

This complaining has got to stop.

Go freaking license your code as (A)GPL and join Richard Stallman's ideology if you don't like companies using your tech.

Or be brave enough to charge proprietary license, but if you do, stop your "we're Open Source" marketing pretense. Such hypocrites.

<comment><USER31> Everyone likes benefiting from Open Source until someone runs with their work and makes a profit.

My take from all of this is that people have no idea what they're doing when they license their work as Open Source.

The go-to process currently is to slap on the MIT license, call it a day, and then forget about it until they find out someone is massively profiting from it.</comment></comment>


An idea for getting rid of the hashbang problem - <AUTHOR> http://www.michaelcgorman.net/2011/02/10/no-more-hashbang-zones-replacesid-and-http/

<comment><USER2> Basically you want to be able to return a diff.

I half remember that this is possible over http for updating a resource (e.g., you looked at example.com/a-page an hour ago and now you want to look at it again; your browser can just download the bits that have changed). But I don't remember the details. Maybe it's part of WebDAV?

Then you only need to hint to the browser that a certain link might be diffable against the current page (or another page that the browser might have cached). That means only one new attribute, one new http header, and no new url structure required.

Adding new mandatory structure to urls icks me out for some reason. It doesn't feel very web-like.

Edit: I think that something like your idea would be helpful, but I don't think your idea would be the best way of doing it.

<comment><USER3> SDCH can save bandwidth, but I think the hashbangs are trying to avoid any page load at all. [http://en.wikipedia.org/wiki/Shared_Dictionary_Compression_O...](http://en.wikipedia.org/wiki/Shared_Dictionary_Compression_Over_HTTP)

<comment><AUTHOR> SDCH looks to be going after a similar problem, but it would cache the templates between browsing sessions rather than just among consecutive pageviews. I'm not sure how well it would work for AJAX-y sites which rely on maintaining Javascript state, though, but potentially you could still use, e.g., localStorage for that purpose.

</comment></comment></comment><comment><AUTHOR> I've been playing around with this idea for a while, and given this week's context, I figured I might as well throw it out there. My main questions for HN are: (a) is this a good idea, (b) do you see any big pitfalls I'm overlooking, and (c) how might we get this to happen?

<comment><USER3> Can you compare this against the HTML5 pushState API?

<comment><AUTHOR> All pushState does, AFAIK, is add things to your browsing history; it alone doesn't affect the content you receive. It does, however, make it possible (for recent browsers with JS enabled) to use the back button to navigate through AJAX-ed sites.</comment></comment></comment>


Government Debt (chart) - <AUTHOR> http://www.economist.com/blogs/dailychart/2011/06/government-debt

<comment><USER2> "By the end of 2011, deficit spending on OEF/OIF (Afghanistan/Iraq) will have raised the ratio of debt to GDP by about 10 percentage points, or between a quarter and a third of the total increase. By 2020, the increase will be 20 percentage points if war spending and the rest of the budget continue as forecast"

[http://costsofwar.org/article/macroeconomic-impact- military-...](http://costsofwar.org/article/macroeconomic-impact-military- spending)

</comment><comment><USER3> I don't get government debt.

How can _every_ country be indebted? Don't mutual debts cancel each other out? How can the whole world be in debt?

Whom do they owe that money? Whom do they pay the interest? Does a government debt mean that each citizen has a debt? Does it mean the government owes that money to its citizens?

If anyone has insight in this, please enlighten me!

<comment> mkr-hn Government debt works like any other investment.

[http://www.treasurydirect.gov/indiv/products/prod_tbills_gla...](http://www.treasurydirect.gov/indiv/products/prod_tbills_glance.htm)

Debt serves an important function in modern economies as a safe place to put money. That's why this debt ceiling showdown is a problem. Investors will take their money somewhere else instead of putting it in T-bills if they think the US might default.

Trillions in debt looks ominous, but it represents the trust (and wealth) of thousands of investors.

<comment><USER4> Safe place? Ask investors in Argentine, Russian, and Greek government debt about that.

The U.S. government is already working to cheat its debt investors by repaying them with devalued printed paper. One could argue that U.S. government debt is one of the least safe investments out there. You will get repaid, but it will be in devalued currency. That dollar you invested 5 years ago, for example, only buys about 1/3 as much oil as it did when you invested it. That is a catastrophic loss.

<comment> mkr-hn Traditionally safe. The recent (past 30 or so years) unwillingness of the congress to invest that money into maintaining growth (things like infrastructure maintenance/improvement--"pork projects" as people like to call them) has started to compromise that. Corporate lobbyists haven't helped much either.

<comment><USER4> Please don't just rely on the nonsense they taught you in econ 101. You need to understand the relationship between debt, the currency, interest rates, and a government's ability and willingness to repay. These factors make government debt potentially VERY risky.

Even in the U.S., which is generally very responsible, government debt has been a terrible investment at times through the years. The U.S. massively devalued during the Great Depression and devalued again after World War Two. An investment in a U.S. ten-year bond in the late-1960s would have been absolutely creamed, I'm talking about losing much of the real value of your investment. That's not anyone's idea of safe.

<comment> mkr-hn Compared to the stock market (Depression) or moving your money overseas (WW2)? It seems like the US was still the best of what was available at the time. Gold and commodities would have been toast in an invasion (the axis invading the US) or collapse of society (French Revolution-style uprising).

Giving the government all your money to keep it afloat in those situations was still superior, even if the government ended up ruining the value.

</comment></comment></comment></comment></comment></comment><comment><USER5> This chart is misleading with regards to the current US debt. It claims our gross government debt in 93.6% of GDP. There are two types of government debt -- debt held by the public (currently about 60% of GDP) and intergovernmental debt (the social security trust fund, medicare trust fund, highway trust fund, etc.). The latter debt can be wiped away by act of Congress -- they simply need to change the payout formula such that this debt remains as pieces of paper in a filing cabinet in West Virginia. We don't even pay interest on this debt -- we send more pieces of paper to the filing cabinet. Our real debt is not at crisis levels yet.

</comment><comment><USER6> Rich countries don't pay off debt, they inflate it away. One of the main reason for European countries to adopt the euro was to get a piece of the reserve currency pie, allowing more debt to be eliminated without severe economic repercussions. Unfortunately, they hit a road block in the Mediterranean, where, due to poor fiscal management, more inflation would lead to economic disasters threatening the stability of the euro. Meanwhile, the U.S. cannot afford much inflation at the moment, since it would threaten the position of the dollar as reserve currency in favour of the euro. Worse yet, they have their biggest creditor breathing in their neck with increasing economic and military power, who would not be pleased by inflationary measures.

Thus begins the biggest poker game of all, in which the U.S. suddenly cares about military spending and is trying to get Europe to spend itself into an inflationary cycle. Meanwhile, Europe is trying to prevent that through severe budget cuts. Ultimately though, neither can get rid of their debt without inflation so both taking inflationary measures is the only logical outcome. China and other countries heavily invested in euros (e.g., Russia) and dollars (e.g., Japan) will not be pleased, but ultimately their wealth is heavily dependent on the economic well-being of their biggest lenders and trading partners. There will be few winners in this game.

<comment><USER7> > The U.S. cannot afford much inflation at the moment, since it would threaten > the position of the dollar as the reserve currency in favour of the euro.

Not buying that, especially not at the moment - Greece is doing far more damage to the Euro as a reserve currency than anything the US could do to the dollar before August 2nd, and if Greece gets its house in order, there's still Portugal waiting in the wings. What's more, the US has an inflation rate right now that's practically nil, and I suspect China's interest is in the US spending again - their bond holdings exist for the sake of currency control, not as a legitimate investment.

<comment><USER6> Obviously the euro zone is bad, but I think you're overestimating how healthy the dollar zone is. Oil supplies are dwindling, the weapons industry is losing importance, the financial industry is not what it used to be, the car industry is no longer competitive, few major infrastructure projects are being undertaken, IT is one of the few industries that is going strong, but it contributes relatively little. Europe has a healthy, productive economy to back up the euro and it's a threat to the dollar.

<comment><USER7> The USD has a century of relative stability behind it - there have been ups and downs, but there's never been a full-fledged currency crisis, and a USD from 1911 can still be used today. The Euro is a stronger threat to the dollar than we've had in some time, but it's still a relatively new currency and it's showing strains already. There's a reasonable scenario which the Euro is a secondary reserve currency, but it's going to take some time and some drastic crises in the US which somehow don't spill to the rest of the world before the USD gets dethroned.

You're right, though - the picture's looked an awful lot rosier. I don't believe there's anything concrete that assures the US's decline, but that's not for lack of trying on the part of Congress.

</comment></comment></comment></comment><comment><USER8> Some interesting data points in there... Japan, government dept is 200% GDP? Never knew that...

Belgium 100.7%? France 94.1%? That's not much better than Ireland (102.4%) and Portugal (103.1%)!

Obviously some countries are much better at managing their debts than others (or at least their public image). Can we learn anything?

<comment><USER9> Japan has had high debt for a couple of decades, and is an interesting example. From what I understand the debt has really hampered growth. However, the debt is mostly held by the Japanese (I think), which is a much different situation because the money is just circulating from the general population to those in the population holding the debt, rather than being sent overseas to foreign creditors.

<comment><USER4> Japan is about to hit the wall though. They've lived off of a culture of savings, and those savers are all retiring, so Japan is going to see a reduced ability to finance its debt domestically. (This is also already beginning to impact the U.S. ability to borrow from Japan). Japan's house of cards is going to collapse in the next decade, and Japan's long slide of mismanagement into the second tier of the global economy will be complete. In just a generation they will have fallen from first place to trailing Korea.

</comment></comment></comment><comment><USER10> Per historical data (one link here: [http://www.deptofnumbers.com/blog/2010/08/tax-revenue-as- a-f...](http://www.deptofnumbers.com/blog/2010/08/tax-revenue-as-a-fraction- of-gdp/) ), the US income from taxes hovers between 15% - 20% GDP. This holds with wildly varying tax policies.

</comment><comment><USER11> It's about survival.

For politics:If everything is well we wouldn't need politics to make it right. Therefore politics must assure that everything is screwed for them to prosper.

For banksters: they must assure that that everyone is indebted and even entire countries in order for them to survive.

Don't expect solutions from those that thrive on the misery of others.

</comment><comment><USER12> First thought: the Swedes are doing well! Why is 60% of GDP the target? What does this do?

<comment><USER13> My understanding is that Sweden has drawn a significant portion of its revenues from North Sea oil, but that they are anticipating significant declines in production (and therefore revenue). They are preparing for this by grdually restructuring (read: cutting) government services and benefits to fit their anticipated revenue, with the added benefit that doing so in advance allows them to build up a decent nest-egg. In other words, they are doing something incredibly rare for a government: budgeting in a responsible and rational manner.

<comment> Tuna-Fish Sweden has practically no oil production at all, and never had any. The Scandinavian country with large oil reserves is Norway.

What you are talking about doesn't even describe Norway properly -- they have largely not spent the revenue they got from oil. Instead, it has been collected into a sovereign wealth fund. At present, the fund holds some $100 000 for every Norwegian. Future decline of oil profits will be covered by the investment income from the fund. There hasn't been any cutting of government services to fit revenues -- there hasn't been any need to.

Sweden, like the other Scandinavian countries, has held a responsible and rational budget for most of it's existence as a democratic country. So while it might be incredibly rare for governments worldwide, for some reason it seems that the Scandinavians can make the idea of a well-run welfare state work. I have yet to figure out why they can do it and most of the rest of the world seemingly can't.

<comment><USER14> > I have yet to figure out why they can do it and most of the rest of the > world seemingly can't.

It helps a lot to have a small country with an ethnically homogenous population, extremely low corruption, high tax morale, and high public trust of government offices and institutions.

<comment><USER15> _It helps a lot to have a small country_

Maybe, or a big but dictatorial country like China.

 _an ethnically homogenous population_

Care to elaborate? Would you consider countries such as Switzerland, Canada, Taiwan, and Singapore ethnically homogeneous?

 _extremely low corruption, high tax morale, and high public trust of government offices and institutions._

Chicken and egg problem - High tax compliance and trust in government come from having a reasonable tax rate from which you get a lot of effective government services. Low corruption is a pre-requisite for trust in government institutions.

I bet what also helps is not having a culture of cousin marriage, that seems to create a culture where is much more difficult to have any kind of trust with people outside of your clan.

I think the modern well functioning democracy is to a very large degree a result of complex game theory.

You don't have to look back much more the 2 centuries, to see thousands of years of European history that's full of corruption, lawlessness, crushing tax rates, all the typical modern day 3rd world problems and then some.

If just being Swedish led to modern Sweden, Swedish history would be a lot less interesting. For one, it would almost certainly be entirely Viking free.

</comment></comment></comment></comment></comment><comment><USER16> Government debt, while not intrinsically bad, is completely unsustainable. I imagine the US is planning the dollar will fall and it's debts will be essentially leveled. Maybe the US should adopt BTC?

</comment><comment><USER17> FWIW, my understanding is that the British government figure is significantly and artificially inflated by having had to bail out Northern Rock in the banking crisis - it ended up meaning that the balance sheets of any NR depositors technically counted against the national debt, but that's hardly what most actually mean when they're investigating the national debt.

<comment><USER18> My initial reaction was "surely the NR liabilities can't be _that_ much" - turns out they are £110 billion out of a total UK public sector net debt of £910.1 billion - which is a pretty big chunk!

[http://www.economicshelp.org/blog/uk-economy/uk-national- deb...](http://www.economicshelp.org/blog/uk-economy/uk-national-debt/)

[http://www.heraldscotland.com/northern-rock-could-be-left- of...](http://www.heraldscotland.com/northern-rock-could-be-left-off-the- balance-sheet-1.845546)

<comment><USER17> Exactly, and will vanish overnight when Northern Rock is later sold, hence why they're not really a part of the national debt in any real sense.

</comment></comment></comment><comment><USER9> Australia is an interesting point on that chart since less than ten years ago its debt was around the same as Spain or Canada. I think it had a minerals boom that brought in large surpluses.

<comment><USER19> It also had really good governance for a while, under Rudd. Shame the mining companies had to take him out.

</comment></comment><comment><USER20> What happens in 2026? Is there a debt collector from a foreign planet coming then?

By definition government debt equals net private savings (+/- exports). If the private sector wants a surplus you have to have a government debt. The alternative, as seen on the chart, is to replace net private savings with welfare state programs.

<comment><USER21> _By definition government debt equals net private savings (+/- exports)._

This is bogus. Sure it's technically true, but it doesn't actually mean anything.

 _If the private sector wants a surplus you have to have a government debt._

There is no such thing as what's meant by a "surplus" here.

There are two things that can be considered "saving". There is stockpiling useful things for your own later use, and there is collecting rights to other people's future work (which can be approximated by stockpiling your chosen medium of exchange).

But when you consider net "savings" of an entire economy, rather than individual "savings" of individual people, there is no "other people" to collect future work from. The government can create more medium-of-exchange for people to stockpile, and it's recorded as a debt of the government, but it's entirely meaningless because there's nothing to exchange it for. The only way to have _net_ savings -- deferring the benefits of current work -- is to stockpile those benefits in a warehouse somewhere for later retrieval.</comment></comment>


A long list of online resources for startups - <AUTHOR> http://www.vccafe.com/startup-resources/

<comment><AUTHOR> Summary of what is on the list: Tools and web services Research, Web prototyping, Testing and Building Tools Useful blogs and links for startups

Plus links for European startups and Israeli startups.</comment>


Uyghur woman forced to marry Han Chinese man in exchange for family's release - <AUTHOR> https://twitter.com/WBYeats1865/status/1176865152676683776 <comment><AUTHOR> Related article "China coerces Uighur women into unwanted marriages" with more context:

[https://share.america.gov/china-coerces-uighur-women-into- un...](https://share.america.gov/china-coerces-uighur-women-into-unwanted- marriages/)</comment>


Three Things We Don't Understand About Climate Change - <AUTHOR> https://medium.com/@aarnegranlund/three-things-we-dont-understand-about-climate-change-c59338a1c435 <comment><USER2> What are things we _do_ understand about climate change?

Here's our project on github, the first open API of atmospheric carbon dioxide data: [https://github.com/giving-a-fuck-about-climate- change](https://github.com/giving-a-fuck-about-climate-change)

It's powered by NOAA data from Mauna Loa Observatory.</comment>


The Fifty Steps to Startup Success - <AUTHOR> https://medium.com/@BlackVClub/the-fifty-steps-to-startup-success-2fcfe490a7da

<comment><USER2> You had me until you tried to sell me v-necks at the end.</comment>


Ask HN: What is your one advice for new programmers? - <AUTHOR> It may be any tip, trick, or hack. <comment><USER2> If you are serious about it: After you learn the basics (what is a string vs an int vs a float vs a loop vs an if statement), grab a pencil and notebook and go through the CS50 MIPS manual ASAP and complete all of the exercises without using stack overflow. After two or so years of initial learning, I never felt like a real programmer, and never felt like I understood the "whys" of what I was doing- going through this manual (it's free online) changed that: you will be forced to plan out everything on paper, and really __think __about what you 're trying to do. I can't think of a time investment that I got more return on than my time going through that manual.

</comment><comment><USER3> Try to make something that works in the real world as soon as you can (a web- page, a mobile app, a web-app, or whatever is most relevant to what you're learning).

Being able to create something that works - no matter how simple - can be a far stronger motivator that simply checking off a curriculum's requirements.

</comment><comment><USER4> Always go beyond the curriculum. The free courses out there are so similar, that it is not enough to differentiate yourself against others who have done the same course.

You also need to do more than just 'code' or 'program' these days in $CURRENT_YEAR if you are starting out as a new programmer.

</comment><comment><USER5> Your job is not "code writer".

When you first come into this line of work you will see your work as synonymous with writing code. Nothing could be further from the truth. Code is a liability, software is an asset

</comment><comment><USER6> leetcode will make or break your career. Ignore all other advice

</comment><comment><USER7> Don't be afraid to fail.

Failure is only when you cannot reflect on what went wrong and learn from your mistakes.

No one became an expert programmer overnight without 1,000's of mistakes unless you are a very gifted individual. (personal opinion)

</comment><comment><USER8> Practice. Practice. Practice.</comment>


Why Did Ancient Europeans Just Disappear 14,500 Years Ago? - <AUTHOR> http://www.livescience.com/53883-ancient-europeans-vanished-after-ice-age.html <comment><USER2> It's always 'climate change'. Dozens of megafauna disappear? 'climate change'. Whole tribes with fortifications suddenly disappear? 'climate change'. Wholesale population replacement? 'climate change'. It's our age's "peoples don't migrate, pots do". It's gotten to the point that when I read about the latest population genetics result show introgression or replacement and they invoke 'climate change', I no longer know if it's a case where it could be climate change or just the usual total refusal to speculate about pre-historic warfare and predation.

<comment><USER3> I mean, reason why humans left Africa, walked upright and grew big brains?

Climate change.

Think about how many times you control your local micro climate around you. Think about how many times you talk about the weather with whomever.

It's a big force in the human condition.

<comment><USER4> "Think about how many times you talk about the weather with whomever."

About as many as the moments in which we have to talk with somebody but lack something better to break the ice!

__________

P.S.: I've read so far about a number of possible cases for which humans started to walk upright and none had climate change as a primary cause.

</comment></comment></comment><comment><USER5> Its peculiar that a group with a specific DNA marker would be wiped out so completely by climate change while another group was not (or at least prospered at a later time in the same region).

Could this more easily be explained by an aggressive disease or plague that wiped out anyone it contacted and the separation of the 2 groups (N and M) in different locations protected enough of the N group for it to prosper. Also isolated pockets of M carriers who had migrated beyond the infected area survived.

<comment><USER4> "Could this more easily be explained by an aggressive disease or plague that wiped out anyone it contacted"

This sounds so familiar! "The previous local populations just vanished and we, the current inhabitants, had nothing to do with it, cross my heart!"

The dramatic change should have been very beneficial to the hunter-gatherers. They survived so long in tundra but they somehow couldn't cope with the abundance of plants and plant eating venison brought along? Come on! It's more likely that the previous harsh conditions weren't that inviting for the better hunters of N type super-haplogroup living somewhere else, and as soon as that changed so did the hunting lands' owners!

<comment><USER6> Disease is nothing to sneeze at! One new virus could quite possibly have wiped out an entire population (as it did so often in more recent human history). The end of the glaciers was the perfect condition to increase mobility and thus disease vectors.

In fact, this is so inevitable we'd have to explain why it couldn't happen, before speculating on less-likely scenarios?

<comment><USER4> "we'd have to explain why it couldn't happen, before speculating on less- likely scenarios"

The colonization of Americas is the perfect counter-example for disease wipe- out explanation. Pre-Columbian American population had no immunity whatsoever to a bunch of serious diseases (some of which managed to become pandemics back in colonizer's homelands), diseases that have accumulated over millennia (since the last Bering overland/over-ice human crossing)! You'd expect that on first Columbian contacts there should have appeared a pandemic so severe over the whole Americas that after a few years the Europeans should have had no one alive to encounter! Not only they still encountered, but long after the first contact they also had a hard time dealing with local resistance of underdeveloped tribes which had worse medical conditions than what was to be had in ancient Rome! Not only that locals haven't been wiped out, but they managed to contribute in a large quote to nowadays's Ibero-America's genotype despite the European settlers' massive migration! So yes, I think it's enough reason to consider the disease a less likely cause for a continental scale wipe-out (which is what the article was talking about).

<comment><USER6> A perfect example! Estimates are that 100 million aboriginal Americans were wiped out by diseases brought by Europeans. Most of the population of North America succumbed. The largest dieoff in human history.

We'll never really know the depth of native culture pre-Columbian because of this.

</comment></comment></comment></comment></comment><comment><USER7> Those who find this interesting might also find the theories of Randall Carlson interesting:

[https://www.youtube.com/watch?v=R31SXuFeX0A](https://www.youtube.com/watch?v=R31SXuFeX0A)

[https://www.youtube.com/watch?v=G0Cp7DrvNLQ](https://www.youtube.com/watch?v=G0Cp7DrvNLQ)

[https://www.youtube.com/watch?v=aDejwCGdUV8](https://www.youtube.com/watch?v=aDejwCGdUV8)

</comment><comment><USER8> In my opinion not always "climate change" but one of the main factor. For me main factor is opportunity to moder human to acclimate for this factores. Yes, climate changes are important but more value is respond for that e.g. better groups organisation, better equipment. In that time if you do not acclimate quick you died

</comment><comment><USER9> Maybe because they didn't learn to write proper headlines explaining the content of an article ?</comment>


How to Beat the Salad Bar - <AUTHOR> http://www.nytimes.com/2011/03/20/magazine/mag-20Subversion-t.html

<comment><USER2> Gee, these places will sell you unprepared vegetables (mostly) cheaper than they'll sell you those same vegetables cut, washed, and available for you to shove in a container in whatever combination you want - who could have imagined such a thing?

<comment><USER3> I like my salads simple. So honestly, half the time I just go into the store, and buy a cucumber and a tomato, or something like that, and eat 'em with a little salt. Cheap, nutritious, etc.

<comment><USER2> That's cool. I've been meaning to try the salad recipe about halfway through this link:

[http://thestonesoup.com/blog/2011/01/reduce-reuse-recycle- ho...](http://thestonesoup.com/blog/2011/01/reduce-reuse-recycle-how-the-3rs- can-help-you-with-the-washing-up)

The thing that bugs me about the NYT article is the pretense that one is somehow being cheated by paying a bit more for convenience. If people only cared about cost, nobody would ever use a salad bar - or go to a restaurant, for that matter.

</comment></comment></comment><comment><USER4> Another approach is to go to Sizzler. Avoid the Carbs, and load up on the Baby-Greens, Spinach, Olives (two kinds), Jalapenos, Radish, guacamole, red onion, mushrooms, broccoli - $8.99 (and they throw in a reasonably well cooked 6 oz (tri-tip) steak to boot - Just say no to the Baked Potato/Fries).

And, if you really want to go to town, for an extra $2.00 you can turn that into unlimited trips to the Salad Bar.

It's amazing how much Salad a guy with his iPad, and a free afternoon can put away. :-)

I used to go to Fresh Choice - until they started charging more for the "Premium" salad bar choices, and I realized 90% of the food beyond the salad bar was low (empty?) nutrition carbs.

</comment><comment><USER5> TL;DR - to get your money's worth at Whole Foods get a Sundried-Tomato and Bacon-Bits Salad. Add a dash of balsamic vinegar if you're feeling indulgent.

<comment><USER6> "Whole Foods doesn't stock the best salad topping of all — bacon bits, obviously — in its salad bar."

A better summary: Most things at the salad bar are 70% more expensive than on shelves. However, there are some exceptions which are much cheaper at the salad bar. The optimal salad consists of Mesclun, Olive Oil, Balsamic Vinegar, Gorgonzola Crumbles, Almonds, Dried Cranberries, Sun-dried Tomatoes, and Croutons (per the infographic included with the article), all items which cost more than $7.99/pound (the cost of a salad) on the shelves.

And a TL;DR for my summary: When buying salads, choose items which cost more per pound than the per pound price of salad.

</comment></comment><comment><USER7> "How to Beat the Supermarket"

I just got home from the supermarket after spending:

      $1.49 for one cucumber
      $2.99 for one pound (about 2) of tomatoes
      $2.49 for one head of lettuce

This is about right for one large salad.

A restaurant up the street has an all-you-can-eat salad bar for $7.29 which includes bread, chili, soup, and dessert.

Looks like it's going to be salad bar for lunch and cooked dinner (without salad) at home here.

<comment><USER2> A salad with a pound of tomatoes _alone_ is pretty darn "large". :)

</comment></comment><comment><USER8> If you are eating salad, you have already lost.

<comment><USER3> What?</comment></comment>


State of the container world, January 2016 - <AUTHOR> http://blog.kubernetes.io/2016/02/state-of-container-world-january-2016.html <comment><USER2> I'm finding it hard to believe that 50% of the individuals using container technologies are using it in production. Any more details on this?

<comment><AUTHOR> this was originally sent out on twitter by a bunch of people (including myself) who are working on containers (and have a container-oriented set of followers) so I'm pretty certain there is a forward looking bias in the data.

I tried to describe this bias at the top of the blog post.

</comment></comment><comment><USER3> the interesting bit is looking at how users of containers are behaving, not the % or marketshare</comment>


Clr-boot-manager now available in Solus - <AUTHOR> https://solus-project.com/2017/03/26/clr-boot-manager-now-available-in-solus/ <comment><USER2> It's a cool idea but I almost wonder if they would have gotten more functionality and the same end result by making use of ZFS snapshots and boot environments. Configuration could have even been kept in ZFS properties. Ikey has previous talked about ZFS not being right for a desktop distribution but I don't agree. Snapshots, rollbacks and checksums benefit everyone everywhere. It could be used transparently in the background so the users of Solus wouldn't even need to know it was being used if they didn't want to. That said, if ZFS can't be used, I can see 'clr-boot-manager' being useful, and I like the fact that Ikey works on projects that can be used outside of just Solus. I've been using Solus on one of my laptops and following along with its development and it's quite interesting project. They seem to be happy to try new things and do things in a different way if it seems like it will benefit the users, but they seem to still be content to use existing software if it looks like it will do what they are looking for.

<comment><USER3> I've not used it, but afaik Suse linux default install have btrfs+snapper ([http://snapper.io](http://snapper.io)) It snapshots before every install and yast configuraton step and have a boot to snapshot feature..

<comment><USER4> Great tool - it's also available for the most common distros (RedHat, Debian, Ubuntu...) and it supports LVM, EXT4...

<comment><USER2> Cool, I didn't realize it supported more than btrfs.

</comment></comment></comment></comment><comment><USER5> I dug around the site, I have no idea what this is. A linux distro? a solaris one? why is their barely any info except a download link?

<comment><USER6> Solus is a rolling release linux distribution aimed at desktops. Its really very impressive. Its starting to feel a bit like a game-changer for linux desktops. Its run by Ikey Doherty who works for Intel's Clear Linux.

<comment><USER7> What makes it a game changer? The website is very sparse -- couldn't even find screenshots for budgie!

<comment><USER8> Solus is an interesting recent distro in that it's written entirely from scratch, not built on top of Debian/Ubuntu/Arch, etc. It also made its own DE, Budgie, which has since been ported to Ubuntu. It's also become the go-to distro for Linux Steam enthusiasts, because the developers put a huge amount of work into making their own package of Steam that seems to work better than many of the existing ones.

You can check out a DistroWatch review of it here: [https://distrowatch.com/weekly.php?issue=20170206#solus](https://distrowatch.com/weekly.php?issue=20170206#solus)

Also, you may wish to listen to an episode or two of Late Night Linux, on which Ikey is one of the hosts. It's not solely about Solus, but it does get quite a bit of screentime by virtue of his connection to it: [https://latenightlinux.com/about/](https://latenightlinux.com/about/)</comment></comment></comment></comment>


HTML 5 data- Attributes - <AUTHOR> http://ejohn.org/blog/html-5-data-attributes/

<comment><USER2> Seems like a bad idea to me. Mixing data into the html seems messy. The same thing can be achieved just by having the data in javascript data structures, and referencing html elements either by ID, or directly.

<comment><USER3> The html _is_ the data, this is the correct place for it. The javascript should be behavior and not contain additional data.

<comment><USER2> I disagree. HTML is a specific serialization of the display/UI. Yes it is data, but it should be data related to display.

Writing a webapp you don't even really need to use html at all apart from to bootstrap your app. Then you can build the UI yourself instead, which I think ends up as a nicer design and more elegant solution often. (Yes it's not accessible or graceful in terms of degrading)

<comment><USER3> The CSS is the display, not the HTML. The HTML _should_ be capable of being displayed in many different ways by just changing the CSS.

<comment><USER2> Yes, but as I said, the HTML should be data to be displayed. eg the content. NOT arbitrary data used in the program.

</comment></comment></comment></comment></comment><comment><USER4> This is a nice feature, but I prefer the XHTML approach. Why invent a new feature when you already have it?

All this repeated feature-ing is why browsers are so huge, memory-intensive, unmaintainable, and unreliable.

</comment><comment><USER5> Cool, didn't know that existed in the spec. Looks very powerful.

</comment><comment><USER6> Anybody have any idea when HTML5 is going to be released?</comment>


"Hearable" devices that do more than improve sound - <AUTHOR> https://www.cnet.com/news/hearing-aids-now-theyre-internet-ai-and-health-devices-starkey-oticon-harman-bose/ <comment><USER2> Despite not having hearing loss, I'd be interested in augmenting my hearing. It seems far more useful than I can ever imagine Google Glass would have been for me.

<comment><USER3> I'd especially love this when working remotely at cafes/busy shared spaces. Being able to tune in and hear someone better if they're in front of me or speaking directly at me or blocking out background noise.

<comment><USER4> These work quite well: [https://www.bose.com/en_us/products/headphones/conversation_...](https://www.bose.com/en_us/products/headphones/conversation_enhancing_headphones/hearphones.html)

I've tested them in demonstration environments only but they seemed to be close enough to real world cases to me.

<comment><USER5> would be interested in reading someone's experience comparing these against the bose qc20. i love my qc20 right now. They reduce a lot of background noise, but I can still hear people talking (slightly reduced, but still can hear a conversation). Is this product mostly just a rebranding of the qc20 noise cancelling tech, or is it a different experience?

<comment><USER4> I don't have the QC20 I use QC35 now, and I have been using Bose NC headphones since the QuietComfort 2 came out like 15 years ago. I never used any of the Bose QC in earones but the "hearphones" are nothing like any of the Bose headphones I ever used even the older ones with a much higher (subjectively better) direct voice / talk frequency and amplitude range crossover than the current ones.

It's more or less the same technology ofc but the profile of the microphone or at least the tunning / post processing they do is completely different.

</comment></comment></comment></comment></comment><comment><USER6> I don't trust the market to treat all signals equally. There's a great opportunity for selling a cheaper (free?) device that inserts ads, or blocks audio from ads whose companies haven't paid. Or hell, we could just sell the data. The possibilities are endless!

I hope someone is working on an open source "augmented" hearing aid.

<comment><USER7> Make it one that uses AI to improve diction and displays speech to text subtitles (with choice of a second subtitle in another language) on your cell phone or another display of your choosing, and imagine the improvement in communication and language learning.</comment></comment>


Writing a C Compiler, Part 1 - <AUTHOR> https://norasandler.com/2017/11/29/Write-a-Compiler.html <comment><USER2> Most compiler tutorials show how to write a compiler, but now how to write an _optimising_ compiler. In fact, it's not that much harder, and is a nice walkthrough in some very cool ideas in graph theory, dataflow analysis,etc.

Which is why I've been writing a mini-LLVM in haskell as a tutorial to explain optimising compiler construction: [Tiny optimising compiler]([https://github.com/bollu/tiny-optimising- compiler](https://github.com/bollu/tiny-optimising-compiler))

<comment><USER3> There is something to be said for having LOTS and LOTS of non optimizing C compilers written in many languages. Even languages like, say, Python. And plenty of C compilers written in C. A virtue of being non-optimizing is that they are easier to study and verify correct operation.

What we need is that the source code for the smaller number of optimizing compilers can be compiled by the many, many different inefficient C compilers.

We need to be sure that the binary of the optimizing compiler is not compromised by the compiler which compiled the optimizing compiler.

You could compile compilers. Including compiling various compilers written in C. At some point you compile the wanted optimizing compiler's source using various different compilers generated through various chains of compilation. Those binaries of the optimizing compiler may all differ, but the code generated by those multiple binary versions of the optimizing compiler should be identical.

Finally use the uncompromised optimizing compiler to compile itself. In fact, using the multiple binaries of the optimizing compiler to compile itself and be sure they all produced the same final binary.

Cross compilation doesn't hurt either. For example a Raspbery Pi compiles the optimizing compiler for x86-64. The binary of that compiler should still match the same compiler compiled from other C compilers.

Now we not only have to be paranoid about the binary of a compiler being compromised by another compiler, but by the Intel Management Engine. Compromise baked right into the hardware.

Every paranoid thing I thought ten years ago turned out to be true.

<comment><USER4> See Ken Thompson's award speech where he talks about the original C compiler backdoor:

[http://wiki.c2.com/?TheKenThompsonHack](http://wiki.c2.com/?TheKenThompsonHack)

[http://vxer.org/lib/pdf/Reflections%20on%20Trusting%20Trust....](http://vxer.org/lib/pdf/Reflections%20on%20Trusting%20Trust.pdf)

<comment><USER5> See David Wheeler's work on "Diverse Double Compiling". What the parent describes seems to be similar.

[https://arxiv.org/abs/1004.5534](https://arxiv.org/abs/1004.5534)

</comment></comment></comment></comment><comment><USER6> A way to do this more directly and without dealing with any assembly syntax and at a deeper level is to compile down to an emulation of a straightforward machine (I really like MSP430 for this), preferably to an emulator you write.

Having had the pleasure of writing a couple emulators and a couple compilers at this point: the emulator is easier than the compiler, by a lot. Relative to parsing and emitting code, emulating a straightforward architecture is easy.

This is a great post!

<comment><USER7> Any reference(s) where I can read more about writing emulators?

<comment><USER8> Here is one [https://fms.komkon.org/EMUL8/HOWTO.html](https://fms.komkon.org/EMUL8/HOWTO.html) I found quickly.

Writing an emulator is almost as much fun as writing a compiler. And when you are done, you will have another level of appreciation of how computers work.

</comment></comment></comment><comment><USER9> _I really like Ghuloum's approach: you start by compiling a tiny, trivial subset of your source language all the way down to x86 assembly. Then you add new language features, one step at a time. In step one, you just return constants; in a later step you handle addition and subtraction; and so on. Every step is small enough to feel manageable, and at the end of the every step you have a working compiler._

That approach sounds like the same one in Jack Crenshaw's _very_ lucid tutorial:

[https://compilers.iecc.com/crenshaw/](https://compilers.iecc.com/crenshaw/)

He uses Pascal to write a Pascal compiler, but the basic steps seem to be the same --- start with a _very_ simple lexer, add pieces to it incrementally, then write a recursive-descent parser on top of that and gradually extend it to handle more of the language.

Another simplification that really helps, once you're used to recursive descent and have noticed that the functions for each operator/precedence level are very similar, is to refactor them into a lookup table and an even simpler function that uses it, creating "precedence climbing":

[https://www.engr.mun.ca/~theo/Misc/exp_parsing.htm](https://www.engr.mun.ca/~theo/Misc/exp_parsing.htm)

That makes it very easy to add/change/extend operators, or even have them be user-defined(!) and dynamically modifiable.

As a demonstration of how absolutely _tiny_ a compiler can be, this one is worth inspecting carefully --- it also uses a handwritten lexer and recursive- descent parser, but doesn't make an AST and just emits code for a stack VM: [https://news.ycombinator.com/item?id=8558822](https://news.ycombinator.com/item?id=8558822)

...and someone modified that one to make an x86 JIT: [https://news.ycombinator.com/item?id=8746054](https://news.ycombinator.com/item?id=8746054)

<comment><USER10> I remember reading Jack Crenshaw's "let's build a compiler" tutorial in the 90's. Though pretty outdated, I can definitely recommend it to everyone interested in compiler construction, because it's written from the perspective of, well, a compiler creator.

</comment></comment><comment><USER11> At uni I went through a module on compilers which basically covered all the Dragon book.

I then followed this on-line course where you actually write a compiler for the COOL language. It has been one of the best educative things I've ever done in my life and I can't help recommending it.

[https://lagunita.stanford.edu/courses/Engineering/Compilers/...](https://lagunita.stanford.edu/courses/Engineering/Compilers/Fall2014/about)

I'd love to do something like that in my professional career but unfortunately 90% of the time I ended up carrying out way less interesting work (writing application for end users).

<comment><USER12> There's no better advertisement for the utility of learning to write a compiler than this introduction from the excellent "Engineering a Compiler" by Cooper and Torczon:

Why Study Compiler Construction?

A compiler is a large, complex program. Compilers often include hundreds of thousands, if not millions, of lines of code, organized into multiple subsystems and components. The various parts of a compiler interact in complex ways. Design decisions made for one part of the compiler have important ramifications for other parts. Thus, the design and implementation of a compiler is a substantial exercise in software engineering.

A good compiler contains a microcosm of computer science. It makes practical use of greedy algorithms (register allocation), heuristic search techniques (list scheduling), graph algorithms (dead-code elimination), dynamic programming (instruction selection), finite automata and push-down automata (scanning and parsing), and fixed-point algorithms (data-flow analysis). It deals with problems such as dynamic allocation, synchronization, naming, locality, memory hierarchy management, and pipeline scheduling. Few software systems bring together as many complex and diverse components. Working inside a compiler provides practical experience in software engineering that is hard to obtain with smaller, less intricate systems.

<comment><USER13> _"A compiler is a large, complex program"_

Writing a compiler isn't much harder than writing an interpreter, if at all.

Assuming your language can be parsed without running it (Perl, for instance, cannot, in general. See [https://www.perlmonks.org/?node_id=663393](https://www.perlmonks.org/?node_id=663393)), it is just a matter of doing that and emitting code for every statement, plus some bookkeeping in order to fix up branches. If the language you are compiling is simple, that's not hard to do.

The result will not be the fastest possible, but it will be compiled code, and it will be faster (sometimes significantly so) than interpreting the program.

<comment><USER14> Are you trying to dispute that compilers are large and complex? It seems like you are saying that if the language is simple, a non-optimizing compiler for it can be simple, which seems somewhat obvious. I think the "microcosm of computer science" and associated complexity become apparent when the source language is not simple, the middle end contains many optimization passes, and the back end requires more than simple statement emission.

<comment><USER13> Yes. The categorical statement "compilers are large and complex" is incorrect, and IMO is detrimental to getting people interested in the field.

We don't say "Web applications are large and complex", either.

Just as you can start simple with a web application and, if needed, grow it, you can start with a simple compiler and, if needed, grow it.

(And, on current hardware, I think the complexity of the language matters less than it used to. Single pass languages such as pascal still are easier targets, but computers have plenty of RAM, so fixing up things is easier than it used to be.

</comment></comment></comment></comment></comment><comment><USER15> Can't wait for the next parts of this!

I think this incremental approach must be the way to do it.

I was inspired by 8cc to write my own C compiler; but I ended up burnt out on that project. (The standard document is just infuriating. There's so much ambiguity and disorganization in the document, and the actual technical specifications are like an afterthought to the novel of text.)

May be time to get back into it!

</comment><comment><USER16> Check out [http://www.craftinginterpreters.com](http://www.craftinginterpreters.com) for something more in-depth.

</comment><comment><USER17> Love it. This is exactly how I wrote compilers for a living back in the day – I always got the minimum in end-to-end demo running because it was always the best way to avoid ugly surprises.

</comment><comment><USER18> To someone who knows more about the subject - am I correct that actually encoding/parsing the language is a small part of creating a compiler comparing to AST optimization?

Optimizing e.g. C to create much more optimal Assembly (especially using some more non-standard instructions) seems like a much more daunting task

<comment><USER19> As others have said, yes, that's mostly the case.

Even parsing C and elaborating it to a correctly typed AST is not _quite_ as simple as others are making it out to be, though. Getting all the implicit type conversions correct is not completely trivial, and is a popular source of bugs (see some examples in [http://www.cs.utah.edu/~regehr/papers/pldi11-preprint.pdf](http://www.cs.utah.edu/~regehr/papers/pldi11-preprint.pdf) for instance).

There are also some annoying ambiguities in the grammar and in name resolution rules that mean that sometimes it's not easy to tell whether something is supposed to be a type name or a variable name: [https://jhjourdan.mketjh.fr/pdf/jourdan2017simple.pdf](https://jhjourdan.mketjh.fr/pdf/jourdan2017simple.pdf)

C is a "simple" language in many senses of the word, but it has a _lot_ of complex details. It's fine to gloss over them when writing a compiler for a language very similar to C for learning purposes, but getting everything just right for _actual_ C is tough.

<comment><USER20> Yes, stuff like the `a * b` ambiguity (declaration or expression?), typedef being just another storage class specifier (thus, `static int var` and `typedef int mytype` being the exact same syntactic forms), having complex rules for implicit promotions and conversions, the mess between signed and unsigned.

All of these design warts of C show up clearly when attempting to write a compiler and are not very obvious to most users of the language.

<comment><USER21> Function pointer declarations is also a fun one, specially in a function declaration as return value and parameters.

</comment></comment></comment></comment><comment><USER22> I find that the assumption that program control requires boolean values (of any kind) facinating, since there are at least two languages that use success/failure as the controlling mechanism - Snobol (in all its incarnations) with the follow up language of Icon/UnIcon.

Neither have the concept of booleans, but use the idea that a computation can succeed and return a result or fail and return no result. This leads to making certain kinds of expressions that normally require booleans and the associated logical operators to be simpler and more clear, as below:

if a < b && b < c && c < d then ...

compared with

if a < b < c < d then ....

Which is easier to mentally parse?

<comment><USER23> I really liked how Icon handles function success/failure separately from return values. It seems very close to Go's error handling conventions, but with a little more magic. Wikipedia gives the following Icon example code:

      if a := read() then write(a)

which would translate into something like the following Go code:

      a, err := read()
      if err == nil {
          write(a)
      }

<comment><USER22> Simplify this to

write(a := read())

where write will not be executed if read fails, nor will a have any value assigned to it. Failure is an option and not an error.

So to copy standard input to standard output, we do

every write(read())

At the end of file, the read fails, the write fails and then the every fails and on you go.

</comment></comment></comment><comment><USER21> Anyone that wants to write C compilers has two good books for it.

"A Retargetable C Compiler: Design and Implementation", from David R. Hanson and Christopher W. Fraser

"Compiler Design in C", from by Allen I. Holub

A bit old, but there are not significant changes to C since they were written, other than optimizers taking advantage of UB.

<comment><USER24> I read that as "A Regrettable C Compiler"

</comment></comment><comment><USER25> Discussion for part 2: [https://news.ycombinator.com/item?id=15861788](https://news.ycombinator.com/item?id=15861788)

</comment><comment><USER26> The "recursive descent parsing" part is not very well explained or thorough. It doesn't have good example...

I still enjoyed reading it though.

EDIT:

I've the recursive descent parsing article on wikipedia, and I'm amazed how the C code example is missing parts... Kinda sad.

</comment><comment><USER27> > You also need to decide whether to write your own parser and lexer or use > automatic parser and scanner generators (e.g. flex and bison).

Really? There are much better choices available in 2017. ANTRL 4 leaps to mind which has been in development since 1989.

<comment><USER28> ANTLR is really only good for the Java ecosystem. The quality of the generated C/C++ code is low and it has a large number of dependencies. C++ development also lagged pretty far behind Java when ANTLR 4 was released.

I don't know of any "real" language written in C or C++ that uses ANTLR.

The state of the art for production languages in C/C++ is either hand-written parsers or yacc/bison. I've never even seen flex used, and I've looked at over 20 lexer/parser implementations for "real" languages.

<comment><USER27> Java, C#, Go, Python, JavaScript, Swift and C++ although everything gets started with Java and filled out to the other targets.

You may be thinking of ANTRL 3. I really didn't much care for the ANTLR 3 C target but I use ANTRL 4 and its C target. It is really much much better than ANTRL 3. TParr says ANTLR 4 is the parser generator he always wanted to make.

However, I'm not still pitching ANTRL as a parser for a real compiler. ANTRL rules the world for little languages, domain specific languages but a production language would almost certainly would use a hand crafted recursive decent compiler.

This is a learning project. So I'd strongly recommend anyone learn modern tools and methods and learn those. bison is C++ native with experimental Java support and nothing else. flex is C and C++ only. Not a fan.

</comment></comment></comment><comment><USER26> I would be more interested in learning how to write a transpiler...

As long as I can compile a language in C, it removes a lot of work.

<comment><USER21> That is just a compiler whose backend generates code in another language instead of machine code.

<comment><USER26> Well I find it more interesting...

Also C compilers can do a lot of micro optimizations that are not trivial, so compiling something to C or maybe LLIR seems like a better choice.

<comment><USER21> That is usually a fallacy, that only looks simple as first step.

The first issue is the impedance mismatch between the original language semantics and what is possible to represent in C.

Two examples are tail calls and continuations, that are hard to get right at C level, either requiring compiler extensions, longjmp/setjmp tricks, or a little bit of assembly.

Then there is the whole issue with undefined behaviour in C, introducing bugs that didn't exist on the original code.

Even if the generated code is clean today regarding UB, you cannot ensure that a new C compiler version won't be introducing such issues.

Having said this, there are languages like Eiffel and Nim, whose main backend relies on C compilers, instead of compiling directly to pure native code.

Skipping Objective-C and C++ here, because although they started as C pre- processors, their code was to be an extension to C and so their semantics also imply C semantics.

Compiling to some kind of IR, like LLIR is more sane.

Then you can do whatever you want with it, generate code in another programming language, machine code, or a plain interpreter for bootstrapping purposes.

<comment><USER29> I agree with most of your points, but...

> Even if the generated code is clean today regarding UB, you cannot ensure > that a new C compiler version won't be introducing such issues.

This is not correct. Undefined behaviour is specified in the standard, and a good compiler-to-C will always generate UB-free code.

<comment><USER21> The standard specifies all known cases, the list is not exhaustive from all possible C implementations.

<comment><USER29> I am sorry but I don't understand... The standard specifies what is UB, and the implementations must behave accordingly to be standard-compliant.

If your code avoids UB behavior as specified by the standard it will run in the same way when compiled with any standard-compliant compiler.

<comment><USER21> Compiler optimization is a tricky business, where every vendor does all they can to beat the benchmarks game.

What gets documented on the standard is what compiler vendors that bother to seat at ANSI C meetings agree to document as such.

By no means does the standard forbid compiler vendors to take advantage of situations not yet documented as such.

Eventually new cases of UB might get added to the standard.

The current set of about 200 scenarios of UB in C11 weren't all there in C89.

Another example, is that you can have code that is perfectly valid, UB free, but thanks to PGO and code re-writting rules on the optimizer stage, gets rewritten with UB side effects at a later stage in the optimizer pipeline.</comment></comment></comment></comment></comment></comment></comment></comment>


Police Tried to Unlock a Dead Man's Phone with His Finger - <AUTHOR> http://time.com/5249765/florida-police-unlock-dead-mans-phone/ <comment><USER2> Isn't there a precedent against using someone's body against their will after they die?

</comment><comment><USER3> Auto-play video + audio.</comment>