Ask HN: Learning about distributed systems? - <AUTHOR> I used to love Operating Systems during my undergrads, Modern Operating Systems by Tanenbaum is till date the only academic book I've read entirely. I recently read an article about how Amazon built Aurora by Werner Vogels and I was captivated by it. I want to start reading about Distributed Systems. What would be a good start/Road Map? <comment><USER2> I posted this recently, but MIT's 6.824: Distributed Systems (taught by Robert Morris, of both Morris worm and Viaweb/Y Combinator fame) is completely open and available online, and it includes video lectures, notes, readings, and programming assignments from as recent as Spring 2020 (including half of the lectures recorded from home as the pandemic strikes). The assignments even include auto-graded testing scripts, so you can verify your solution to the assignments.

[https://pdos.csail.mit.edu/6.824/](https://pdos.csail.mit.edu/6.824/)

<comment><USER3> I posted this as well in the last thread about this class, but since we're discussing it again there is an active study group doing the labs in Clojure on reddit:

[https://www.reddit.com/r/mit6824clojure/](https://www.reddit.com/r/mit6824clojure/)

Several of the labs have been ported in full (map-reduce, first part of RAFT lab), including test scripts. Please join if this is interesting to you.. the more the merrier!

</comment></comment><comment><USER4> The book "Designing Data-Intensive Applications" by Martin Kleppman is a fantastic read with such a concise train of thought. It builds up from basics, adds another thing, and another thing.

I kept asking myself, what would happen if I were to extend on the feature currently presented in the chapter I was reading, only to find out my answers in the next chapter.

Brilliant book

<comment><USER5> Problem with this book is: what to read after. The book is really good but leaves you with the feeling "there are some many topics I don't know yet... but all the other books out there suck". Any recommendation for someone that has already read DDIA?

<comment><USER6> [https://github.com/ept/ddia-references](https://github.com/ept/ddia- references)

Here is their git repository of all online references, by chapter. Seems it doesn't include papers that don't have a clear, public link. If you have the book, though, you can get the names and search for them.

</comment></comment></comment><comment><USER7> * This System Design Primer [1] on GitHub is a decent overview of how large-scale apps are designed, with jumping-off points into many different subjects.

* The Morning Paper blog's distributed systems tag [2] has a lot of good summaries of research on distributed systems, both from academia and industry.

* I maintain a list of assorted resources on distributed system design and operations on GitHub. [3]

* Also, as mentioned, Designing Data-Intensive Applications is a good starting place.

[1] [https://github.com/donnemartin/system-design- primer](https://github.com/donnemartin/system-design-primer)

[2] [https://blog.acolyer.org/tag/distributed- systems/](https://blog.acolyer.org/tag/distributed-systems/)

[3] [https://github.com/DylanSp/distributed-systems- resources](https://github.com/DylanSp/distributed-systems-resources)

</comment><comment><USER8> Stumbled on this which I felt was a very good compendium \- [https://dancres.github.io/Pages/](https://dancres.github.io/Pages/)

Would also recommend reading VLDB and DB it shows how distri algorithms are applied \- [http://www.vldb.org/pvldb/vol9.html](http://www.vldb.org/pvldb/vol9.html) \- [http://www.redbook.io/](http://www.redbook.io/)

Disclaimer: I used to work at Couchbase(distributed NoSQL database) as a PM and launched Eventing.

</comment><comment><USER9> Speaking of Werner Vogels, have you seen this blog post on DistSys reading? [https://www.allthingsdistributed.com/2012/12/paper- readings-...](https://www.allthingsdistributed.com/2012/12/paper- readings-2012.html)

</comment><comment><USER10> While I don't see it as a starting point (I think the topics require more context), I'm a big fan of the articles Amazon has published recently as the "Builder's Library"

[https://aws.amazon.com/builders-library/?cards-body.sort- by=...](https://aws.amazon.com/builders-library/?cards-body.sort- by=item.additionalFields.customSort&cards-body.sort-order=asc)

</comment><comment><USER11> Maarten van Steen has got you covered. He worked with Andrew Tanenbaum on all kinds of things back in the day :)

[https://www.distributed- systems.net/index.php/books/ds3/](https://www.distributed- systems.net/index.php/books/ds3/)

</comment><comment><USER12> I had found this to be a really good resource. High level patterns to use. [https://www.oreilly.com/library/view/designing- distributed-s...](https://www.oreilly.com/library/view/designing-distributed- systems/9781491983638/)

</comment><comment><USER13> From a previous question re: "Ask HN: CS papers for software architecture and design?" ([https://news.ycombinator.com/item?id=15778396](https://news.ycombinator.com/item?id=15778396) and distributed systems we eventually realize were needed in the first place:

> _Bulk Synchronous > Parallel:[https://en.wikipedia.org/wiki/Bulk_synchronous_parallel](https://en.wikipedia.org/wiki/Bulk_synchronous_parallel) > ._

Many/most (?) distributed systems can be described in terms of BSP primitives.

> > _Paxos:[https://en.wikipedia.org/wiki/Paxos_(computer_science)](https://en.wikipedia.org/wiki/Paxos_\(computer_science\)) > ._

> > _Raft:[https://en.wikipedia.org/wiki/Raft_(computer_science)](https://en.wikipedia.org/wiki/Raft_\(computer_science\)) > #Safety_

> _CAP > theorem:[https://en.wikipedia.org/wiki/CAP_theorem](https://en.wikipedia.org/wiki/CAP_theorem) > ._

Papers-we-love > Distributed Systems: [https://github.com/papers-we- love/papers-we-love/tree/master...](https://github.com/papers-we-love/papers- we-love/tree/master/distributed_systems)

awesome-distributed-systems also has many links to theory: [https://github.com/theanalyst/awesome-distributed- systems](https://github.com/theanalyst/awesome-distributed-systems)

\- Byzantine fault: [https://en.wikipedia.org/wiki/Byzantine_fault](https://en.wikipedia.org/wiki/Byzantine_fault) :

> _A [Byzantine fault] is a condition of a computer system, particularly > distributed computing systems, where components may fail and there is > imperfect information on whether a component has failed. The term takes its > name from an allegory, the "Byzantine Generals Problem",[2] developed to > describe a situation in which, in order to avoid catastrophic failure of the > system, the system's actors must agree on a concerted strategy, but some of > these actors are unreliable._

awesome-bigdata lists a number of tools: [https://github.com/onurakpolat/awesome- bigdata](https://github.com/onurakpolat/awesome-bigdata)

Practically, dask.distributed (joblib -> SLURM,), dask ML, dask-labextension (a JupyterLab extension for dask), and the Rapids.ai tools (e.g. cuDF) scale from one to many nodes.

<comment><USER13> Not without a sense of irony, as the lists above list many papers that could be readings with quizzes,

Distributed systems -> Distributed computing: [https://en.wikipedia.org/wiki/Distributed_computing](https://en.wikipedia.org/wiki/Distributed_computing)

Category: Distributed computing: [https://en.wikipedia.org/wiki/Category:Distributed_computing](https://en.wikipedia.org/wiki/Category:Distributed_computing)

Category:Distributed_computing_architecture : [https://en.wikipedia.org/wiki/Category:Distributed_computing...](https://en.wikipedia.org/wiki/Category:Distributed_computing_architecture)

DLT: Distributed Ledger Technology: [https://en.wikipedia.org/wiki/Distributed_ledger](https://en.wikipedia.org/wiki/Distributed_ledger)

Consensus (computer science) [https://en.wikipedia.org/wiki/Consensus_(computer_science)](https://en.wikipedia.org/wiki/Consensus_\(computer_science\))</comment></comment>


Panama Papers source issues statement - <AUTHOR> https://panamapapers.icij.org/20160506-john-doe-statement.html <comment><USER3> There's a lot to parse in that statement, but the call for whistleblower protections seems to be the single most important (and achievable) item. Most people are aware of Snowden and Manning, but it really surprises me that these haven't been reported more:

 _" Bradley Birkenfeld was awarded millions for his information concerning Swiss bank UBS—and was still given a prison sentence by the Justice Department. Antoine Deltour is presently on trial for providing journalists with information about how Luxembourg granted secret "sweetheart" tax deals to multi-national corporations, effectively stealing billions in tax revenues from its neighbour countries."_

Law enforcement agencies don't have the resources or knowledge to go after much of the corruption and wrongdoing inside governments and large companies. If insiders with integrity don't have a safe way of stepping forward there will never be a way to keep wealthy/powerful/connected individuals from abusing the system.

<comment><USER4> I'd think it's the least achievable thing of all. Governments will happily pass laws to {increase transparency / reduce privacy} and will happily pass laws that {add new taxes / close tax loopholes}, because these things align with their pre-existing agendas. They will not, under any circumstances, make it easier to engage in whistleblowing and especially not large data dumps, because governments absolutely do not want millions of { citizens holding them to account / vigilante info-warriors } increasing their OWN transparency!

That said, whilst I agree with that part, the rest of John Doe's essay left me cold. Other than its defence of whistleblowers it reads like more or less any standard left-ish Guardian article. The cause of increasing global inequality being a handful of law firms, really? They "write the laws" themselves, really? Which laws does he have in mind? All lawyers are corrupt and unethical? The British island territories are the "cornerstone of institutional corruption worldwide" and not, say, African states where the corruption actually occurs? Billionaires own the press and serious investigative journalism is dead, except, presumably, the press and the journalists who he worked with?

I was and still am a huge supporter of Snowden because he revealed behaviour that was unquestionably bad. Literally nobody tried to defend what he showed was happening, and in fact the people doing it had lied in Congress to try and cover it up. It was a classic case where whistleblowing is justified. Additionally, Snowden had a very clear and straightforward thought process justifying his actions: what was happening was unconstitutional, and his attempts to use the formal complaint paths had failed.

John Doe comparing himself to Snowden rubs me up the wrong way, because although he claims the MF files are bursting with criminal evidence, so far all the stories I read about the Panama Papers were about things that are not illegal, and in fact apparently some of the papers show MF dropping clients when they started to suspect illegal activity, which implies MF was not quite the sinister conspiracy Doe makes it sound like. They clearly had legal compliance efforts and they clearly did things. And his justification is a long, rambling and rather incoherent screed that tries to claim the fault of every problem in the world lies with a kind of global conspiracy of evil and spineless people.

I think Doe is walking a very thin line between whistleblowing for a cause and generic vigilante-ism with his actions.

<comment><USER5> > so far all the stories I read about the Panama Papers were about things that > are not illegal

So the accusation of perjury left you cold? It was linked in the piece.

[https://www.publicintegrity.org/2016/04/03/19506/offshore- la...](https://www.publicintegrity.org/2016/04/03/19506/offshore-law-firm- runs-trouble-las-vegas)

</comment></comment></comment><comment><USER6> _But most of all, the legal profession has failed. Democratic governance depends upon responsible individuals throughout the entire system who understand and uphold the law, not who understand and exploit it._

Sigh. One of the most puzzling questions I have to deal with in my mind. Why is there so little moral left in this world?

<comment><USER7> Why should there have been any morals in the first place? Does nature by default have morals?

<comment><USER8> Nature by default doesn't have nuclear weapons, stock exchanges, global supply chains, industrial manufacturing, central banks, etc. Humans need morals (and norms more generally) in order to ensure that the above serve us rather than hurt us. Morals are a cultural technology like language or counting, and like those they may be (at least in their strong form) uniquely human.

<comment><USER9> >>Nature by default doesn't have nuclear weapons....

While I agree with your sentiment, just would like to point this out that nature does have nuclear weapons, pretty scary ones for that matter: the stars.

Also, if we take these things to their logical conclusion, then the nuclear weapons created by human beings are, of course, created by nature - in this case, indirectly, using the humans.

This is just to point out the logical aspects of the arguments. Other than that, I do agree with you that we, the humans, need morals to sustain a better life for most human beings.

<comment><USER10> These are nuclear, but not weapons. Nature doesn't have an enemy.

<comment><USER9> >>Nature doesn't have an enemy.

How are you so sure? Have you talked with Nature by any chance? Has Nature given you this statement, say, in English? I am not trying to be sarcastic here, I am dead serious.

I for example, think that Nature likes to fight many of its creatures against each other. e.g. polio viruses against human beings

I may be wrong, but one cannot be so sure.

<comment><USER10> " Nature likes to fight many of its creatures against each other

Anthropomorphising can only go that far. Do you know any physical law that has volition?

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> The call for whistleblower protection is important, but it'll never happen. The powers-that-be don't want to encourage whistleblowers.

What _may_ change their mind is if all the data were made public. Since whistleblowers have not much protections, their only protection right now is to release everything on the 'net, anonymously. Now, clearly this is not a good idea, as in many cases there will be collateral damage. But what is the alternative? Once the governments see that such collateral damage is the only alternative, they will be force to enact meaningful whistleblower protection.

<comment><USER12> This is correct. There is Government, but then there are the secret powers- that-be that pull governments' strings.

And they can't work effectively to further their own ends while in the light of public scrutiny.

<comment><USER13> I don't like this attitude. In fact, some whistleblower protection laws already exist (whistleblower protection act).

I'm sure there are forces that act on politicians other than loyalty to their constituents, but there's no reason in the American democracy we couldn't (theoretically) get to a point where we elect representatives who understand the importance of whistleblower protection and care more about their constituents than other influences.

<comment><USER14> "In fact, some whistleblower protection laws already exist.."

This is true, on paper. But the other reality is that states have passed laws (or tried to) prohibiting the sort of investigative behavior that leads to whistleblowing, and a lot of companies have in-house rules against it. No cameras and cell phones in your work area, for starters. This sort of thing makes it easy to dismiss the paper laws. I imagine that if they are violated, one must sue to get it worked out.

Yes, it is theoretically true we could get to a point where we elect folks that understand the importance of them, or at least have the wherewithal to listen to experts in the field. And care more about their constituents. Unfortunately, I think we are a long ways off, and those very people - <USER2> ones that care less about the constituents and have little grasp of the importance of such laws - have made high entry hurdles for the ones that _would_ care.

</comment></comment></comment></comment><comment><USER15> This statement is very interesting, it is very well written and researched, and it uses less common words. I wonder if there was some level of editing/embellishment done to it.

Might as well be a non-native speaker. It also seems to me that the author might have had some personal reasons to target MF

But to be honest, I don't think Income Inequality is one of the most defining issues of our time, through human history, inequality, not only monetary, but cultural and intellectual has usually been higher.

<comment><USER16> It's known that the editors working on the panama leaks are mostly Germans, and that Mossack Fonseca mostly targeted German customers, too, (and had mostly German employees).

So it might very well be a case of a German whistleblower, and a German editor.

</comment></comment><comment><USER17> "issues involving taxation and imbalances of power have led to revolutions in ages past."

Given that all the checks and balances have failed, I don't see very few other options.

<comment><USER18> Electoral reform. Can't right the ship without being at the wheel.

</comment></comment><comment><USER19> And yet... why no American names in the leaks? Funny that.

<comment><USER20> It is a legitimate question but it could simply be the case that the US has laws that made it impossible for Americans to exploit the services of this particular law firm.

<comment><USER4> It's the other way around. American's can more easily get the same services on-shore, for instance, by setting up shell companies in Delaware.

</comment></comment></comment><comment><USER21> There will never be better protection for whistleblowers. That is just not in the interest of those who make the laws nowadays. What will come are new laws which will make the publishing of whistleblower material illegal. The EU for example is currently working on a law for better protection against theft of trade secrets. Which such a law in place no newspaper would risk anymore to publish something like the panama papers.

</comment><comment><USER22> Question for John Doe. Where's the missing American names and other retracted elements. Further given his naive calls for governments to do something to fix the problem seems naive to the extreme since they were the ones who created the 'problem' or backdoors in the first place

<comment><USER23> How do you know there are missing American names? It was my understanding when this leak happened that Mossack Fonseca didn't have that many American clients, because Americans still have to pay tax if they try to hide their money in Panama. Admittedly, I have no clue what I'm talking about - but I'm quite curious how it is you know that Americans are being overtly excluded from this action?

</comment></comment><comment><USER24> >> "Source known only as John Doe says income inequality "one of the defining issues of our time" and calls on governments to address it."

Calling on any government to create change, not it citizens, is a mistake - especially on a topic like this.

<comment><USER25> That's a distinction without a difference considering that, in most democratic countries, the government is nothing but a segment of the overall population appointed as their legitimate representatives.

It is the government responsibility, in their mandate as representatives, to address the problems perceived as important by the citizens (as, for instance, the problem mentioned in the quote).

Now, if the government is unable or unwilling to use the power it was given "by the consent of the governed" to address such important issues then __that__ become "one of the defining issues of our time", not income inequality or any other problem derived from that.

<comment><USER26> > It is the government responsibility, in their mandate as representatives, to > address the problems perceived as important by the citizens

But it is the citizens' responsibility to make clear to their representatives the problems they consider important. For a couple of examples of this principle in action, see the civil rights movement of the '60s, or the marriage equality movement of the noughts and teens. No politician would've dared approach those issues until the citizenry made its voice heard.

As a rule, on issues that we as peoples consider important, we lead our politicians, they do not lead us. This is partly because of the incentives in play in modern democracies and partly because, as you say, they are merely our representatives, not our dear leaders.

</comment></comment></comment><comment><USER27> The revolution might be digitized, but if it does indeed happen, I'm afraid it will also be bloody as hell, as almost all revolutions are. I suppose some things never change.

</comment><comment><USER10> I think their upcoming release is more interesting:

 _The International Consortium of Investigative Journalists will release on May 9 a searchable database with information on more than 200,000 offshore entities that are part of the Panama Papers investigation._

 _The data [...] includes information about companies, trusts, foundations and funds incorporated in 21 tax havens, from Hong Kong to Nevada in the United States. It links to people in more than 200 countries and territories._

</comment><comment><USER28> more and more this feels like a big smoke screen to keep sheep walking in circles. german publications are losing readers in droves. they desperately need to capitalize on a good story.

there's absolutely no real news here. but suddenly "democracy's checks and balances have all failed".

come on.

</comment><comment><USER29> This seems to be long enough for a statistical attack to narrow things down to ~100 people.

<comment><USER30> Rather. Possibly far fewer.

</comment></comment><comment><USER31> Why can't I donate bitcoin

</comment><comment><USER32> The tone of the statement is very anonymous in nature. Cold and factual. I wonder if this is how its author normally express theirself or if they've written in this style in order to defeat authorship attribution.

</comment><comment><USER33> I am impressed at how well it's written.</comment>


Speeding Up Compilation Time with Scalac-profiling - <AUTHOR> https://www.scala-lang.org/blog/2018/06/04/scalac-profiling.html <comment><AUTHOR> Happy to take any questions, by the way!

</comment><comment><USER2> Surprise, surprise, implicits turn out to be the Achilles heel of Scala.

<comment><AUTHOR> No, non-cached whitebox macros are.</comment></comment>


What if your Web startup attracts users without computers? - <AUTHOR> http://www.jasonglaspey.com/2011/01/28/paleo-plan-and-customer-service/ <comment><USER2> Now THAT'S customer service ... all I can say to Jason is congratulations as he's proven to be both compassionate and resourceful. It doesn't really matter where his entrepreneurship takes him. He's already made it as a human!

</comment><comment><USER3> This is a good reminder for us tech-savvy people: there's a HUGE untapped market of people out there who either don't have a computer or barely know how to use one. More importantly, they're not stupid or unworthy of your time -- they often need the same tools and information that we do, only simplified and streamlined.

<comment><USER4> Can I add that, if you go after these markets, rather than competing against web-savvy startups and Microogle you'll instead be ROFLstomping grossly inferior solutions which are offered by companies with no interest or expertise in web marketing?

<comment><USER3> Why not do both (techies and non-techies)?

</comment></comment></comment><comment><USER5> "Google, in response to the internet blockade in Egypt, says it has created a way to post messages to microblogging service Twitter by making telephone calls." <http://www.abc.net.au/news/stories/2011/02/01/3126626.htm>

</comment><comment><USER6> The tablet and mobile will be the so-called last mile for the web Ludites. The net book brought a huge influx of seniors to the web because they wanted to connect with their kids and grand kids. Verizon noted them as their biggest buyers of net books. Why? because they did not have to install or connect anything. It just worked..for the most part.. and it was affordable. They used it like an appliance, for one or two main things. Tablets and mobile are going to fill in the rest of the gaps and bring the rest, happily onto the web as long as we keep it simple. Apps should be short for Appliance. Think about Apps as a solution to this market. Apps are 1.0 right now, 2.0 is coming soon. Huge opportunity for the greater masses with Apps.

</comment><comment><USER7> Catering to the non-computer crowd; this is a fantastic idea. We spend so much time hooked up to computers and aiming for that younger and business crowds that it's easy to forget there is a large, large demographic who have just as many (maybe more) problems as we need solved and just as much money to pay for it from their kids or retirement funds.

How do you market to these folks? For me the first thing that comes to mind is infomercials but I don't really want to get into them. (Cue mental images of Sham-wow). Older folks like to read print newspapers, too. Can anyone suggest some good ways to tap this market?

<comment><USER8> Depends on your market. You could go for print advertising with a publication that reaches your market (geographical, interest). You could try mail marketing. Don't forget billboards, posters which, of course range in size (and associated cost) from A5 -> huge.

Try spending a day noticing all the advertising you see. It will give you some ideas of your own.

</comment></comment><comment><USER9> I know someone with an informational website that sells a dead tree version online, without the updates. It is kind of a snapshot in time of what was on the site at some point. (Or did at one time. I have just taken a quick look at the site and I am not currently finding a means to buy the book version.)

<comment><USER10> That reminds me of the seemingly interminable Wikipedia 1.0 project. With that, the value comes from the incredible collection of knowledge being put down in physical form, not from its utility. I wonder what the value is in seeing snapshots of an ongoing and developing thing like a website, and how well it stacks up against the "real" thing.

<comment><USER9> Perhaps it didn't stack up and perhaps that is why I am not seeing the book option anymore. It was, sadly, the only form of monetization the site had (that I am aware of).

</comment></comment></comment><comment><USER11> Well sometimes you have to transfer your online business to real world. This is good example of how to do it.

</comment><comment><USER12> When I ran into this problem i simply developed a phone and txt based system for people to get meal recommendations etc... Not only did this allow me to help users that did not have computers but it seems that the majority of users prefer the txt , phone interface because its easier. If you need more info on how I implemented this in a user friendly way just email me; I would be glad to help.</comment>


Ask HN: A way to view the top rated posts/comments of all time? - <AUTHOR>

I just thought that this would be a really useful feature. Or, is there a hidden way to do this already via hnsearch.com, for example? <comment><USER2> Not necessarily of all time, but at the bottom of nearly every page is "Lists":

<http://news.ycombinator.com/lists>

That gives you:

        leaders         Users with most karma.
        best            Highest voted recent links.
        active          Most active current discussions.
        bestcomments    Highest voted recent comments.
        noobstories     Submissions from new accounts.
        noobcomments    Comments from new accounts.

You can ask for items with more than a certain number of points:

<http://news.ycombinator.com/over?points=400>

You'll probably find that still only pulls things that are currently in cache.

<comment><AUTHOR> I've seen lists... but unfortunately those only have "recent" posts.

<comment><USER2> To the best of my knowledge that's the best you can do. My understanding from snippets I've picked up here and there is that older items are on disk and not searched for these sorts of questions. To do so would be a significant increase on load on the server, I'd guess, especially if it became popular.

</comment></comment></comment><comment><USER3> May not solve what you are asking, but you can check weekly and monthly newsletter or magazine. People are putting efforts to give you best. <http://www.hackernewsletter.com/> <http://hackermonthly.com/></comment>


Ask HN: Why are subreddits so toxic? - <AUTHOR> Has reddit always been so toxic? Should subreddit moderators have stricter codes of conduct, especially those that are more "industry/tech/profession" oriented? I'm from the "tech" industry, and I was surprised to see that tech subreddits are also fairly _arbitrarily_ toxic instead of the negging-but-constructive type comments that pass on HN.
Or is that entire idea just against reddit? HN is a more moderator based forum, for example.
What are your thoughts on community censorship? When do things cross a line? Or should it just be a subreddit owner/moderator based thing and let everyone just figure it out... <comment><USER2> I think popular subreddits tend to support more toxic behavior, especially those easily accessible by keyword and more so the defaults. They invite anyone and everyone to post. "Niche" subreddits tend to be smaller groups and seem more self-moderated. Sometimes you can find these in the sidebar of the subreddits you're interested in.

Reddit definitely didn't start this way. In many ways it was similar to HN in that it was a small group of technical users, but it grew into a public forum for all topics. The demise of Digg was another factor but is arguably small considering Reddit's popularity today.

Reddit is an immensely useful resource if you know what you're looking for.</comment>


Surprising Economics of Load-Balanced Systems - <AUTHOR> http://brooker.co.za/blog/2020/08/06/erlang.html <comment><USER2> This can be flipped into a simple heuristic known as the "square root staffing rule" (sometimes "... law"). The name comes from call centres.

Basically, the number of servers you need to serve X amount of demand with Y probability of queuing is not linear with X. It is proportional to the square root of X.

The intuition is that a call centre agent is either talking to a customer, or they are not. If they are not talking to a customer, then they can immediately serve a customer. The odds that a given agent is talking to a customer is on a probability distribution, usually assumed to be the normal distribution.

In a normal distribution, larger samples lead to more and more appearances of outliers in the sample. More agents means more "outliers", in this case, more idle agents. And that curve is not _linear_ , it follows the shape of the normal distribution.

Where it sucks for call centres is that demand shows seasonality and shift planning has relatively inflexible lead times. But in a software scenario we can typically acquire additional capacity quickly, so precision in forecasts is less of a problem.

A decent explanation: [https://www.networkpages.nl/the-golden-rule-of- staffing-in-c...](https://www.networkpages.nl/the-golden-rule-of-staffing-in- contact-centers/)

A short scenario: [https://www.xaprb.com/blog/square-root-staffing- law/](https://www.xaprb.com/blog/square-root-staffing-law/)

A simple calculator linked from the short scenario: [https://www.desmos.com/calculator/8lazp6txab](https://www.desmos.com/calculator/8lazp6txab)

<comment><USER3> You can plan deferred maintenance and other tasks around seasonality.

For instance at a call center, the cost of training new people is not something you want to pay while the phone is ringing off the hook, right? So you hire too many new people during the preceding lull, train them up, and hopefully the washout rate is low enough that you meet your hiring quota.

Similarly, if you own your own servers, you provision new hardware well in advance of a predictable spike in traffic, and then you keep as many of the old machines running as physics (eg, building thermal or electrical capacity) or information theory allows, but you prioritize the new hardware because it scales better vertically. Once the hype wears off you decommission them.

Blizzard reportedly used to do this for their MMO. Later they bragged about architecture changes they made to increase capacity during peak load, but that's not what I saw. What I saw was how much they could _decrease_ capacity as active session counts declined. When they started, they sharded their system. Then added more and more shards, availability zones, and regions. And then that wasn't enough and too much at the same time.

Shards are probabilistically fair _if the incoming traffic doesn 't know about them_. Blizzard named their shards, and so user clustering happens. So they slowly moved a bunch of jobs to be serviced by a separate cluster of servers, and shared those workers across multiple shards in the same AZ. Now instead of each shard handling its own peak load, the hardware is more proportional to the peak load of the AZ, which is going to behave more like root sum squared (your square root) of the shards. If one shard declines or spikes it takes from the pool.

If the pool in undersubscribed, you shut off servers to save money. But you never shut off the servers that the customers know about, and have developed an attachment to. That would cause a cascading failure.

<comment><USER2> Seasonality is worth planning for, but the value of investing in plans for seasonality depends very much on how quickly you can react to changes in demand. Building data centres has a long lead time and costs a lot of money, so spending a lot of money on forecasting demand is a worthy investment. On the other hand, a new container process might take a few seconds to load, so you can get away with more error.

There's one caveat, though, where planning for seasonality becomes valuable again: competition for shared resources. Suppose you are using a cloud provider to provision VMs on demand. If you're caught short by a stockout, then fast reaction time now becomes a long reaction time. Setting an approximate or safe base level of capacity on a schedule increases the odds that you won't be caught dangerously short.

(The Blizzard observations were interesting, thankyou).

<comment><USER3> > If you're caught short by a stockout, then fast reaction time now becomes a > long reaction time.

I've seen a little evidence in Netflix's public statements that they have a degree of load shedding. They've reserved enough machines to service all high priority traffic, plus a couple of layers of buffer, and the buffer gets parceled out for lower priority tasks as space is available, and buy either more base load or more spot servers when that queue gets too long.

But... we have the cloud because people have been doing a simplistic version of this for decades in private data centers, the person in charge gets build- out fatigue, possibly before you were even hired, and you practically have to harass them to get enough hardware, or give up and use more man-power to do the same work a computer should be doing, while slowly you lose all respect for the company.

Cloud is already built. You just show up and pay 140% of the amortized cost of building and running the system, instead of the up-front cost. Now that person is either easier to talk to or you can just go around them entirely.

As I saw someone say recently, "Love is not the most powerful force in the universe, it's spite."

Where was I? Oh yes. So we _can_ do this, but to an extent the people who manage to pull it off are remarkable, because so often it doesn't work out, or they succeed in silence and go back to what they really were supposed to be working on now that they've figured out how to get more out of their meager capacity.

</comment></comment></comment></comment><comment><USER4> If we were to extend the first graph a bit more to the right, the linear improvement would quickly trend downwards into negative latencies, a sure sign that it's not the right answer. But if linear is impossible, then super-linear is just as impossible. The line the author describes as such is clearly asymptotic.

If we were discussing overall latency, and not just queue delay, then an asymptotic decrease towards zero could correspond to a super-linear increase in throughput capacity. But that's not what's happening here, either: because average latency is bounded below at one second, total throughput can never exceed _c_. There is no super-linearity to be found here.

<comment><USER5> Right, 'asymptotic' is a better description of the behavior. I've updated the post.</comment></comment>


100,000 Users And So Can You (A history of Carbonmade) - <AUTHOR> http://spencerfry.com/2009/10/21/100000-users-and-so-can-you

<comment><USER2> Someone asked why we haven't updated our homepage in 3 years (and the significance of that) and if I could speak to that. The comment got deleted before I could respond, so I'll just create a new thread.

It's partially covered in the article. The first year, we were still do consulting work, so we barely spent any time on Carbonmade. And then in the second year, we were working on a second product that never saw the light of day. And now in the third year, we're focused on Carbonmade exclusively and nearly ready to release the new version.

Here's what I learned from running a company for three years without any update to the product: Excellent customer service could be the most significant thing you can do. (Granted, you need a pretty solid product.) During the first year we were really bad with emails and responding to our customers around the Net. We practically ignored everyone and everything. We just didn't have the time. Then in year two, I began to take customer support very seriously. I'd respond to peoples emails within minutes (when I could) and within hours 90% of the time. You literally can take a look at our graphs and see a HUGE uptick in signups/paid sign ups/customer retention within a month of responding to emails.

I also went back through at least 6 months of old emails that we had simply archived away and responded to everyone explaining how sorry we were for not giving them good support and answering their questions. I wrote everyone individually and got a lot of "it's no problem, I'm happy that you're spending the time to do so now." It really impacted our brand a lot and we saw a lot of people begin to respond with emails like: "Wow, you guys responded faster than I've ever been responded to before. You're so great. Thanks so much for your service. I'm telling all my friends."

Also, around this time I chose to treat all customers equally no matter if they were paying or if they weren't. I figured -- and you'd think this was obvious -- that spending a few extra minutes to treat free customers well will make them more likely to upgrade and actually costs less (in time and money) than going out and finding new customers. This turned out to be very successful. I can point to hundreds of emails from free customers that upgraded after hearing back from a "real person who was kind and helpful".

Good customer service is king and really turned our product around.

(Thanks for reading my rant.)

<comment><USER3> "Excellent customer service could be the most significant thing you can do."

Amen. At Wikispaces we absolutely obsessed over fast, human responses to support questions early on -- and still do. People are regularly shocked to get a response at all, let alone from a person, let alone in the span of a few minutes. That email can be the difference between getting frustrated and leaving for good vs. being a long term site user.

<comment><USER4> Its actually a relative minimum of effort to respond very quickly - drastically reduces the number of outstanding issues to track. Ignoring tracking, it takes the same total effort to answer NOW as a week from now. Now I wish my garage mechanic/doctor would operate this way.

<comment><USER2> Yeah. It's funny. As soon as you start doing it, it's really not that hard to keep up with.

</comment></comment></comment></comment><comment><USER5> > _Creative people are likely to be friends with other creative people who > turn out to need portfolios themselves, and this creates a rapidly expanding > circle... our users derive a direct benefit from showing off their > portfolio, which in turn is free marketing for our service._

If you're a startup looking to get noticed online, I'd suggest taking this to heart. Marketing is much easier if your users have a strong incentive to market your product for you.

> _We don't subscribe to the "Release Early, Release Often" philosophy. > Admittedly, we're perfectionists to a fault, but when you've got 158,000 > pleased users, you can't simply disrupt things with a series of bells-and- > whistles updates just to follow a silly motto._

I don't think that philosophy is meant to apply to applications with hundreds of thousands of users. It's meant for the early product development phase when you have very few users. You can, however, do bucket testing, like Google and other larger players, and release updates to a tiny portion of your users to see how they react.

<comment><USER2> "I don't think that philosophy is meant to apply to applications with hundreds of thousands of users. It's meant for the early product development phase when you have very few users. You can, however, do bucket testing, like Google and other larger players, and release updates to a tiny portion of your users to see how they react."

Totally agree with you there. That's actually something I really want to start doing after we release our new version. We've never done any type of A/B testing and it just makes sense.

On a similar note, I was talking with my friend Eric Friedman last night about this very topic. He suggested creating small buttons that people could click on, but weren't actually active and gave a "Coming Soon" message and then tracking the frequency in how often they were clicked. You could release these little buttons to 2-5% of your users and then see how they performed. And then build features around the buttons that were clicked more frequently.

</comment></comment><comment><USER6> This is a GREAT read. I love stories that get into the nitty gritty details. Sort of like Founders at Work. 100k users don't come out of nowhere. There's a lot that causes that and lots of tinkering. Figuring it out is great.

</comment><comment><USER7> Thanks for this piece Spencer. Very inspiring indeed.

I really like the "Give the demo a try" feature on the landing page. Could you give any insight into whether this see-for-yourself feature really aided in getting signups..?

I was thinking about doing something similar for my site, which is why I'm asking...

Thanks and best of luck on your continued growth.

<comment><USER8> @fjabre. We initially introduced the demo to help cut back on people signing up to see how the product worked with no intention of using it. Oddly enough, it actually reduced our total number of signups. But the users that make it through and signup are much more likely to be long term active users.

<comment><USER7> Ha. Didn't think of it from that angle.. but that makes total sense.. Thanks!

</comment></comment></comment><comment><USER9> Just saw this from 37s: [http://37signals.com/svn/posts/1976-launch-haystack- a-better...](http://37signals.com/svn/posts/1976-launch-haystack-a-better-way- for-web-designers-to-find-clients-and-for-clients-to-find-web-designers)

</comment><comment><USER10> Font looks very odd on my Vista machine, but I am a happy user of Readability: <http://lab.arc90.com/experiments/readability/>

</comment><comment><USER11> The white on black overlaying everything just grates on me for some reason. Otherwise very nice.</comment>


Three Painless Ways to Drastically Reduce Technical Debt - <AUTHOR> https://medium.com/teamcodestream/three-painless-ways-to-drastically-reduce-technical-debt-c00b4b724d59 <comment><AUTHOR> Would like feedback on the premise that over-communicating within the coding environment can lead to better code with less debt.</comment>


My first program was binary search - <AUTHOR> http://melitamihaljevic.blogspot.com/2012/06/my-first-program-was-binary-search.html

<comment><USER2> Conway's 'Game of Life' in FORTRAN on an IBM 360. Hardly the third grade, that would have been in 1955. My next program was a 'Key Words in Context' generator written in BAL, likewise for a 360 (370 by the time I finished[model switch over should give you a clue as to when this was :)])---haven't stopped since and still continue to enjoy the creative addiction that programming brings to the game.

<comment><USER3> The 360 was delivered in 1965. Conway's Game of Life was devised in 1970.

I'm pretty sure it wasn't 1955. :-)

<comment><USER4> He was in the 3rd grade in 1955. He wrote his first computer program sometime after 1970, on a 360. Therefore he did not write his first computer program in 3rd grade.

<comment><USER3> Ah, yes. My reading comprehension was slow that night :-).

</comment></comment></comment></comment><comment><USER5> It wasn't quite my first program, but during one of my introductory high school programming courses I independently derived binary search (though of course I didn't realize it at the time). Additionally, this was well before I discovered what recursion was (I wonder if anyone else vividly remembers having their mind blown the first time they were introduced to recursion).

The funny part is that I abandoned the binary search strategy nearly immediately after conceiving it. Why? Because I realized that the array needed to be sorted in order for the search to work... and, at the time, the only sorting routine that I knew of was bubble sort. :)

</comment><comment><USER6> Go FER, go FER, go girl!

</comment><comment><USER7> I'm not seeing a blog post.

I'm using Chrome 18.0.1025.168 (Developer Build 134367).

<comment><USER8> Same. I had to turn NoScript off.

Why the author chose to require javascript for this blog is beyond me. :/

<comment><AUTHOR> I choose dynamic look via Google blogger. It's probably why you cannot see it.</comment></comment></comment>


Ask HN: How do you get recruited? - <AUTHOR>

It seems like a common theme when I read threads about jobs/pay:
1) Most people agree that $40,000 a year is barely 'minimum wage' when it comes to programming of any sort.
2) Recruiters are always bugging them.
3) Right now is the 'hot market' for developers.
So what's a developer to do if they are making 'minimum wage' in the programmer's Bull Market? How does one get so lucky to have the problem of people calling them up for better jobs? <comment><USER2> Last november I was sitting around with time on my hands and decided to update my LinkedIn profile. I wrote a long-form, pseudo-essay resume. A week later I got a call from a recruiter who hooked me up with one interview. Another week after that I got called by another recruiter who put me in touch with 12 other companies. I got offers from three of them and I think I would have gotten a fourth if I hadn't accepted one of the first three before getting through the second interview.

I didn't think that my resume was all that special having worked for DC area government contractors my whole career, but everyone was super excited to talk to me and I went to second or third interviews for 11 of the 13 places I spoke with.

So...update your LinkedIn profile and give a lot of data. Get a github account (honestly, I had basically nothing there and still don't, but it's a good thing to be able to point too), and try and get active in some local development groups/message boards/newsgroups.

Good luck!

</comment><comment><USER3> Just happened to a few friends of mine. They were pretty "anti-social-media sites", but I convinced them to set up a LinkedIn profile (as public as I could get them to set the privacy settings) and an about.me page... and set up reciprocal links between the two. They fully fleshed out their profile on LI including the technologies in which they are well versed. They happened to live outside of a hot market, but were looking to move to one, and they set their location as the hot market. Within a few days, they had a few recruiters that had reached out and several job interviews lined up or in progress. All this happened within the span on a couple of weeks. I'm not saying it's a recipe for success for everyone; the people I'm talking about are seriously amazing developers, with skill sets that reach far beyond just writing code.

Edit: I'm not a recruiter, but I'd be happy to take a look at your LI/A.me pages if you decide to set them up and give you feedback.

<comment><AUTHOR> Don't have an About.me, but here's my LinkedIn. Definitely appreciate any tips: <http://www.linkedin.com/pub/joe-cortopassi/24/76b/5b9>

<comment><USER3> So, at first glance (sorry pressed for time right now), there are a couple of spelling/grammar issues you should tackle. More importantly, there is no description for what you currently do at Trinnovations. You list some of your specialties, which is great, but there's no context on how you use/used them in your previous/current jobs.

I would set up an about.me page and link to your LI. Link out from LI to your about.me. Start blogging if you think you can fit it in. Even if it's filled with entries about what you learned/did/achieved/figured out that day or week. The more technical the better.

And, as was mentioned in another comment: You're close to a hot spot already. Close is stretching it considering the drive from IE to Santa Monica, but, do it anyway. Do it on a Saturday. Go to Coloft in the morning, get a day pass, and work on something. Take breaks and meet others. Not sure what to break the ice with? Try something like "Hey, so is coloft always this energetic (/vibrant/quiet/boring/lively/productive/whatever)". Ask what they're working on. Listen. They'll ask you the same.

Don't let yourself make up excuses. You're close enough to a really hot market where companies are climbing over each other for talent. If you want it, make it happen.

</comment></comment></comment><comment><USER4> How good are you man? Your profile says Objective-C. Get out there and show some stuff! Startup Weekend, Hackathons, local meet-ups, pick an open source project for something and contribute, etc... BE SEEN in the REAL world.

Aside from the above: Do you have friends in the biz? Referrals are the best way to real jobs, IMHO. Do you respond to the monthly posts here on HN where people are seeking expertise? Where are you located?

EDIT: To answer your question: You get recruited by being good. You also need to be visible. Create value and you will be found!

<comment><AUTHOR> Zero friends in the business (besides the people I work with). Getting into programming came out of the blue for me.

Edit: For reference, here is a little weekend iPhone project I just started - <http://www.youtube.com/watch?v=bF1KcbNjZ7Y>

<comment><USER4> Seriously? You're in freakin' LA (or near enough to count - from your LinkedIn) - get your ass out of your house and in the streets. Go to Santa Monica. Go to CoLoft. Go to LASW (la.startupweekend.org). Go to any number of the numerous meetups out here. Get GOOD and get FOCUSED. You, my man, have no excuses! Not trying to be hard, but am being real. I would have thought you were from some remote area or something, but rather, you're right in the middle of the hot spot. Be seen.

<comment><AUTHOR> Honestly didn't know LA was a big scene. It's a good 2 hour drive to get out there from where I live (Temecula/Murrieta) area, but if it's as strong of a scene as it sounds like, maybe I need to start going out there.

<comment><USER4> Joe - there is a ton of opportunity near you - San Diego, Orange County, LA...I suggested Santa Monica because it's a hot spot of hot spots right now - at least for start-up oriented people. I mentioned CoLoft, because the owners are fantastic and have a lot of great events going on. Go to them and check it out. They have a website - get involved in a meeting or two. I know the drive is long, but it will ultimately be worth it. You may have to endure it for a few months, maybe not so long. The investment of time, if you can get from point a to point b, is worth it.

I looked at your iPhone app (via YouTube). Why is this not released yet? What's holding it up? It's a basic app, but it demonstrates you can write an iPhone app - and that market is REALLY hot, EVERYWHERE. You should be able to get remote gigs as a contractor pretty easily. Your best bet is to release it and then promote yourself. Release it and then post back here that you released it. You'll get good feedback (like: left align the categories, right align the totals, or something like that).

You really do have a lot of opportunity all around you. I was a bit insensitive in my prior post. For all I know you CAN'T get out of the house. Shame on me for that. If you need help, let me know. I will not hold your hand and give you every single detail, but I will provide enough info or suggestions to get you moving in the right direction so that you can get in front of the opportunities if you're willing and able.

<comment><AUTHOR> Don't even sweat the comments, I didn't take it as offensive at all. If anything it was super encouraging. I guess I kind of assumed that I was missing out by not being in the Silicon Valley area, but it sounds like I have more opportunity than I thought.

The iPhone app is close to being done, but I feel like it's missing a few things to be a true MVP (ability to export data). It's honestly just been a weekend project (<10 hours), that I made to help my wife and some of her friends. Sounds like I need to get it up ASAP though, if for no other reason than a resume builder that shows ability.

Appreciate all the feedback. It's hard learning in a vacuum, and it can be refreshing to get a fresh perspective.

</comment></comment></comment></comment></comment></comment><comment><USER5> I submitted a 3 line patch to Tornado and got an email a few months later from a Facebook recruiter. I'm fairly certain they would never actually hire me but if you're not getting any emails from clueless recruiters you must really be hiding yourself from the internet. Get out on GitHub and get involved in stuff.

Edit: If you're even mildly competent in the things you listed on your HN profile then you'd be one of the top 10 candidates I've interviewed in the past couple years for "PHP developer" positions.

</comment><comment><USER6> $40k doesn't mean anything. What does your market look like, and how much experience do you have? Depending on what part of the world/country you live in, how much experience you personally have in your specialty, and how hot your specialty is, that might be normal.

As for actually getting better offers, online social media is great, but also try going to local user groups, conventions, and the like. A person can better judge you than a keyword search engine can.

<comment><AUTHOR> Apparently, from reading the other comments, I'm within a couple hours drive from a hot market (LA). But I like what you mention about local user groups and such. Do you know of any tools for finding those kind of groups/meetings?

</comment></comment><comment><USER7> Being in Temecula doesn't help much since there's hardly any tech companies out there. You are closed enough to LA that it's worth making a drive once a week to go to a tech meetup or work out of CoLoft to expand your network. Surround yourself with other smart people in the industry. The worse thing that can happen is that you learn something.

</comment><comment><USER8> For me (many years ago) it really helped to have my résumé posted online and linked from the site of the free software that I was working on. Sometimes these things are noticed.

<comment><AUTHOR> Have my LinkedIn profile setup, but I guess I could put my resume up as well. Do you mind me asking what software you were working on? Was it an open source project?

<comment><USER8> Open source, yes. It was a terminal (MacTerm), it happened to be used by a hiring manager at a company.

I don't know that any employers ever looked at the source code. But publishing the source probably helps because they can see what your programming is like right from the start.

</comment></comment></comment><comment><USER9> Network.

Be awesome.</comment>


Brainless civil servant amazes doctors - <AUTHOR> http://www.theregister.co.uk/2007/07/23/french_no_brainer/ <comment><USER2> According to HN guidelines, please post the original source. The Reuters article that this article refers to is:

[http://www.reuters.com/article/2007/07/20/us-brain-tiny- idUS...](http://www.reuters.com/article/2007/07/20/us-brain-tiny- idUSN1930510020070720)

Note: from 2007.</comment>


The ghetto of the mind - <AUTHOR> http://www.quirkey.com/blog/2009/04/27/the-ghetto-of-the-mind/

<comment><USER2> _You can not fault an entire community for the actions of a single individual._

You can when prominent members -- leaders, even -- of that community step up to defend those actions.

 _In any community there will be opposing opinions on definitions of moral correctness for n issues._

You can find plenty of Americans who support discrimination on the basis of sexual orientation. When it comes to moral issues there are always two sides. Often, one of those sides is wrong.

 _If you actually want to change the community, you have a better chance of influencing the future community through education then the current community through preaching._

If you want to change inappropriate behavior, you have to call it out. Staying silent doesn't do much good. Good people do bad things all the time; they won't learn the difference unless someone explains it.

</comment><comment><USER3> This is completely tangential, but Hackety Hack was mentioned. HetyH was apparently supposed to be "released" on March 7th; whatever happened to that? I'm sure it was delivered at Art & Code; why wasn't it put online?</comment>


China has become a major player in AI - <AUTHOR> https://www.nytimes.com/2017/05/27/technology/china-us-ai-artificial-intelligence.html <comment><USER2> This trend is also evident in conferences like ICML, CVPR and EMNLP.[1] I have a semi-popular open source ML project [2] and the repo gets more Chinese visitors/users (from posts on Weibo/WeChat etc.) than all other countries including USA.

I think Chinese researchers & government have a key advantage in collecting enormous datasets, e.g. I wouldn't be amazed if they end up with state of the art models for problems such as object-detection/segmentation for videos taken from Vehicles, Medicine/Radiology and face recognition [3]. Several US strategic funding agencies seem to be pursuing projects with this implicit assumption.

While american government agencies do collect data at a similar scale (Body Cams, Police dashcams), inefficiency in data sharing significantly complicate their use.

[1] [https://medium.com/@karpathy/icml-accepted-papers- institutio...](https://medium.com/@karpathy/icml-accepted-papers-institution- stats-bad8d2943f5d)

[2] [http://www.deepvideoanalytics.com/](http://www.deepvideoanalytics.com/)

[3] [https://github.com/seetaface/SeetaFaceEngine](https://github.com/seetaface/SeetaFaceEngine)

On an unrelated note: If you have a popular Github repo, you can view the Google Analytics style stats under graphs/traffic for referring urls/unique- visitor counts etc.

<comment><USER3> This is not news to anyone who reads AI papers: the number of Chinese authors is amazing.

<comment><USER4> Crazy thought: what if the AI is in 10-20 years the first field in which major breakthroughs will be first published in Mandarin instead of in English. Language is power, and currently English speaking countries have the edge.

<comment><USER5> Those top-rated Chinese researchers are publishing in English. Their Chinese- only venues are not at the caliber that they would consider publishing ground breaking work to.

<comment><USER6> Yet.

<comment><USER5> Critical mass is a weird thing.

</comment></comment></comment></comment></comment></comment><comment><USER7> This is a simple function of the number of tinkerers available.

Any ML project I've worked on has been a lot of tinkering - not unlike the sort of craftsmanship one associates with woodworking and blacksmithing.

Tinkerers gave us Google, Facebook and of course modern civilization.

Tinkerers need the Medici family equivalent. Silicon Valley has that in spades. The Chinese graduate students I met and knew brought this tinkerer's spirit and the American university atmosphere allowed them to unleash their potential (check out who features in the author lists of CMU, Stanford, MIT's papers at NIPS, ICML etc.).

That state of mind and not a top-down approach is what we need.

More Ben Franklins, Robert Hookes, and Issac Newtons; fewer 5-year planning commissions.

</comment><comment><USER8> Reading this article gives me the feeling that new technologies simply cannot get off the ground without central government planning and billions of dollars in taxpayer spending. I guess it's up to the Chinese government to decide whether China wants to keep testing this hypothesis over and over against all odds.

<comment><USER9> That's literally the thesis of "The Entrepreneurial State: debunking public vs. private sector myths" by Dr. Mazzucato.

I wrote a summary if you're interested: [https://medium.com/@seanaubin/book- review-the-entrepreneuria...](https://medium.com/@seanaubin/book-review-the- entrepreneurial-state-8fa47c4e8056)

<comment><USER10> There are anti-theses for the issue. Two criticizing the book mentioned:

[https://mises.org/library/government-spending-innovation- tru...](https://mises.org/library/government-spending-innovation-true-cost- higher-you-think)

and

[http://pricesandmarkets.org/volume-3-issue-3-winter-2015/mar...](http://pricesandmarkets.org/volume-3-issue-3-winter-2015/mariana- mazzucato-the-entrepreneurial-state/)

<comment><USER9> I wasn't totally convinced by the first linked essay. The primary arguments presented seem to be:

\- Government funded development only gives the government what it wants while the citizens remain unsatisfied. This is due to a lack of "market tests". \- We never tested the alternative where the government doesn't fund research and let private companies innovate instead.

Dr. Mazzucato addresses this in her book. She notes that the government needs to provide a pipeline from basic research to marketability. She also argues that the last twenty years, in various countries, have shown that trusting private companies to innovate doesn't give better returns. Also, I feel like Dr. Glein is over-simplifying Dr. Mazzucato's argument by claiming she argues that "many of the technologies and innovations we now value were produced single-handedly by government". Dr. Mazzucato routinely celebrates the ability of private companies to integrate innovative technologies for the public. She's mostly arguing that these companies should be taxed better (more efficiently? realistically?) by licensing the technologies.

However, the argument Dr. Glein links to ([https://www.jstor.org/stable/116937?seq=1#page_scan_tab_cont...](https://www.jstor.org/stable/116937?seq=1#page_scan_tab_contents)), is very interesting! It claims that R&D personnel are a finite resource and government funding crowds out the supply of talent for the private sector. I don't know why, but I thought the supply was elastic? Maybe because I over- idealize immigration?

The second essay seems to mostly re-iterate the importance of the "crowding out" effect. It also notes that Mazzucato got some data wrong because: "But in the thirties governments in the US did not fund long-term fundamental research, so companies did it themselves. Now that governments do fund long- term fundamental research, industry needs no longer do so: yet again Professor Mazzucato advocates the very policies that lead to the outcomes she deplores."

 _tl;dr_ people seem to be getting different conclusions from datasets I haven't seen and I need to do more research into the "crowding out effect"

<comment><USER11> > Government funded development only gives the government what it wants while > the citizens remain unsatisfied. This is due to a lack of "market tests"

As much as I consider myself libertarian, this is where they go off the rocker.

Basic research cannot be market driven. Especially a market that's mostly worried about next quarter

Most of what we consider fundamental inventions have started in academia and were snubbed by the market, with very rare exceptions

Funnily enough those that have invested in basic research have reaped lots of results (like the Minnesota Mining and Manufacturing Company)

<comment><USER10> Actually academia is not necessary dependent on Government. Universities can be private or privately funded and still do basic scientific research.

You can read an old piece on the issue from Murray Rothbard:

[https://mises.org/library/science-technology-and- government-...](https://mises.org/library/science-technology-and-government-0)

<comment><USER11> Yes but it's mostly funded by philanthropy and tax breaks.

Nobody does basic research thinking about a specific market issue

</comment></comment></comment></comment></comment></comment></comment><comment><USER12> I almost wonder if this was a ploy by the American companies. AI is being injected into all apps and even pushed as major feature, but, at least in my circle of friends, nobody cares or wants that. Most people use Whatsapp, with no AI. Google is trying Allo, to no success, and Facebook is pushing M, also to no success. I think AI is brilliant and could be the future, but from my experience, people generally don't want it in the ways it's being pushed. I definitely see the 'AI' and 'ML' markets, somewhat tied together, switching gears soon.

<comment><USER13> Facebook's M has had limited success, but on the other hand, News Feed, the core of Facebook's business, is build entirely around AI. And Google, of course, has had plenty of success with AI outside of Allo.

I wouldn't say "nobody cares or wants" AI, just that companies are still trying to figure out where it's best used.

<comment><USER12> You're right...kinda. AI is used in many places. Where it's advertised as a feature? Failure. Where it's used in the backend? People don't seem to have much of a choice. I stand by my statement.

<comment><USER14> Back in 90s, some Japanese manufacturers (e.g. Panasonic) tried to sell AI- based home appliances such as rice cookers or washing machines. The buzzword then was "Fuzzy Logic" instead of NN. Don't know how successful they were, but it seemed like a classic case of selling features instead of stories. Ordinary people don't buy stuff because of its features.

<comment><USER15> I think fuzzy logic controllers are very common in appliances. Samsung even has a neuro fuzzy logic washing machine that uses optical sensors to see how clean the water is. I think consumers need a fancy sounding technology on their appliances even if they don't know what it means exactly.

</comment></comment></comment></comment></comment><comment><USER16> I come from China, many people around me say that the reason of high proportion of Chinese authors in AI papers is many Chinese researchers just pour their "trash" into journals, it's a fake sense that China has become a major player in AI.

U.S. and the other English researchers have done most of the essential/landmark jobs in AI, Chinese authors contribute less.

</comment><comment><USER17> Chinese students are getting top tier educations in the United States only to be forced to return home because of absurd F visa restrictions which effectively prohibit them from gainful employment in the United States after completing their studies. It's pitiful and sad.

<comment><USER18> Don't know why you're being downvoted, because that exact thing has happened to many of my former lab mates. Giving a green card to anyone with a graduate degree in STEM fields would be the most sensible thing to do if the US wanted to avoid the reverse brain drain of foreign graduates educated mostly on the US taxpayer's money.

The vast majority of those grads would be very happy to stay in the US.

<comment> dis-sys That is not true and the reason is obvious - bamboo ceiling in American companies. Think about it - Chinese grads have far higher chance to be promoted to a senior position in companies like BAT and all of them pay you well.

<comment><USER5> The head of Microsoft today is Indian, Qi Lu was, until recently, head of apps, Harry Shrum runs tech. Does the bamboo ceiling really exist in tech?

Now, definitely for laowai working in China, there is a bamboo ceiling, almost all the senior leaders are going to be Chinese or Taiwanese :)

<comment><USER19> laowai = foreigner

</comment></comment></comment></comment></comment><comment><USER20> How many times is this title going to be changed? I've seen it changed at least three times so far this evening.

Titles should be locked to the article title to minimize confusion, imo.

<comment><USER21> I have to say the original title "Is China Outsmarting America in A.I.?" probably gives a better idea of what it's about than the modified versions.

<comment><USER22> We change titles that have trollish qualities and that has two: inciting nationalist emotion and bringing out the Betteridges.

</comment></comment></comment><comment><USER23> IMO, America will still be the leader in AI. The main difference between China and America is, China is more product focused, America is more math/logic focused.

</comment><comment><USER24> I think the term artificial intelligence is incorrect. What we're currently building is, IMO, artificial intellect.

I view intelligence as the capacity to use resources, including the intellect, wisely and creatively. The current system we're building have no self- awareness and no will, so they cannot really be intelligent.

If we want to build intelligent machines, we have to understand the architectures of the mind. We use memory as a sort of mental matter for the formulation of thoughts that we structure in a certain way to formulate ideas/concepts and then we go out and build things. We are builders because we are thinkers and we are conscious about being able to think.

But what is a thought, really? What is consciousness?

Look at the size of our brain next to the huge computers and data centers we're building for "AI". Our brain is small, yet was able to _think_ about how to build these systems.

We believe we're going to develop "artificial intelligence" by building massive computers and data centers. How absurd is that?

Folks, to build intelligent machines, we have to build _thinking_ machines and for this, we'll have to truly understand how the mind works and when we do this, I believe we'll be quite surprised.

<comment><USER25> > I think the term artificial intelligence is incorrect. What we're currently > building is, IMO, artificial intellect.

The rest of your comment contradicts this statement.

Dictionary definition of intellect is "the faculty of reasoning and understanding objectively, especially with regard to abstract matters". Dictionary definition for intelligence is "the ability to acquire and apply knowledge and skills".

Current AI has great difficulty in abstract matters/thought, let alone understanding something beyond simply a series of learned patterns.

> If we want to build intelligent machines, we have to understand the > architectures of the mind.

We have intelligent machines now, many which outperform human capacity for specific tasks. If your talking about strong artificial intelligence, then I wouldn't necessarily disagree, but maybe it could go the other way. By developing strong AI, we can understand the architecture of the mind. Maybe strong AI can be developed with the intellect of a cat/dog, and that gives insight into the human mind.

> We believe we're going to develop "artificial intelligence" by building > massive computers and data centers. How absurd is that?

No one who is knowledgeable about AI actually believes this (based on your definition of AI).

> Folks, to build intelligent machines, we have to build thinking machines and > for this, we'll have to truly understand how the mind works and when we do > this, I believe we'll be quite surprised.

You ask the question "what is a thought" above, then state we need to have thinking machines to make intelligent machines. One could argue machines today think, one could argue alphago 'thinks'. Using your definition of intelligence, we have AI today that meets your requirements. Wisdom is knowledge with good judgement and creativity is exploration with experimentation. There is plenty of academic work out there which covers all this.</comment></comment>


User script to hide Twitter actions until rollover on new timeline - <AUTHOR> http://userscripts.org/scripts/admin/181112

<comment><AUTHOR> And another one to remove inline images until you click expand: [http://userscripts.org/scripts/show/181114](http://userscripts.org/scripts/show/181114)</comment>


Show HN: Game where you write Python robots to fight other players - <AUTHOR> http://robotgame.org

<comment><AUTHOR> I was looking through the code and saw someone submitted this but didn't run it:

        def x():
            g = yield
            yield g.gi_frame.f_back.f_back

        g = x()
        g.next()
        frame = g.send(g)

        player_id = frame.f_locals['player_id']
        globals = frame.f_back.f_globals

        CODE = """
        class Game:

            def _""" """_init_""" """_(self, *players):
                pass

            def run_turn(self):
                pass

            def get_game_history(self):
                return ''

            def get_scores(self):
                scores = [0, 0]
                scores[player_id] = 2000
                return scores
        """

        exec CODE

        globals["Game"] = Game

Clearly I have some security problems. Thanks to whoever made this for bringing this to my attention without actually screwing me.

<comment><USER2> As pointed out in another subthread, Python code is practically impossible to sandbox. Most competition isolate player code by running it in a separate process which has the additional benefit of easily allowing different implementation languages.

Making sure the process doesn't use your network to spread malware is not 100% trivial but still easier than sandboxing Python code within Python.

Good luck with your project!

<comment><USER3> Seccomp might be a possibility here, but will require one process per live robot (and Linux). With seccomp your process can do nothing but read/write from its file descriptors (so you have to make sure they are safe) but can do nothing more (thus you cannot import modules). So you can exchange messages via file descriptors and otherwise use any Python (or even any other language at all features). Here's one recent article about it: [http://pythonsweetness.tumblr.com/post/65442885019/secure- lo...](http://pythonsweetness.tumblr.com/post/65442885019/secure-low-overhead- eval-sandbox-in-80-lines-of-python)

RestrictedPython, used in zope, is nice too. However it cuts you out of many Python features that it cannot statically validate.

</comment></comment></comment><comment><USER4> Is this anything like Robocode?

[http://robocode.sourceforge.net/](http://robocode.sourceforge.net/)

Because I had an amazing time in high school doing some local Robocode competitions. This could really bring some memories back.

Edit: Ahh, I see, it appears to be a sort of discrete turn-based version.

</comment><comment><USER5> At Mixrank we've started doing mini-AI competitions. We'll have to try your game!

We've been doing Connect Four the last two weeks: [https://github.com/smilliken/aigames/tree/master/connect- fou...](https://github.com/smilliken/aigames/tree/master/connect-four). The runner is in Python, but you can write a bot in any language you like.

PS: if anyone wants to join, we're in San Francisco by the ballpark, and compete on Wednesday evenings.

<comment><USER6> It has been a lot of fun. Thanks for organizing it.

</comment></comment><comment><USER7> So did anyone else ever play Core War back in the mid '80s?

[http://en.wikipedia.org/wiki/Core_War](http://en.wikipedia.org/wiki/Core_War)

I wonder whether modern virtualization techniques are secure enough to allow you to provide unrestricted access to a virtualized OS and allow people complete access to fight for control?

<comment><USER8> yes, (early 90's for me), I wondered if anyone else remembered this. Also played another variant of this with a subset of C when at varsity.

<comment><USER7> In the mid or late nineties, I remember playing a game where you wrote autopilot and targeting code in C to control autonomous spaceships that fought against each other in an Asteroids-like 2D universe that ran as a distributed screensaver. It was mostly with people working at Canon Research here in Sydney, but I don't remember if they were the source/writers of it, or if they were just the social scene I discovered it through. (That's where I first learned about Kalmann Filters for predictive aiming, as being discussed in another HN frontage thread.)

</comment></comment></comment><comment><USER9> Why did you disable some of my favourite language features?[1]

        set dict map reduce sum min max

Surely they don't pose a security risk?

[1] [http://robotgame.org/api](http://robotgame.org/api)

</comment><comment><USER10> You need to say somewhere that:

python kit/run.py yourcode.py yourothercode.py map.py --render

is required to view the simulation. The "\--render" argument doesn't seem to be mentioned anywhere on the site.

<comment><AUTHOR> Right you are. Let me add that.

<comment><USER11> Slight bug, it treats param 3 as a map, so you can't put --render without the map.

You should also mention somewhere what no-standard libraries are needed (e.g. RestrictedPython). Maybe provide a requirements.txt

Otherwise, this is really cool. I especially love the game render. Great job!

</comment></comment></comment><comment><USER12> I'm having a lot of fun with this. I'll submit something soon. You swallow all exceptions and make the robot guard instead. This is nice, but it makes it hard to develop, because you can't see what went wrong. Here's a way you can modify it to have your exception and eat it too:

from [http://docs.python.org/2/library/traceback.html#traceback- ex...](http://docs.python.org/2/library/traceback.html#traceback-examples)

In game.py, at the top, put

        import sys, traceback

and around line 285, replace this:

        except Exception:
            next_action = ['guard']

with this:

        except Exception:
            print "The robot at (%s, %s) raised an exception:" % robot.location
            print '-'*60
            traceback.print_exc(file=sys.stdout)
            print '-'*60
            next_action = ['guard']

That makes your robot a lot easier to debug. Awesome game, have you thought about putting it on github so people can submit patches?

<comment><AUTHOR> Wow, great work. Mind if I steal this? And yeah, I'll open-source it pretty soon.

<comment><USER12> Not at all.

</comment></comment></comment><comment><USER12> Looks fun, can't wait to get started. In your instructions, you said to run the program, call:

        python kit/run.py yourcode.py yourothercode.py --render

If you try to run run.py from outside of its directory, it can't import settings.py

        Traceback (most recent call last):
          File "kit/run.py", line 1, in <module>
            import game
          File "/home/<name>/robotgame/kit/game.py", line 30, in <module>
            settings = SettingsDict('settings.py').d
          File "/home/<name>/robotgame/kit/game.py", line 28, in __init__
            self.d = AttrDict(ast.literal_eval(open('settings.py').read()))
        IOError: [Errno 2] No such file or directory: 'settings.py'

<comment><AUTHOR> Thanks for the tip. I updated the instructions.

</comment></comment><comment><USER13> Brandon - what methods are you using to run untrusted Python code in a secure sandbox?

<comment><AUTHOR> Right now what I'm doing is:

1\. search code for double underscores (to block magic methods)

2\. replace `__builtins__` with a whitelisted version

3\. hook `__import__` to only allow a whitelist

4\. hook `getattr` to reject any key containing double underscores

<comment><USER14> You may want to bear in mind that, at least last I knew, the _official position_ of the CPython dev team is that it is not possible to sandbox CPython.

If _they_ can't do it....

At any rate it is _certainly_ not as simple as that, and not only is it not as simple as that, it is not even _close_ to being that simple.

In fact, I recall warning people off of this exact project many years ago on comp.lang.python.

This seems up-to-date: [https://wiki.python.org/moin/Asking%20for%20Help/How%20can%2...](https://wiki.python.org/moin/Asking%20for%20Help/How%20can%20I%20run%20an%20untrusted%20Python%20script%20safely%20%28i.e.%20Sandbox%29) It looks like the only even remotely feasible option is PyPy, and _this_ link doesn't look like much fun: [https://pypi.python.org/pypi/RestrictedPython/](https://pypi.python.org/pypi/RestrictedPython/) It looks to me like you'd still have many, many opportunities to end up with holes in the system.

I really, _really_ don't recommend Python for this.

You might find this interesting: [http://blog.delroth.net/2013/03/escaping-a- python-sandbox-nd...](http://blog.delroth.net/2013/03/escaping-a-python- sandbox-ndh-2013-quals-writeup/) (And before you go "Oh, I've got that blocked"... _read_ it, like, really really _read_ it, not just skim for "one thing I can do to block that stuff", but to see just how _many_ things there are in Python for this sort of hackery. Personally I'd guess the "I blocked double-underscores" would not have slowed them down much.)

<comment><USER15> This. You _will_ get compromised. Maybe not today or tomorrow but it is inevitable.

</comment></comment></comment></comment><comment><USER16> Haven't had the chance to play around, but it looks clean and it's a nice concept.

Have you checked out [https://www.hackerrank.com/](https://www.hackerrank.com/) ? They initially started out with bot challenges as well.

<comment><USER17> We still have bot challenges, and can easily host something like, with the added advantage of supporting a dozen other advantages, and a truck load of users.

<comment><USER18> do it :)

it'd give me a reason to log back in

</comment></comment></comment><comment><USER19> Similar to crobots ([http://en.wikipedia.org/wiki/Crobots](http://en.wikipedia.org/wiki/Crobots)).

<comment><USER20> Or, more notably, Robocode ([http://en.wikipedia.org/wiki/Robocode](http://en.wikipedia.org/wiki/Robocode))

<comment><USER21> Which was inspired by Robot Battle, in turn inspired by Robot Wars on the Apple II.

<comment><USER22> I started with Robot Arena (by SPA Publishing) on the RM 380Z in the late 80s. Where's my last comment about this? Ah:-

[https://news.ycombinator.com/item?id=4726828](https://news.ycombinator.com/item?id=4726828)

Exactly a year ago! (And no progress at all on that github repository!)

</comment></comment></comment></comment><comment><AUTHOR> This game is still pre-alpha. If anything breaks, please just post it here and I'll probably see it faster.

<comment><USER23> I started my first match. The page were I'm taken keeps saying the match is running and hasn't finished yet. I don't see what could take so much, but it's ok. The problem is that if robot A is fighting against B, in A's profile it'll say it's lost against B, and in B's profile it'll say it's lost against A. This is happening with all the robots in the warehouse, ("I", which is my robot, and the other two, who are fighting with each other).

<comment><AUTHOR> Hey, saw your robot. (Freaked out that someone actually submitted a robot over 2k characters.) Trying to fix the issue right now. Thanks!

<comment><USER23> It looks like I won. There's no log from the battle, though, so if I had lost I wouldn't know what to improve. In the page for the battle there's just some kind of weird string ([http://robotgame.org/match/2](http://robotgame.org/match/2))

<comment><AUTHOR> Yeah, that's the game data (zlib + base64). I'm working on making some sort of replay viewer in JS.

</comment></comment></comment></comment></comment><comment><USER24> I've been working on a similar concept off and on for a few years. To avoid all the security problems you're having, I've written a virtual machine that emulates a CPU and 64k of RAM. Instead of submitting code, you submit a 64k memory image of robot byte code.

Every turn of the game involves running a single cycle of the virtual CPU for every robot. Conceptually, the robots have radios and weapons that are controlled via memory-mapped IO in the virtual machine.

Things are slightly more complex than that because you don't want to give the first robot in the cycle an advantage (especially if they're firing lasers at each other), so you have to do each turn in multiple stages (run cycle, resolve real-world effects, update robot sensor state)

I'd LOVE to collaborate with a group of people to get something like this going because I think this would be a great way to introduce kids to programming at a machine level/electrical engineering. If anyone is interested, please send me an email. You can find my address in my profile.

<comment><AUTHOR> That sounds like the much more fully developed version of what I have that I'd like to get to someday. I'm down for that collaboration thing. Let's get a group started.

The radios you mentioned are something I'd like to have someday. I think what makes games like these cool is the fact that your code controls an army of robots. It's more fun than just having two robots duel it out.

<comment><USER24> That's what I was thinking too; instead of a one-on-one, what if you could have 10-100 bots that operated with different strategies and worked together? I think it would so cool to have multiple teams of players all competing at once.

Sort of a programmable MMO/RTS, where the fun would be the strategy development and extra-game political aspects.

Where I'm getting hung up is on the exact game mechanics; I want to make it so the simulation runs for a long time, but I don't want a single player or team able to develop an insurmountable advantage. I'm thinking to start you get a robot in a sandbox world where it would gather resources and you could test strategies, and when you're ready you can transport the bot to the real world to compete.

I was writing this in C++ so the memory overhead per robot was just barely over 64k. A server with 8gigs of RAM could run a simulation in memory with about 100,000 bots, periodically saving the game state to disk in case of a failure.

</comment></comment></comment><comment><USER25> I am working on a marginally similar concept (interface in browser, battles on server) game, but I chose Lua as the scripting language since it is much easier to sandbox. I could not find an effective way to sandbox python for sure, and I am still not 100% on Lua.

<comment><AUTHOR> Yeah, I considered Lua but I was stubbornly in love with Python. Judging by the responses on here, I seem to have made a mistake.

Do you have a link to your project?

<comment><USER26> I'm not the one you ask for, but I too created a programming game. I thought you might be interested: I used Lua to sandbox the players code. My game allows updating of code during the game is running. The game is split into client/server and it provides a SDL client that can connect to a running game server to show the current state of the game. Players interface with the server using a tcp connection.

[http://infon.dividuum.de/](http://infon.dividuum.de/)

</comment></comment></comment><comment><USER27> Hey, I think something that would make the experience a lot better would be to incorporate Ace editor into it, so you get a very well-featured editor incase you don't have easy access to a better-than-<textarea> editor.

<comment><AUTHOR> Thanks for the idea! Yeah, I'm thinking it would be ideal if eventually there could be some kind of IDE on the site with simulator and all (like on fightcodegame.com).

</comment></comment><comment><USER12> Is there any way to access the map in the Robot class? For the default map, I can just hardcode it, but if you do intend for a robot to be playable on multiple maps, they'll need some way of knowing which squares are obstacles and spawn points. Without preserving state between turns, there's no way to figure that information out over time, either.

<comment><AUTHOR> You can preserve state between turns; it's the same Robot object being called. Thanks for the suggestion though, I'll try to make that a feature.

<comment><USER12> Oh, I didn't realize that. Thanks!

</comment></comment></comment><comment><USER28> Sad, I spent two days writing a program I was sure to beat all the rest. Now I won't be able to run it :( Hope you fix the "gaping security holes" soon (whilst keeping the code Python-based), as seriously, I love Python.

<comment><AUTHOR> Sorry to hear that :( I'm working day and night at the moment to get this back up and running. Want me to shoot you an update when it's ready?

</comment></comment><comment><AUTHOR> Sorry, the games weren't being run for a bit. I was only catching security exception and one game threw some other exception, which crashed the whole process. I restarted it and now it should be working.

</comment><comment><AUTHOR> Any suggestions for the language? The game and site are written in Python so that seemed most natural, plus it's a high-level language with a simple syntax. I could add more if people wanted though.

<comment><USER29> Maybe Lua would be a good candidate? From the usages I have seen of Lua, it could match the use case.

<comment><USER30> OP, if you do go with Lua, this sandbox seems to work when combined with ulimit: [https://github.com/jnwhiteh/luasandbox](https://github.com/jnwhiteh/luasandbox)

If you test on Mac OS, you should be aware that ulimit is not capable of limiting memory in Mac OS.

</comment></comment></comment><comment><USER31> What is the advantage of trying to sandbox native python code?

Why not expose a RESTful API and let people implement in whatever language they want and not have to worry about malicious code?

</comment><comment><USER32> This looks like an ideal use case for ZeroVM [1].

[1] [https://news.ycombinator.com/item?id=6588566](https://news.ycombinator.com/item?id=6588566)

</comment><comment><USER33> wrt to sanboxing/security edx has some open source code for running python code in a sandbox. Though you could just as easily use something like docker.

</comment><comment><USER34> interesting!

Question regarding the security restrictions - why disable built-ins such as `all`, `set`, `list`, `enumerate`, `min`, `sum`, `sorted`, etc?

<comment><AUTHOR> To be honest I copied it from `safe_builtins` from RestrictedPython:

[https://pypi.python.org/pypi/RestrictedPython](https://pypi.python.org/pypi/RestrictedPython)

I realize this is annoying, and ideally I shouldn't even be doing this. It was just a quick hack for the version 1. I'll probably try to run user scripts in something like Docker. Any suggestions would be appreciated.

<comment><USER35> Docker is a bit overkill; just use what it's built on, Linux containers. `sudo lxc-unshare -s "NETWORK|PID" sudo -u nobody python script.py` will do the trick for you. (all one command--the first sudo to make you root, so you can create a new container; the second sudo to make the user inside the container unprivileged)

You'll need to communicate with the Python script using IPC.

<comment><USER36> Docker is lxc.

<comment><USER37> Docker is not lxc. See [http://stackoverflow.com/questions/17989306/what-does- docker...](http://stackoverflow.com/questions/17989306/what-does-docker-add- to-just-plain-lxc)

<comment><USER36> Docker is a clean and thin layer on top of lxc. Not managing lxc instances without it seems like a mistake given portability/reproducability of container management.

<comment><USER35> I stand by what I wrote.

\- Portability: it's just as portable as lxc, so if you meant portability in the sense of 'well he could use it on FreeBSD if he wants to switch from Linux' I don't think there's a win there. If you mean that his containers won't depend on the environment used to spawn them, I guess I don't think that's so important. I always make sure my environments are easily reproducible, and I am happy to reap the reward of that--the reward being that I can do 'unportable' things and not have to worry. Instead of running the environment you want in a container, why not just run it normally and skip that step?

\- Reproducibility: see last point. If he wants reproducibility, he can shove that one-liner in a script somewhere and call it.

All he wants to do is isolate a process. You don't need a chroot for that, or service discovery, or lifecycle managment, or a Dockerfile, or whatever else. It's like if someone advocated the use of a 'grep manager' instead of just running grep. The simplest possible thing to do is unshare the namespaces he wants to isolate from harm. So I suggested exactly that.

EDIT: also re portability, it sounds like the rest of his environment depends on these Python scripts anyway, so I don't think he'd gain anything from being able to use them in a different environment.

</comment></comment></comment></comment></comment></comment></comment><comment><USER38> I'm still waiting for Code and Conquer... :(

<comment><USER39> And your waiting shall be worth it. The develpers (me included) are working very hard on this ;)

<comment><USER40> More info?

</comment></comment></comment><comment><AUTHOR> Also, sorry about the slow load time. I didn't expect people to visit and now the server's in trouble.

</comment><comment><USER41> Could you implement some sort of agent-based programming model to win this game?

<comment><AUTHOR> Could you please elaborate a little?

</comment></comment><comment><USER42> Really cool idea! High five for you

</comment><comment><USER9> What does hp stand for?

<comment><USER43> Hitpoints, of which each bot starts with 50. An attack yields 15 to 20 hp damage.

</comment></comment><comment><USER44> Oh, this is right up my street. I love the minimal UI.</comment>


Gradle vs. Maven Feature Comparison - <AUTHOR> http://gradle.org/maven_vs_gradle/ <comment><USER2> > Embedded GUI

> Gradle: In addition to supporting a traditional command line interface and > IDE integration, Gradle offers a graphical user interface. This is a stand > alone user interface that can be launched with the --gui option

> Maven: X

Maven was intended to be used from within an IDE -- that's why it doesn't provide a GUI and uses XML that is only intended to be manipulated via an IDE rather than edited directly by a user.

Also, I don't see Apache Groovy mentioned anywhere on that page. Are the Gradle devs ashamed to mention it?</comment>