AlphaGo Zero: Learning from scratch - <AUTHOR> https://deepmind.com/blog/alphago-zero-learning-scratch/ <comment><USER3> The fact that they only used self play with no outside input here is really interesting. I wonder if this system produced more new styles of play. While I am not that familiar with Go, I know in some of the other articles they talk about things like Chinese starts that are specific to certain cultures. I wonder if the fact that it had no outside reinforcement made it produce movements that we have already seen that are somehow inherent to the game, or if it produced many more new moves that were a result of it learning without any possibility of cultural interference. According to the article it did invent some unconventional and creative moves, but I also wonder how much it rediscovered.

I also wonder how much it's style of play changes if it were re trained, due to the random start that it is given. Maybe that would produce something like seeds for procedurally generated worlds in games. Like if they could find a seed for a Chinese or japanese players, or ones that more aggressive styles. This is some pretty cool work and may open up even more doors for pure reinforcement learning

<comment><USER4> > I wonder if this system produced more new styles of play.

One thing Alpha go has told us clearly is that it thinks human players over value the margin of victory vs the probability of victory.

<comment><USER5> I'm not 100% sure I agree. It values probability of victory because that's it's goal. For humans, aiming _only_ for probability of victory might not be as good, because we're much worse at estimating probabilities. So aiming for maintaining a large margin at all times is conceivably the best proxy that we can use _in practice_.

<comment><USER6> Agreed. I know I'm winning by 4 points but I have no idea about my probability of winning. However if I'm winning I know that I should play low risk moves and refrain from starting complicated fights. That increases the probability of winning. IMHO the exact value is out of reach for human beings.

<comment><USER7> It would be interesting to play human go, assisted by a go computer that doesn't say anything about moves, but rather just spits out, for each player, their current likelihood of victory if all further moves by both players were "what it would do."

That way, each player could know, at all times, (one major factor that goes into) their probability of winning. They'd still have to mentally adjust it for the likelihood of them and their opponent making an error, and how that can be controlled by making intimidating moves, etc. But it could lead to much tighter control on the abstract flow of the game.

It'd almost be like the computer was the general, issuing strategy, picking battles; and the human player the tactician, fighting those battles.

<comment><USER6> There is computer go program (maybe Crazy Stone?) that analyzes a game record and annotates it with the winning percentage for every move.

Knowing that the opponent's winning probability changed from 52 to 57 was interesting only because it hints at a mistake. In case of such a large change the program suggests the move it would have played.

I saw an annotated game record and there were no variations: I remember a suggested move that made me wonder "why!?".

Another benefit of seeing the value of the winning probability is an assessment of who's ahead. However that's already possible with the score estimation that programs and go servers provide. Sometimes is crude, sometimes is good, but it's the score, not the winning probability that humans can estimate when playing. The best probability estimate I can make is: if the score is close and the game is still complicated, it's 50-50; if the score is close but the game is almost over, it's 95-5 for who's ahead. If the score is not close, the player with more points will probably win.

</comment></comment></comment></comment></comment></comment><comment><USER8> I'm reminded of Eliezer Yudkowski's article "There is no fire Alarm for Artificial General Intelligence." Is this smoke?

[https://intelligence.org/2017/10/13/fire- alarm/](https://intelligence.org/2017/10/13/fire-alarm/)

Yes, this is not an AGI. But the hockey-stick takeoff from defeats some players, to defeats an undefeated world-champion, to defeats the version of itself that beat the world champion 100% of the time is __nuts __. If this happens in other domains, like finance, health, paper clip collection, the word singularity is really well chosen--we can 't see past this.

<comment><USER9> You don't even need to produce an AGI for this kind of intelligence to be frightening.

At some point, a military is going to develop autonomous weapons that are vastly superior to human beings on the battle field, with no risk of losing human lives, and there is going to be a blitzkrieg sort of situation as the relative power of nations shifts dramatically.

If we have two such countries we could have massive drone and cyberwars being fought faster than people even can comprehend what's happening.

Right now most countries insist on maintaining human control over the machinery of death. But that will only last for as long as autonomous death machines don't dominate the battlefield.

It's a fun challenge right now to build a machine that can win in Starcraft, but it's really a hop skip and a jump from there to winning actual wars.

<comment><USER10> Nuclear ICBMs already push us past that boundary. The world can no longer afford to fight a war seriously.

</comment></comment></comment><comment><USER11> > Previous versions of AlphaGo initially trained on thousands of human amateur > and professional games to learn how to play Go. AlphaGo Zero skips this step > and learns to play simply by playing games against itself, starting from > completely random play.

So technically this version has lost every game it's ever won.

Jokes aside, it's pretty interesting to note that they were able to combine the "policy" and "value" networks. Good SO answers on the difference ([https://datascience.stackexchange.com/questions/10932/differ...](https://datascience.stackexchange.com/questions/10932/difference- between-alphagos-policy-network-and-value-network))

> accumulating thousands of years of human knowledge during a period of just a > few days

It'd be interesting for what this would mean when things like a neural lace become a reality.

As an aside, anyone have any other links or references to others investigating learning algorithms with a 'tabula rasa' approach?

<comment><USER12> TD-gammon is a well known version of this technique (with 2 ply lookahead, vs a 1600 deep mcts) [https://en.m.wikipedia.org/wiki/TD- Gammon](https://en.m.wikipedia.org/wiki/TD-Gammon)

Temporal difference learning was previously consider weak at 'tactical' games, ie ones with gamestates that require long chains of precise moves to improve position (like many checkmate scenarios in chess) .

For anyone more familiar with this technique, is it clear how the mcts/checkpoint system overcomes this? How sensative is the system to the tuning params for those parts of the alg. Like is Go a particularly good candidate because of the ~400 play positions resulting in a (relatively) small tree seach requirement? (I kinda cant believe im saying that go has 'a small search tree'!)

We us td learning for the ai in our game Race for the Galaxy, so it's neat to hear about possible avenues for improvement!

<comment><USER12> After digging a bit deeper into the paper, it seems a key part of the new scheme is the NN is trained to help guide a deep/sparse tree search (as opposed to TD-gammons fully exhaustive 2-ply search). It's somewhat surprising to me that the simple win/loss is a strong enough signal to train this very 'intermediate step' in the algorithm - a spectacular result! It begs the question what other heuristic based algorithms would be improved by replacing a hand rolled non-optimal heuristic function with a NN?

<comment><USER13> It's estimating the probability of winning from the position based on what it has already seen. So basically it's a giant conditional probability distribution. Is it mistaken to interpret this as a bayesian network?

</comment></comment></comment></comment><comment><USER14> "It uses one neural network rather than two." and "AlphaGo Zero only uses the black and white stones from the Go board as its input, whereas previous versions of AlphaGo included a small number of hand-engineered features."

This is amazing! The technology they came up with must be super generic.

<comment><USER15> Also, unsupervised. Also, no rollouts. They got rid of a lot of complexity. At this point it looks like a reasonable challenge to write a superhuman Go AI in 500 lines of unobfuscated python.

<comment><USER16> I was wondering about this: can we study AlphaGo Zero and other nets created in the same way for similarities, extract and study them? Or are we limited to observing the behavior and learning from that?

</comment></comment></comment><comment><USER17> I'm wondering if once one of these algorithms comes along that has been perfected if it is going to "burn in" the domain it was built for as the target of problem reductions, similar to 8086 assembly or the qwerty keyboard living on today despite them being ancient relics.

For example, after this result it seems if you can reduce your problem domain onto Go (or a similarly structured game) you now have a way to create a superhuman solver. It may just be easier to do that then try to even figure out how to design and tune a new network.

I could imagine waking up in 10 years being confused at why all software efforts in the AI space are focused on just figuring out clever ways to map real problems onto a hodgepodge of seemingly random "toy" domains like Go and Chess and Starcraft. Hell, maybe the Starcraft bot will immortalize Starcraft in a way the game never would have been able to if it becomes a good reduction target for a lot of domains.

It kind of reminds me of how SVMs were "abused" by twisting non-linear domains into them via kernel methods, or by proving the NP-equivalence of a problem by reducing it onto 3-SAT, or how ImageNet's weights are being re-purposed for other image oriented prediction tasks.

<comment><USER18> I think the only thing about go that enables this technique is "turn-based perfect information 0-sum game".

<comment><USER19> Also it has fairly limited input. A real world problem may have much more possible inputs at any time step as opposed to placing just 1 stone

</comment></comment></comment><comment><USER20> Looks like the performance improvement comes from two key ingredients:

1) Using Residual networks instead of normal convolutional layers

2) Using a smarter policy training loss that uses the full information from a MCTS at each move. In the previous version, I believe they just ran the policy network to the end of the game and used a very weak {0, 1} reinforcement signal over all of the moves played. Here, it looks like they use each run of MCTS to provide a fully supervised signal over all moves it explores.

<comment><USER21> How is it different to apply the loss on each actual move at the end of the game VS on each rollout (which is itself a tiny game)? Does it help reinforce learning towards the end game as shorter rollouts are needed? Is the more accurate information then propagated to earlier moves as well?

<comment><USER22> I think the difference is that under 1/0 policy gradient loss, it gets feedback only on the actual chosen move. Under MCTS-rollouts-each-move, it gets feedback on every move on the board whether its value estimate was slightly too high or low plus the ultimate outcome of the 1 move it did make.

</comment></comment></comment><comment><USER23> So this is fun:

"AlphaGo Zero is the program described in this paper. It learns from self-play reinforcement learning, starting from random initial weights, without using rollouts, with no human supervision, and using only the raw board history as input features. It uses just a single machine in the Google Cloud with 4 TPUs (AlphaGo Zero could also be distributed but we chose to use the simplest possible search algorithm)."

<comment><USER24> Single machine?

Stunning.

</comment></comment><comment><USER25> I remember reading ages ago in Scientific American about a much more interesting (and useful) AI application of this technique.

Genetic algorithms were used to evolve new, more efficient variants of existing electronic circuits. I dug it up - it was: [https://www.scientificamerican.com/magazine/sa/2003/02-01/#a...](https://www.scientificamerican.com/magazine/sa/2003/02-01/#article- magnetars) Article "Evolving inventions". I have no idea if there is an open- access version anywhere.

As far as I remember, that approach led to some patents, because some of the inventions were better than existing solutions. One of the examples in the article was a low-pass filter (I dont remember if AI version was actually better or worse than human-made).

The essential element of this approach was that in electronics (as in go) there exist a well defined set of rules, that allows researchers to build a simulation engine with optimization/evaluation function that the AI targets by itself, without supervision. It's great to see that this approach is still alive, although in my humble opinion, application in electronics is much more interesting than Go.

<comment><USER26> Somebody needs to dig this up and apply it to an open FPGA toolchain like ICEstorm.

The other SA article on this was The Darwin Chip which I think went into more detail.

One of the limitations was the lack of documentation for the actual bitstream.

</comment></comment><comment><USER27> Very impressive, the original implementation relied a lot on feature engineering.

I'm surprised they're able to prevent a self-play equilibrium with such a simple loss function.

It's sort of like they are using auxiliary outputs but instead of using them to fit features, they are fitting to multiple ways of arriving at 'best play', through predicting value (SL) and predicting probability for best outcome (RL). In principle, they're doing the same thing but in practice it seems like they are making up for each others shortcomings (e.g. self-play equilibrium with RL).

</comment><comment><USER28> > If similar techniques can be applied to other structured problems, such as > protein folding, reducing energy consumption or searching for revolutionary > new materials,

Protein folding sounds like a nice idea for their next challenge.

<comment><USER29> [https://www.bloomberg.com/news/articles/2017-10-18/deepmind-...](https://www.bloomberg.com/news/articles/2017-10-18/deepmind- s-superpowerful-ai-sets-its-sights-on-drug-discovery)

Indeed.

</comment></comment><comment><USER30> When things will start getting _interesting_ is when we figure out how to get move simulation and search into the network itself, rather than programming that on the outside. As far as I know, no-one has even the faintest idea of how to do that. We have an existence proof that this should be possible.

The networks are great at perception and snap-prediction. Anything a human can do in 200ms is fair game. And with clever engineering, we can make magic happen by iterating or integrating those things.

But it's after that first 200ms that humans get really _intelligent_. When we can come up with an architecture that lets the networks themselves start simulating possibilities, backtracking, deciding when to answer now or to think more -- when the network owns the loop -- then it will get _interesting_.

<comment><USER7> > We have an existence proof that this should be possible.

Not guaranteed. The human brain has diffusion signalling (i.e. neurotransmitters passing out of the synaptic cleft, into a neighbouring one, and activating a receptor on some other spacially-local axon as a result.) And one of those signalling molecules is thought to represent, in its intensity, a confidence-interval bias adjustment (i.e. a pruning bias factor for MCTS.) So the brain's MCTS-equivalent process may rely on some extra-graphical properties of the brain-as-embodied-meat-thing.

<comment><USER31> That will be a couple of additional terms in activation function. Or am I missing something?

<comment><USER7> "Neighbouring" is defined in terms of embedding in a metric space and inverse- cube diffusion, rather than anything to do with graphic connectivity.

Also, these signals pile up in the synaptic cleft until they're picked up, so it's not just about instantaneous transmissivity as if these were radio signals.

But _also_ also, other stuff like monoamine oxidase is floating about in its own diffusion patterns, cleaning up these signals.

It's basically like a "scent" communication embodied-actor model, but a very complex one where things like redox reactions with the atmosphere occur.

Oh, and there are "secondary messengers": signals that trigger other signals that, among other things, inhibit the release of the original signal when received back at the sender, such that an dynamic equilibrium state is reached between the two signal types.

</comment></comment></comment></comment><comment><USER32> Why don't use the same approach for chess?

It's very interesting to see if it is able to handle much more advanced and tuned engines that exist for chess, game with considerable much more complicated rules?

<comment><USER33> I think chess is less compelling because, in a sense, it is a "solved problem" \- superhuman AI chess players already exist.

And chess, while it does have more complex base rules, has a much lower combinatorial complexity than Go.

<comment><USER34> Well, I'd love to see NN solution beating top chess engines. It might also introduce novelty to the game, just as regular engines did

<comment><USER35> It'd be particularly useful to have a chess bot that can play badly in the same way a human does.

The problem with the current chess bots is that they play badly, badly. They choose a terrible random mistake to make every few moves, while some of their other moves are brilliant. They cannot accurately mimic beginner or intermediate level players.

<comment><USER29> This seems like something DeepMind could create, given the incentive. They were able to train AlphaGo to predict human moves in Go at a very high accuracy (obviously not with AlphaGo Zero, but the inferior human-predictive version is how they determined that AGZ is playing qualitatively differently).

</comment></comment></comment></comment></comment><comment><USER36> This is pretty incredible, especially the power dissipation results. Only 4 TPUs? Humans are toast.

<comment><USER37> What is a TPU?

<comment><USER31> [https://en.wikipedia.org/wiki/Tensor_processing_unit](https://en.wikipedia.org/wiki/Tensor_processing_unit)

<comment><USER37> Thanks!

</comment></comment></comment></comment><comment><USER38> Comparing the top player's ELO with Zero's ELO (assuming numbers are accurate, etc):

 _Your rating: 3664

Opponent's rating: 5000

Probability of winning: 0.000456879355457417_

So 1 in 2,200 games... ouch

<comment><USER39> I don't think you can apply this to alphago. I think probability for a human to beat alphago now is zero.

Lee Sedol's single victory is the first and the last.

<comment><USER24> I disagree. This is precisely what the ELO predicts, and it has been pretty accurate over time - it's a good metric.

<comment><USER39> For humans yes. For humans against machines? I don't think so. Can any human beat a modern chess computer? The chance is zero.

<comment><USER24> Zero means zero, yes?

Alpha particles can flip bits and cause erratic behavior, can they not?

"[The probability of] at least one bit error in 4 gigabytes of memory at sea level on planet Earth in 72 hours is over 95%"

<comment><USER39> Oh hey maybe they use ECC? Are we really arguing this? Pedantry on a weird level.

</comment></comment></comment></comment></comment></comment><comment><USER40> Is AlphaGo Zero the first Go program without special code to read ladders? I'm curious how a pure neural net can read them, given how non-local they are.

<comment><USER29> The concept of locality is nothing but a human weakness in Go, the best AI must read the whole board with every move.

EDIT: From the paper: "Surprisingly, shicho ("ladder" capture sequences that may span the whole board) – one of the first elements of Go knowledge learned by humans – were only understood by AlphaGo Zero much later in training" I'm surprised by the author's use of the word "Surprisingly" here.

<comment><USER41> AlphaGo is still based around layers of 3×3 local convolutions.

That represents a strong assumption about locality in the network design. I would expect AlphaGo to perform poorly on the game "Go with the vertices randomly permuted".

<comment><USER42> Well they didn't use inception. If their inception units have, say, a 7x7 conv, then ladders will probably be found much earlier.

</comment></comment></comment></comment><comment><USER43> The catch is that this isn't _quite_ zero human knowledge, since the tree search algorithm is a human discovery, and not one that came easily to humans. It also massively cuts down on the search space for an appropriate policy function.

That means that this setup isn't necessarily general. How applicable is MCTS to games with asymmetric information, a la Starcraft? What about games that can't quite be modeled with an alternating turn-based game tree like bughouse?

<comment><USER44> There's a Dota 2 bot by OpenAI that played games with itself and managed to beat a lot of pros in the scene. It's still SF mid only no runes and some restricted items, but it shows that there is also potential for Starcraft.

[https://blog.openai.com/dota-2/](https://blog.openai.com/dota-2/)

<comment><USER45> Maybe you know something we don't...

"We're not ready to talk about agent internals"

What makes you think it uses a tree search?

</comment></comment></comment><comment><USER46> How I wish Marvin Minsky would have stayed alive for one more year and seen this. He would have been so happy!

<comment><USER47> In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6.

"What are you doing?", asked Minsky.

"I am training a randomly wired neural net to play Tic-Tac-Toe" Sussman replied.

"Why is the net wired randomly?", asked Minsky.

"I do not want it to have any preconceptions of how to play", Sussman said.

Minsky then shut his eyes.

"Why do you close your eyes?", Sussman asked his teacher.

"So that the room will be empty."

At that moment, Sussman was enlightened.

[http://www.catb.org/jargon/html/koans.html](http://www.catb.org/jargon/html/koans.html)

<comment><USER48> So Sussman was right the first time?

<comment><USER45> A random net has some random preconception. That doesn't mean it's a bad idea to try random preconceptions.

</comment></comment></comment></comment><comment><USER49> One idea occurs to me is to now evolve the Go game itself in a direction that adds more challenges for an AI to solve, and then solve those problems. How about being able to handle different and randomized board shapes? How about being allowed to say one move the opponent cannot take when you play a piece? It would be interesting to keep track of what variations the algorithm handles well automatically, and which it falls flat on, etc.

<comment><USER50> Arimaa was a chess inspired game intended to be difficult for computers. It "fell" in 2015.

</comment></comment><comment><USER51> This is such an impressive result, and so general, I bet many people (including me) wish they knew exactly how to duplicate this result. It would be great if they created an online course that explained all algorithms in detail right up to the creation of AlphaGo Zero itself. The paper gives the impression that it shouldn't be too hard for them to create such a course.

</comment><comment><USER52> So in terms of training, they went from nothing to "Go Singularity" in about a month? Impressive.

</comment><comment><USER53> Slightly scary how it went from zero to superhuman play in three days. I wonder if general AI will go that way one day.

<comment><USER54> General AI relative to an individual human, or billions of humans? The sum total of human beings, or organizations of humans is superhuman relative to an individual. We've had superhuman organizations for millennia. I'm not sure how much general AI will be different, other than the large scale automation of jobs which would happen.

As Rodney Brooks pointed out, all technology happens within a context, not a vacuum. A general AI will come to exist in a world with a lot of other superhuman capabilities already in existence.

<comment><USER55> One of the more interesting things the success of "starting with zero" suggests is that the idea that some mystical "human consciousness" is the end goal for AI might be laughable in the long term. AI might just casually bypass human consciousness, say "oh, hi!" and wave us goodbye a day later. Also, a factor of 7 billion "happens" in computer science.

This is getting rather creepy to think of, even if it's still science fiction. At this point, I could see a computer that out-thinks humanity within decades. What would it think? What would we even do with its findings? Would we understand it? Would it understand itself? Would it know how to manipulate us?

</comment></comment></comment><comment><USER56> Amazing results, though I am somewhat frightened by how generic this model is and how it achieved such amazing results. I can't help but think that these same techniques can be used to learn how humans react in certain situaties and how they can be, very subtely, be worked to think in a certain way - one that fits the agenda of whatever party is behind it.

With the mass surveillance that is Google it's quite doable to test for human reactions on certain things. They got the tools to execute a certain plan and evaluate the effectiveness. Ofcourse it can also go in a benelovent way: like what kind of policy will benefit the most people? (semantics of 'benefiting' aside)

I atleast certainly hope these kinds of generic algorithms will be used to generate effective, meaningful policies that truly help the people. Still a far away future but one that gets closer by the day.

<comment><USER57> I'd only worry if it can outperform humans when there are not rules per se. That is, if I put a queen down in the GO board, and start knocking off stones, moving three times a turn, then take a lighter and burn the go board, the AI responds by decapitating my head.

<comment><USER54> Ha! I do wonder about using a board game where the rules periodically change in simple ways at random. A human could easily adapt to the rule changes while playing and adjust their strategy accordingly. Would a Deep Learning algorithm be able to do this?

If we keep the board and pieces digital, then the board could change shape, the pieces could change color indicating a random association with a rule change, and what not.

</comment></comment></comment><comment><USER58> What's fascinating (and admittedly somewhat worrying) about Self-play is that an agent can accidentally become adept at tasks other than intended via transfer learning. The "wrestling spiders" in OpenAI's demo quickly mastered the art of Sumo Wrestling. And whatever skills they learned in resisting an opposing force to stay standing on a platform, were immediately applicable to myriad different domains. In this case, being subject to hurricane force winds, and not as any normal spider may, be hurled into the sky!

It's more difficult to see how Go playing skills can translate to other domains. But for tasks in robotics, cybersecurity or fintech the power of self-play trained transfer learning becomes more apparent.

<comment><USER59> It is clear that these "self-play" scenarios depend on simulation - <USER2> there is an appropriate stage for self play to take place on, there can be no play. The question is - how do we stand with simulation for robotics, self driving cars, etc.

My bet is that simulation is going to be the crowning jewel in the AI field, replacing static datasets and supervised learning with "dynamic datasets" and rewards. It would help with data sparsity as well (where can you find an image of a donkey riding an elephant for the new ImageNet? - but you can sim that or any possible combination).

Not to mention that humans are fallen head over heels with simulation as well - VR headsets and games in general. I see a great future for simulation with both AI and humans. It will be our common learning/playing/research sandbox.

</comment></comment><comment><USER60> I would be willing to spend an entire lifetime to perfectly understand how this algorithm works. Currently I can barely write Djikstra's algorithm.

<comment><USER59> You can watch the RL course given by one of the inventors of AlphaGo, David Silver.

[https://www.youtube.com/playlist?list=PLzuuYNsE1EZAXYR4FJ75j...](https://www.youtube.com/playlist?list=PLzuuYNsE1EZAXYR4FJ75jcJseBmo4KQ9-)

<comment><USER60> This is pretty cool. Thanks!

</comment></comment></comment><comment><USER61> Would be nice if there was an open source attempt at an alpha go clone on a 9x9 board, so it could be run on commodity hardware and maybe trained in more reasonable time. Also would be interesting to see if human would still win on a 190x190 or some arbitrary size board against alphagozero trained appropriately.

</comment><comment><USER62> Is this evidence of a broader leap forward in machine learning, or are these advancements domain-specific? In other words, could these innovations be applied to other fields and applications?

<comment><USER63> I think the fact that it's no longer using Monte Carlo tree search is a huge step forward in the generalizability of the technique. But go is still

\- a perfect information game

\- with a relatively small input size (vs. arbitrary computer vision)

\- cheap to simulate

\- discrete action space

\- deterministic

This isn't to take away from the magnitude of the achievement, but the nature of the problem itself makes the result less applicable to many tasks we might want to use RL for.

<comment><USER64> Math research shares those qualities, except small input size if you include the body of all the already-known theorems as an input. I don't know if we'll see much smarter proof assistants soon, but it doesn't seem absurd to me as a possible development.

<comment> dane-pgp Actually, just converting all the already-known theorems into a form that can be computationally verified (not just convince a skilled human) would be an interesting starting point. This would really help the metamath project, and perhaps make peer review of mathematical research papers easier:

[http://us.metamath.org/mpeuni/mmset.html](http://us.metamath.org/mpeuni/mmset.html)

</comment></comment></comment></comment><comment><USER65> _Extended data figure 2_ contains something really cool: how long it takes for AG0 to discover a joseki, and how long it takes it to later discard it as non- jokeki. So you could, in theory, evaluate a human joseki by plotting AG0's probability of playing it against training time (or against its Elo rating). What's also cool is that the differences that cause it stop playing a joseki must be really minute, but it can see them anyway.

</comment><comment><USER66> I think that we cannot full understand the implications of that in our present/near future. The use of tecnhology will make the difference. There is a small book in amazon called something like Alpha Go Zero - 10 prophecies for the world. Interesting point of view of the use of it.

</comment><comment><USER67> > The system starts off with a neural network that knows nothing about the > game of Go. It then plays games against itself

I might have missed this, but: Where are the actual rules of Go encoded? Mustn't there be some enumeration of what constitutes "capturing," how the win condition of the game is calculated, correct?

<comment><USER68> Yes, I assume you start with an objective function ("winning Go") that includes the rules.

<comment> dane-pgp Surely if they are really starting with "zero", then all the AI is given is the arrangement of stones on the board (which starts empty) with the opportunity to select a position for its next stone after its opponent has placed one, until the game is over. (Let's assume that there is another piece of software responsible for determining when the game has finished, and who has won). As such, the only "rules" the AI needs are that it can only place one stone at a time, only in an empty position, and only when it is not the opponent's turn.

To start with "less than zero", though, it would be interesting to see them give the AI a 3D simulation of a room with a simulated Go board and a simulated stone, and give the AI a fixed amount of time for it to have its turn. Just by using the pixel data from a simulated camera, it could learn to use a simulated arm to place the simulated stone on the board in a legal position. The reward function would just have to say, at the end of each allotted time period, whether a legal move had been made or not, and the AI could bootstrap up from that.

</comment></comment></comment><comment><USER69> Very impressive. I wonder if the reduction in computing costs has to do with pruning the resulting trained networks.

</comment><comment><USER70> I wonder if it could be applied to SC2 (since they already started the research: [https://deepmind.com/blog/deepmind-and-blizzard-open- starcra...](https://deepmind.com/blog/deepmind-and-blizzard-open-starcraft-ii- ai-research-environment))

<comment><USER45> As is, no. There are too many possible actions and too little time between decisions.

I wouldn't discount it entirely though, some sort of clustering of actions may be able to reduce continuous action spaces to a manageable branching factor.

</comment></comment><comment><USER71> Did they publish any papers on AlphaGo Master ?

AFAIK the 2016 Nature paper is Alpha Go Lee. And now we skipped to AlphaGo Zero.

<comment><USER22> No. Skipped straight to Zero from AG Fan Hui (not AG Lee).

</comment></comment><comment><USER72> On the bright side, it means the several thousand years of humans playing Go, we were actually going "in the right direction" in terms of optimal strategy, despite not having reduced the game down into provably-optimal mathematical theorems.

<comment><USER24> I sometimes think about boats.

We built damn good boats, even before we knew anything formal about fluid dynamics, or even AIR.

</comment></comment><comment><USER73> Is there anywhere to see the games? I'm curious if the AI is superior to humans or just human trained AI. It'd also be interesting to see the source, but that is apparently not being released for some reason.

<comment><USER74> _It 'd also be interesting to see the source, but that is apparently not being released for some reason._

Last time this was brought up, someone implied it's closed source so as not to boost the Chinese competitor.

<comment><USER73> If all their methods for success are in the paper the Chinese competitor can just copy that.

<comment><USER74> Theoretically, yes. But anyone who has tried to implement scientific papers will tell you it's far harder. The papers often lack critical details and implementation hacks: all the little rough edges that go into making a production system work. They also lack context in many cases, so you spend more time reverse engineering the paper than figuring out how to make it work.

<comment><USER73> It seems odd they make a big deal about tabula rasa, but won't release the source for verification.

</comment></comment></comment></comment></comment><comment><USER75> I would love to know how this adversarial training doesn't end up overfitting. Or, put another way, I'd love to see another piece of software (or even a human being?) exploit Alpha Go's overfitted strategy.

</comment><comment><USER76> One thing interesting here is how close the best humans appear to be to the asymptotic theoretical max skill level.

</comment><comment><USER77> Deep reinforcement learning is interesting and has plenty of potential. But highlighting AlphaGo as an example of reinforcement learning is like undermining the concepts of reinforcement learning.

<comment><USER78> How so?

</comment></comment><comment><USER29> A great resource for "watching" AlphaGo's games: [http://www.alphago- games.com](http://www.alphago-games.com)

</comment><comment><USER79> this is quite an achievement that it plays accurately. Does this have implementations for Strategy? No two games are alike, yet we are able to learn from our experience. Human's players be somehow comparing local positions from various games and deciding (sometimes wrongly) that they can be played similarly. Can it recommend not just individual positions, but general modes of play for classes of positions?

</comment><comment><USER16> Serious question: could it be possible, even in theory, to translate a neural network to a high level programing language? So we can see what it's doing?

</comment><comment><USER80> Absolutely amazing. I'm dying to see the game records!

<comment><USER81> You can find them here : [http://www.alphago-games.com](http://www.alphago- games.com)

<comment><USER82> Weird how they anthropomorphized and capitalized "her Final Form" there. Creepy!

</comment></comment></comment><comment><USER83> How do they evaluate that AlphaGo Zero is better than the previous AlphaGos? By playing them against each other? Or playing AlphaGo Zero against humans?

<comment><USER51> The same way they make AlphaGo Zero better, by playing against other AlphaGos.

<comment><USER83> But what if the other AlphaGos are blind to some tactic that humans could take advantage of? Not saying this is likely given how the Lee Sedol match and others went but I'm curious how they come up with the rankings.

<comment><USER51> At the end of the paper they describe how the come up with ELO rankings, and that in order to avoid some bias due to self-play only they include the results of the AlphaGo's versus Fan, Sedol, etc.

</comment></comment></comment></comment><comment><USER84> Anyone know why they trained on TPUs? My understanding is that the main benefit of a TPU is inference. Is this not true?

<comment><USER85> The first generation TPUs were focused on inference. Second generation TPUs are focused on both training and inference.

[https://www.blog.google/topics/google-cloud/google-cloud- off...](https://www.blog.google/topics/google-cloud/google-cloud-offer-tpus- machine-learning/)

</comment></comment><comment><USER86> What if a top human player could train like AlphaGO Zero during billions of games and record everything?

</comment><comment><USER87> Algorithms > Big data in this particular machine learning scenario makes it very impressive.

</comment><comment><USER88> Anyone happen to have a link (or can PM one to me) to a copy that isn't behind a paywall?

<comment><USER89> From the Deepmind blog post: [https://deepmind.com/documents/119/agz_unformatted_nature.pd...](https://deepmind.com/documents/119/agz_unformatted_nature.pdf)

</comment></comment><comment><USER90> Would love to see some gameplay!

<comment><USER29> Here's a self-play game: AGZ v AGZ - [http://eidogo.com/#u2UdsDFJ](http://eidogo.com/#u2UdsDFJ)

</comment></comment><comment><USER91> Can self-improving AI predict behaviors in emergent environments? Will they?

<comment><USER45> Is Go strategy not an emergent property of the rules of Go?

</comment></comment><comment><USER92> How does that timeline have any meaning?

<comment><USER93> It does if you know that they used 4 TPUs.</comment></comment>


Handling Self Doubt and Battling Inactivity - <AUTHOR> http://www.thebuzzmedia.com/handling-self-doubt-and-battling-inactivity/

<comment><USER2> "What, your life-dreams aren't worth $10k to you?"

Well... the problem is often that money is just part of it. Just because you spend some time and $10k (or some mixture) doesn't guarantee success. But... staying put in a job for those same months will very likely guarantee that $10k. And most people need that money because they're living, if not check to check, then dangerously close.

Additionally, it's typically _not_ just that $10k. When that's spent, and the idea hasn't taken off, what should you do? The logical/safe thing and go back to the w2 world and keep working? Or do you dig in a "refuse to give up!" like so many self-help books/cds would tell you to do, and what so many tech/successporn blogs talk about.

"Look at all those 'founders' who are wildly successful!" you can say to yourself. " _They_ didn't give up! When things went bad, they did a pivot!" And so you'll pivot, and spend more money, etc.

For some people, the journey _is_ the reward, and that's fine for them. _Personally_ , for me, having a long runway and low expenses would be important before trying to focus on any one idea (and that's what I'm working towards). So... in my case, and perhaps for many others, the "micro-actions" are (or should be) getting your finances in such a state that you can fund your own idea rather than pining away for investment money.

The "more criticism you receive means you should push on because it's a good idea" is... frankly, bizarre. There's probably a lot of counter examples to scenarios I'm thinking of, but often times people really do criticize something because it truly is bad. Not everyone is a negative dream stealer full of "stinkin' thinkin'".

As you said, you need to apply some 'common sense', but that's what's truly uncommon. People closest to their idea often are too emotionally invested in their baby, and to tell them "criticism == you have a good idea" would cause many people to spend a lot more of their time and money in pursuit of a bad idea. People also need to learn to separate criticism of the core idea/concept from criticism of the execution of that idea, to the extent that's possible.

Anyway, lots of reactions to that piece - thanks for posting it :)

<comment><AUTHOR> Michael, thank you for the well thought out feedback.

</comment></comment><comment><USER3> This struck a chord with me today.

" -I want to start my own business… -Oh it will never work, I don't know what it takes ... -I guess I should just hope for that promotion at work… -Good, that sounds safe "

I decided a few weeks ago that I'd had enough at my job. Some new management came in and have really messed up a good place to work. And I missed out on that promotion...

I finally decided that I needed to do something. I've wanted to start my own business, but it takes a lot of effort, and I'd rather play with my kids. But there is time, you just need to find it.

<comment><AUTHOR> I'm glad to hear you took the next step in realizing the future you want, that is exciting/scary/wonderful.

What kind of business are you starting?

<comment><USER3> I'm just going to do software development. It's what I love. But I've always done it for a big company and I'm tired of that. I'm in the process of figuring out how I can work for myself - some software product I can sell maybe. I started learning Android development this week with the thought that I might find a little success there as a starting place. The details are fuzzy, but as the article pointed out, sometimes you just need to start.

</comment></comment></comment><comment><USER4> I find that active self-deception works for me frankly.

I concentrate on pretending that the thing i want to do is something I cannot not do. Whats one activity that i do every day that I can't avoid doing?

Going to the bathroom.

So whenever I try to talk myself out of doing that activity I ask myself, "Can you avoid taking a piss for an entire day?" the answer is no.

I found that for me the feeling of having to take a piss is the one thing that conveys both urgency and inevitability at the same time. and that any activity that i catalogue as "the same" as taking a piss by association also becomes urgent and inevitable.

anyway, this is how I got myself to run 4-5 km everyday for the last year and a half.</comment>


Java vs. Scheme in Education - <AUTHOR> http://www.innoq.com/blog/st/2011/03/java_vs_scheme_in_education_1.html <comment><USER2> Asking as a CS major who first learned C, then C++, and didn't touch LISP and Prolog until the upper divs, how difficult is it to learn a functional language as your first programming one? Wouldn't it be trippy? When I first learned that Berkeley students had to learn Scheme first, it made me think of them as hardcore... though I may be overestimating its difficulty.

<comment><USER3> > how difficult is it to learn a functional language as your first programming > one? Wouldn't it be trippy?

As easy as learning a dysfunctional one if not easier.

Now scheme may not be the best example of this as its syntax is entirely alien (then again all syntax is, when you've never developed), but the canonical is the following statement:

        x = x + 1

If you've never developed before, the only place where you've seen something like this is maths, and in maths _this is completely nonsensical_.

So learning imperative programming does require lots of deprogramming and reprogramming of your brain. Which make up _most_ of the hurdles of learning an FP (or logic, or dataflow, ...) language later on: your brain (or its section that has to do with development) has been baked in imperative idioms and methods, FP languages not only don't use these but use completely different techniques to solve the same problems, so you have to "unlearn" the old ones and "relearn" the new ones.

<comment><USER4> I can corroborate that for people with no prior programming experience, but a significant math background (engineers, stat and math majors), the statement "x = x + 1" is befuddling to them. They have no concept of _time_ in a sequence of imperative statements. That is, they don't intuitively understand that statements get executed, and that the state of the program changes (potentially) after each statement. It takes real effort on their part to internalize this idea.

Now, I'm sympathetic to your final paragraph, but having never taught functional programming to anyone, I can't corroborate it. Personally, I went through something similar when first learning how to think functionally.

</comment></comment></comment><comment><USER5> "As I see it, a software engineer who hasn't worked through SICP with Scheme, a basic editor and command line, is like a surgeon who has never dissected a frog and faints at the sight of blood." -- Priceless !!! Great comment, Stephen.

<comment> bulte-rs I'd better get my mint copy of SICP off the bookshelf then. Being a SE able to dissect frogs and not faint at the sight of blood apparently doesn't mean anything these days ;-)

</comment></comment><comment><USER6> I do remember playing with Scheme in freshman CS. It was fun as hell. You type in (+ 2 7) and it spits out 9. No

void main(String args[]) { ...

<comment><USER7> That's not an insurmountable advantage. In the freshman CS class I took they started us out with a GUI that had an interactive prompt.

(DrJava)

</comment></comment><comment><USER8> What is Scheme? Sounds cake. We used Putty, Pico, and javac. I take it that this school is not an engineering school?

</comment><comment><USER9> So scheme is better because there is no tool support and with no tools there is less to be confused by. Hm.

<comment><USER10> I think the point is that, with Scheme, the tool you use is less important than the thinking you do. With Java, you have to worry about your environment before you can worry about programming.

<comment><USER11> _With Java, you have to worry about your environment before you can worry about programming._

You have to download the jdk and set an environment variable. Then open a text editor. That's about all the 'worrying about the environment' that you have to do before you can start programming in Java.

<comment><USER3> No, you also have to think about various command-line tools invocation (java and javac for starters) and all the boilerplate that goes around what you want to _write_ (create a file, name it correctly, put the same name inside with a `class` statement whatever that means, then put braces everywhere and a `main` and a `String[]` which you don't know jack about) and then you have the joy of wondering why you don't see anything when you run your program (assuming you got it to compile and run) and learn about `System.out.*`.

With scheme, you download Racket or whatever (equivalent to downloading the JDK), you open it, and you start typing and getting instant feedback in your window.

<comment><USER11> I'm not sure I understand you. You can't just start typing and getting instant feedback in Java simply because Java is not interpreted (well, actually, it is, but you have to compile to bytecode first). So, what you're saying about Java goes for just about any compiled language.

 _No, you also have to think about various command-line tools invocation (java and javac for starters)_

Use "javac" to compile to byte code. Use "java" to run your compiled program. This is all you need to know about these tools to start programming. There are various options for using java and javac, of course, but it's not like you can't compile and run a simple program without these options.

As for this part of your comment:

 _create a file, name it correctly, put the same name inside with a `class` statement whatever that means, then put braces everywhere and a `main` and a `String[]` which you don't know jack about_

Yeah. You have to know a bit of Java before you actually start programming in it. If that's one of Java's shortcomings, well then, there are _a lot_ of languages with the same shortcoming.

<comment><USER12> _> I'm not sure I understand you. You can't just start typing and getting instant feedback in Java simply because Java is not interpreted (well, actually, it is, but you have to compile to bytecode first). So, what you're saying about Java goes for just about any compiled language.<_

This is not true. There are statitically typed languages, usally functional, with a read-eval-print-loop. Haskell, Ocaml, SML, F#, Scala are the main ones I can tink of. There are many more. I will also note that your parenthetical is slightly inaccurate in that jvms usually JIT bytecode and don't just inteprete it straight.

<comment><USER11> After googling for a while, I found this great answer on SO:

[http://stackoverflow.com/questions/3265357/compiled-vs- inter...](http://stackoverflow.com/questions/3265357/compiled-vs-interpreted- languages/3265680#3265680)

Lunaryom is correct. There are no 'interpreted' and 'compiled' languages, just different implementations. So, I was wrong in my original post. It would appear that you _can_ just type and get instant feedback in Java (if you use Beanshell, for example).

<comment><USER12> I noticed the possibility of that semantic ambiguity, that is why I switched to noting static/dynamic typing instead of retaining your distinction on compiled/intepretated.</comment></comment></comment></comment></comment></comment></comment></comment>


Evaluating persistent, replicated message queues - <AUTHOR> http://www.warski.org/blog/2014/07/evaluating-persistent-replicated-message-queues/

<comment><USER2> It would be nice to see the strongest of the various Apache contenders represented here too. Once upon a time, there was just ActiveMQ to worry about:

[http://activemq.apache.org/](http://activemq.apache.org/)

It's long been very popular, and has all the flexibility and enterprisey goodness you could want, but has not had a reputation for speed. However, i believe there was a release a year or two ago that significantly improved performance. I vaguely remember this being based on an infusion of technology from the Qpid project, which itself has a broker:

[http://qpid.apache.org/components/java- broker/index.html](http://qpid.apache.org/components/java-broker/index.html)

Or maybe that was what led to Apollo, which is the faster, more reliable successor to ActiveMQ, but which hasn't, er, succeeded ActiveMQ:

[http://activemq.apache.org/apollo/](http://activemq.apache.org/apollo/)

Or maybe i just imagined the link to Qpid.

I'm fairly hazy on how these bits of software relate, and i can't promise you there aren't more somewhere in the vast Apache mansion. Even figuring that out would be useful, never mind the benchmarks!

<comment><AUTHOR> I'm also not sure how the above projects relate. As far as the scope of the blog goes, Apollo seems to be ruled out as it is not replicated.

ActiveMQ does have replication, but they don't really sound convincing in the docs that it's first-class ;)

</comment></comment><comment><USER3> I am not very familiar with messaging or message queues, so wanted to seek out some advice before spending time reinventing some well known wheel poorly.

I would like to have a broker that sits between an application and several endpoint services to which the application makes requests via the broker. These endpoint services have rules, typically throttling rules (i) you cannot submit more than x number of request per second (ii) you cannot queue up more than y number of requests at our end (iii) you must put a delay of z seconds after we have responded to a request of yours, etc., etc.

Is there a defacto standard open source software for this situation, that would take a config file describing the policies to follow for an end point. My requirement is pretty toy'ish, small to medium scale only. Just taking a look at Redis docs (am new to that too) it seems it will not be that hard to write out a simple solution. Would that be the way to go ? Or is there something out there that already handles this.

<comment><AUTHOR> HornetQ has flow control ([http://docs.jboss.org/hornetq/2.4.0.Final/docs/user- manual/h...](http://docs.jboss.org/hornetq/2.4.0.Final/docs/user- manual/html_single/index.html#flow-control)). Not sure about e.g. Rabbit.

However I'm not aware of a standard solution in that space.

Take a look at Reactive Streams ([http://www.reactive- streams.org/](http://www.reactive-streams.org/)) as well.

</comment></comment><comment><USER4> What this article fails to talk about are primarily the failure scenarios of the queues under partitions. Does the queue become write-unavailable, read- unavailable, inconsistent, or otherwise during network partition? Can the queue remain partially available during network partition?

These are the situations, in a scaling, production system that really matter.

<comment><AUTHOR> These are very good questions, however for a different post. Here I was mainly interested in the throughput.

Jepsen post provide some insights (into Mongo and Rabbit). With Hornet you can easily end up with two primaries. SQS is untestable in that respect :).

</comment></comment><comment><USER5> required reading for anyone using distributed systems: [http://aphyr.com/tags/Jepsen](http://aphyr.com/tags/Jepsen)

in particular:

[http://aphyr.com/posts/284-call-me-maybe- mongodb](http://aphyr.com/posts/284-call-me-maybe-mongodb)

[http://aphyr.com/posts/315-call-me-maybe- rabbitmq](http://aphyr.com/posts/315-call-me-maybe-rabbitmq)

<comment><AUTHOR> Agreed, a must-read :). Bottom line is all queues can loose messages (we don't know much about SQS in fact).

</comment></comment><comment><USER6> Would love to see Kafka represented here.

<comment><AUTHOR> Yes, my bad, should be there :)

Stay tuned, I'm out soon till end of next week, but will definitely add it.</comment></comment>


The Role of Trust in the Stellar Network - <AUTHOR> http://www.hubbledev.com/blog/2014/08/22/the-role-of-trust-in-the-stellar-network/

<comment><USER2> > Stellar is a brand new decentralized protocol

Nope, it's just a clone of Ripple by the same author. He just dumped all his XRP, rebranded the whole thing, and started a new pump and dump scheme. He tried really hard to remove all the Ripple references from the code, but the whole concept looked so similar (nothing interesting, just another premined, centralized in practice service) that people caught him in no time.

<comment><USER3> Nobody ever tried to hide anything. Stellar was rebranded because it was launched as a forked collaboration with additional people and additional code. Look at the front page of stellar.org:

Stellar Development is led by Jed McCaleb (resuming development of the open- source technology he created at Ripple) and Professor David Mazières (head of Stanford's Secure Computer Systems group, creator of SFS and bcrypt).

<comment><USER2> Yes, yes, they now admit it because they got caught. That doesn't change the way they presented it when it was launched.

<comment><USER4> Joyce from Stellar here. We have not changed any content from our launch post since it was published for launch. Please feel free to read over our post here. Also adding Stripe's if anyone missed both announcements a month ago. (Side note: Stellar is 1 month old today!)

\- [https://www.stellar.org/blog/introducing- stellar/](https://www.stellar.org/blog/introducing-stellar/)

\- [https://stripe.com/blog/stellar](https://stripe.com/blog/stellar)

Thanks!

<comment><USER5> Hi Joyce, what I'm really interested in are the details you promised in that first blog post:

      We'll be releasing a paper soon
      documenting and exploring a
      provably-correct version of this
      algorithm.

When should we expect this paper about Stellar's consensus model to be made available?

Thanks.

<comment><USER6> Here is an explanation of Ripple consensus which was forked by Stellar: [http://dev.ripple.com/consensus- whitepaper.html](http://dev.ripple.com/consensus-whitepaper.html)

</comment></comment></comment></comment></comment></comment><comment><USER7> Here are the 'trust lines' that have been extended to one Gateway based in New Zealand (Coinex)

[https://www.yett.co/a/gs9HHU3pmkKBuvykhNm6xiK1JKrput9i3K](https://www.yett.co/a/gs9HHU3pmkKBuvykhNm6xiK1JKrput9i3K)

</comment><comment><USER8> How coupled together is the native currency and assets that need trust? Meaning, suppose you wanted to swap in a different crypto coin for the native currency one day, is that a major rewrite or more like a plug in?

</comment><comment><USER9> How do you create a Stellar node?

<comment><AUTHOR> I would recommend downloading the code and building from source on your own hardware or an EC2 instance (a medium sized instance should be good enough).

Stellard repository: [https://github.com/stellar/stellard](https://github.com/stellar/stellard) You can follow this wiki for build instructions: [https://wiki.stellar.org/Building_Stellard](https://wiki.stellar.org/Building_Stellard)

Some people may have had luck with Vagrant or Docker, but since I have not I can't vouch for those methods.</comment></comment>


The Problem with Lisp - <AUTHOR> http://heinrichhartmann.com/blog/problem-lisp.html <comment><USER2> I don't understand the author.

The evaluation order in Lisp is not "highly unpredictable", it is dependent on the form.

I don't understand why he even constructs this strawman of "you must start in the innermost sub-form".

That's not a rule that anyone ever made. That's the rule for arithmetic expressions, sure, but what does that have to do with Lisp?

The rule in Lisp – as he later admits – is "look at the first symbol in the (outer) form".

And the whole point of special forms is to make your own rules about evaluations. In order to _help the programmer_.

Can you obfuscate the meaning? Hell, yes! Is that a reason not to allow it and to force obfuscated code by using unsuitable evaluation orders everywhere? Hell, no!

And even in this bad example Lisp is certainly not more unpredictable than any other language.

Let's take C:

      int func(int a, int b)
      {
        return a + b;
      }

If you start at the innermost point "a + b" you get exactly the same questions: "Oh gosh, what is the + operator?", "How do I apply + to a number?" (really?) and "where do I store the result?"

I think the author has misled himself into some extreme notion of the divine simplicity and purity of Lisp that nobody ever claimed to exist.

<comment><AUTHOR> Author here. Thanks for your comment. I appreciate your feedback!

Let me first comment on your C example:

> "a + b; // Oh gosh, what is the + operator?"

There is no problem with the "+" operator. It's a language builtin. You can look that up in the manual.

In the Lisp case, I did not pick on the "+" operator, at all. The problem is a line below:

> "(sum x) ;; ??? Where does "sum" come from?"

There is no function "sum" at all! It just looks like a function call, which is it not. You have to know what the enclosing special form "let" does to it's arguments.

You could have those problems with C macros as well, but not in this example.

> Can you obfuscate the meaning? Hell, yes!

> Is that a reason not to allow it and to force obfuscated code by using > unsuitable evaluation orders everywhere? Hell, no!

I am not picking sides here. I am pinpointing a source of confusion. The Python/Ruby authors took another path and made the language less powerful, so they don't give developers the power to obfuscate the code in this way.

<comment><USER3> Your article is confused between code walking (recognition of forms) and evaluation order. Code is generally walked top down for the purposes of expansion of macros, compilation or interpretation. Evaluation of nested expressions is generally bottom-up, and in well-behaved Lisp dialects, left- to-right.

You cannot simply let your eyes randomly land on some _(a b c)_ in Lisp code and assume that a is a function being called with arguments b c. You have to read the whole sentences and phrases, so to speak.

No language is immune to this. Can you assume that __* X is a pointer dereference? No, it could be a declarator. Or a fragment of a multiplicative expression. Or the interior of a comment as in / __* X.

(a, b) could be a comma expression, or the argument list of a function call.

I can't think of any language in which I can extract _any_ substring of a sentence that could be a syntactic unit on its own, and correctly understand its meaning without knowing the rest of the sentence.

Lisp has to be scanned from the outside in: we have a defun, which encloses a let, and so on. When you're familiar with the piece of code you're working on, you can then take considerable shortcuts. You know you're in the middle of a function, even though the _defun_ is off the screen. You know you're in the middle of the _progn_ -like body of some construct (a loop, or let or whatever), so all the lines of code you see are evaluated forms.

<comment><AUTHOR> > Your article is confused between code walking (recognition of forms) and > evaluation order.

Yes, to some extend.

> Can you assume that * X is a pointer dereference?

[Aside: The C declaration syntax is particularly terrible. In a declaration 'star' means "address of", elsewhere it means "dereference". [https://books.google.de/books?id=bXGhBAAAQBAJ&printsec=front...](https://books.google.de/books?id=bXGhBAAAQBAJ&printsec=frontcover&dq=21st+century+C+pointer+declaration+syntax&hl=en&sa=X&ved=0ahUKEwjil_ShpYTeAhWNiKYKHfZ5CAYQ6AEIJzAA#v=onepage&q=21st%20century%20C%20pointer%20declaration%20syntax&f=false)]

> I can't think of any language in which I can extract any substring of a > sentence that could be a syntactic unit on its own, and correctly understand > its meaning without knowing the rest of the sentence.

Well, most languages have expressions/statements that roughly correspond to source lines. When I read a line of python like this:

        self.raw_requestline = self.rfile.readline(65537)

[https://github.com/django/django/blob/master/django/core/ser...](https://github.com/django/django/blob/master/django/core/servers/basehttp.py#L139)

I can be 100% sure that:

\- `self` is an object with an attribute rfile

\- `=` is the assignment operator

\- there is a method `readline` that get's called with an integer argument 65537.

All this without checking any enclosing socpes.

A random line of lisp:

       (setcdr current (list :args (cdr current)))

[https://github.com/emacs- mirror/emacs/blob/master/lisp/wid-e...](https://github.com/emacs- mirror/emacs/blob/master/lisp/wid-edit.el#L742)

could mean _anything_ depending on the enclosing scopes. Pretty sure setcdr,list,cdr are function calls but who knows. You don't have enough information to decide this from the snippet provided.

<comment><USER4> > I can be 100% sure that: `self` is an object with an attribute rfile, `=` is > the assignment operator, there is a method `readline` that get's called with > an integer argument 65537.

I think this is overstating things, eg, the class `self` belongs to might have defined __getattribute__, __setattr__, etc, so the things you know for 100% certain about that line of code in isolation don't really tell you anything about what it does (self does not definitely have that attribute, and that might not be assignment, at least); most of that you assume based on common sense, conventions, figuring the code is "normal", etc, the same as in Lisp. And that's ignoring a trivial case like if the previous line was `f('''`, where f may or may or not call eval.

</comment></comment></comment></comment></comment><comment><USER5> It's a bit like claiming that riding a bicycle is hard because steering, moving forward and keeping balance at the same time is too difficult. Yet, people learn it and do it just fine. They just don't think consciously about it. Riding a bicycle takes a bit of training but after that the body movements are virtually effortless.

Lisp is slightly more difficult than some other languages because lists/symbols are both used in code and data, because there are a bunch of different types of lists, and because code is a token tree.

      (defun silly (a b c) (let ((my-sum (+ a b c))) (* my-sum (+ 1 my-sum))))

Now what's wrong with that? Nothing. But people don't write that.

Lisp is written like that:

       (defun silly (a b c)
         (let ((my-sum (+ a b c)))
           (* my-sum (+ 1 my-sum))))

There are a limited amount of tokens on a line. Syntactic constructs are not one line.

Any definition has this pattern:

       (DEFsomething name ...)

Oh, it begins with DEF, then it is a macro, followed by the name.

Function definitions have this pattern:

       (DEFun name arglist body)

The code layout is:

       (defun name arglist
         body
         ...)

A method defintion:

        (DEFmethod name arglist
          body
          ...)

The difference between a function and a method is in the arglist.

LET is:

       (LET   binding-list
         body
         ...)

Now you need to know what a binding list is:

          ((arg0 value0)
           (arg1 value1)
           ...
           (arg2 value3))

Such a binding list is used in other places, too.

If you give shortcuts to it:

      df function definition
      bl binding list
      nm name
      bd body
      al arglist

DEFUN:

      df nm al
       bd

LET

      LET bl
       bd

PROG

      prog bl
       bd

lambda

      LAMBDA al
       bd

DESTRUCTURING-BIND

      DESTRUCTURING-BIND al
         form
        bd

BLOCK

       BLOCK nm
        bd

CATCH

       CATCH nm
        bd

etc.

Basically much of Lisp consists of a small number of basic patterns which are being combined. lists, assoc lists, property lists, arglists, binding lists, body, ...

You've learned then a new alphabet of visual/structural patterns. Beyond that, code reading gets intuitive. Names give a clue what kind of pattern combination follows.

The IDE can then help with syntax highlighting, syntax documentation display, etc.

Lisp can even show you useful code layout:

      CL-USER 68 > (let ((*PRINT-RIGHT-MARGIN* 30))
                     (pprint '(defun silly (a b c) (let ((my-sum (+ a b c))) (* my-sum (+ 1 my-sum))))))

      (DEFUN SILLY (A B C)
        (LET ((MY-SUM (+ A B C)))
          (* MY-SUM (+ 1 MY-SUM))))

<comment><AUTHOR> Thanks for this elaboration!

Obviously there is lot of community know-how, that I am not aware of. Thanks for explaining them in detail. Learning to spot all those pattern is crucial if you want to be come effective at reading lisp. For the untrained eye lisp code often looks horrid.

I don't think we disagree, though. All I am saying is that: "Riding the Lisp bicycle has been difficult for me. Here one thing I struggled with a lot." Maybe other people have the same difficulty and find this helpful.

<comment><USER5> Right, you need to train yourself with these visual/structural tree patterns. The code does no longer look horrid and the parentheses will go away.

You were not knowing that there is actual structure and thus you could not see the forest through the trees.</comment></comment></comment>


Is it OK to have kids? - <AUTHOR> https://aeon.co/essays/do-people-have-a-moral-duty-to-have-children-if-they-can <comment><USER4> Pain and suffering are part of the human condition. If you want to blame them on the immoral act of some creator, blame God, not those who choose to have children.

<comment><USER5> But considering the really poor chances there being a god... Anyway, you are responsible for your own choices.

Also "Pain and suffering are part of the human condition" does seam like a poor, religiously ingrained, philosophy. I do not see why it has to be the case.

<comment><USER6> "You are responsible for your own choices" is just as much of a religious philosophy as "pain and suffering are part of the human condition". Both are non-empirical, irrational expressions of your personal observations and beliefs.

</comment></comment></comment><comment><USER7> Aeon used to be interesting but it's just gone off the deep end recently.

</comment><comment><USER8> Under the heading of "creating people seems like a necessary but not terribly nice thing to be doing":

Less us ponder the subject of having children in the face of the existence of aging coupled with the possibility of progressively defeating aging - <USER2> to the point where some of us alive today will escape age-related death by the skin of our teeth. Or perhaps not if we don't get our act together here and now. Evidently we need to have children in order to have the chance of incrementally defeating aging by building ever better versions of a biological repair kit to reverse ever more of the damage that causes degeneration and death. This task is one of decades, long enough that it may be today's researchers who start the job, but it'll be younger hands that finish it - their children and grandchildren. Yet creating people is somewhat like drafting them into a war and a human condition that they didn't ask for:

"There's a task we need you for, son, you and the rest of your generation. We may or may not manage to complete it, but we certainly won't without your help - and if we don't get this done, we're dead all too soon, a slow death, heavy on the pain and suffering. We'll be dragged away first so you get to see the end in all its horror, with plenty of sleepless nights to think it over before it happens to you as well. Oh yes, and most people don't see the need for any of this work and think the pain and suffering and death is just dandy. So that's the deal, a raw one all round - welcome to the asylum, son. No need to thank me."

I'm sympathetic to the hedonistic imperative view of pain and priorities in technological development, and I also think there's a fair but short-sighted argument to be made for nihilism along the lines of voluntary species extinction. It runs something along the lines of a utilitarian consideration of suffering, slavery, existence, natural rights, and similar concerns. I call that short-sighted because, if we're going to be utilitarian, we should consider that the point and beneficiary of all this technological development - not to mention the bone mountain of suffering and corpses we stand upon and continue to build - is very much not us. Our own longevity and diminished future suffering is a tiny side-effect on the way to providing massively greater benefits to our future descendants, be they biological or machine intelligences. They will be so greatly endowed by the cumulative efforts in advancing technology that ensuring their existence (and ensuring that it comes about as soon as possible) will far and away outweigh our needs in any utilitarian consideration. We are short-lived, small in number, small in mind, and planet-bound evolved intelligences, while our descendants of future centuries will not be any of those things. There will be trillions of them, a near infinite variety of forms of mind, ageless, absent suffering, and hopefully wiser than us for it. They will exist because we, our forebears, and our children suffered the limitations and risks of our present existence in order to build the road that little bit further - and because we chose to inflict the same on others by bringing them into being.

So having children still looks to me largely like throwing new people into a horrible situation in order that some of them will try make it better - <USER3> with some hope that they might benefit as individuals, but also the considerable risk that they will not, and suffer greatly as a consequence. Beyond that, there is an abstract grail that will be enjoyed by people yet to come - our descendants made in biology or machinery - who we will likely never know, and whose era will be brilliant and golden beyond our imagining, but only if we strive to lay the foundation stones here and now.

<comment><USER9> Making an argument that not having kids is an extremely bad decision for whatever subgroup you are part of is trivial. The only exception would be if all possible subgroups agreed this, which seems impossible. Any group of people choosing to do this would, of course, be doomed. So let's analyze what happens to the number of kids.

Since there is competition you're even making it easier for other groups to have kids, and therefore more kids will come from them. So the result of having less kids results in more kids, especially in the subsequent generations. (assuming you're unwilling to do what China does to limit kids)

So for any individual the choice is simple, but indirect. 2 options:

Either one doesn't believe your argument about less kids is better. Obviously those will have kids.

Or one does believe your argument about having less kids, as less kids will minimize suffering. In this case, you should have kids.

So there is no moral argument either way. But of course we know reality : kids are a burden. People are refusing to have kids because it would (at the moment) affect them negatively. I have a large family, and so I can compare the situation of older people having (lots of) kids to the situation of older people who are alone. I don't agree that for an individual not having kids is better, in fact it is far worse.

But a shortsighted view that for 10-20 years having kids will be a burden that prevents them from realizing their dreams is quite common. And I can certainly understand that argument even if I do disagree with it.

But "the good of the human race" is a horrible, misguided reason not to have kids. It is good for the human race to have a minimum number of kids (I'd guess about 2.1 on average).</comment></comment>


Ask HN: How much math one needs to know to learn programming? - <AUTHOR> I am learning Python, and I have no prior experience of any language. I am switching career.<p>I want to know if math is required to learn programming for web dev. <comment><USER2> Maths for web dev? You don't need any maths beyond what you'll learn in junior school:

Arithmetic

Basic algebra

Basic Boolean logic

That's it. You'll mostly be wrangling interfaces and/or api/DB plumbing. Making things testable and repeatable. Reliable. Etc.

In my experience most programmers are useless at maths. Mathematicians and programmers are mostly separate populations and it's relatively rare to find a solid programmer that's also a solid mathematician or vice versa: totally different set of aesthetic preferences.

Meanwhile you will find plenty of programmers that skim a book on graph theory and think they know something. Or mathematicians that write crap matlab code and say they can programme.

<comment><USER3> Also, knowing math will make you think about how you could solve the problem, while everyone else's first reaction is "there is probably already a library for that". And there usually already _is_ a library that does a similar enough thing.

Therefore, knowing math will leave you endlessly frustrated, because the kinds of tasks you would like to do are exactly the kinds of tasks you shouldn't be doing.

</comment></comment><comment><USER4> In general, nothing more than grade school math is needed to learn web development. Obviously, if your developing a statistical healthcare application you'll need more math skills, but for most websites and web applications all you need is basic math skills.

<comment><USER5> Indeed, I've seldom needed more than GCSE math (level of a 16 year old in the UK) in fact I can think of only a handful of times I've needed A-Level equivalent math (on notable occasion was solar insolation calculations).

It's just not a big issue in the life of a typical enterprise developer.

</comment></comment><comment><USER6> I would learn the basics in a few fields that they don't generally cover in high school: graph theory, number theory, set theory, statistics, and calculus. Just getting a few of the basic ideas from each of those in your tool kit will make it easier to think about certain classes of problems. Plus, you'll have a better idea where to look for an answer when you hit a problem that requires you to go deeper.

For web dev, statistics will help you spot A/B tests that are broken or meaningless. And set theory and combinatorics will help with database queries.

</comment><comment><USER7> Cliff Notes: Learn Math, I didn't, but still can do dev work, but I really regret it.

I know almost nothing, really. I stopped taking math classes around 16/17 years old, I just wasn't able to grasp it and gave up.

Double majored at Uni in CS & Geography which allowed me to pass over all the mathematical CS courses. If you can't substitute out the math heavy papers with papers from another course, over your whole `X` years studying, then you better upskill as best as you can or it will be tough going.

But here I am in a dev job, that I got because it's hard to find geographers who can code, with my biggest regret being that I did not get enough of a mathematical understanding earlier on. It is such a useful and universal skill, it is a pity not to be confident in applying it. But the best thing is that if you work with a solid team you will have people around you to fill in your weaknesses, and you can fill in theirs.

You can get by without it, if you find a niche you can specialize in where coders are hard to come by, but you will find yourself really needing it and having to pass work on to someone else or finding some library.

</comment><comment><USER8> If you are going to learn programming on your own and not planning to go through a computer science curriculum, then I would say you don't need much more than basic algebra and arithmetic. The majority of fields in demand (Web dev, mobile, etc.) don't involve much math if any.

If you are going to a CS curriculum then I would advise learning up to Calc 1 pretty well before hand in order to be ready because I had to take all of calculus (1, 2, 3), linear algebra, and a few CS classes that involved math like (Discrete Mathematics, Theoretical CS, etc.). It wasn't fun because I am not strong in math but I got through them.

Most importantly, if math is not your strongest subject, don't be discouraged to still learn programming because I was in the same situation and now after going through a CS degree and working at a few companies, I can confidently tell you that everyday work will very rarely involve any math beyond basic arithmetic.

</comment><comment><USER9> Database Applications Also require very low levels of math. And depending on whether you use Artificial, Convolutional Neural Networks for solving PDE's, or Image Classification will depend on the level of mathematical training you will need to verify or analyse your results.

<comment><USER10> Depends what you mean by database applications. If you want to get your hands dirty and understand SQL theory, you'll need to learn relational algebra.

</comment></comment><comment><USER11> You don't need much math, in terms of specific knowledge. However, it is helpful to be able to reason in a mathematical kind of way.

</comment><comment><USER12> Of all the maths I use pushing code, statistics are likely the skills I keep using the most. Not required, but helpful.

<comment><USER13> I believe a basic understanding of statistics is a good life skill which everyone should know.

</comment></comment><comment><USER14> If math is not a big part of the underlying domain for your software you won't need it at all. But being good at math (like not having problems with it) is a strong signal you'll be able to pick programming fairly easily.

</comment><comment><USER15> Just recently I've had to implement camera preview on Android. This common task has required some knowledge of how transform matrices work. For me personally maths has been indispensable during my career.

</comment><comment><USER16> Basic arithmetic and algebraic substitution.

Set theory, graph theory, stats, discrete math and linear algebra all rear their heads in various fields, but you could go your entire web dev career without having to understand them.

</comment><comment><USER17> Basically none beyond arithmetic, though more is helpful.

You need some to learn computer science in any depth, though they can be learned togethe and quite a bit for some particular kinds of programming or application domains.

</comment><comment><USER18> Honestly very little. Just the basics. Simple Algebra, arithmetic and geometry are enough. I would suggest start with a language without worrying about math. If you need for anything, learn it on the fly

</comment><comment><USER19> Mathematics isn't just another toolbox - it also represents an attitude. Well- structured code, modularization, and generality are natural by-products of theorem proving. So interest in any area of mathematics will make you a better programmer.

</comment><comment><USER20> Math is not required to learn web dev.</comment>


Help this Google Glas related Kickstarter project get off the ground - <AUTHOR> http://glass-apps.org/on-kickstarter-google-glass-adapter-for-prescription-glasses

<comment><USER2> At first I thought this was a joke, but they seem to be serious. Looks like they should reach their goal easily.

<comment><AUTHOR> it is something and they told me they will make a special version for google glass. They need a google glass explorer to take a look and measure the dimensions and make the best possible adapter</comment></comment>


Avaaz website under massive attack, from a government or large corporation? - <AUTHOR> https://secure.avaaz.org/en/massive_attack_on_avaaz_a/

<comment><USER2> Call me a cynic, but phrases like:

 _"An expert is telling us that an attack this large is likely coming from a government or large corporation"_

 _"Many wondered when these people would come after us."_

Make me seriously doubt the integrity of the claim. Speaking of Murdoch, this is the exact same trick tabloid papers pull when fabricating or exaggerating their own reporting to fit their agenda.

</comment><comment><USER3> Site seems pretty snappy to me. Maybe people donated to their "defence fund" and kept the site running, hahaha. Seriously though, 36 hour DDOS attacks from the government or a large corporation? Why would people bother.

</comment><comment><USER4> I have two reactions to this.

1) Welcome to the Internet; if you're high profile, you're going to get attacked. This is the new reality and why companies like Facebook or Google support/tolerate cybersecurity legislation. The growth in threats just in the last 12 months has been significant.

2) Your site loaded instantly for me, and you're asking for money...color me suspicious. At minimum I'd like to see some actual evidence before believing that Rupert Murdoch or Exxon are DDOSing your site.

<comment><USER5> I second that. Just-ping.com says the website is running fine (except in vancouver and china).

[http://just-ping.com/index.php?vh=http%3A%2F%2Favaaz.org%2Fe...](http://just- ping.com/index.php?vh=http%3A%2F%2Favaaz.org%2Fen%2F&c=&s=ping%21)

But wait... maybe just-ping.com is in on the conspiracy!

</comment></comment><comment><USER6> One-one-thousand, two-one-thousand, loaded.</comment>