Los Alamos says it can manufacture gasoline from CO2 in the air at a price of $4.60 a gallon - <AUTHOR> http://dotearth.blogs.nytimes.com/2008/02/13/federal-lab-says-it-can-harvest-fuel-from-air/index.html?ref=science

<comment><USER2> Does this really make sense? Nuclear processes are used to generate power to drive a process which produces gasoline from CO2 in the air. That gasoline is then used... to generate power. Why not just cut out the middle step?

I don't think fully-electric cars are too far off, and when that is the case, the nuclear energy will be able to power the cars directly (or at least their batteries), along with solar power or whatever else.

Electricity is simply much more flexible since it can be produced and used in so many ways and distributed relatively cheaply. Gasoline only works for combustion engines, and it has to be manually transported. Our current infatuation with gasoline is going to look pretty stupid when we finally get to the point that it's hardly used anymore.

<comment><USER3> Agreed. Electric cars need the high performance Li-Ion batteries, the 3 minutes charge = 3 hour play kind that you get in MP3 players. I mean, if they merely got it to the point where you could go to a fuel station, plug it in to charge and go in to grab a coke and by the time you get to the cashier your car is charged.

I mean, combine this with regenative braking and thin-film solar cells (so as not to add unnecessary weight and lower mileage), you could greatly improve over most current full-electric cars.

<comment><USER4> I assume that you guys have seen the Tesla Roadster (<http://teslamotors.com>).

What I'm hearing from them is that the critical feature that is holding electric cars back is the power system. This field is pretty experimental but someone who could come up with the magic battery that charges in 5 minutes would probably have a valuable system on their hands.

Another angle would be to have cars that run on batteries that can be swapped out at gas stations as needed. There are a bunch of issues that need to be considered for this approach but it's still a neat idea.

<comment><USER3> Yeah I've seen the Tesla Roadster, and all electric cars need to be manufactured like that. I have a Li-Ion drill, it takes 30 minutes to get a full charge yet it takes me over 4 hours to run it down with constant use, so I don't get how some electric cars can barely get you around town and take 8 hours to charge on a night.

I'd accept 2 hour charge time for in city driving, but most of my driving with work isn't in a city, and thus electrics are useless until they're as fast to charge as a petrol is to fill.

<comment><USER5> Maybe because moving an electric car takes slightly more energy than turning a drill bit. It's not really the same class of problem.

</comment></comment></comment></comment></comment><comment><USER6> How much is $4.60 a gallon in $ per liter?

And when are you going to convert to the metric system like any other sane country?

<comment><USER7> Google says about 1.21519144 U.S. dollars per litre

[http://www.google.com/search?q=4.60+USD+per+gallon+in+USD+pe...](http://www.google.com/search?q=4.60+USD+per+gallon+in+USD+per+litre)</comment></comment>


w2ui - <AUTHOR> http://w2ui.com/web/home

<comment><USER2> The demo page starts with 'auto-resize', but resizing the demo page does not resize the demo layout.

"Once you have rendered layout into a container, it will attach a listener to the window and will make sure it takes up all the width and height available from the container."

This should be done with css media queries, not in javascript. Why would you want a window resize listener when all this can be done with css much more efficiently and with more fine-grained control?

Then going to the 'grid', I was expecting the search to be live search of the fields, but I had to click the 'search' button to get it to update.

I'm using Chrome 39 beta, no errors are coming up in my console.

With other modern tools, I don't see where w2ui is really offering.</comment>


Ask HN: How did you overcome your fear of death? - <AUTHOR> I'm not talking about death as an accident but the inevitable thing happens to everyone. I'm 21 and the feeling of being unconscious after death for unlimited time  scares the sh*t out of me sometimes. Have you had this feeling , how did you overcome it?<p>PS : obviously not everyone has this feeling, I'm just asking people who had this feeling. <comment><USER2> I think that the fear of death is natural, normal, and impossible to overcome. The only thing you can do about it is to stop thinking about it all the time and being stressed because of it (or trick yourself into thinking that death is too distant/not that painful/positive in some way). I cope with the fear of death by just not thinking about it too much since I fail to find anything positive about future death. And when I do, I strive to use thoughts about death as a tool for making my life more productive/efficient.

</comment><comment><USER3> Dying is part of living. You cannot live without dying eventually (at least for now). Being born ultimately leads to dying somewhen. There is no way around that, that's just how living works.

Since you cannot change anything about that you just have to accept the fact of being dead in the future. So what to do about it? First thing is to realize that everything is temporary. You, humanity, earth, the universe. It's just a huge coincidence that you were born in the first place (you can try doing the math ;) ). So while you cannot prevent yourself from dying, you can do the best of the time when you are not dead.

You cannot know, what happens when you are dead. Maybe there is nothing, maybe there's some kind of paradise, maybe you can f*ck 42 virgins or maybe you are born again depending on how much karma you collected. Maybe the universe and existence itself is just a loop and when you die, you just start over where you started last time. Who knows and since you cannot know, there is not much sense in worrying about. Just be the best human you can be and accept the fact that someday it will be over.

</comment><comment><USER4> What that called is Thanatophobia. I have had this for quite a long time. The earliest I remember was when I was about 7-8 years old. I am 25 now.

I have not overcome it really. It just goes and comes with periods - makes it incredibly hard to sleep when it does.

The periods are worst when I am stressed but they seem to get a bit better as I get older (ironically I suppose).

A thing that helped greatly is talking about it with my wife and going to sleep together, it relaxes me enough to fall asleep relatively easy..

I am not sure what would work permanently as I tried various things over the past 17 years but nothing made it go away entirely.

If you want to talk more in detail about it, you can sent me an email (it is in my profile).

Good luck with it!</comment>


Coming Soon: EC2 Dedicated Hosts - <AUTHOR> https://aws.amazon.com/blogs/aws/coming-soon-ec2-dedicated-hosts/ <comment><USER2> It's interesting that AWS markets this as a trick to reduce enterprise software license costs.

I would guess that the more common use case will be to get more predictable performance by preventing "noisy-neighbor" issues.

Will the next step be AWS supporting non-virtualized instances?

<comment><USER3> It's less a trick, than a sad reality for certain types of software. I'm sure there are others, but for Oracle, for example, currently they require you to have a perpetual license for every core that's running inside your Amazon VM, which makes the pricing crazy.

I'm sure Oracle said, "Hey, either price it like this, or you can price it based on the underlying hardware, which is how we do it with VMware".

So at least now it might be feasible for enterprises to run some of these traditionally licensed software packages without either an all you can eat license or blatantly violating the rules.

</comment></comment><comment><USER4> This sounds like a fully-baked and useful version of Dedicated Instances ([http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/dedica...](http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/dedicated- instance.html)).

</comment><comment><USER5> What next, actual, physical, bare metal servers for better performance? I am actually doing this already for my test instances - the server running in our office is running Linux and docker with no intermediate vm layer.

</comment><comment><USER6> This falls into the "announcement of an announcement" category that is generally off-topic for HN. There's no harm in waiting for the actual release.</comment>


Simple OAuth login with JWt and JRuby - <AUTHOR> http://blog.elevat.eu/2012/04/simple-oauth-login-with-jwt-and-jruby/

<comment><AUTHOR> Constructive criticism is welcome!</comment>


Alan L. Hart - <AUTHOR> https://en.wikipedia.org/wiki/Alan_L._Hart <comment><USER2> > "Each of us must take into account the raw material which heredity dealt us > at birth and the opportunities we have had along the way, and then work out > for ourselves a sensible evaluation of our personalities and > accomplishments".

Bravo, sir. All respect for a hard life well lived.

</comment><comment><USER3> > Doctors, including Hart, were realizing that myriad illnesses—consumption, > phthisis, phthisis pulmonalis, Koch's disease, scrofula, lupus vulgaris, > white plague, King's evil, Pott's disease, and Gibbus—were all in fact cases > of tuberculosis (TB). TB usually attacked victims' lungs first; Hart was > among the first physicians to document how it then spread, via the > circulatory system, causing lesions on the kidneys, spine, and brain, > eventually resulting in death.

> Hart became interested in [X-rays] potential for detecting tuberculosis. > Since the disease often presented no symptoms in its early stages, x-ray > screening was invaluable for early detection. Even rudimentary early x-ray > machines could detect the disease before it became critical. This allowed > early treatment, often saving the patient's life. It also meant sufferers > could be identified and isolated from the population, greatly lessening the > spread of the disease.

Incredible. Developed a fast means to accurately identify the most common killer in America, allowing doctors to treat it and isolate the infected.

How many more would have died if no one has figured this out until even a year later.</comment>


Spew: UI-less Firefox extension feedreader - <AUTHOR> https://github.com/akkartik/spew?take2

<comment><USER2> Author here. Here are some screenshots that may help convey what it's like to use: [http://imgur.com/a/kT3NN](http://imgur.com/a/kT3NN)

<comment><USER3> Can you please add link to a binary? Or even link to FF plugin directory... SDK install as a pre-requirement is bit too much to 99.9% of people.

<comment><USER2> Here you go: [https://addons.mozilla.org/en-US/firefox/addon/spew-the- ui-l...](https://addons.mozilla.org/en-US/firefox/addon/spew-the-ui-less- feedreader)</comment></comment></comment>


Alpha Go Zero: How and Why It Works - <AUTHOR> http://tim.hibal.org/blog/alpha-zero-how-and-why-it-works/ <comment><USER2> The main reason AlphaGo Zero learns so much faster than its predecessors is because it uses temporal-difference learning.[1] This effectively removes a _huge_ amount of the value network's state space for the learning algorithm to search through, since it bakes in the assumption that a move's value ought to equal that of the best available move in the following board position, which is exactly what you'd expect for a game like Go.

A secondary reason for AlphaGo Zero's performance is that it combines both value and policy networks into a single network, since it's redundant to have two networks for move selection.

These are the two biggest distinguishing characteristics of AlphaGo Zero compared to previous AlphaGos, and the OP doesn't discuss either of them.

[1] [https://en.wikipedia.org/wiki/Temporal_difference_learning](https://en.wikipedia.org/wiki/Temporal_difference_learning)

<comment><USER3> Interestingly, the idea behind temporal difference learning is more or less the intuition behind how people price derivatives in finance.

The expected value of a contract at time T, estimated at some time t < T, is assumed to be equal (up to discounting) for all t -- e.g. if today we think the contract will be worth $100 a year later, then we also think that the _expected estimate, made n months from now, of the value [12-n] months later, will also be $100_. This allows you to shrink the state space considerably.

You can usually work out the payoff of a derivatives in different scenarios given rational exercise decisions by all contract participants. The calculation assumes that every market participant makes the best possible decision given the information they had available at the time by either explicitly or implicitly building a tree and working backwards, back- propagating the 'future' value back to the root.

This closely resembles the modeling of a discrete adversarial game, except the payoffs need to make reference to random market variables like the stock price, so the tree nodes are not just indexed by participant action, but also by variables.

There's actually a nice resemblance between the Longstaff-Schwarz method of pricing American options and MCTS + Alphago, except that the former is using kernel regressions instead of deep neural nets and we sample from a continuous space with an assumed probability distribution instead of a discrete space guided by a policy network [1].

[1] [https://people.math.ethz.ch/~hjfurrer/teaching/LongstaffSchw...](https://people.math.ethz.ch/~hjfurrer/teaching/LongstaffSchwartzAmericanOptionsLeastSquareMonteCarlo.pdf)

<comment><USER4> I think the bellman equation (which is used extensively in reinforcement learning) is also taught in stochastic calculus for finance (except in the continuous form?). [https://en.wikipedia.org/wiki/Hamilton%E2%80%93Jacobi%E2%80%...](https://en.wikipedia.org/wiki/Hamilton%E2%80%93Jacobi%E2%80%93Bellman_equation)

My memory is hazy so there might not be a real connection here.

<comment><USER5> Yup, and a lot more! The Hamilton-Jacobi-Bellman equations come up in anything that can be formulated as an optimal control problem.

I'm not an expert, but he is: [http://www.athenasc.com/dpbook.html](http://www.athenasc.com/dpbook.html)

</comment></comment></comment></comment><comment><USER6> Some additional discussion: [https://www.reddit.com/r/reinforcementlearning/comments/778v...](https://www.reddit.com/r/reinforcementlearning/comments/778vbk/mastering_the_game_of_go_without_human_knowledge/) [https://www.reddit.com/r/MachineLearning/comments/7apvr4/d_i...](https://www.reddit.com/r/MachineLearning/comments/7apvr4/d_it_seems_like_alphago_zeros_biggest_successes/)

</comment><comment><USER7> If the author's here: some of the math formulas don't render correctly. In particular, 10^170 is parsed as 10^{1}70, and $5478$ shows up without TeX applied to it.

<comment><AUTHOR> Thanks, fixed those

<comment><USER8> Another minor fix:

> A new paper was released a few days detailing a new neural net

I believe you mean "a few days _ago_ "?

<comment><AUTHOR> kk

</comment></comment></comment></comment><comment><USER9> They are two things a human brain does when playing chess or go: evaluating a position and mentally playing some positions (by doing a search tree).

The AlphaGo neural network is able to do the first part (evaluating positions) but the search tree is still a hand crafted algorithm. Do they have plans to work on a version with a pure neural network? (i.e. a version which would be able to learn how to do a search tree.)

<comment><USER10> Who is to say that the human brain doesn't have a different cell type to do this kind of search. The neocortex is shaped in a different way than other neural cells so I imagine search and evaluation are different architectures.

But you're right. A NN way to do Montecarlo search in GPUs would make things even simpler.

</comment></comment><comment><USER11> Would be really cool to see a generic framework for this, where you can plug in the rules of your discrete-deterministic-game-with-perfect-information and get a superhuman bot. Does something like this already exist?

<comment><USER10> If anyone's interested who understands alpha zero in depth. I would love to start a github project to implement a super not that players checkers, tic tac toe, chess, go etc and all related games on the browser via an openai like interface.

</comment></comment><comment><USER12> Given that AlphaGo Zero was trained on several million games of self plays, each game involving hundreds of steps, each step with 1600 MCTS simulations, the total number of board positions it has considered is on the order of trillions. While impressive it pales in comparison to the number of possible board positions of 10^170 ([https://en.m.wikipedia.org/wiki/Go_and_mathematics](https://en.m.wikipedia.org/wiki/Go_and_mathematics)). So its amazing performance tells us that:

1\. Possibly the elegant rule of the game cuts down the search space so much that there is a learnable function that gives us optimal MCTS supervision;

2\. Or CNN approximates human visual intuition so well, so while Zero has not evaluated so many board positions it has evaluated all the positions that human has ever considered - so it remains possible that a different network could produce different strategies and be better than Zero.

</comment><comment><USER13> > It is interesting to see how quickly the field of AI is progressing. Those > who claim we will be able to see the robot overlords coming in time should > take heed - these AI's will only be human-level for a brief instant before > blasting past us into superhuman territories, never to look back.

This final paragraph is just editorializing. A computer will never care about anything (including games like Go and domination of other beings) that it is not programmed to imitate care about, and will thus remain perennially unmotivated.

Also, my intuition says that gradient descent is an ugly hack and that there HAS to be some better way (like a direct way) to get at the inverse of a matrix (not just in specific cases but in the general case!), but I digress, and not being a mathematician, perhaps someone has already proved somehow that a general method to directly and efficiently invert all possible matrices is impossible

<comment><USER14> What do you mean with inverting a matrix? In the usual definition it is well studied and understood [https://en.m.wikipedia.org/wiki/Invertible_matrix#Methods_of...](https://en.m.wikipedia.org/wiki/Invertible_matrix#Methods_of_matrix_inversion)

<comment><USER15> They might be talking about an interpretation of gradient descent that this article provides: [http://blog.mrtz.org/2013/09/07/the-zen-of-gradient- descent....](http://blog.mrtz.org/2013/09/07/the-zen-of-gradient-descent.html)

<comment><USER13> I elaborated here: [https://news.ycombinator.com/item?id=15639863](https://news.ycombinator.com/item?id=15639863)

</comment></comment></comment></comment><comment><USER16> I wonder how the STYLE of Alpha Go Zero is regarded by human experts. Is it far different from AlphaGo? Why bother learning from AlphaGo if they can learn from AlphaGo Zero?

Did they unleash a second "Master" program?

I am wondering if the "better" strategy moves are now super wacky and weird and break all theory.

<comment><USER17> At least initial reports are that alphaGo Zero is more human-like than Master. Zero packs even more of the inhuman ability to pick the most critical part of the board for each move, but less weird looking stuff.

In fact, one of the obvious differences between AlphaGo Zero and top human players, is much more play on safe opening spots, which has been out of fashion among human pros for a hundred years or so.

<comment><USER18> >At least initial reports are that alphaGo Zero is more human-like than Master.

Empirically, this is not correct. The original AlphaGo achieved a 57% accuracy at predicting expert moves.

I can't find an exact number, but based on the graph in the nature article, AlphaGo Zero has a less than 50% accuracy at predicting human moves. Eyeballing the graph, it looks like the supervized learning variant of AlphaGo Zero scored <55% at predicting human moves. Better than reinforcement learning AlphaGo Zero, but still worse then the original AlphaGo.

Of course, it is not clear that ability to predict _the_ move humans will play is the best metric to measure how human like a computer plays. It is just the only objective metric we have. Although, if this were an actual research question, we could probably come up with better metrics.

[https://storage.googleapis.com/deepmind- media/alphago/AlphaG...](https://storage.googleapis.com/deepmind- media/alphago/AlphaGoNaturePaper.pdf)

[https://www.nature.com/nature/journal/v550/n7676/pdf/nature2...](https://www.nature.com/nature/journal/v550/n7676/pdf/nature24270.pdf)

<comment><USER19> I'd have thought the metric to watch was "% of moves made which seemed kooky to human experts", not "% of human experts moves with which the program agrees."

</comment></comment></comment></comment><comment><USER20> someone posted an attempt at an open source implementation of alphagozero [https://github.com/yhyu13/AlphaGOZero-python- tensorflow](https://github.com/yhyu13/AlphaGOZero-python-tensorflow)

anyone try it yet ?

<comment><USER21> There's also Leela Zero [1].

[1] [https://github.com/gcp/leela-zero](https://github.com/gcp/leela-zero)

</comment></comment><comment><USER22> Saw the AlphaGo movie at a festival recently.

Been following the AlphaGo Zero developments, which leap-frog what was going on in the movie (although still very much worth seeing).

One thing I was curious about is if Go would be considered solved, either hard or weakly solved, since AlphaGo Zero at this point doesn't seem to be able to be beat by any living human. Wikipedia does not list it as solved in either sense, and I was wondering if this was an oversight.

<comment><USER18> Go still has not been Ultra-weakly solved (eg. we do not know who is supposed to win).

If AlphaGo has weakly solved go, then it should either have a 100% win rate when playing against itself as white, or a 100% win rate when playing against itself as black.

<comment><USER10> Or 100% rate of a draw. Like tic tac toe.

<comment><USER23> Due to komi, a go game cannot be drawn. That is, the 2nd player is awarded some number of points for the disadvantage of moving 2nd, and that number of points typically has a fractional 0.5 in it to break ties.

The most common komi values these days are 6.5 or 7.5, depending on ruleset.

<comment><USER24> Depending on the rules, a game may be drawn due to an "eternal life" (an endless forced repetition).

[https://senseis.xmp.net/?EternalLife](https://senseis.xmp.net/?EternalLife)

In Go this is very rare (as opposed to e.g. chess).

</comment></comment></comment></comment></comment><comment><USER25> I don't get what is new in the set of attributes that this article describes.

Monte Carlo was already used in 2005 in AIs playing on KGS. Gradient Descent is a basic algorithm is a basic algorithm that I saw in an AI class in ~2008 as well. I bet both are even a lot older and well known by all experts.

This is not what makes AlphaGo special or Zero successful. The curious thing about Zero is that usually with Gradient Descent you run a huge risk of running into a local maximum and then stop evolving because every evolution makes you not better than the current step.

So one question is actually how they used these same old algorithms so much more efficiently, and the second question is how did they overcame the local maximum problem. Additionally there may be other problems involved that experts know better than me.

But an explanation of basic algorthims can't be the answer.

<comment><USER6> > The curious thing about Zero is that usually with Gradient Descent you run a > huge risk of running into a local maximum and then stop evolving because > every evolution makes you not better than the current step.

No. The curious thing is that you can train a godawful huge NN with 40 layers via pure self-play with no checkpoints or baselines or pretraining or library of hard problems or any kind of stabilization mechanism, and it _won 't_ diverge but will learn incredibly rapidly and well and stably. As Silver says in the AmA, _all_ their attempts at pure self-play ran into the usual divergence problems where the training explodes and engages in catastrophic forgetting, which is what the RL folklore predicts will happen if you try to do that. Local maximums are not the problem - the problem is the self-play can't even find a local maximum much less maintain it or improve it.

> So one question is actually how they used these same old algorithms so much > more efficiently, and the second question is how did they overcame the local > maximum problem.

Er, this is exactly what OP is all about: the Monte Carlo tree search supervision. That's how they used them.

<comment><USER26> If you read the AGZ paper closely, they actually use checkpoints during training. Specifically, during training they only perform updates to the "stable" set of parameters when the current "learning" set of parameters produces a policy which beats the stable set at least 55% of the time. The current stable parameters are what they use for generating the self-play data which they use to update the current "learning" parameters. I believe this is only mentioned in the supplementary material...

<comment><USER6> I did and I would point out that while they use checkpoints, the training curves indicate this is not necessary, and what I meant is that they do _not_ use the usual self-play (and evolutionary) mechanism of checkpoints from throughout the training history which is necessary to combat catastrophic forgetting (and which apparently wasn't enough to stabilize Zero on its own as it is the single most obvious thing to do but Silver notes all the pre-Zero self-plays diverged until they finally came up with that of MCTS supervision). The checkpoint mechanism there appears no more necessary than checkpoints in training any NN - it's critical to avoid a random error or bug wasting weeks of time but does not affect the training dynamics in any important way.

<comment><USER6> (And Anthony et al 2017 don't use checkpoints at all, noting that it slows things down a lot for no benefit in their Hex agent.)

</comment></comment></comment></comment></comment><comment><USER27> Are there any plans to do this for Chess?

I imagine that this is an iteration of the Alpha Go engine, people working on this are very current with Alpha Go.

If Chess is similar, then wouldn't DeepMind be able to bootstrap game knowledge. Perhaps this isn't a big goal, but Chess is Chess after all.

<comment><USER20> There was a project for chess, called giraffe [https://bitbucket.org/waterreaction/giraffe](https://bitbucket.org/waterreaction/giraffe) whose author shut it down after joining google deep mind: [http://www.talkchess.com/forum/viewtopic.php?t=59003](http://www.talkchess.com/forum/viewtopic.php?t=59003)

He thinks it's only a matter of time till machine learning beats hand crafted systems like stockfish even in chess.

[http://arxiv.org/abs/1509.01549](http://arxiv.org/abs/1509.01549)

<comment><USER10> Does alphazero beat stock fish purely by self play. That would be huge. Stock fish is the result of so many hand crafted optimizations over a large game play dataset.

<comment><USER28> Intuitively you'd expect deep learning to improve on Stockfish's evaluation of positions, but perhaps not at the same level of throughput. I'm also intrigued as to whether a purely self-taught AI system can compare in the endgame to an engine with access to a 7-piece endgame tablebase, whose evaluation of those positions is obviously _perfect_.

</comment></comment></comment></comment><comment><USER29> Go has been studied for hundreds of years. In many cases, by people who study the game since their childhood and work on it as a full-time occupation.

The consequence of Alpha Go Zero is that it can, in a matter of days, disregard and surpass all human knowledge about the game.

Maximizing a score margin has been equated for a long time with maximizing your probability of winning. Alpha Go doesn't play like that... it's a substantial paradigm shift. If you see the early commentaries you will see that human players were initially saying Alpha Go moves were mistakes because they were slow and wasted opportunities to get more territory, to then realize that Alpha Go was actually winning.

<comment><USER30> I think people have always understood the difference between maximising probability of victory versus score. Even in amateur games you'll get the feeling "I could fight here, and it would be complicated, but I might get a huge advantage, or I could just play safe and keep my advantage."

</comment></comment><comment><USER31> Are there adversarial examples for Alpha Go Zero?

<comment><USER17> What do you mean?

AlphaGo Zero has played itself, and they have published 20 of those games.

<comment><USER32> the OP is referring to the use of a GAN, which is a different type of setup.

GAN's were not used for alphaGo, as the article points out Deepmind uses reinforcement learning and MCTS.

<comment><USER33> They're most likely referring to adversarial attacks where degenerate inputs are constructed that could cause AlphaGo Zero to perform sub-optimally or catastrophically fail (see OpenAI Research [0]). This is distinct from generative adversarial networks (GANs) or adversarial self-play (which I guess AlphaGo Zero is an example of).

[0] [https://blog.openai.com/adversarial-example- research/](https://blog.openai.com/adversarial-example-research/)

</comment></comment></comment></comment><comment><USER34> "On a long enough timeline, everything is a discrete game." (With apologies to _Fight Club_)

Personally, I look forward to the day when the software I own works for me to the extent of optimizing the decisions I make during the day, even many mundane ones. Properly executed, such a system could make a big difference in my quality of life. I believe that a big piece that is missing is a solid life-model, a life-representation, that can be optimized. Once that is defined, an RNN or MCS can optimize it and I can reap the benefits.

<comment><USER35> If it is optimising all your decisions, even the mundane ones, is it really _your_ life any more? If it plugged into your brainstem and took over, by definition nobody else would notice the difference (you always did what it said anyway, or you'd be suboptimal), so why not just flood the 20 watt carbon-based neural network with bliss drugs and let the silicon-based neural network be the person instead?

<comment><USER34> I think sometimes it's hard for us to zoom out and solve bigger problems that we might not see. Like, if I don't enjoy the chore I'm doing, I can make it easier, or I can _eliminate_ it. (Paying a bill in person is a good example). Or, maybe there is an object that is generating a lot of work for me, like a house, and there is an alternative object (an apartment). Heck, sometimes it would be nice to have a record of all the things I enjoy and just have an AI path-find to maximize revisiting those things.

</comment></comment></comment><comment><USER36> What makes this different from a minimax algorithm with alpha-beta pruning?

<comment><USER6> Aside from MCTS being a different tree search method, there is no 'closing of the loop'. In regular MCTS, it is far from unheard of to do the random playouts with instead some 'heavier' heuristic to make the playouts a little better estimators of the node value, but the heavy playouts do not do any kind of learning, the heuristic you start with is what you end with; what makes this analogous to policy iteration (hence the names for the Zero algorithm of 'tree iteration' or 'expert iteration') is that the refined estimates from the multiple heavy playouts are then used to improve the heavy playout heuristic (ie. a NN which can be optimized via backpropagation in a supervised learning of board position -> value). Then in a self-play setting, the MCTS continually refines its heavy heuristic (the NN) until it's so good that the NN+MCTS is superhuman. Then at play time you can drop the MCTS entirely and just use the heavy heuristic to do a very simple tree search to choose a move (which I think might actually be a minimax with a fixed depth but I forget).

</comment></comment><comment><USER37> I wonder how AlphaGo Zero would fare against the others if they were all using the same search algorithm, and I wonder how the search depth vs breadth changes in Zero compared to earlier variants.

</comment><comment><USER38> Mageek, any reason why they haven't applied this to chess yet?

<comment><USER39> MCTS has done very poorly on chess compared to alpha-beta. Chess has a high number of forcing moves in capture sequences, and it's been very difficult to evaluate positions that are not settled. Traditionally an algorithm called a quiescence search is used, but it relies on doing an evaluation at each node of the search, which would be prohibitive with the latency for a network evaluation.

One of the things that amazed me the most about AlphaGo Zero was that they didn't do any tricks to minimize the latency of the network evaluation!

Still, it's certainly worth a try, I'd be extremely interested to see what style of chess a self-trained MCTS chess version would have :).

</comment></comment><comment><USER40> Can this technique be used to write a strong chess engine?

<comment><USER41> Yes. As described, all that's needed is a way to imagine all possible moves from a game state, and check if a game state corresponds with win/tie/loss. That is possible in Tic-Tac-Toe, Chess, and Go.

<comment><USER38> Surprisingly, they haven't done it though. Wonder why. Is it because of the number of different state changes that can occur from a particular position? Maybe Go is easier to solve than chess.

It would be very interesting to see a computer improve on chess openings.

<comment><USER42> First of all, Go is not 'solved'. While the AI plays better than the best humans, it does not play perfectly (nor will any AI program in the forseeable future).

If by 'solving' you mean, developing a strong AI program to play chess, I think the reason they haven't done it because chess is easier to solve than Go. The best computer chess engines are already a lot better than the best humans. Thus, there is not much to gain for Google by finding a new approach.

There have been attempts to use similar approaches for chess, but they al seemed to perform worse than the traditional approach for chess. (Alpha-beta search with a hand-written evaluation function and null-move pruning).

The differences between Go and Chess that might explain this are:

\- Chess has a smaller branching factor (the number of legal moves is around 40 in a typical position in chess, whereas it's over 100 in a typical position for Go)

\- A chess position is easier to evaluate. Simply counting the material already gives a decent evaluation in a lot of cases. This supplemented by hand-written rules gives a solid evaluation of a position

\- The average game of chess is shorter. A game of chess lasts between 20 and 80 moves, whereas a game of Go lasts around 200 games. This makes it a lot more feasible to extend the search far enough to oversee the consequences of each move in chess when compared to Go

</comment></comment></comment></comment><comment><USER43> As someone who has played many a game of Tic-Tac-Toe, I found the numerical examples really hard to follow. s(0,5) is obviously the winning move for the X player, but for some reason all examples seem to favor s(0,1).

<comment><USER44> Winning move how? With which board?

with

        O__
        _X_
        O_X

Then X can't win in the next move and must choose (0,1) between the two O's the left column in order to not lose.

<comment><USER43> D'oh, thanks. I suppose I just misread the oh's.</comment></comment></comment>


Ravi: A dialect of Lua with optional static typing and JIT - <AUTHOR> https://github.com/dibyendumajumdar/ravi <comment> as-j Ravi catches my interest, then frustrates me. The Lua 5.4* compatibility statement. "Well we're 5.3, we forked 5.3 I don't agree with a minor 5.4 detail so we're stuck on 5.3" It's similar to LuaJIT stuck on 5.1.

I don't think it does the Lua world any favors to be stuck on so many versions. It causes extra work for library developers, makes for poor code samples, and confusion for new adopters. It's not like Lua is moving quickly with versions either. Every ~5 years for a minor version?

*) yes I know 5.4 is brand new.

<comment><USER2> Lua does not use semver. Every minor version is a breaking change like changing the way function environments work, adding integers and integer division, or adding <const> and <close>

</comment></comment><comment><USER3> "My motivation is somewhat different - I want to enhance the VM to support more efficient operations when types are known. Type information can be exploited by JIT compilation technology to improve performance. At the same time, I want to keep the language safe and therefore usable by non-expert programmers."

Seems very promising. I will try it.

</comment><comment><USER4> Astrologers proclaim the week of Lua. Amount of Lua-related posts doubles!

</comment><comment><USER5> Anyone know how it compares to Luau, which is also on the front page right now?

<comment><USER2> One major difference is that you can download it, run it, and edit the source code.

</comment></comment><comment><USER2> This is a cool project, and I find the documentation quite useful. However, it is a tough sell to say that the JIT only works on the main stack, where almost none of my code is run.

</comment><comment> ravi-delia I am...interested.</comment>


Ask HN: What Makes Projects "Unfun"? - <AUTHOR> Hi everyone,
I'm surveying data in an effort to make an article I'm interested in writing. I'd like to know an answer to a simple question:
"What has made projects 'unfun' for you?"
-
To jog your memory, here are so disparate subjects that have made projects unfun for me:
1. Interpersonal communication with coworkers/managers
2. ill-defined requirements (bad scope of work, nebulous requirements/needs)
3. Lack of recognition from leadership (no 'good job!' after doing a weekend of work)
4. 'bad' code with no budget to improve it, only budget for more features
-
Please respond with as many stories as you'd like! It's most helpful if you do the Google survey below, but feel free to respond in comments too:
https://forms.gle/w8pGuSVZwmRk9rt2A
Thanks so much! I'm very excited to write on this topic and share it with others. <comment><USER2> no documentation for legacy code. need to spend a long time to understand the code before the project can start.</comment>


Startup Marketing 101: Why You Need to Feed Yourself - <AUTHOR> http://thestartupfoundry.com/2011/03/06/startup-marketing-101-5-reasons-why-you-need-to-feed-yourself/

<comment><USER2> In summary: start a blog.

Isn't this pretty much startup 101 by now? I would imagine anyone on Hacker News who founds a startup and has even just an inkling of an idea on what to write about, would start a blog about it as the most basic of marketing effort.</comment>


Sacked by AI, tech worker found humans could do nothing - <AUTHOR> https://www.itwire.com/business-it-news/business-intelligence/83323-sacked-by-ai,-tech-worker-found-humans-could-do-nothing.html <comment><USER2> I don't see the AI part, just next-level automation/integration and some invalid input.

When AI has the authority to make screw ups of this nature, it is gonna be spectacular.

OB AI: I'm sorry Ibrahim, I'm afraid I can't do that.

Plot twist, title gore written by AI.</comment>


Ussia's fast-track coronavirus vaccine draws outrage over safety - <AUTHOR> https://www.nature.com/articles/d41586-020-02386-2 <comment><USER2> One thing I don't quite understand: Why not combine the testing with the using? As in pretty much everyone who wants to, can apply for the vaccine testing, but they then are gonna be monitored.

Based on my current understanding, they pretty much cherry pick the testers, so they only have a small group.</comment>


Making network authentication simple in a Bring Your Own Device environment - <AUTHOR> https://firstname.lastname@example.com/making-network-authentication-simple-in-a-bring-your-own-device-environment-9080baf39617 <comment><USER2> Very interesting take on modern 802.1x authentication !</comment>


Losing a Fortune Often Comes Down to One Thing: Family - <AUTHOR> https://www.nytimes.com/2017/02/19/your-money/losing-a-fortune-often-comes-down-to-one-thing-family.html <comment><USER2> Good read.</comment>


Zulip 2.1: open-source team chat - <AUTHOR> https://blog.zulip.org/2019/12/13/zulip-2-1-released/ <comment><USER2> We use zulip for our project. I like that their threads make discussions much more structured and easier to sift through. It takes a while to get used to though. It's also open source and I like being able to self host it. Can't say I miss slack too much. The bots were fun but they were mostly gimmicky than anything useful.</comment>


Show HN: Physically Based Rendering – Online Edition - <AUTHOR> http://www.pbr-book.org/ <comment><AUTHOR> Matt, Greg, and I are super-excited to release a free online edition of the Physically Based Rendering Book with hyperlinked cross-referencing, beautiful SVG figures and equations as well as interactive image comparisons. The details of how all of this came to be are explained in the preface to the online edition. ([http://www.pbr- book.org/3ed-2018/Preface_to_the_Online_Editi...](http://www.pbr- book.org/3ed-2018/Preface_to_the_Online_Edition.html))

<comment><USER2> Is there a place to report issues?

In section 2.9.1 Quaternions, 4th paragraph reads as if it's missing an equation for the second representation of a quaternion:

      A quaternion can be represented as a quadruple [EQ present] or as [EQ missing] , where is an imaginary 3-vector and  is the real part. We will use both representations interchangeably in this section.

This is what it looks like for me:

[https://imgur.com/a/3Mfanma](https://imgur.com/a/3Mfanma)

<comment><USER3> Nice find!

In general, you can send errata/bugs to email@example.com, but we got this one. :-)

It looks like an equation that MathJax wasn't happy with. I've got it fixed now and will include that fix in a push to the site in a little bit.

If you want to be listed as something other than "OnACoffeeBreak" in the errata credits, send me an email (firstname.lastname@example.com).

Thanks again!

<comment><USER4> There's a typo in the second sentence on the landing page, a redundant "the":

> where the an accurate modeling

<comment><USER3> Thanks! Also fixed and will also go live in a little bit.

</comment></comment></comment></comment></comment><comment> s-macke Thank you for this detailed description. I will definitely take a look and read a few chapters about the light transport.

Just a remark. When I first read the title, I was interested in the approximations you use because you emphasize physics so much in your book and your title. I read the preface and found a suspicious sentence.

 _When configured to do so, pbrt can compute images that are physically correct; they accurately reflect the lighting as it would be in a real-world version of the scene._

That is not correct. You use a phenomenological model for the light-matter interaction and approximate the effects of light with raytracing. That still produces (almost) photo realistic images for our eyes, but is not "physically correct". A paragraph on the limitations of technology could be worthwhile.

<comment><USER5> I'd love to hear a bit more about what you mean. What parts of the model do you consider incorrect, and what would it take to be correct? Are there specific things you read in the book, or are you talking about rendering and/or ray tracing generally? Do you have some specific limitations you're thinking of?

Honestly curious, since as a graphics person and not a physicist, I've been under the impression that most of our "physically based" rendering framework these days can be derived from, explained by, or validated against first principles, and that only some of the reflectance functions we use might be described as "phenomenological" (and we might call those "hacky"). And as @wjakob said, there are certainly optical effects we don't normally see, and don't spend time computing.

Is it possible to have phenomenological models that are correct? The word means the model hasn't been derived from first principles, but it doesn't mean the model is wrong... right?

Personally, I think of the term "physically correct" within the context of computer graphics history. It's not a technical term, and its meaning historically is referring to what came before now, and maybe not as much of a strict literal absolute as your interpretation(?). The 80's and 90's were full of fabulous graphics tricks that are even less physically correct than what we have now. Video games still have lots of them too.

Calling our newer techniques "physically correct" is perhaps kinda like how we call our TVs now "high definition", or our colors "high dynamic range". They're not "high" in any absolute sense of the word, they're just higher than before. In 10 or 20 years, what we call "high definition" today is probably going to feel like pretty low definition.

<comment><USER6> Physicists here. I would be careful about calling it "first principles" (you can look up that the meaning of that term within today's physics literature, it's typically called "ab initio", and usually refers to quantum mechanical treatment without a bunch of high level approximations). To a physicist, a first-principles calculation of light-matter interaction would in practice mean starting from a second-quantized form of the electromagnetic field + the lattice of the bulk or surface material + itinerant electrons.

What PBR people are doing is to imagine the matter at small scale looks like a patchwork of many small walls (perfect reflectance or ballistic transport), and assume that light and matter behave and interact the same way in a macroscopic setting, which is further simplified to Snell's law, neglecting all classical wave-like characteristics. Beyond that, obviously, all quantum mechanical effects are neglected (which aren't that exotic, for daily-life examples, think laser pointers or solar panels or crystals).

That is a far cry from an ab initio calculation and is, at best, a very incomplete toy model or cartoon description of light and matter interaction which might barely be enough to deceive human eye for most everyday objects.

<comment><USER7> What you seem to be describing are micro facet models which while commonly used in PBR renderers are not really fundamental to PBR. One of the major things that distinguishes PBR from earlier approaches is trying to ensure BRDFs for non emissive surfaces aren't adding energy by reflecting more total outgoing radiance than incoming radiance. Micro facet models help achieve that but there is also research that tries to more or less directly use measured BRDFs as well as trying to fit micro facet based and other models' parameters to measured data.

Of course there are non local scattering effects that are not captured by BRDFs and that can be perceptually important for some materials so there are also extensions like BSDFs used in some PBR renderers. Overall the PBR approach is to try and understand the underlying physics and drop or approximate (in a more or less principled way) the parts that are impractical to simulate or have small perceptual effects in most situations. It's not really about any specific such approximation like micro facet models. "Physically Based" seems like a perfectly good name for this approach, perhaps "Physically Inspired" would have been an appropriate alternative.

<comment><USER6> Microfacets is just one example (which _is_ what PBR approximations today ubiquitously use BTW). The very idea of a B*DF function for light is a rough approximation, and can never capture a variety of quantum effects. You can simply never recover the first-principles quantum dynamics of a macro-scale matter + light.

Approximations and models need to be made, that's how it works. Yes, you can make it better and better as the hardware improves. And calling it "physically based" sounds fine to me. What doesn't sound fine is to claim that any PBR- related work in the graphics literature today starts from "first principles". First-principles or "ab initio" in physics is a very specific technical word reserved for calculations which really start from first principles (see [https://en.wikipedia.org/wiki/Ab_initio_quantum_chemistry_me...](https://en.wikipedia.org/wiki/Ab_initio_quantum_chemistry_methods) for an example) that doesn't get thrown around cheaply.

<comment><USER5> > What doesn't sound fine is to claim that any PBR-related work in the > graphics literature today starts from "first principles".

The use of that term was entirely my sloppiness alone. I used "first principles" only to mean the opposite of a phenomenological model, as @s-macke put it. What's the right term there? Theoretical model?

You're okay with "physically based", but not "first principles", @s-macke was okay with "first principles" but not "physically correct". Personally, I wouldn't mind being able to talk about graphics in a way that doesn't immediately rub physicists the wrong way, but I'm not sure how, and I'm bound to cause some problems flinging around terms that have a precise technical meaning in physics. Sorry about that. ;)

It's definitely true that in graphics we are not usually setting out to simulate quantum effects, unless the result is visible at the macro scale. Despite that, we're trying to incorporate all the physics that matter at the macro scale.

<comment><USER6> I think an accurate description would be classical (meaning not quantum mechanical) model of light (which covers both geometrical optics and wave optics) and phenomenological model (in physicist's sense: [https://en.wikipedia.org/wiki/Phenomenological_model](https://en.wikipedia.org/wiki/Phenomenological_model)) of matter and light-matter interaction. It is a phenomenological model, because it's either derived from a cartoon description of matter at microscopic scale and how light bounces off it, or fitted to some experimental data, and is without an underlying actual first-principles physical theory either way.

BTW, I hope you don't think this is just me being picky about words; if you talk to any physicists in condensed matter matter physics and say the magic words "first principles", they'd immediately start thinking that you did ab initio calculations. First-principles calculations and DFT is a sub-field, and entire careers and communities are built on it!

<comment><USER5> > I hope you don't think this is just me being picky about words

Not at all, thank you for taking the time to explain!

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> Ever since I saw this book being sold, I've been eagerly wanting to read it. The physics-based approach is definitely the way I would prefer to learn any subject, and rendering is just one of the areas I wish I knew more about. Delighted to know that you guys put it online, thank you!

</comment><comment><USER9> This is the book that won Matt Pharr, Greg Humphreys and Pat Hanrahan an Academy Award

[https://www.oscars.org/sci-tech/ceremonies/2014](https://www.oscars.org/sci- tech/ceremonies/2014)

>> To Matt Pharr, Greg Humphreys and Pat Hanrahan for their formalization and reference implementation of the concepts behind physically based rendering, as shared in their book Physically Based Rendering.

>> Physically based rendering has transformed computer graphics lighting by more accurately simulating materials and lights, allowing digital artists to focus on cinematography rather than the intricacies of rendering. First published in 2004, Physically Based Rendering is both a textbook and a complete source-code implementation that has provided a widely adopted practical roadmap for most physically based shading and lighting systems used in film production.

</comment><comment><USER10> There is a Patreon to help pay for the servers [https://www.patreon.com/pbrbook](https://www.patreon.com/pbrbook)

</comment><comment><USER11> What is the best way to attack this book as a graphics programming beginner (but with plenty of C++ experience)? I saw several people praise it a while back in HN and took a cursory look. It seemed complete and thoroughly explained but missing examples or "checkpoints" along the way where I could turn around and test what I had been building.

<comment><USER7> One option is to dive in with Peter Shirley's ray tracing intros linked at the top of [http://www.realtimerendering.com/raytracing.html](http://www.realtimerendering.com/raytracing.html)

These will get you a quick overview and get something rendering. PBRT is a great follow up to really understand the physical underpinnings and get more into the theory of how to do things correctly and efficiently.

<comment><USER11> Fantastic! Thanks for the suggestion.

</comment></comment></comment><comment><USER12> what technology have they used to create the website? I'm guessing some pandoc to html workflow plus a static site engine?

<comment><USER3> It's pretty much a big hack.

The book was originally written in TeX, with lots of macros on top. We hacked together a custom translator (~3k loc, written in golang) that parses the TeX and generates the site.

Then MathJax to render the equations, Bootstrap 4 for some navigation, and jeri.io for the interaction with images.

<comment><USER13> Thank you, this is the first time I've seen a "literate programming" text presented on the web. It's really well done, and makes up for some of the drawbacks of on-paper LP: being able to expand the definitions of sections (modules?) in-place, and being able to follow function definitions, are both very useful.

I had some plans to present some other such written-in-TeX-literate-programs in HTML; could you perhaps share somewhere your "big hack" that you used here (the translator, the javascript)? Totally ok if you don't want to :) Thanks,

</comment></comment></comment><comment><USER14> Awesome work guys! [http://www.pbr- book.org/3ed-2018/Introduction/Photorealistic...](http://www.pbr- book.org/3ed-2018/Introduction/Photorealistic_Rendering_and_the_Ray- Tracing_Algorithm.html) I am going to read this part when I get home.

</comment><comment><USER15> This makes me super happy. I bought the book a few years back and always meant to go through it with rigor to build my own renderer. Now I have that free time but as a digital nomad, I couldn't bring the book. (its up there in size).

</comment><comment><USER16> Thanks for putting this online, my physical copy is currently buried in storage and I've been dying continue reading it!

</comment><comment><USER17> I got my physical copy a few months ago, glad to see more people having access to this great resource now!

</comment><comment><USER18> Thank you so much. I love my physical copy but can't always take it with me places.

</comment><comment><USER19> This is amazing news. Congratulations and thanks so much for this present to humanity.

</comment><comment><USER20> Wooow thanks guys!</comment>


Show HN: Weekend Project - My MVB (minimum viable business)  - <AUTHOR> http://thehodge.co.uk/blog/business/hodgetastic/my-weekend-project-a-mvb-minimum-viable-business.php <comment><USER2> I wouldn't really call this a "minimum viable" anything—as you said, you had already purchased the necessary equipment to run your own events with; at that point, you had invested 99% of the start-up cost for an events company, that just happened to not exist yet. I would say this is more of an extension to your current business (it's re-using its assets, after all) than a new venture.

I mention this because it makes the advice you've given a bit less general than you've made it out to be: the "step zero" of your MVB seems to be "have some uncommon set of skills and knowledge, and own some expensive equipment, that could have already been a full-time business operating at scale if you had just gotten off your lazy butt earlier and marketed it as such." That's a bit different than the nominal idea of an MVP, which is more "put up a page offering X, and only bother to invest in the resources+knowledge required to make X at scale if there's demand for it."

<comment><USER3> An MVP actually offers crude but working versions of the features you're evaluating. A page offering something that doesn't exist at all is not any kind of product, merely false advertising that can only tell you what people _think_ they might want _without_ having tried it. It also poisons your reputation among people like me who care about an efficient market that isn't filled with chaff.

<comment><USER2> You missed the words "at scale." An MVP is when you offer a service X, but only build the capacity to handle, e.g., one customer, until you have 2+ customers. Sometimes this is done by not building anything at all, but rather by manually creating what the service would create for the person. The idea is that you only invest in automating/scaling/tooling up when you actually determine that there is demand, rather than building something for no reason. It's sort of like TDD: you have a facade (your storefront) that presents a finished product, and have customers make requests through it—then your business does the simplest thing that could possibly work to satisfy each request, only adding code/equipment/people/etc. when a request can't possibly be satisfied without it.

In the case of the article, an actual MVB (as done by someone who doesn't already own event-staging equipment) would be to put up the page, and then only rent the equipment needed if-and-when people actually set up an event.

<comment><USER3> Thank you. I did assume "put up a page offering X" was advocating a dry test and you needn't be prepared to offer some kind of X prototype.

</comment></comment></comment></comment><comment><AUTHOR> Evening HN'ers, Thought I'd share my weekend project with you and see what you think, I run a few events in the UK (Conferences, Barcamps, Hack Days) and I decided to take my learnings and see if I could help other people who want to run events (we have a reputation for doing things a bit differently).

I'm still learning rails so projects like these are as much about learning a new language as they are looking at a potentially new business.

I hope this interests someone and if you have any questions, let me know

</comment><comment><USER4> I scanned the post, and I have no idea what service you offer.

<comment><AUTHOR> Hmm I see what you mean, I guess I was trying to make the post more about how/why I built it in a weekend rather than trying to sound like I was trying to sell the company to the readers... I take your point though, thanks :)

</comment></comment><comment><USER5> Very Informative post. It would be much better if can share how would you scale it in future and what's your business model.

Thanks and best of luck!

<comment><AUTHOR> Scaling is easy for this, I have a team of people we use for events who have agreed to come onboard and help with any contrats we get for this.

Our business model is to charge people for organising, hiring or providing items or advice</comment></comment>


Show HN: Get NYC Subway and Transit Status via Alexa Flash Briefing - <AUTHOR> https://www.amazon.com/dp/B0733BVTQZ/ <comment><AUTHOR> This skill has individual feeds for NYC's Subway system, Busses, Long Island & Metro North Railroads, Bridges and Tunnels. Turn on the feeds you need and ask Alexa for your briefing. The skill reports a concise list of transit options which are running smoothly, those being worked on, and those which are delayed.

Here's a video demo: [https://youtu.be/tyXaR12SZE8](https://youtu.be/tyXaR12SZE8) Here's the code: [https://github.com/nealrs/mtafeed](https://github.com/nealrs/mtafeed)

Right now, this is running on a free Heroku gear, but I'd love to get your feedback on it before I refactor it (and add some much needed caching) . Thanks!</comment>


Inside the NSA's War on Internet Security - <AUTHOR> http://www.spiegel.de/international/germany/inside-the-nsa-s-war-on-internet-security-a-1010361.html <comment><USER2> This would be a good time to wait and let security professionals analyze the documents and take what you read in this article lightly, as I've found a number of sensationalist examples.

For example, they claim Canada is monitoring hockey sites:

> Canada's Communications Security Establishment (CSEC) even monitors sites > devoted to the country's national pastime: "We have noticed a large increase > in chat activity on the hockeytalk sites. This is likely due to the > beginning of playoff season," it says in one presentation.

But if you look at the actual slide [https://i.imgur.com/2GO8H6L.png](https://i.imgur.com/2GO8H6L.png), it is clearly a fake sample report of what a real one might look like. It even uses the name 'Canukistan' as the country name.

There are 44 slide decks, one of the biggest leaks so far. It will take time to make sense of the noise. And any misinformation from reporting by non- technical journalists doesn't help the cause.

<comment><USER3> If I didn't know that the government is manipulating social media all the time, I totally would not think you're a shill trying to discredit these news reports by claiming that Jake Appelbaum is a non-technical journalist.

* Revealed: US spy operation that manipulates social media ([http://www.theguardian.com/technology/2011/mar/17/us-spy-ope...](http://www.theguardian.com/technology/2011/mar/17/us-spy-operation-social-networks))

* How Covert Agents Infiltrate The Internet To Manipulate, Deceive, And Destroy Reputations ([https://firstlook.org/theintercept/2014/02/24/jtrig-manipula...](https://firstlook.org/theintercept/2014/02/24/jtrig-manipulation/))

<comment><USER4> Reprising a relevant comment:

The United States and allies do use the internet to spread Western culture and ideas, start revolutions, and kindle insurrection.

The United States CIA attempted (and nearly succeeded) in inciting a revolution against Castro by pretending to be a series of grassroots movements on a Twitter-like platform and by inciting anti-administration feelings within the Cuban population. That was earlier this year.

"USAID effort to undermine Cuban government with fake 'Twitter' another anti- Castro failure" [1]

The United States has an ongoing effort to use Internet media to 'deradicalize' the next generation of Middle Easterners and actively manipulates public opinions in Jordan, Cairo, Syria and other Middle Eastern states. Here are some quotes from one DoD MINERVA paper:

"...it is imperative that we develop empirically-based procedures for countering messages that promote violent extremism and anti-Western beliefs..."

"...Neural predictors of Twitter impact in Cairo (UCLA & Egypt). Our prior work (Falk et al., 2012), indicates that neural responses of a small group can predict which persuasive messages will be more successful in mass media campaigns..."

"... Defense Group Inc. already tracks Twitter trends specific to Egypt and will identify which of the selected Twitter topics went on to be highly influential over the next month and which did not..." \- Matthew Lieberman, UCLA, September 30, 2012, Department of Defense MINERVA Initiative [2]

Here's one US company that does it. MARAYA MEDIA - "Driving Intelligent Dialog". [3]

The United States engages in targeted mass media and social manipulation to stir dissent in target nations, and to quell dissent where destabilization would hurt policy objectives. The DoD's MINERVA project specifically looks to understand the cultural components of stability of various countries and mechanisms to encourage or disrupt that stability. Among a great number of social studies you will find DoD research on how to seed information inside of specific Asian countries, including China, for the targeted introduction of instability. I will leave speculations of possible connections to the Hong Kong protests to the reader. [4] During the Iraq war US officials were known to detain Iraqi journalists and bloggers and force them to write articles in favor of the American efforts or to spread misinformation useful to ongoing campaigns. The CIA purposefully slipped misinformation into American media outlets to fool counterinsurgents who were reading American media (the infamous "Fallujah PysOp").

This should not come as a surprise given the history of the US: The United States and allies are known to target media in other countries to stir dissent. Radio Free Europe, "Voice of Iraq" (cough American), the Lincoln Group infiltrations and partnerships, etc.

But now with global interconnectedness it is easy to set up 'foreign media', blogs and other politicizing content to influence other nations' populations.

In the past decade it has become a global issue.

This year Egypt sentenced Al Jazeera journalists that they believed were partnered with geopolitical interests of other states. Putin's administration is now requiring bloggers to register if they have a certain number of readers, so that his administration can curtail international influence. China blocks many American services including Facebook and Google. The usual story in America is that they are censoring free speech. The truth is that they do not want foreign influence to destabilize their population and that they do not want their citizen's data in America's PRISM program (there's a reason it's called the FISA "Foreign Intelligence Surveillance Act" court).

The Snowden revelations showed us how intelligence agencies are involved in PsyOps - the term for 'psychological operations' used by the CIA and others. The GCHQ's BIRDSONG/BADGER/GATEWAY/SLIPSTREAM/ETC and partnership with the NSA are used to influence online polls, discussion forums and to vote up and down content that aligns with policy goals. [5][6][7] The giant meta-data graph created by the NSA is also particularly valuable for 'influencer' and 'social contagion' analysis (leaks showed they do use it to understand internal chain- of-command and organization structure for target selection). It's why metadata matters. A nice illustration of this is the article "Finding Paul Revere."

And so we have issues here with the use of targeted social influence in America as well. First there are instances where other countries are trying to incite disruption in the US - the US wants to study and curtail it. [8]

A number of journalists have called out that the state has been extremely aggressive to dissenting opinions, even to go so far as labeling current policy on the issue "War on Journalism". American officials have exported a number of journalists with Middle Eastern descent and journalists like Ayman Mohyeldin have been pulled from Gaza and other conflicts when reporting has erred on the side of other state interests. The crackdown on journalism is worth another post I don't have time to write.

Just look at how central a role controlling internet dialog is for running a modern US presidency. A Google search for "Obama internet campaign" [9] results in headlines "How Obama's Internet Campaign Changed Politics", "How Obama won the internet", "Barack Obama and the Facebook Election", "Propelled by Internet, Barack Obama Wins Presidency" \- this isn't because of grassroots discussion but because both Obama and McCain (and Romney before him) had cyber centers in control of internet PR engaging tens of millions of dollars in Twitter messages, etc.

You can nudge public opinion by bombarding them with an influx of the same message, slightly disguised in one way and then another. The MINERVA program has plenty of good reading with regard to this. Anyway, the USG does this overseas and, to a limited degree (you decide how limited) presidential campaigns and journalistic partnerships (anyone want me to write a blurb on that...?) have them doing it inside the United States as well.

[1] [http://www.washingtonpost.com/lifestyle/style/usaid- effort-t...](http://www.washingtonpost.com/lifestyle/style/usaid-effort-to- undermine-cuban-government-with-fake-twitter-another-anti-castro- failure/2014/04/03/c0142cc0-bb75-11e3-9a05-c739f29ccb08_story.html)

[2] [http://minerva.dtic.mil/doc/samplewp- Lieberman.pdf](http://minerva.dtic.mil/doc/samplewp-Lieberman.pdf)

[3] [http://www.marayamedia.com/company.php](http://www.marayamedia.com/company.php)

[4] [https://firstlook.org/theintercept/2014/02/24/jtrig- manipula...](https://firstlook.org/theintercept/2014/02/24/jtrig- manipulation/)

[5] [https://firstlook.org/theintercept/2014/02/24/jtrig- manipula...](https://firstlook.org/theintercept/2014/02/24/jtrig- manipulation/)

[6] [http://www.theguardian.com/commentisfree/cifamerica/2011/jun...](http://www.theguardian.com/commentisfree/cifamerica/2011/jun/22/hacking- anonymous)

[7] [http://www.dailykos.com/story/2011/02/16/945768/-UPDATED- The...](http://www.dailykos.com/story/2011/02/16/945768/-UPDATED-The-HB-Gary- Email-That-Should-Concern-Us-All)

[8] [http://www.washingtonpost.com/opinions/truthy-project-is- unw...](http://www.washingtonpost.com/opinions/truthy-project-is-unworthy-of- tax-dollars/2014/10/17/a3274faa-531b-11e4-809b-8cc0a295c773_story.html)

[9] [https://www.google.com/?q=obama+internet+campaign](https://www.google.com/?q=obama+internet+campaign)

Comment reprised from here: [https://news.ycombinator.com/item?id=8709976](https://news.ycombinator.com/item?id=8709976)

<comment><USER5> All of this makes seems and is well sourced. But there is a huge difference between inciting revolutions that are beneficial to the US in foreign countries, one of the stated purposes of the intelligence community, and "shilling" comments on hackernews or reddit. Covert US involvement in swaying public opinion against opposing ideologies is a proud tradition that goes back to at least the 1930s and beyond. Using social media is just an extension of that. Most of the sources you provided are about creating fake social media sites to be used in foreign countries or broad discussions about psychological influence techniques that mention social media. There are no detailed plans that mention hackernews, reddit or any other online social news aggregator. It would be incredibly costly for the NSA to have agents posting pro-government comments on every thread that pops up on the hundreds of online tech communities and I don't see how they would benefit from it in any way. The idea is frankly laughable.

It's fun to think that we're so important that the US government cares enough to intervene in our political discussions. But we are not, not a single one of us. If pg himself called for open insurrection in his next essay, no one in the NSA would lift an eyebrow or raise a finger. Until this or any community becomes known as a hotbed for muslim extremism or communist agitation we're simply not on the radar in any way. As far as hackernews and reddit are concerned, "shill" is a synonym for "someone who disagrees with me" and always will be.

<comment><USER4> We do know that the United States Government and allies manipulate both foreign and domestic press.

I agree with the sentiment that this does not imply reddit or hackernews are subject to influence by the United States Government or allies.

I do not agree that the idea is preposterous or laughable. This is because we do know that the NSA infiltrates domestic technical groups as they did with the IETF to affect standards discussions, that they infiltrate activist groups inside the United States to disrupt them, that they are aware of social contagion theory and its usefulness in affecting public opinion, that they have done studies with at least the UCLA on viral messaging for Americans (to compare to, with and against foreign countries), that political campaigns use social targeting techniques without branding and will comment on news articles (to be 'first to post') to color conversation on hot button issues during the races, and that companies with political interests and who share a revolving door with elected office also advertise political discourse online in this way. Thinkst researchers studied how easy it is to manipulate online social conversation, news media outlets and platforms. We know that the GCHQ have JTRIG capabilities to perform internet manipulation and that there are documents from Snowden that specifically mention their use in derailing conversations on online forums. There have been reports of PR firms of private companies astroturfing reddit and others. And we know that HBGary Federal and other cyberoperations contractors for the US Government sell astroturfing services.

What we don't know is that reddit or hackernews are targeted specifically or for domestic purposes by the US Government. We have a few indications that this is done for large media outlets (recently Judith Miller, Ken Dilanian, CNN on Bahrain) in tandem with other leverage like access to officials, exclusive press passes and permission to report at the edges of no-reporting zones. Unfortunately there isn't enough evidence to be conclusive yet about the reddit/HN case as there have not been leaks that speak directly about it, so any debate in this area is bound to be speculation versus speculation.

</comment></comment></comment></comment></comment><comment><USER6> THESE DOCUMENTS CONTAIN EVIDENCE OF ATTACKS ON VPN, SSL, TLS, SSH, TOR. What do we do now? No seriously, what do we do?

The full list of documents: [http://www.spiegel.de/international/world/nsa- documents-atta...](http://www.spiegel.de/international/world/nsa-documents- attacks-on-vpn-ssl-tls-ssh-tor-a-1010525.html)

The accompanying lecture: [http://streaming.media.ccc.de/relive/6258/](http://streaming.media.ccc.de/relive/6258/)

Also, obligatory: [https://eff.org/donate](https://eff.org/donate)

<comment><USER7> Earlier this year at goto copenhagen I heard a good talk by Tim bray:

[http://gotocon.com/cph-2014/presentation/Privacy%20and%20Sec...](http://gotocon.com/cph-2014/presentation/Privacy%20and%20Security,%20Policy%20and%20Technology)

Where he argues that even though we can not achieve complete security there is great value in raising the bar. If we continuously make it increasingly harder for NSA, MOSAD, GCHQ and the rest of them to spy on us, we can achieve good enough privacy. Where most communication will be secure. But he also argues that if one of these agencies really wants to target YOU specifically they will get to the information. By breaking into your house and installing cameras, if necessary.

<comment><USER8> It's Michens' MOSSAD/not-MOSSAD question[1]. Any half-decent encryption will protect you from bulk collection and monitoring, but if you're targeted, you lose.

[1] [http://research.microsoft.com/en- us/people/mickens/thisworld...](http://research.microsoft.com/en- us/people/mickens/thisworldofours.pdf)

<comment><USER9> That has to be the best paper I've ever read

<comment><USER10> I thought you were joking, but I had to click anyway. You're right, this paper is fantastic. I thought Micken's stopped writing, I'm so glad he didn't.

</comment></comment></comment></comment></comment><comment><USER11> I mostly read this thinking "good news". No, seriously, the documents suggest that the NSA hasn't made fundamentally important advances in decryption or uncovered significant weaknesses that academia doesn't know about. Now, that's not too much of a reassurance, because what academia (and the NSA) know is that HTTPS is in pretty terrible state, end-point security remains a significant problem, IPSec is a terrible protocl and so on.

It does raise the question what all the mathematicians are doing at NSA, and why they don't seem to have come up with any meaningful results. Suggests they are a waste of money, but then that's all of the NSA.

I suggest all of you check the original material (powerpoints w/ screenshots). A lot of people here suffer from the _action movie_ mentality where they think the NSA is not like any other government agency, i.e. inefficient, behind the times, filled with horrible middle managers, deadweight, .. you get the idea. Things like the enterprise Java web interface, the CSV mass data export and "genericIPSec_wrapper.pl" can quickly dispel that myth.

<comment><USER12> Or at the very least they have compartmentalized serious mathematical cryptanalytic capabilities.

For instance:

* We know that the NSA has a novel md5 collision capability since they have used it in their malware. None of the Snowden docs, that I have seen, have talked about this.

* It is likely based on public research that the NSA can break 1024-bit RSA, but this has not showed up in the documents either.

My personal belief is that we are missing compartments dealing with cryptanalysis because Snowden did not have access to them. His work and access were focused on Computer Network Operations and not cryptanalysis.

<comment><USER13> _It is likely based on public research that the NSA can break 1024-bit RSA, but this has not showed up in the documents either._

It would be expensive though. This is one reason why I consider 1024-bit end entity certificates much less of a threat than 1024-bit CA roots.

<comment><USER14> I don't totally agree. I think that factoring in the risk of exposure leaves a CA root with a worse price / performance ratio versus an individual cert.

While you could use a faux CA root to sign faux certs for any site you want (ideally ones who are customers of that CA), in practice your use is severely limited. If faux certs are spotted and no one knows where they came from, suspicions are going to be raised. Not only is your faux CA root compromised, but now you may have tipped your hand regarding your capabilities.

To limit that possibility, your attacks would have to be extremely targeted. The more often a fake cert is used and the more people exposed to it, the higher the likelihood that someone will notice what is going on.

It also doesn't help you decrypt the real traffic to the site, or historical traffic, which busting the site's actual SSL key can yield. This presumes that you have a way of intercepting said traffic, but I think it's pretty clear that that is not out of the question (public wifi / ISP cooperation / fiber optic taps / malware). It's more work to bust individual certs, but you're leaving a smaller trail and you aren't sending out examples of your RSA cracking capabilities to your opponents over the public Internet.

Lowering the risk of exposure will let an attacker use the same methods over a much longer period of time, which I think is the goal here.

As to how to combat this: there is a lot of low hanging fruit. Besides the obvious, I would love to see much shorter expiration times for certs become the norm (as in weeks, if not days). For this to realistically happen in a widespread fashion, at minimum CAs need to embrace the concept from a pricing perspective.

<comment><USER13> Yea, if one was signed for www.google.com it would be a serious problem. If it is targeting specific obscure domain names where the customer is willing to accept the risk, that is a different matter.

</comment></comment></comment></comment></comment><comment><USER15> Has anyone found which docs say how they attack SSH? The intro slides don't go into any detail. It could just be known SSH-1 vulnerabilities.

My overall impression is that this doesn't reveal any new attacks. They are most likely using known vulnerabilities. For example, they decrypt PSK IPSec by exploiting routers and getting the keys, not breaking the encryption.

<comment><USER16> In the talk they say that the papers only suggest that it can be broken into.

</comment></comment><comment><USER17> One topic I find missing from the privacy and security debate following Snowden's revelations is an explicit consideration of the adequate threat model.

If the public thinks that the most prominent attackers on their privacy, security or identify are the best founded intelligence agencies on the planet, then the likely outcome will be grumpy resignation and consequent failure to protect against more mundane (and more likely) threats. Security and encryption are considered difficult and tricky. Even for software engineers. Raising the bar by highlighting the scale of resources of the most competent attackers is counterproductive.

I think a practical threat model for an average internet user should highlight cyber-criminals, accidental misconfiguration, and careless handling of private information. Not NSA or GCHQ.

Edit: The discussion of mischief by NSA and GCHQ belongs to the debate on public oversight of government agencies. The article above is about using encryption on the internet.

</comment><comment><USER18> Most proprietary and mainstream software and protocols are insecure. If you care about your security use open source and open standards so that security professionals can test and verify its security.

Skype insecure Cloud email popular ones used by end users insecure Whatsapp insecure Facebook messenger insecure Email insecure Dropbox insecure

So in conclusion they are tapping into mainstream communication channels, its their job.

People have become a bit lazy with cloud solutions and proprietary software because of their fast setup and convenience. People pay with their privacy for the convenience/laziness.

<comment><USER19> WhatsApp's recent integration of TextSecure [1] makes it one of the most secure communication options available to lay users. The vast majority of people simply can't manage without hosted tools. This lack of sophistication shouldn't damn them.

[1] [https://whispersystems.org/blog/whatsapp/](https://whispersystems.org/blog/whatsapp/)

</comment></comment><comment><USER20> Domestic spying + Immunity from insider trading laws... It's a good time to be in the government.

<comment><USER21> To give some credence to your comment about insider trading immunity, I performed a quick search. (My guess is that you got downvoted for not providing any evidence to support your, arguably acerbic and snarky, comment.)

[http://www.forbes.com/sites/kylesmith/2011/06/01/insider- tra...](http://www.forbes.com/sites/kylesmith/2011/06/01/insider-trading- rules-that-dont-apply-to-congress/)

The money quote: _" There is no limit to how much money you can earn on insider trading in the House or Senate. Lawmakers and their staffers are specifically exempted."_

While I would consider it unlikely that NSA feeds senators with free stock market tips, the members of the oversight committee are sure to have an extensive advance view to foreign (and domestic) market-changing intelligence. There has been systematic resistance to reforms - the members probably consider making a quick and safe half-million on the stock market a necessary perk of the job.

</comment></comment><comment><USER22> There was also just a talk about this at 31C3. Recording is still running, you have to seek back to the beginning: [http://streaming.media.ccc.de/relive/6258/](http://streaming.media.ccc.de/relive/6258/)

</comment><comment><USER23> As a google security engineer once said, "f-ck these guys"

 _According to an NSA document, the agency intended to crack 10 million intercepted https connections a day by late 2012._

 _By the end of 2012, the system was supposed to be able to "detect the presence of at least 100 password based encryption applications" in each instance some 20,000 times a month._

</comment><comment><USER24> It makes you wonder if the NSA/five eyes is actively working to keep topics/threads such as this one down played in the media, or even on HN.

I did my part and upvoted the story to get it more exposure here:)

</comment><comment><USER25> The fact that they broke some but not all the OTR messages in the log suggests to me that their attack is not a MITM, but instead a compromise of the 1024 bit DH or CTR mode AES.

<comment><USER26> Do you really think NSA has compromised AES-CTR? That would have to be a pretty fundamental attack, wouldn't it?

<comment><USER25> I am not trying to draw any conclusions. Just exploring what the data seems to support.

Another alternative (mentioned on otr-dev) is an implementation which uses a low quality rng feeding the ECDH might result in some messages being recoverable and others not.

An attack on CTR would indeed be pretty fundamental. Though some of the other documents appeared to support some level of cryptanalysis capability against some implementations of at least some symmetric ciphers.

<comment><USER26> Can you think through a scenario in which CTR _could be_ broken? CTR, in particular. What's a hypothetical here?

<comment><USER25> Sure.

Improve the existing key-recovery attacks ([http://research.microsoft.com/en- us/projects/cryptanalysis/a...](http://research.microsoft.com/en- us/projects/cryptanalysis/aesbc.pdf)) on AES from 2^126 to 2^80 (through unknown methods, potentially exploiting the trivial relation of CTR plaintexts), which is a scale at which a state level party could perform computation, especially on specialized hardware. Observe a CTR block on known plaintext and recover the key.

Practical key recovery attacks have existed against many block ciphers. AES is pedantically weaker than it should be (since an attack exists at 2^126).

Do I think this is likely? I don't have enough information to answer, and in the absence of information I'd default to "probably not". It wouldn't be inconceivable, however.

</comment></comment></comment></comment></comment><comment><USER27> Even though we know that the NSA does collect data on Americans, let's assume that they didn't. By using a network such as a VPN or a Tor Node that is located outside of the United States, would they "legally" be able to use the data collected on you received from those networks as if you weren't an US Citizen?

<comment><USER28> No, but that's where the Five Eyes concept comes into play, previously known as ECHELON. The idea is that agencies such as CIA and NSA can work around the limitations imposed by their own charters by cooperating with foreign agencies who have no such prohibitions. Then, the doctrine of parallel construction takes care of any remaining legal hangups.

</comment></comment><comment><USER29> does anyone know why spiegel leaked these docs - and not greenwald via the intercept?

<comment><USER30> They were released by Jacob Appelbaum and Laura Poitras who did a 3C31 talk about this too.

</comment></comment><comment><USER31> Ok, now that I've finally read the slides. They take usernames and passwords out of SSH.

People, if the private botnets didn't made you disable password authentication already, do it for the NSA.

</comment><comment><USER32> The comments that try to reassure the reader seem to have become more frequent. Scary.

</comment><comment><USER33> In the age of information this is how an overpowered super power gets "nerfed":

By "targeting" its information gathering capabilities by providing information about its activities.

Information is power.

<comment><USER33> Anyone care to explain why the downvotes? I still think all this revelations will lead to a more balanced state of affairs.</comment></comment>


How much you need to earn to be part of the 1 percent - <AUTHOR> https://www.cnbc.com/2018/02/27/how-much-you-need-to-earn-to-be-part-of-the-1-percent.html <comment><USER2> Warning: the site plays a loud video.</comment>


Lisp as a replacement for XML -- any ideas? - <AUTHOR>

If I want to use Lisp syntax to store tree-like data structures (as a replacement for XML), are there any libraries, standards, or just ideas how to do it? <comment><USER2> Here's how we do it:

("as_2586" ("as_2587" ("as_2588" ("as_2637" ("as_2638" ("as_2639")) ("as_2640")) ("as_2595")) ("as_2589")))

Equals:

Sub Projects

        * 200705_PSC_Trinity (Construction)
              o as_2587 (Client Approval)
                    + as_2588 (Building Permit)
                          # CD Set (Deliverable)
                                * A-110 (Deliverable)
                                      o 1. Site Plan (Deliverable)
                                * PSC Survey Info (Building Surveying)
                          # as_2595 (Systems Coordination)
                    + as_2589 (Programming)

<comment><AUTHOR> I don't quite get it -- can you show an example with tags, attributes and text?

<comment><USER2> I just showed the hierarchy structure. You can put anything you want in the nodes of the hierarchy. In the example, these are object names that then generate the example hierarchy below in our application. Hope this helps.

</comment></comment></comment><comment><USER3> Until all Lisps standardize on Unicode (at least UTF-8 and UTF-16), Lisp will never replace XML. It's been slow going just getting Unicode _support_ into Lisps, let alone having internal data structures (symbols, etc.) represented in Unicode.

</comment><comment><USER4> Read this essay for some ideas:

<http://www.defmacro.org/ramblings/lisp.html>

</comment><comment><USER5> I think you can do it directly. No? I've just started to study List, but I think you could do sth like:

      '(root (child-1 attr1: val1 attr2: val2)
             (child-2 attr1: val1 attr2: val2)
             (child-3 attr1: val1 attr2: val2))

And then recurse over it?

BTW, If I'm wrong please teach me :D

</comment><comment><USER6> XML and Scheme <http://okmij.org/ftp/Scheme/xml.html>

</comment><comment><USER7> You're just replacing the data storage. XML has a lot more features, like schema validation and querying.

<comment><AUTHOR> True, but syntax doesn't matter, I think. Everything can be re-defined in Lisp style if needed.

What I'm looking for is a clearly defined syntax, character sets, metachchacters, etc for Lisp-like representation of data.

<comment><USER8> _syntax doesn't matter, I think. Everything can be re-defined in Lisp style if needed._

Yes, you're right.

You might be interested in this: <http://www.pmsf.de/resources/lisp/expat.html>

If you use Mai's modified expat, you can convert any feed into s-expressions, preserving all attributes and namespaces.

Here's a quick example.

This is the beginning of the xml for the news.yc feed:

<rss version="2.0"><channel><title>Startup News</title><link>[http://news.ycombinator.com</link><description&#...](http://news.ycombinator.com</link><description>The) most interesting startup-related links, as determined by readers.</description><item><title>PG: If starting viaweb today, would you use lisp?</title><link>[http://news.ycombinator.com/item?id=36905</link><...](http://news.ycombinator.com/item?id=36905</link><comments>http://news.ycombinator.com/item?id=36905</comments></item>);

And here's the expat-parsed, s-expression:

(("rss" "version" "2.0")(("channel")(("title")"Startup News")(("link")"<http://news.ycombinator.com>")(("description")"The most interesting startup-related links, as determined by readers.")(("item")(("title")"PG: If starting viaweb today, would you use lisp?")(("link")"<http://news.ycombinator.com/item?id=36905>")(("comments")"<http://news.ycombinator.com/item?id=36905>"))

<comment><AUTHOR> Thanks.

I wonder if quotes can be omitted to make it slightly more readable. Anyway, this idea obviously needs some effort for improvement before it can be brought to market, so to say.</comment></comment></comment></comment>


Jeff Bezos beats Elon Musk's SpaceX in the reusable rocket race - <AUTHOR> http://www.engadget.com/2015/11/24/blue-origin-reusable-rocket-landing/ <comment><USER2> A click-baity title, but undoubtedly cool achievement. Watching the rocket toggle around in the last second hover before touching down was really neat.

<comment><USER3> Amazing accomplishment by BO! However, headline is misleading, as a sub- orbital (100km) flight, not technically the same magnitude / mission as SpaceX. Elon tweeted about this: [https://twitter.com/elonmusk/status/669129655597731841](https://twitter.com/elonmusk/status/669129655597731841)

</comment></comment><comment><USER4> I think they might have beat SpaceX with their video production at least... great job! Also: the landing gear on this rocket is 10x more scifi than SpaceX's, so bonus points there.

I didn't know Blue Origin's goal was 4 minute suborbital floaty experiences for rich people though. Has that been common knowledge? Are they even interested in going orbital with non-human payloads?

</comment><comment><USER5> What a fine example of great reporting. 50% success rate in TWO launches, definitely indicates a clear winner. #irony</comment>


Placebo Effect Produces Higher Test Scores - - <AUTHOR> http://www.psmag.com/blogs/news-blog/placebo-effect-produces-higher-test-scores-54276/ <comment><USER2> It's interesting to think of how trained we are to rely on external stimulants for productivity. From caffeine to adderall, steroids to blood doping, it seems that people are as attracted to the idea of enhancing drugs to actually using them.

Thanks for the share.</comment>