The Markov Property, Chain, Reward Process and Decision Process - <AUTHOR> https://xaviergeerinck.com/markov-property-chain-reward-decision <comment><USER2> Here are a couple of other Markov Chain articles that are worthwhile:

[http://setosa.io/ev/markov-chains/](http://setosa.io/ev/markov-chains/)

[https://towardsdatascience.com/a-zero-math-introduction- to-m...](https://towardsdatascience.com/a-zero-math-introduction-to-markov- chain-monte-carlo-methods-dcba889e0c50)

<comment><USER3> These are great! Thank you!!

</comment></comment><comment><USER4> > Xavier works on enabling everyone to construct products that make a life > changing impact on world scale.

I would remove/change that sentence if I were you. I suppose it was made to make what you do look impactful and serious but I have exact opposite impression after reading it.

<comment><AUTHOR> Hi Lossolo, thanks a lot for this feedback, I took it into account and refactored this sentence a bit :) (deploy will be done in +-2min)

</comment></comment><comment><USER5> there is a lot of info out there about markov chains, but very little about markov decision processes (MDP).

How popular are MDP? What are their strengths? weaknesses?

Is there really a connection between MDP and reinforcement learning as the author claims, as I've not heard this connection before?

This is a very welcome article.

<comment><USER6> Consider reading "Reinforcement Learning, an Introduction" by Rich Sutton[0]. It is very accessible (and probably the most used textbook in the field), and goes into detail about the connection between RL and MDPs.

If you want a high-level answer right now about how the two are related: reinforcement learning focuses on predicting or maximizing the discounted sum of rewards, G_t = R_{t+1} + γ R_{t+2} + γ^2 R_{t+3} + ... which is called the return Crucially, the process generating the rewards (the environment) is assumed to be memoryless[1]. So we can reformulate the return as a recursive equation: G_{t} = R_{t+1} + γ G_{t+1}, since the return from "t+1" doesn't depend on the reward you got from the previous time step ("R_{t+1}").

This in turn allows you to define Bellman equations (which define optimal solutions to the problem of maximizing return, which is what you want to do). More generally assuming the problem is Markovian makes things substantially easier to reason about, because you don't have to worry about complex histories leading up to the agent's arrival in a state.

While it's honestly a pretty strong assumption, it's one that we make across many fields and particularly in RL. It's actually not too inaccurate; for example most games are Markovian (or close), and in physical problems (e.g. robot locomotion) if you've got position+velocity then controlling the robot can be viewed as a (continuous-time) MDP.

The weaknesses are obvious: not every environment is Markovian, sometimes there is long-term dependence on the past. For example, translating prose or poetry in such a way that preserves the point across is _very_ tough[2], not really something you can formulate as an MDP. Other times, the weaknesses are easier to overcome (think of a poker game, where you can combine the present state information with the betting patterns from the past; this augmented state should be able to tell you everything you need to know).

\---

0\. [http://incompleteideas.net/book/bookdraft2017nov5.pdf](http://incompleteideas.net/book/bookdraft2017nov5.pdf)

1\. Another way of phrasing this is that each state provides all the useful information, and knowing about previous states does not tell you anything new about the environment. Think of perfect information games, like Chess. It doesn't really matter how you got into a particular position, just the moves you make from there.

2\. Compare the poetry translations of Jerry Lettvin with what you'd get using existing translation software ([https://sites.google.com/site/lettvingroup/Home/Projects- His...](https://sites.google.com/site/lettvingroup/Home/Projects- History/biographies/jerry-lettvin/creative-writing/gallows-songs)).

<comment><USER7> Wow, what a great answer -- thanks!

I have a question -- in a card game, it appears that knowing which cards have been played is important (or can help, anyway). This seems to violate the idea of being memoryless.

Can you "cheat" and make reason on the cards left instead? Thanks for any clues.

<comment><USER5> Card games e.g. Poker isn't a perfect information game, so history does matter, so it's not markovian.

<comment><USER7> Sorry, should have given a more concrete example, more what I'm interested in.

I was thinking Hearts or Spades, which aren't perfect information, but since the whole deck is dealt out, you can (with a lot of memory :-) know what cards are left.

<comment><USER6> Yeah, so here we can "solve" the problem without using a POMDP by really expanding what constitutes the state space.

Initially, your state is your hand plus the initial rules (who goes first, which suit is trump, etc depending on what kind of game you're playing). After each player's turn you transition into a new state, which is <<starting hand>> \+ <<history of cards played>>, so at each point in time the state has all the available information. Obviously this leads to a combinatorially huge state space, because there's O(52!) different possible sequences for a standard deck of cards.

Typically you have to use some sort of approximation architecture because evaluating all different possible responses to a given sequence of cards would both take too long and even if you could do it, the memory requirements would be enormous. See for example DeepStack[0], which uses a neural net to approximate the quality of positions + possible responses (along with other techniques), rather than having a lookup table for what to do in each possible game state.

So you can formulate such games as MDPs, but the question then becomes: is this a useful way to think about it? Sometimes it is, because your approach to the problem scales and so you can throw enough resources (training data, compute power) at it to learn a useful strategy; other times there is a better way of formulating the problem[1].

I have some implementations of reinforcement learning algorithms and code for working with MDPs if you're interested in that sort of thing[2], and of course I highly recommend Rich's book.

\---

0\. [https://www.deepstack.ai/](https://www.deepstack.ai/) \-- note that it's not really an RL system, but it does illustrate using neural nets to learn complicated functions in a large state space.

1\. For example, I am not sure that such an MDP for hearts is well-posed: the convergence results that I'm familiar with rely on the MDP being ergodic and aperiodic, but the suggested construction is obviously periodic.

2\. The algorithms: [https://github.com/rldotai/rl- algorithms](https://github.com/rldotai/rl-algorithms) An MDP library: [https://github.com/rldotai/mdpy](https://github.com/rldotai/mdpy) (check out notebooks/example_notebook.ipynb for a basic demo).

</comment></comment></comment></comment></comment></comment><comment><USER8> Hi Xavier, in the discount factor equation, shouldn't the last term of the expansion be gamma^n.Rn instead of gamma^2.Rn? Or am I missing something?

<comment><AUTHOR> You are completely right! Thanks for spotting this! :)

<comment><AUTHOR> Update: Fixed it, the caching might need to expire on your side though (ctrl + shift + r should do the trick)

</comment></comment></comment><comment><USER9> For my taste, the article is a bit too high-level to build a meaningful intuition. I think a more complex example would be beneficial.

<comment><AUTHOR> Thanks a lot for your feedback! Currently I am working on the high-level definitions of Reinforcement Learning, so that i can build on that to go deeper in the more advanced ones. I'd like to have a series that explains the OpenAI examples :)

</comment></comment><comment><USER10> Nice work! Any good pages for pomdp?

</comment><comment><USER11> Doesn't the typo "3th" annoy anyone when reading that article?

<comment><USER12> No it doesn't.

<comment><AUTHOR> also fixed it to "third" :) keep the feedback coming</comment></comment></comment>


Milton Friedman's Thermostat - <AUTHOR> http://worthwhile.typepad.com/worthwhile_canadian_initi/2012/07/why-are-almost-all-economists-unaware-of-milton-friedmans-thermostat.html <comment><USER3> Am I the only one who read Friedman's paper? [1] The answer to OP's question [2] - "why are economists unaware of milton friedman's thermostat" - <USER2> because the OP is misinterpreting Friedman's paper, and therefore has misattributed this nonexistent "theory".

Friedman just said that the Fed acts like a thermostat, and in the linked paper, claimed that the period subsequent to 1985 had increased price stability owing to improvements in their understanding of inflation and it's relation to the quantity of money, hence their "better thermostat".

AFAICT, the OP has made up this "Friedman's Thermostat" insofar as it relates to economic forecasting, and now he wants to know why no one has heard of it.

As an aside, "exception that proves the rule" is an almost universally misapplied phrase, that should only be used if you're really really sure you know what it means (kind of like "begs the question").

[1] [http://online.wsj.com/article/0,,SB106125694925954100,00.htm...](http://online.wsj.com/article/0,,SB106125694925954100,00.html)

[2] OP = Nick Rowe - [http://worthwhile.typepad.com/worthwhile_canadian_initi/abou...](http://worthwhile.typepad.com/worthwhile_canadian_initi/about- nick-rowe.html)

<comment><USER4> The delivery is so emotionally charged too. It took several paragraphs of "I can't believe no one gets this!" before we even got to Milton's theory.

<comment><USER5> It's worse than that. He rambles on, and then immediately shifts the metaphor to a more complicated and less clear one. He should have just stuck with the thermostat: if the temperature inside a house is determined by a thermostat, the outside temperature has no correlation with the inside temperature.

<comment><USER6> I suspect this is because he's already written several other posts (linked at the bottom) with the thermostat example, so is trying a different example here, in the hope that people might pay more attention this time.

</comment></comment></comment></comment><comment><USER7> The level of discourse in policy-oriented economics papers is shockingly low. My pet policy issue is spectrum management (allocation of radio frequency ranges). Its a subject that lends itself to "neat" economic arguments. Ronald Coase wrote an instrumental paper in 1959 suggesting we should propertize spectrum and allow trading, which would lead to efficient allocations. It saddens me how many policy papers I see from "think tanks" that completely ignore the second part of that paper, as well as Coase's earlier work: the efficient allocation assumes that trading spectrum has negligible transaction costs. This works for say big TV stations in a city, not so much for say allocating spectrum for personal wireless devices. In my research for a seminar paper, I came across people who really should know better who ignored crucial features of the problem domain, ignored transaction costs, ignored considerations of lock-in, etc.

I'm tremendously skeptical of economics-based policy papers. A biologist works on a paper for years, and concludes that this kind of gene expression might be linked to this kind of nutrient deficiency. An economist at a "think tank" works on a paper for a fraction of that time and gives advice on how to structure a tens of billions of $ industry...

<comment><USER8> I like the saying, the economy does not need economist, just like evolution does not need biologists.

The things we talk about today are not really that diffrent from what Adam Smith argued for/against a couple hundert years ago. Today you have people like Stigliz arguing against markets with much more sophisticated models then the people that argued against Adam Smith. The people that argue against Stigliz use much better models too. So where does that leave us?

I conclude that we should let the market be the market and not try to influence it, its a far more complex thing then economist understand. Trying to influence in to one direction will probebly just cause ten more things that are unexpected. The market is not perfect, not even close, read Stigliz as a good example of this but I highly doute that the goverment can improve the end result in most cases. If you look at the pridictions of TOP economist at every time X, the are almost always wrong and you get some other people that get it right, these people are then wrong the next time.

I barly trust the goverment to do the easy things write, tweek a huge economy is far from easy.

<comment><USER7> I agree we shouldn't try to tweek the economy with government, but the corrolary to this is that we shouldn't then raise up economic arguments to keep the government from solving real problems.

Also, the knee-jerk cynicism about the effectiveness of government is misplaced. E.g. government funds about 1/3 of all R&D in this country (and it was as high as 2/3 in 1964). It is definitely not the least productive third of the R&D expenditures. We crow about all these advances in medical technology, etc, but many of those advances are the direct result of enormous NIH funding. Our world-leading universities get well over half their R&D funding from the government.

<comment><USER9> _but the corrolary [sic] to this is that we shouldn't then raise up economic arguments to keep the government from solving real problems._

I disagree: The economic argument against price controls is a very good one, and it's completely valid and useful to raise against the idea when governments try to use them to "solve real problems".

<comment><USER8> Its kind of funny how even the few say 99% of economist agree on get disregared in politics. Sure price controls are not that common anymore but once a storm comes they are back. Ifs funny as economist would argue that, especially in situation like that a flexible price system is imprtend.

Seams to me the goverment does what it does and if the can find economist that agree with them then they are just happe they can clame that 'economist' agree with what there doing. They are not actually listening to them.

</comment></comment></comment></comment></comment><comment><USER10> OK, I think I'm missing something. To quote the article:

> And no, you can not get around this problem by doing a multivariate > regression of speed on gas pedal and hill. That's because gas pedal and hill > will be perfectly colinear.

This is one of the reasons why you check for multicollinearity [1] when performing multivariate linear regressions; aside from introducing significant instability into the model, if your predictors are correlated, then you're essentially measuring the same thing twice. The stated problem is actually a good example as to why you should _avoid_ having highly correlated predictors in a multivariate model; by blindly pursuing the regression despite the inputs being correlated, we miss out on the fact that there is a relationship present (i.e., the dependent variable (speed) is actually a factor of _both_ of the independent variables (pedal height and hill slope)).

[1]: <http://en.wikipedia.org/wiki/Multicollinearity>

<comment><USER11> You aren't missing anything. The author's article isn't about the optimal ways to do statistics, it's a criticism of the informal economic discussion that dominates policy, blogs, and pop-economics--and, according to the post, an unfortunate number of actual econ papers--that says things like "economic plan X doesn't work because there hasn't been a change".

The criticism generalizes to any statistical methods, though. You always need to think about the underlying model before doing any kind of statistics in any science.

<comment><USER12> Yes, but I wouldn't say that "almost all" economists don't understand the principle (as in the title). I would be more likely to say that "almost all" reporters don't understand basic economics & statistics.

<comment><USER13> THIS. The problem lies with reporters lacking the basic econmic and statistical background to either report the news and/or challange others they are interviewing.

We are often surrounded by "armchair specialists" who know little about what they are talking about, yet carry large opinions about said topic. I'm all for constructive discourse, but that seems like a rare thing when talking about hot topics.

</comment></comment></comment></comment><comment><USER14> Really interesting. Biggest complaint is the author takes a long time to explain the thermostat model. Here is the definition:

Everybody knows that if you press down on the gas pedal the car goes faster, other things equal, right? And everybody knows that if a car is going uphill the car goes slower, other things equal, right?

But suppose you were someone who didn't know those two things. And you were a passenger in a car watching the driver trying to keep a constant speed on a hilly road. You would see the gas pedal going up and down. You would see the car going downhill and uphill. But if the driver were skilled, and the car powerful enough, you would see the speed stay constant.

So, if you were simply looking at this particular "data generating process", you could easily conclude: "Look! The position of the gas pedal has no effect on the speed!"; and "Look! Whether the car is going uphill or downhill has no effect on the speed!"; and "All you guys who think that gas pedals and hills affect speed are wrong!"

<comment><USER15> I'm really looking forward to the comments on this one. Overall it is a variation on the "correlation is not causation" theme and that defining variables in a social setting is super tricky, if not to say impossible.

<comment><AUTHOR> It seems to me that when people say "correlation is not causation", they typically mean you can't learn about a causal relationship by just looking at the correlation. The article makes the converse statement: you can't infer the _absence_ of a causal relationship from the absence of correlation, or equivalently (contrapositive), even when you know there is a causal relationship, it won't necessarily lead to a visible correlation in the data.

<comment><USER16> Well, more to the point: The data analysis game changes entirely once you've introduced an intelligent actor (in this case the driver).

<comment><AUTHOR> Feedback processes don't have to be intelligent, e.g. the thermostat mentioned in the title.

</comment></comment></comment></comment></comment><comment><USER17> I think the better question is why are (almost all) economists unaware of basic economic theory?

Not to go off topic, but it's hard to take any "economist" talking head seriously these days. I'm currently re-reading _The Road to Serfdom_ by F.A. Hayek (highly recommend) and it's scary to see that even 60 years ago we understood cycles and events that are still being ignored today.

The sad fact remains that real economic policy is no longer in vogue in Washington or the L.A. TV sets. What _is_ popular these days is hand wavy gestures that seem to please the most amount of people... the science be damned.

Rather scary stuff.

<comment><USER18> I'm not sure why you think that the lessons of _The Road to Serfdom_ are being ignored. Does anybody in the halls of power seriously think that Soviet style central planning is a good idea anymore?

But the real reason that nobody pays too much attention to Hayek is that the way countries recovered from the Great Depression worked entirely counter to Hayek's predictions. Going off the gold standard worked. It worked _really well_ for every country that tried it! The way that the events of the 1970s conclusively disproved paleo-Keynesianism.

So now we have neo-Keyesianism as endorsed by people like Krugman and Monetarism from people like Friedman and they agree about a lot more stuff than Hayek and Keynes did. Now, they use different terms and have different ideas about the transmission mechanism by which monetary and fiscal policy influences aggregate demand, but everybody these days thinks of aggregate demand as the thing that drives fluctuations in the business cycle. That's progress, maybe in another 100 years economics will be a real science.

And please don't take the talking heads that show up on TV seriously. Mainstream reporting of economics is just as bad as it is of every other technical field.

<comment><USER17> Hayek addresses redistribution of wealth as well as central style planning. He even specifically notes that while they may differ at (sometimes important) points, the general cycles have very similar results. So while not many people still believe in central planning, they seem to have grasped the new theory du jour of "spread it around", and in the end that is just as dangerous. Even so, to ignore the lessons in the book simply because of a superficial difference in ideologies probably isn't wise.

I think you're rather over simplifying things. I'm no economist, but I make it my business to be at the least knowledgable about such things, and I feel recovery from the Great Depression was more of a combination of New Deal infrastructure, WWII profiteering and the resultant increase of wealth in US that was then transferred to Europe to aid in the rebuilding. But then again, that's probably one of the most complex economic time periods in modern history so I don't pretend to know everything.

To address the gold standard, there are indicators that removing the gold standard may have benefitted in the short term, but will hurt us in the long term.

<comment><USER8> You have very strang opinions. One the one hand you speak well of hayke on the other point you use the exact opposit of what hayek belived to explain the recovery from the great depression.

I dont want to go into the hold Great Depression discussion, I just wanted to point out that hayek would have made the argument that the New Deal made the GD much longer, same with the WW2. The idea that a war can help the economy would made hayek cry.

Watch this modern Rap-Video of Hayek vs. Keynes and listen closly when the talk about the war.

<https://www.youtube.com/watch?v=GTQnarzmTOc>

</comment></comment></comment></comment><comment><USER19> A similar problem happened at Harvard Business School. In the mid 80s they found no correlation between GMAT scores and success at the school, so they stopped requiring applicants to submit scores. This was a thermostat-unaware view.

The real answer was that they were giving it the right amount of weight. If they were overweighting the GMAT, unqualified people with high GMATs would get in. If they were not underweighting it, the opposite.

Approximately 10 years later they did a correlation of student GMATs (which weren't used for admissions purposes) and school success, and found that there was indeed now a correlation.

The lesson was that if you're properly managing a factor, you won't see it in the outcome. Or in the thermostat example, if you're properly managing the thermostat, you won't see the changes.

</comment><comment><USER20> I am an econometrician... exactly the sub-population that the author faults most for not knowing about Friedman's thermostat.

We haven't heard of the thermostat because it does not introduce new ideas to us. If the data includes hill height, this is an example of multicollinearity. If your data does not include hill height, this is an example of endogeneity.

Both concepts are taught in considerable depth in an undergraduates first course in econometrics. It's a nice examples, but the author's claim that these phenomena are outside are awareness shows a striking ignorance of basic econometrics.

<comment><USER21> Okay, fine, but then what do you think of the Casey Mulligan column that the author links to (third paragraph)? It certainly seems at first glance to contain instances of the error the author is describing.

(I am not an economist and have no position on this; just curious.)

<comment><USER20> That's a great question, and I'm afraid I'm not going to do it justice. But the place where I think you see the thermostat concern is in the paragraph:

 _A 1983 study by Lars Peter Hansen of the University of Chicago and Kenneth Singleton of Stanford showed that short-term rates on Treasury bills and short-term returns on stocks traded on the New York Stock Exchange had very little correlation with consumer spending. Many empirical studies have confirmed this sort of result (this comparison of inflation-adjusted Treasury bill returns and business sector profitability is a recent example)._

This summarizes "many empirical studies" in a single sentence. Unfortunately, it doesn't include citations.

There are a number of ways to address the "thermostat" problem. Each potential solution only works in specific circumstances. I'd hope the underlying research adequately addresses this concern, but it's hard to say without seeing the research.

In general, summarizing "many" 40 page papers in a paragraph is inherently difficult, and Mulligan seems to be especially vague here. If the claims here have any credibility, it would only be through an "appeal to authority," which most of us won't find very compelling.

So to answer your question, I'm unimpressed with the Mulligan column... in part because we have no indication what problems there are in the underlying analysis.

</comment></comment></comment><comment><USER22> >And no, you can not get around this problem by doing a multivariate regression of speed on gas pedal and hill.

I'm no expert on statistics, but this seems like a pretty basic scenario for a statistical analysis to provide insight into. These two phenomena would be perfectly correlated in both the time of appearance and the degree.

You wouldn't be able to derive anything about the effect of hills or gas pedals on speed, but you'd be given a powerful clue as to what's going on.

The second there's even a tiny imbalance between the two, you're given the relationship to speed.

<comment><USER23> Well, in this contrived scenario, you can't tell what would happen if the driver _didn't_ push down on the gas pedal while going up the hill. The relationship between the gas pedal and the speed of the car is entirely obscured.

However, in a real world system, the chance that there is zero imbalance is also nearly infinitesimal. You could take the most skilled driver in the world and you would still be able to pull out the relationship between gas pedal and speed out of his driving pattern. Similarly, with the right variables and enough data, you could pull a relationship like the one described here out of economic data.

(As a side note, I think the way in which this article is written is pretty obnoxious. The first 500 words are dedicated to the author congratulating himself for being aware of an idea that he hasn't even explained yet.)

</comment></comment><comment><USER24> As a meta-comment, this article dances around and only after an entire page of text, does it get to the actual idea.

</comment><comment><USER25> It sounds like the problem boils down to the difference between understanding a scalar (speed) and a vector (velocity). So economics would need a way to represent economic forces similar to the way a free body diagram is used in engineering and physics. And in order to do that, you would need an economics coordinate system. And therein lies the problem of economists building models without known or absolute reference points. Economics != Physics, QED.

<comment><USER26> Nope, absolutely not. The problem is that they have a feedback system which is attempting to maintain a constant value. This could be a driver pressing the gas, a thermostat (as listed below), or the Fed trying to modify interest rates to achieve a result. The problem is, if you do everything right, and there's no economic collapse, people wonder why you need to exist. If there is, you're clearly useless.

I fall into this a lot, but I really hate the attitude of engineers and scientists believing that everything can be solved by approaching a problem a certain way. We have this idealized, fetishized approach that if you define all your axioms and have a rigid framework in place, you can solve anything. Of course, as soon as you try it in real life, especially in a field like sociology or economics (which, of course, we deride as not being scientific enough), it all goes to hell.

I would recommend that anyone who is of this opinion - that economics, psychology, linguistics, et al are not scientific enough - actually take a course in that area. What you find are: people who are experts in a field are really smart, yes, they have thought of that already, no, it didn't work. It's a very humbling experience.

<comment><AUTHOR> > _The problem is, if you do everything right, and there's no economic > collapse, people wonder why you need to exist. If there is, you're clearly > useless._

Interesting. Can we come up with some general method of answering questions like "does institution X really play a part in keeping parameter Y stable?"

<comment><USER26> The problem is: the only way to detect this, as the article points out, is for them to stop doing their job well intentionally. You either wreck the perfectly good situation you have locally, or you find some other place where everything is terrible, and you compare the two. This is one of the reasons why scientific methods are hard to apply in non-experimental situations: you can't necessarily destroy the national economy 'to see if it would really work'. It's the same as informed consent in medical experiments, just on a larger scale.

</comment></comment></comment></comment><comment><USER27> I have no idea if the author describes "friedman's thermostat" correctly, but to say that Economists are unaware of the phenomenon he describes is beyond absurd (disclosure: I'm an Economist!).

There's another way to find the relationship between the gas pedal, slope, and speed that the author did not mention (but is somewhat related to some of his suggestions): write down a model of the driver that assumes he or she uses the gas pedal to try to keep speed constant:

E_{t-1} speed(gas_t, slope_t, parameters) = const (eq1)

where E_{t-1} is the conditional expectation of the period-t term, given the information available in period t-1. Maybe speed should be nonparametric, but that will introduce some new problems in estimation; maybe you know enough about physics to write down a parametric formula for speed; whatever, the exact details of speed(.,.) are kind of beside the point for the author's argument.

Now,

speed(gas_t, slope_t, parameters) - E_{t-1} speed(gas_t, slope_t, parameters) (eq2)

is a martingale difference sequence when "parameters" is set to their true value, so the sequence equals zero in expectation for all t, assuming the model is true, and that can be the basis for estimation through, say, Generalized Method of Moments. Because combining (eq1) and (eq2) gives us

E speed(gas_t, slope_t, parameters) = const,

so you can estimate the parameters as

\hat parameters = argmin (average_over_t speed(gas_t, slope_t, parameters) - const)^2

Everything I laid out is an extreme simplification of the DOMINANT STRATEGY in applied macro (with, in all likelihood, some errors due to sloppiness). Now, for any realistic economy, it's going to be hard as hell to write down a sensible formula for "speed(.,.,.)" and since there aren't that many years since WWII (which is kind of seen as the beginning of the "modern" economy) there isn't a lot of data to estimate the model, but neither of those issues have anything to do with the author's "critique" and are basically the direction of almost all research in macro.

<comment><USER28> pseut: "so you can estimate the parameters as

\hat parameters = argmin (average_over_t speed(gas_t, slope_t, parameters) - const)^2"

No you can't.

Think about it.

1\. If the driver has as much information as the econometrician, setting both parameters on slope and gas equal to 0 will fit the data equally well.

2\. If the econometrician observes slope and gas and speed with error (as will almost always be the case), then GMM will estimate both parameters on slope and gas as zero (if those 3 errors are independent of each other), even if the true parameters aren't.

3\. If the driver observes gas or slope with error, and the econometrician doesn't, then the econometrician can indeed actually estimate the parameters.

How likely is 3? Only a stupidly arrogant econometrician would assume a priori that he knows better how to drive the car than the guy actually driving it. (OK, maybe the econometrician has final revised data, and the driver has only real time data, and the final revised data is better than the real time data. But even then the driver will be observing other indicators that the econometrician doesn't have data on, so the econometrician will interpret the driver's response to those omitted variables as "gas pedal shocks", and will screw up the estimation royally.

Yep, all that Sims VAR stuff is wrong. Here, read this.

[http://worthwhile.typepad.com/worthwhile_canadian_initi/2011...](http://worthwhile.typepad.com/worthwhile_canadian_initi/2011/10/james- hamilton-on-christopher-sims-and-identifying-monetary-policy-shocks.html)

You have just proved my point: economists don't understand Milton Friedman's thermostat.

Forget your fancy stuff. Just STOP AND THINK about what you are really doing when you try to estimate parameters.

<comment><USER27> I should have been a little more specific; the approach I described assumes that the driver makes some errors, so the speed is not actually constant in practice. That can be because the driver slightly errs in seeing the slope or gas, or because of other sources of error (gusts of wind, maybe). Without that variation, you're right, the system is unidentified. The larger the variation, the better the identification. But none of this is news to economists, and none fundamentally requires the econometrician to have better information than the driver (although measurement error can matter in some circumstances, it matters in a different way).

As I'm sure you know, these arguments (i.e. mainstream Economics is wrong and ignorant!) are more convincing when they're accompanied by a model. DSGE models have a lot of limitations, but they're pretty good for demonstrating failure of identification. Monetary policy is tricky to identify, and I'd be sympathetic to an argument showing that deviations from a Taylor-rule are bad for identifying the effects of monetary policy shocks (lots of people would agree with this, the interesting question is whether they're bad in empirically important ways or just conceptually bad) but your car+driver analogy seems like it's aiming to be broader than that.

<comment><USER28> Thanks psuet. I'm not one of those guys who goes around saying "mainstream economics is wrong and ignorant". Well, OK, we are often wrong and ignorant, but people who aren't in mainstream economics are usually even more wrong and ignorant!

This is just a particular beef I have. Because I keep on seeing examples where economists make mistakes by missing this point. Like when economists try to test whether headline or core inflation is better at forecasting future inflation, and use those results to give policy advice on whether inflation- targeting central banks should respond to core, headline, or both indicators. All they are modelling is the central bank's mistakes, in responding too strongly or too weakly to those indicators.

I think proper identification _does_ require the econometrician be (in some sense) a "better" driver than the driver. If you can see the hills better than the driver can, then you can see the effects of a hidden hill that you know he doesn't see.

</comment></comment></comment></comment><comment><USER29> Why does it 11 paragraphs repeating himself about how important this idea is? I thought this was some set-up for a joke in which he never tells us about the thermostat.

</comment><comment><USER30> In addition to klochner's comment, this is why economists rely on exogenous variation. Any applied econometrician worth his or her salt would not say "the gas pedal has no effect", but that the data doesn't rule out the hypothesis that it does.

</comment><comment><USER31> I thought he was never going to get to the point. This entire post should be about 1/5th the size.

</comment><comment><USER32> There are exactly 500 words before a single bit of substance in this article.

</comment><comment><USER33> find that explanation shorter & faster, although polsci

[http://themonkeycage.org/blog/2012/07/31/milton-friedmans- th...](http://themonkeycage.org/blog/2012/07/31/milton-friedmans-thermostat/)

</comment><comment><USER34> This is a subset of David Hume's analysis of causality, see his "An Enquiry Concerning Human Understanding."

</comment><comment><USER35> > And no, you can not get around this problem by doing a multivariate > regression of speed on gas pedal and hill. That's because gas pedal and hill > will be perfectly colinear.

Um, what? For a constant speed, which is this person's example, this claim is completely false.

<comment><USER36> When the incline increases, the driver pushes the gas pedal harder. Inline up correlates with gas pedal up. It's a statement of relation between gas pedal and hill incline. The claim is correct.</comment></comment>


Why can't techie's sell? - <AUTHOR> Embarking on a side business while I work. What is the most frustrating thing about sales from a techie's point of view? Do we really have to use hired guns to promote our vision? <comment><USER2> Most people don't buy on numbers or how it works. Price is only part of the equation.

</comment><comment><USER3> No. Find one person who will indeed use what you are building. Then find another. And so on.

Good luck.

</comment><comment><USER4> Because techies spend their time learning how tech work, not how people do?</comment>


My Medical Choice - <AUTHOR> http://www.nytimes.com/2013/05/14/opinion/my-medical-choice.html <comment><USER2> I find it fascinating that just over a hundred years ago proactive teeth removal was a standard practice. Women would get all their teeth removed to save money for the prospective husband. And left untreated for too long, rotten teeth will definitely kill you through infection.

These days, women are getting proactive mastectomies, not so much because of the costs involved in treatment, but because cancer is a much scarier disease than teeth rot. At least to us. To us proactively removing your teeth is a bad joke.

In 100 to 150 years, will people think about a proactive mastectomy as a bad joke? I sure hope so.

<comment><USER3> _I find it fascinating that just over a hundred years ago proactive teeth removal was a standard practice._

What would people eat then? Smoothies, cream of wheat?

<comment><USER4> They used false teeth.

<comment><USER3> Hmm, I guess that makes sense. Seems so primitive compared to modern dentistry though.

</comment></comment></comment></comment><comment><USER5> _I am fortunate to have a partner, Brad Pitt_

Took me a few seconds to process and understand who the author is.

<comment><USER6> I don't get why they don't make a bigger thing of who the author is on this one.

<comment><USER7> she's credited in the byline

<comment><USER6> Yes - but you miss that because it's not normally important - i.e. it's no one you've heard of.

<comment><USER8> Breast cancer is more important in "someone important, like an actor"?

<comment><USER9> It's a braver decision from someone who depends on their looks for livelihood. Also creates a mindset "if she could do it, then so could I"

<comment><USER8> I'd expect it's less brave for someone from an industry and social circles comfortable with implants, injections scalpels, etc, and with a bit more than a "livelihood" to ease the experience. The experience would surely be more foreign to the average woman, requiring more bravery. Angelina isn't dependent on the look of her breasts for her "livelihood". 87% risk is a no brainer.

<comment><USER9> It's easy to talk about 'no-brainers' in the abstract. A different story if it's your own favourite bit.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> Preventive medicine like this saves lives and money. But try convincing the pencil pushers. Looks like in both privatized and nationalized healthcare, they are often only willing to pay when there's a disease to diagnose. Not when there's an 80% chance you'll get it later when it's much harder and expensive to cure. Unless you have Jolie money and can simply afford to pay for it yourself.

<comment><USER11> _Preventive medicine like this saves lives and money._

Sometimes it does, sometimes it doesn't. Lots of preventative medicine doesn't save either lives or money - mammograms below age 50 are a good example. The increased risk of infection from doing a biopsy on a non-cancerous mass (and similar complications) outweigh the rare early breast cancer cases prevented.

[http://www.uspreventiveservicestaskforce.org/uspstf/uspsbrca...](http://www.uspreventiveservicestaskforce.org/uspstf/uspsbrca.htm)

[http://www.bloomberg.com/news/2012-07-31/in-mammogram- debate...](http://www.bloomberg.com/news/2012-07-31/in-mammogram-debate- politics-trounces-science.html)

I don't know the specific details of this case. Maybe she reduced her risk, or maybe she traded a 5/1000 risk of cancer for a 6/1000 risk of infection.

But this "preventative medicine saves lives and money" meme needs to die. Sometimes it's worthwhile, sometimes it's not. Numbers >> political memes.

<comment><USER12> Needs to die? What are you talking about? Obviously administering tests that haven't been shown to be effective prophylactically and can even cause harm to a given population is a stupid idea. That's why we do research and weigh benefits/risks. That's like saying antibiotics need to die because many times they don't work and can cause resistant strains, when the problem is that physicians often administer them incorrectly.

Appropriate preventative medicine supported by research and numbers is so, so important.

<comment><USER13> It's more akin to saying "Antibiotics for every cold and allergy needs to die", which, is true.

The other poster said the meme needs to die, not _all_ preventative medicine.

</comment></comment></comment></comment><comment><USER14> Note that the BRCA test costs $3k when purchased through a physician, but 23andme violates the patents and tests for BRCA for $99. <http://23andme.com>

<comment><USER15> Have you done this? I'm interested in knowing my genetic risk factors, but I'm afraid of how I would handle it if there was something really bad and unexpected.

<comment><USER16> I have done 23andMe and had my full genome sequenced. Unfortunately, we are still in the early stages of understanding our genome and there is often little of medical importance revealed.

The most significant of the "something really bad and unexpected" are the risk factors for Alzheimer's and Breast Cancer. I believe that there are certain markers for both those diseases which could reveal that you have a much higher lifetime risk (for some cases of breast cancer, >50%), which is obviously scary. Don't quote me on this, but I believe in both situations you are very likely to have had some family history in these diseases, so it should not be a total surprise. And if you have a family history of breast cancer, you should be getting Myriad's full BRCA test (see my other post). I would probably look into a clinical test for Alzheimers too if I thought I was at risk.

There are a few other diseases that also have big impact (>50% lifetime risk), but they are rarer, and I'm not sure if there are any where you would have had no family or personal history to clue you in on the possibility.

The vast majority of the diseases 23andMe reports on give you very little actionable data. For example, it might say I have a 2x risk of prostate cancer, from a background risk of 2% chance to a personalized 4% chance. I view that as pretty unhelpful. Furthermore, it's based on SNP-associations currently known. It's possible, I have another SNP that's associated with a .25x risk, and therefore I actually have only 1% chance overall, i.e. half the general population's risk.

23andMe can be useful for other reasons. For example, it will tell you if you are a recessive carrier of certain diseases, which is helpful if and when you decide to have children. It's also fun, has ancestry info, etc.

<comment><USER17> Shameless plug for a friend of mine, but <http://www.snpedia.com/> and <http://promethease.com> can help you further in your analysis (if you don't already know those tools).

</comment></comment></comment></comment><comment><USER18> I assume a lot of women would want to wait until after they were done having children (particularly since ovaries would also be removed). It might be a harder choice between "mastectomy/oophorectomy and sterility" and "increased risk of cancer" for a 22 year old.

Also surprised the patents on BRCA didn't come up in this article, just the cost.

<comment><USER19> On the other hand, given a genetic risk factor so potentially life threatening that it warrants having body parts preemptively removed, you might think twice before reproducing.

<comment><USER20> That would certainly be a great way to prevent genetic disorders for future generations, just disallow anyone with a pre-disposition from reproducing and/or being born.

That seems like an odd way to look at reproduction. Would you rather like 50 years and die of cancer or not have lived at all?

It's one thing to make a decision when there's medical evidence during the pregnancy that can be tested but when you're only talking about a statistical risk that something may or may not occur...

<comment><USER19> I was implying a personal decision to avoid becoming pregnant in the first place. I guess if you want to knowingly risk passing genes that cause terrible disease and suffering to all your descendants… yay freedom!? There are other options if having kids is important to you.

</comment></comment></comment></comment><comment><USER21> At first I thought, this wasn't relevant to HN. But then I realized that A. Jolie is hacker. She is trying to hack her life.

<comment><USER22> ><http://www.imdb.com/title/tt0113243/>

<comment><USER21> 1995!

<comment><USER23> RISC is good.

</comment></comment></comment></comment><comment><USER24> Elizabeth Iorns, the CEO of Science Exchange (YC S11), is currently conducting independent research to find a potential cure specifically for the BRCA mutation.

([https://www.microryza.com/projects/can-we-prevent-the- transm...](https://www.microryza.com/projects/can-we-prevent-the-transmission- of-brca-mutations))

</comment><comment><USER25> This is generally covered by insurance in the US, including the reconstructive surgery.

[http://breastreconstructionnetwork.com/will-my-insurance- com...](http://breastreconstructionnetwork.com/will-my-insurance-company-pay- for-a-mastectomy-to-reduce-my-risk-of-breast-cancer/)

<comment><USER26> Generally, maybe. My friend had this surgery recently and she had to fight (her insurance company) for it.

</comment></comment><comment><USER27> My mother lost both breasts, her gallbladder and her fallopian tubes to 4 separate cancers. When they discovered the BRCA gene mutations, she was tested and turned out positive. My sister tested positive and had proactive masectomies and hysterectomies. I was tested as well, but came up negative for the BRCA mutations ( _phew_ ). I'm male, but if I was positive it would have meant changes for my daughter.

One advantage of the human genome project was that it lead to discoveries like the BRCA mutations. Now, the women in my life could decide ahead of time to prevent these cancers.

</comment><comment><USER28> Do any of my fellow HN readers know enough to say that Angelina Jolie processed the available information of her extremely high risk of breast cancer correctly? Is the mathematics logical?

Something I wondered is, that given that 1 in 3 American women will have cancer sometime during their life, (and 1 in 5 will die from cancer), how much has she improved things by reducing from 87% to 5% the chance of having one specific cancer?

<comment><USER29> Wikipedia has a good summary of the risks:

<http://en.wikipedia.org/wiki/BRCA_mutation>

 _A 25-year-old woman with no mutation in her BRCA genes has an 84% probability to reach at least the age of 70.[16] Of those not surviving, 11% die from either breast or ovarian cancer, and 89% from other causes._

 _Compared to that, a woman with a high-risk BRCA1 mutation, if she had breast cancer screening but no prophylactic medical or surgical intervention, would have only 59% chance to reach age 70, twenty-five percentage points lower than normal. Of those women not surviving, 26% would die of breast cancer, 46% ovarian cancer, and 28% other causes.[16]_

 _Women with high-risk BRCA2 mutations, with screening but with no prophylactic medical or surgical intervention, would have only 75% chance to reach age 70, nine percentage points lower than normal. Of those not surviving, 21% would die of breast cancer, 25% ovarian cancer and 54% other causes.[16] The likelihood of surviving to at least age 70 can be improved by several medical interventions, notably prophylactic mastectomy and oophorectomy.[16]_

<comment><USER30> But, given that she is a Hollywood celebrity? Does the likelihood varies on this condition?

</comment></comment></comment><comment><USER31> See also <https://news.ycombinator.com/item?id=5607618> (discussion of "Our Feel-Good War on Breast Cancer"). That article does not address women with a BRCA mutation however.

</comment><comment><USER32> Google: "Angelina Jolie complies diet, which combines a high percentage of carbohydrates and proteins with a low percentage of fat intake."

Meanwhile, in PubMed: "Women in the highest glucose quartile were 63% more likely to develop breast cancer."

</comment><comment><USER9> Her going public about this has got a great potential to raise awareness and make other women more likely to consider this course of action. The sad reality is, however, most women won't be able to afford the reconstructive surgery (or to have it done to the standard Angelina Jolie has).

<comment><USER24> I think more importantly it brings up the very real human implications of the debate against patenting human genes, which has most recently been centered around the BRCA gene.

[http://www.npr.org/blogs/health/2013/04/15/177035299/supreme...](http://www.npr.org/blogs/health/2013/04/15/177035299/supreme- court-asks-can-human-genes-be-patented)

</comment></comment><comment><USER33> i think it's brave and commendable of ms. jolie to open up about something so personal and difficult, using her fame and notoriety to help others who come into similar situations.

much respect sent her way.

</comment><comment><USER34> I think this is a great example of how a celebrity can use star power to impact the lives of other women. I only wish she had not done this in secret...if there are tabloid reports of pregnancy and weight loss, why not on such an important issue that can have an impact?

I also wish (for the first time ever) that she had introduced herself earlier in the article. Just like many readers, I didn't realize this was Angelina Jolie until very late in the article (which doesn't minimize her story, it just puts it in a different context).

<comment><USER35> | I only wish she had not done this in secret.

Not done what? She published an account of her medical decisions in the NYT... hardly secret.

</comment></comment><comment><USER36> I will add that breast cancer runs very heavily in my family. My sister has had two mastectomies and four occurrences of breast cancer. My mother has had two occurrences of breast cancer, one mastsctomy, one lumpectomy. I took care of my sister after her first mastectomy. I know how severely and permanently impaired my two workaholic relatives are. Angelina Jolie may be less impaired than my relatives because she might not have had lymph nodes removed. But this is major surgery and she is really downplaying that.

My sister likes to garden. She used to dig in the red Georgia clay with a pick axe. These days, she can't even vacuum. Her arms are prone to swelling. She does not have the strength she once had. Her life is far more limited than it once was. For years, my mother owned two pathetic, falling apart bras. Anything new with the eladtic still working made her arm numb and caused it to swell. She cannot tolerate anything snug, so she also wears a hugely oversized one piece bathing suit. They carry on and are not complainers. But the reality is they are both very seriously and permanently impaired.

Having seen what my relatives have been through, I understand why Angelina would make such a radical choice. I have done some rather extreme things as well for health reasons. But I am disturbed by the way this is being glossed over. I hope she remains satisfied with her choice. But with her fame and beauty and social influence, I think she is being negligent and irresponsible to make light of the consequences of this surgery. It may be a perfectly reasonable thing to do, but it is not without consequence.

</comment><comment><USER36> I have mixed feelings about seeing this. On the one hand, I do understand why she would make such a choice and I am cool with her using her fame to try to promote awareness and discussion. On the other, I am not happy with the way so much of modern medicine mutilates people instead of making them genuinely healthy and her example helps encourage people to feel that is okay.

</comment><comment><USER37> Best of luck to Crash Override. You're the best!!

<comment><USER38> ahem, acid burn.

</comment></comment><comment><USER39> The quality of the surgery is important too. If a plastic surgeon does it they tend to be less aggressive in scraping every last bit of tissue off before reconstruction. Breast surgeons tend to be more aggressive. Source: breast surgeon in the practice next door who I was discussing this with. Due to the source there may be a bit of profession bias.

</comment><comment><USER40> is this on the front page of hn because jolie starred in hackers?

<comment><USER41> It is on the front page because enough readers of hacker news considered it interesting and [hacker]newsworthy, and therefore up-voted it.

<comment><USER40> yeah, and by extension i'm asking why enough readers of hn considered it interesting.

<comment><USER42> Could be said that not enough people make health decisions based on risk. Someone quite public has done exactly that.

</comment></comment></comment></comment><comment><USER43> And what do women without insurance, money, and BRCA mutations do?

</comment><comment><USER44> utterly pointless... By that logic some say we were born to the grave.

</comment><comment><USER45> You can run but you can't hide. Body is a 100% accurate signal of your condition. She's messing with it to remove reception: it's like taping your car's warning light black so that you won't have any problems.

<comment><USER46> Erm, no.

She can't have breast cancer if she doesn't have any breasts anymore!

What she is doing is prevention. Like fixing the engine before the warning light comes up.

<comment><USER47> Actually, she can, breast cells are not limited to the breasts, neither are ovarian cells limited to the ovaries, hence the 5 - 10% remaining risk. Woman who have this procedure cannot expect to never have to check for ovarian or breast cancer again, it is not a 100% preventative.</comment></comment></comment>


Solve This Math Problem, Win a Million Bucks - <AUTHOR> http://newsfeed.time.com/2013/06/11/solve-this-math-problem-win-a-million-bucks/?hpt=hp_t3 <comment><USER2> I came across this rather amusing discussion on google groups: [https://groups.google.com/forum/?hl=en&fromgroups#!topic/sci...](https://groups.google.com/forum/?hl=en&fromgroups#!topic/sci.math/Mc2aYUCnQ_U)</comment>


The Night Watch (2013) [pdf] - <AUTHOR> http://scholar.harvard.edu/files/mickens/files/thenightwatch.pdf <comment><USER2> There will always be someone at a lower level than you. As a systems programmer he pokes fun at UX people. It is indeed hilarious. But technology and science is all about collaboration and building on other people's work. Remember that a generation ago nobody could be expected to use a program successfully without reading documentation first, and this even included most games. The proliferation of computers into the hands of everybody at your local bus stop is as much because of the work of the HCI people as the systems people.

A chip designer could write a similar rant about how systems programmers live a life of luxury sipping champagne with their pinkies held up. YOU USED AN ASSEMBLER? I SHOULD BE SO LUCKY! Or a chip designer could be mocked by someone else because he uses verilog.

Or the UX researcher could scoff at the systems programmer, because at least CPU's are rational and predictable, while people are at varying levels of intelligence and have different kinds of cultural biases.

<comment><USER3> Its intended as humour, not a rant. Maybe read some of his other stuff, for context.

</comment></comment><comment><USER4> Mickens has some great material. He gave a talk at Monitorama a few years ago that's one of my favourites: [https://vimeo.com/95066828](https://vimeo.com/95066828)

The article paints the plight of systems programmers in a hilarious way. I think programmer humour is rarely this well written, although a recent piece comes to mind by aphyr: [https://aphyr.com/posts/340-acing-the-technical- interview](https://aphyr.com/posts/340-acing-the-technical-interview)

<comment><USER5> Mickens is my favorite. His piece about theoretical vs practical security in tech is awesome.

<comment><USER6> Link?

<comment><USER7> [https://www.usenix.org/system/files/1401_08-12_mickens.pdf](https://www.usenix.org/system/files/1401_08-12_mickens.pdf)

</comment></comment></comment></comment><comment><USER8> Having done that sort of work early in my career, I think that's a bit much.

My first job in California started out with two 6-foot stacks of octal dumps of operating system crashes, for a mainframe OS that was running 1-2 hours between crashes. I worked through those slowly, with colored pencils and highlighting markers, figuring out what had gone wrong, then putting a fix into the OS, in assembler. Gradually the stacks got shorter and the time between crashes increased. When I left that job, we were going a month between crashes.

Then I changed jobs to work on operating system design and proof of correctness.

This is why I'm so negative on unsafe code in Rust. Memory safety is essential. There are programmers who think they are so good that they don't need bounds checking. They are wrong. The most recent CERT advisory of a security hole for a buffer overflow was 8 days ago.[[http://www.kb.cert.org/vuls/id/214283](http://www.kb.cert.org/vuls/id/214283)]

</comment><comment><USER9> _I'm glad that people are working on new kinds of bouncing icons because they believe that humanity has solved cancer and homelessness and now lives in a consequence-free world of immersive sprites. That's exciting, and I wish that I could join those people in the 27th century. But I live here, and I live now..._

How often I feel like this describes almost every human endeavor. "Oh, your toaster can phone home? Marvelous..."

</comment><comment><USER10>
        Syntax error: unmatched thing in thing from std::nonstd::__map<_Cyrillic, _$$$dollars>const basic_string<epic_mystery,mongoose_traits<char>,__default_alloc_<casual_Fridays = maybe>>

</comment><comment><USER11> His pieces in Login are great. There's links to the rest of them and his others works at [http://mickens.seas.harvard.edu/wisdom-james- mickens](http://mickens.seas.harvard.edu/wisdom-james-mickens)

</comment><comment><USER12> Mickens usenix articles helped me get through several all nighters in undergrad. He has the perfect balance of technical material and humor in my opinion.

</comment><comment><USER13> MySQL: You want answers?

htop: I think I'm entitled.

MySQL: You want answers?

htop: I want the truth!

MySQL: You can't handle the truth! Son, we live in a world that has data, and those data have to be queried by processes that takes load. Who's gonna do it? You? You, Lieutenant htop?

I have a greater responsibility than you can possibly fathom. You weep for 4 cores and you curse the Queries. You have that luxury. You have the luxury of not knowing what I know: that the core's load, while tragic, probably runned queries.

And my existence, while grotesque and incomprehensible to you, runs queries. You don't want the truth because deep down in places you don't talk about at parties, you want me on that processor, you need me on that processor.

We use words like load average, slow running queries, query execution plan. We use these words as the backbone of a life spent developing db applications. You use them as a punchline. I have neither the time nor the inclination to explain myself to a man who rises and sleeps under the blanket of the very database power that I provide, and then questions the manner in which I provide it!

I would rather you just said, "Thank you," and went on your way. Otherwise, I suggest you pick up a server and run a query. Either way, I don't give a damn what you think you are entitled to!:

htop: Did you order the Code SELECT * ?

MySQL: (quietly) I did the job I was sent to do--

htop: Did you order the Code SELECT * ?

MySQL: (shouting) You're goddamn right I did.

[stunned silence] htop: Please the court, I suggest the members be dismissed, so that we can move to an immediate article 'Use Redis' session. The witness has rights.

<comment><USER14> Loved it! Absoolutely hilarious!

<comment><USER13> Thanks! For those who didn't get the joke, I want to point you to this moment of decision in A Few Good Men.

(spoiler alert) - [https://www.youtube.com/watch?v=9FnO3igOkOk](https://www.youtube.com/watch?v=9FnO3igOkOk)

</comment></comment></comment><comment><USER15> > I mean, when a machine learning algorithm mistakenly identifies a cat as an > elephant, this is actually hilarious.

But when the ML algorithm mistakenly identifies a black person as a gorilla, it's a disaster.

[https://fusion.net/story/159736/google-photos-identified- bla...](https://fusion.net/story/159736/google-photos-identified-black-people- as-gorillas-but-racist-software-isnt-new/)

<comment><USER16> Someday, people will live in a world where both errors are funny for the same reason. Right now we've got a lot of cultural baggage we unintentionally picked up over the years. I'm glad most people are aware enough to see it as a disaster, and maybe they're able make some adjustments to their values, slowly influencing our collective cultural values. I doubt i'll ever see it, but hopefully 10 generations down the road, it'll be funny for everyone.

I wish there was a way to make things like that go faster, but it seems like most people have a hard time changing at that level. On the upside, my parents are less racist than my grandparents. I like to think i'm less racist than my parents. The only effective method for that kind of change seems to be the people holding old values just dying off.

<comment><USER17> > Right now we've got a lot of cultural baggage we unintentionally picked up > over the years.

"Unintentionally"? That's an optimistic take on history (recent and non- recent), considering the institutionalized/codified nature of the said "baggage".

<comment><USER16> Individual intention. I didn't build those institutions or codes. Yet those institutions and codes influenced my development, and continue to manipulate me.

History is history. It's how we got here and what built the culture we deal with today. For example, 250 years ago those institutions came to the conclusion the 3/5ths compromise was a good idea. 150 years ago they reversed their opinion. We're still dealing with a zillion consequences of those decisions.

No one alive today had any influence over any of that. We're told this is how the world was, this is how the world is. Some people recognize, no, the world doesn't have to be this way and make changes. But i think that's rare. Most people just sort of accept what they're told and go on with their lives.

</comment></comment></comment></comment><comment><USER18> actually. you can* lay a copy of steele's lambda papers on top of an intel processor and the blinding light of godhood will shine forth

<comment><USER19> *symbolics

</comment></comment><comment><USER20> I was impressed until about this:

 _As a systems hacker, you must be prepared to do savage things, unspeakable things, to kill runaway threads with your bare hands, to write directly to network ports using telnet and an old copy of an RFC that you found in the Vatican._

So if I can use netcat and ethereal, I'm ready for the zombie apocalypse? I can sleep better then. :-P

<comment><USER21> Netcat/telnet doesn't write the payloads for you.

</comment></comment><comment><USER22> From now on I'm calling my UI/UX guy ZeusHammer

</comment><comment><USER23> Good to have candid information online.

</comment><comment><USER24> Title should have a (2013) tag.

Some previous discussion : [https://news.ycombinator.com/item?id=9671020](https://news.ycombinator.com/item?id=9671020)</comment>


Blues of Many Skies - <AUTHOR> https://www.lrb.co.uk/v41/n04/joyce-chaplin/blues-of-many-skies <comment><USER2> Can we get a "paywall" or "login required" tag? And the ability to filter those out from my HN listings? It's getting pretty annoying.

<comment><USER3> NoScript solves this one though.</comment></comment>


Why Are the Rich So Interested in Public-School Reform? - <AUTHOR> http://ideas.time.com/2011/12/09/why-are-the-rich-so-interested-in-public-school-reform/

<comment><USER2> I don't think that this is something particular to the rich. I'm not rich, but I'd like to see true reform, where the public schools phased out altogether. Truancy laws should be abolished.* The good schools (the profitable ones) should be sold off to private investors, and the bad ones should simply be shut down. True morality (non-enslaving, non-sacrificing) says that taxes should not be taken from Peter to pay for Paul's children. Reforms such as the ones I've outlined will get us there so we can all get on with our lives.

*Most people will send their kids in anyway. Parents universally tend want their children to have better lives than they do, unless they're in their natural, pre-industrial state, which is poverty. But here in America, where even the poor have cars, cable, microwaves, etc., no one is as poor as the poor were in the pre-industrial world. So what I'm proposing won't result in widespread ignorance (in fact, with parents having greater choice and responsibility, standards and student achievement will go up.)

And for those that simply can't pay to send their kids anywhere? They can homeschool, or rely on charity. Also, if you care so much, you're free to donate your own money to help them, and to try to make rational appeals to others to donate. Using a gun to extract money from some to give to others, regardless of the supposed end is not a rational argument. That gun is exactly what the current compulsory school system is based on (if don't pay your property taxes, you'll see the government point a gun at you eventually) and that gun is precisely what reforms must abolish if things are going to get better.</comment>


Ask HN: If you have a short time to learn and do, what will be your strategy? - <AUTHOR> I am interested to know how others learn and build something in a short time. For example, if you do know about using cryptography protocols, what will be your approach if you need to use an authentication library? What will be the balance to aim between depth and pragmatism ? <comment><USER2> Personally, I might start by reading up on some background literature but ideally I'd move into a phase of building out an initial system as soon as possible.

It's only when you really try to build something that you uncover assumptions you made and aspects of the task that you don't truly understand. From there you can read further on a topic or try watching a lecture or two on the specific issue you encountered. Otherwise you may spend hours/days/months just passively learning without gaining fundamental understanding.

Eventually, you scrap everything from your first couple of attempts and actually put together a working solution.

</comment><comment><USER3> For a well defined field, the best text book is going to balance depth and breadth. For a research subject, you will need to read scientific papers until you reach the cutting edge. For a trend in the arts, you will need to follow the trendsetters on social media until you catch up to them.</comment>


Ask HN: Will this community avoid an Eternal September? - <AUTHOR> If so, how?
Is it not a risk for this community? Why?
Is the idea too nebulous for this to be a meaningful question?
Reference: https://en.wikipedia.org/wiki/Eternal_September <comment><USER2> Regret to inform you that HN's Eternal September passed circa 2011. Before that, according to my recollection, the community was mostly YC founders and employees. Discussions were mostly about web startups, ideas, and tech. Tone was professional, comments were substantive, and people used their real names. Best of all, people were free to share their experiences without being harangued at every turn by the "citation needed" and "my science is more science than your science" cargo cults that became entrenched here at some point.

<comment><USER2> Almost forgot: Paul Graham used to comment frequently, and now he comments never.

<comment><USER3> He's too busy since he joined the resistance...

</comment></comment></comment><comment><USER4> We're something like 8 years beyond HN's Eternal September, and seem to be doing fine.

Early on, this was a place for entrepreneurs to share actionable advice about building startups and software businesses. Over time, the developer-friendly nature and hostility-free discussion attracted refugees from Slashdot and Reddit.

Over time, other people from those communities trickled across, and many of them were just average developer folk without any real interest in all this entrepreneuring. But there was the occasional Haskell story here that they could discuss, so they stuck around.

Eventually, the less civil members of those now empty communities arrived, bringing their attitude and inclination to snark and pun threads with them. You'll notice them voted down by the grown-ups, but less frequently than before.

Still, it's a generally good vibe, and nothing yet has emerged to take its place.

But it's a little late to worry about the word getting out. This is now the most popular place on the internet to discuss tech.

</comment><comment><USER5> > _" Every September, a large number of incoming freshmen would acquire access > to Usenet for the first time, taking time to become accustomed to Usenet's > standards of conduct and "netiquette". After a month or so, these new users > would either learn to comply with the networks' social norms or tire of > using the service."_

This phenomenon is unlikely to occur in a moderated forum. Articles and comments that violate HN's social norms (off-topic, uncivil, etc.) get flagged pretty quickly. Users whose behavior is persistently bad get banned.

</comment><comment><USER6> Your question assumes it hasn't happened already...

</comment><comment><USER7> I think that has already been answered by [http://www.n-gate.com/hackernews/](http://www.n-gate.com/hackernews/) rather conclusively.

<comment><USER8> I think the author doesn't read the same comments that I do. It is easy enough to have a deeply cynical view of the world, especially if you chose to ignore all things that contradict your cynicism.</comment></comment>


Stoicism 101: A Practical Guide for Entrepreneurs - <AUTHOR> http://www.fourhourworkweek.com/blog/2009/04/13/stoicism-101-a-practical-guide-for-entrepreneurs/# <comment><USER2> "Seneca, who enjoyed great wealth as the adviser of Nero, suggested that we ought to set aside a certain number of days each month to practice poverty."

What about those of us that don't have to 'practice' poverty. (well, more like we get plenty of practice as it stands?)

</comment><comment><USER3> Man, I wish a 4 hour work week was for real.</comment>


When You Should Quit a Startup Job - <AUTHOR> http://rogerhuang.co/when-you-should-quit-a-startup-job/ <comment><USER3> When I joined my first (and only) startup, I promised them I would be there for a year. At the end of that year, if I didn't feel my career wasn't headed anywhere (raises, stock options, etc) then I would head back to fintech.

Almost exactly one year, one month later I left the company with no promotions or stock options in sight, although the workload and my duties were significantly larger than we agreed on.

In addition, that year made me realize that I had absolutely no passion for the work I was doing (automotive industry) and while the startup culture looks glamorous, it's far too stressful in the end and negatively affected my emotional and physical health.

I'm back in "big corporate" fintech and happy to say I was able to bring a lot of what I learned in the startup world back with me. I've been advancing my career far faster than I would have done in the land of startups and now own a significant amount of stock in my current company.

<comment><USER4> This aligns with Reed Hoffman's "tour of duty" philosophy:

[https://hbr.org/2013/06/tours-of-duty-the-new-employer- emplo...](https://hbr.org/2013/06/tours-of-duty-the-new-employer-employee- compact)

Basically, a manager and employee agree to work on a particular project for a set period of time. At the end of that period of time, the employee is offered the option of switching to a new kind of project or leaving. If an employee leaves between tour of duties, it is considered a respectable departure and they receive the full support of the company in terms of referrals and positive reference checks.

I wish that more startups treated jobs as tours of duty. Your case highlights how having such an agreement with your employer could have been more mutually beneficial.

<comment><USER5> > I wish that more startups treated jobs as tours of duty.

Is that even feasible for startups? I guess it depends on size and stage, but I imagine most startups don't have their mission and product to a solidified- enough stage where they can think in year-long phases...or to even have more than one major project in which a year-long engineering tour is possible.

<comment><USER6> No, it's definitely not feasible for a startup. Contracts are very common in a lot of jobs, but the friction they generate don't make sense in a startup. That's why you get all that startup equity, to deal with that risk.

<comment><USER7> "[A]ll that startup equity" for normal employees is typically sub 0.1%.

</comment></comment></comment></comment></comment><comment><USER8> This is good general advice for any job. It's not healthy to work at any company where you don't like your superiors, company culture, and the work itself.

<comment><USER9> One crucial difference - in a corporate your superiors may quit, get promoted, transferred, re-orgged. If nothing else there is always skip level manager whom you can take your complaints to. In quite a few progressive corporates I have heard managers very actively manage (in a good way) favorable feedback from their reports during performance review time.

</comment></comment><comment><USER10> 4\. Their MVP doesn't even work _cough_ Is essentially vaporware

5\. Founders are focusing on raising another round instead of getting customers

[my experience at a "tech startup" in the midwest]

<comment><USER11> _" 4\. Their MVP doesn't even work"_

Well, hold on a second here. This really depends on what stage you're joining in. If you're joining a super-early, pre-product/market fit startup, this is almost to be expected. Rare is the MVP that immediately and conclusively achieves fit in the market. More often than not, there are several iterations, if not major pivots, before a startup achieves any measure of fit. The product itself has to work from a technical perspective——but it may well be janky and unreliable for the first ~6 months of its existence, and that's ok. (So long as people seem to like it, and are willing to put up with its rough edges.)

Now, if you're joining a well-funded and more ostensibly established company that doesn't seem to have fit, but is focused nevertheless on raising even _more_ money...that's a bright red flag. Stay away. Stay far away.

These are two different scenarios. There's also a lot of grey area in between them. There is such a thing as educated risk-taking (early early startup; you know it might not work) and unnecessary risk-taking (company has raised $$$ without fit, and seems hell-bent on "scaling" what it doesn't actually have). Knowing the difference makes all the difference.

<comment><USER12> The OP said "doesn't even work", which I took to mean "not viable, not _minimally_ viable". That "V" is quite an important differentiator. Sure, it's not perfect, but it should be _minimally viable_ , and that includes... "working".

<comment><USER11> I interpreted the OP's use of "MVP" to mean the founders' self-proclaimed "MVP," not necessarily an _actually_ viable product. To your point, if it were actually viable, then a "viable" product that "doesn't even work" would be a contradiction in terms.

<comment><USER13> There are many edge-cases where a product can be viable without actually working. They mostly come down to markets where the buyers are affected by principal-agent problems, and thus are willing to buy something that "doesn't work" for the principal's use-case, as long as it matches the incentives the principal has put upon them.

For example, it's very easy to sell things to defense [sub]contractors. They're not spending their own money; and, in fact, the more money they hand out to others, the more they can consume as a margin. As long as what they get in return matches the strict definition of a few check-boxes, they can't be blamed.

</comment></comment></comment></comment></comment><comment><USER14> you don't quit a startup job - the startup job quits you.

Like, literally, the way a startup is supposed to be set up is that either it takes off, or the whole vehicle crumbles around you and you are left sitting on the pavement while the remains of the startup you worked at are strewn around you. Then you get up and get in a different car, and hopefully it gets up to speed rather than disintegrate again.

Like, this is the structure all startups are supposed to have. If you don't like it, get in a real company. nobody makes you work for a startup. it's also why you get equity in case the car gets up to speed.

<comment><USER13> That's how _bootstrapped_ startups work. Bootstrapped startups have wonderfully aligned incentives that way—there's only as much cash as there is traction, and it will run out the moment you do anything stupid.

VC startups, on the other hand, can chug along "on fumes" for years, neither gaining traction nor running out of money, every once in a while doing a new investment round or merging with some other zombie startup.

</comment></comment><comment><USER15> My favorite to know when to do something binary like this is to flip a coin with choice #1 equal to heads and choice #2 equal to tails.

When the coin is in the air, you will naturally hope for one side to come up. Choose that one.

<comment><USER16> That is a huge "cliché", not to mention that we are talking about things that may have a long term impact on your life and a rational choice must be done. Going impetuously like that is sub-optimal in nature. You may have bills to pay, and hope that the startup work/culture/founders will get better, while at the same time being miserable with all aspects of it.

<comment><USER17> Getting out of dodge doesn't mean immediately upend your life. But if you're unhappy with your work, your boss, etc. it might be a good idea to _start_ looking for a new job.

</comment></comment></comment><comment><USER18> There are three ways to get out of a startup:

1) jump 2) go down with the ship 3) get pushed

There is also the mythical "startup hits the bigtime and you get rich", but the odds of that happening are so low as to be not worth talking about.

So if you join a startup you should be prepared to experience one of those exits. Jumping may be worth it if the startup is going sideways in a variety of ways, including but not limited to:

1) persistent low traction

2) anyone in the company ever says anything resembling "we must educate the customer"

3) frequent pivots (one or two is OK, after that it's probably just flailing before failing)

The bad culture and poor pay issues aren't as likely to come up if you've chosen well in the first place, and are reasons why you should leave sooner rather than later, but startups are mostly experiments aimed at answering two questions:

1) Can we build this thing?

2) Do enough people want to pay for this thing to make it worth our while?

The answer to the first question is usually--but not always--yes. The answer to the second question is almost always "no".

</comment><comment><USER19> I would be careful to note that the inverse isn't always true though. Even if none of these things attributes rings true, that doesn't mean it's not a good idea to leave. I think the decision is trickier when that's the case, actually.

</comment><comment><USER20> My issue with this advice is that things change. Things may not be great now but in 4 months time it may be different.

It's also an opportunity for you to try and make the change with culture and founders, rather than just giving up and running away.

<comment><USER21> In my experience people don't change. Two out of three items on that list are about people.

<comment><USER22> Three out of three. If you don't have passion for the work, don't expect a change of heart anytime soon.

<comment><USER13> In startups, your passions might stay the same, but the work can change radically. Have you ever joined a software startup and had it turn into a social-lending nonprofit two months later?

</comment></comment></comment></comment><comment><USER23> I get that for some "big corp" jobs could be more rewarding, and I am sure some are. However, my experiences at "big corp" jobs have been nothing but disappointing. For example, I've worked for/with incompetent managers who are good at "gaming" the system to be where they are and totally suck at managing people, incompetent colleagues who create more problems than solving any and do not seem to want to learn new things or correct their mistakes.

Startup jobs are definitely more stressful, and co-founders have enormous impact on the company, but I would always take startup jobs, where I can work with and learn from smart people (as long as there is a good team chemistry and no one is an asshole), over "big corp" jobs.

<comment><USER13> > where I can work with and learn from smart people

People always say this, but in every startup I've joined, I've ended up being the most experienced tech person on the team. Never really ended up learning anything (in my field) that I didn't just teach myself on my off-hours.

Now, I've learned quite a bit _about business_ from working for startups—with just as many "don'ts" as "do"s—but that hasn't helped me become better in the ways I was looking to become better. If I wanted to start a company, I'd be much more ready—but if I just want to code something, I'm no more ready than if I had just spent the last five years working on my own hobby projects.

<comment><USER24> Heh, same as me.

For all this talk about SV talent, and startups being filled with smart people, from experience, it's crap. I've been consulting with various SV startups recently and between crappy code and more crappy code, I've learned a lot more working for 'big bad companies' than startups

</comment></comment></comment><comment><USER25> Good list. Also, BEFORE any of these:

4\. Company starts firing people

5\. Company shuts down

<comment><USER18> Riding a company into the ground can be an interesting an educational experience, so I kind of recommend doing it at least once if you're going to stick with the startup world. It isnt' pleasant, but after the first time you'll find your attitude toward work and life will likely be quite different than they were previously.

<comment><USER13> Amusingly, every startup I've joined, I've joined effectively in the middle of it running headlong into the ground. In fact, these startups were either drastically misspending their remaining cash in the bank (and hiring me in the first place was part of that misspending), or they were hiring me because the original engineering lead had already seen the blood in the water and quit.

The annoying thing is that, because of this pattern (paycheques always evaporating, etc.), it's been really hard to get paid, so for the longest time I've had no savings, and was thus forced to stay at these jobs for the money... which they don't have. Kind of a vicious circle.

</comment></comment></comment><comment><USER26> Number 0 is: evaluate the character of the founders and the people you're working with.

I had a contract to build a website - the general idea seemed sound and I was lead it would serve as a calling card to attract more investment; it was just one building block of a larger picture. But I didn't do any background checks on the founders and the guy who recommended me to them I had known and worked with for a decade.

Fast forward 4 years later - I'm still quasi-employee #1 (and the only one, the 2 other part time contractors quit because as I learned later, they were tired of getting paid late), the company is now selling data to enterprise level customers (instead of to 'consumers' in this market) so the workload has increased... and I'm working on weekends trying to keep up with the IT needs... while constantly being paid late.

I finally wised up / burnt out when it became apparent the CEO had 'negotiated in bad faith' with a enterprise customer and they were threatening to sue unless they got exactly the data they'd paid for. Yes, I made a ton of mistakes and I hope no one else ever does what I did! I ignored all the warnings signs over the years because I got so caught up in the work due to various personal stuff. The whole company (me and 2 salesguys) quit in the space of 2 months and had to go thru months of hell trying to get what we were owed.

The guy who introduced me to them finally decided he should have told me of the fraud the CEO pulled 30 years ago, the other founder (who has a criminal record as I recently found out) told me of the CEO's fraud in his other business, one salesguy told me of the multiple identities the CEO was managing to make it seem like there were more employees... the list goes on.

Character matters. Contracts are worthless in comparison unless you have the resources (i.e. enough lawyers and money) to tangle with the other side - <USER2> bad actors know this fact, and how to twist contracts into balloon animals. Business ethics may be an oxymoron in the real world, but ... my story is a perfect example of why they should be fundamental / mandatory and the inherent flaws of capitalism.

Now the CEO has divorced his wife, who funded most of the company, and is trying to walk off with it having it valued at $350k, while a real estimate is that it's worth a few million.

</comment><comment><USER27> This seems like good advice, but I wonder how people are supposed to find companies that they like in the first place. After all, for whatever reason its looked down upon to keep switching companies, so after a while you probably will be pressured to keep your job if only to say you stayed a sufficient amount of time.

<comment><USER28> I try to seek insider information from friends who work at these companies to see if they'd urge me to join. They have credibility and relationship risk and are thus incentivized to keep things honest.

</comment></comment><comment><USER29> You should quit your job if you hit even one of those items on the list.

<comment><USER13> Well damn; I've literally _never_ had a job (big company or small) where I truly enjoyed the culture or connected with the founders. I guess I'm unemployable.

<comment><USER29> Nah, not at all. You just need to keep looking. It's certainly not easy. But I think that life is WAY too short to work anywhere you aren't entirely happy with.

</comment></comment></comment><comment><USER30> Everyone reading this thread - you will absolutely love (or at least like) a book called "Who moved my cheese?"

It's sort of a must read for anyone who ever worked any job ever.

</comment><comment><USER31> These kind of seem like general rules for leaving any job.

</comment><comment><USER32> For non startup I used to say leave asap when checked 2 of 3:

1\. No longer supports the ambiance/atmosphere/vibes of the team

2\. Salary is a problem

3\. No more passion for the work

<comment><USER33> I'm not sure what you mean by #1 - could you restate it with different wording? Your coworkers start to annoy you?

<comment><USER13> If the team is encouraging a culture of [x], but you don't support [x]. For x, substitute: workaholism, frattish sexism, elitist we-know-better-than-the- management cloistering, etc.

</comment></comment></comment><comment><USER34> OP referenced a Quora answer – is this your answer, or someone else's? Do you have a link?

<comment><USER35> Here is direct quora link: [http://www.quora.com/I-just-joined-an-early-stage- startup-as...](http://www.quora.com/I-just-joined-an-early-stage-startup-as- the-first-employee-and-I-think-I-have-made-a-huge-mistake-Should-I-take-time- to-reconsider-it)

In jest, this is what I was thinking of sharing: [http://lmgtfy.com/?q=%22You+should+get+out+of+Dodge.+I%27ve+...](http://lmgtfy.com/?q=%22You+should+get+out+of+Dodge.+I%27ve+been+through+it+myself.+%22)

<comment><USER34> Hah, that's not jest, that's helpful. I wasn't sure if this was his exact answer or a summary of some other Quora answer, so I didn't bother searching, shame on me :)

Thanks!

</comment></comment></comment><comment><USER36> multiplier: You're trading adequate compensation for some potential upside.</comment>


How Obama Got Rolled by Wall Street - <AUTHOR> http://www.newsweek.com/2010/08/29/how-obama-got-rolled-by-wall-street.html

<comment><USER2> Whole article, single page:

[http://www.newsweek.com/2010/08/29/how-obama-got-rolled- by-w...](http://www.newsweek.com/2010/08/29/how-obama-got-rolled-by-wall- street.print.html)

</comment><comment><USER3> Of course, this was all obvious at the time (check the comments at Calculated Risk at those times), but a major outlet like Newsweek can only report on it now, when all the loot's already walked.

</comment><comment><USER4> It's not just Obama - look at RBS (84% owned by UK taxpayers after their bailout):

[http://www.guardian.co.uk/business/2010/feb/25/bonus-rbs- ban...](http://www.guardian.co.uk/business/2010/feb/25/bonus-rbs-bankers)

Part of me understand the reasons for it - they need to keep the banks as going concerns so we can sell them off again. But I fear that may be establishing a precedent for bailing out the "too big to fail" banks periodically.</comment>