f7d9395c_atings_and_reviews___Gwern_net__Review
[Title] Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics, and Society [Author] Jim Manzi [Rating] ★★★ [Year] 2012 [Read] 2013/01/25 [Review] 
Speaking as a die-hard believer in the value of randomization and meta-analysis, I’m not entirely sure how much I got from this book other than some useful assertions and interesting claims. To go through it roughly in order: 1. the first few chapters are a serviceable philosophy of science primer. We get discussion of how the Scientific Revolution was a break, we get some Popper and falsificationism, and as a very important correction, some Duhem-Quine; we get some cultural and vocation material, and discussion of the value of prediction even in non-experimental sciences. Chapter 6 is a bit of a waste as Manzi makes the standard criticism of frequentism that the basic ideal - probability as the limit of some particular set of events - is extremely imprecise since it gives absolutely no guide as to which set of events, since you can choose arbitrarily complicated events, but he does so on his own terms and without any reference to competing paradigms such as the many flavors of Bayesianism. 2. then we get into genuinely important material on the development of RCTs (he uses the term “RFT”, which I find silly). Personally, I could wish for a whole book on the gradual development and refinement, and many more examples of the superiority of RCTs to other approaches; his summaries of things like the US social program experiments is short and in many ways inferior to, say, Rossi’s “metallic laws”. ch9’s example analyses read like columns folded into the chapter, but I still enjoyed them as fun examples of critical thinking and how analyses can reach any goal. 3. with ch10 we finally get into Manzi’s own career. This section is… weirdly lacking in many examples, even though it’s the section where you would expect Manzi to just lay the smack down with countless scores of anecdotes and stories and statistics about how experimentation is the ne plus ultra of epistemology and exactly how much money it made all these corporations. There’s a few, like Capital One and some Internet companies, but it is not very thematically separate from point #2. 4. he issues some policy recommendations. Hard to argue with some of them: why not let in some immigrants as part of randomized experiments? It can hardly be worse than the current system. Fairly anodyne. So overall good, and maybe great for people who aren’t familiar with the topics. But I think in general if I wanted a layman to appreciate statistics and its pitfalls and experiments, I might actually be better off giving them Nate Silver’s The Signal and the Noise (although they’re not totally comparable, of course). People seem impressed by his Hayekian libertarian arguments using genetic algorithms as arguments. I don’t think they’re as well supported as they may seem. Just to name the obvious, society has changed massively over short time-spans, corporate mortality is astonishingly high, corporations do not replicate with high fidelity (“corporate culture” is fragile) etc; evolution, as it happens, can only filter out so many mutations per generation so past a certain point, a genome just decays. This, along with other considerations, strongly suggests that corporate ‘evolution’ is nothing but a poetic metaphor.
Ratings and reviews - Gwern.net
Review