Ask HN: I want to learn statistics and data mining - <AUTHOR> I want to learn statistics and data mining. I can't afford to buy many books right now, so I'm asking for one or two on the topic, and also one or two available online, as ordering through amazon would take a month for shipping.
I'd like to get your opinions before spending time on random books. Thanks a lot.
P.S. My math background is not very strong but I'm willing to learn. <comment><USER2> I'll make a non-mathematical observation about data-mining. Real world data sets are messy. Humans are just not the most reliable data entry machines. Data sets are not always gathered for the purpose you intend to use them. You have to validate/scrub/realign your data set before wasting time on an analysis that could be rendered meaningless by these issues.

It's the old "Garbage In, Garbage Out" principle, but it's easy to forget until the real world hammers it into you.

</comment><comment><USER3> If your math background is weak enough that you don't feel totally at home using calculus and linear algebra, I'd recommend "Principles of Data Mining", which they use for the intro to ML class at Princeton. The ratio of English to equations is pretty high, which might help you to get into the swing of things.

If you think that you can learn enough math to follow the material, you should watch the Stanford lectures given by Andrew Ng. They're available freely online through iTunes U. Andrew is also one of the most respected experts in the field. As he points out, you will eventually need to understand the math behind the methods. Otherwise, a day will come when you start wasting time coding algorithms that plainly don't apply to the problem at hand.

</comment><comment><USER4> This is probably not what you're looking for, but while trying to figure out if you could still download a free copy of Drake's out-of-print _Fundamentals of Applied Probability Theory_...

[HINT: The answer is yes: [http://ocw.mit.edu/OcwWeb/Electrical-Engineering- and-Compute...](http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Computer- Science/6-041Spring-2005/RelatedResources/index.htm)]

...I found this potentially interesting page full of references:

[http://www.cscs.umich.edu/~crshalizi/notebooks/probability.h...](http://www.cscs.umich.edu/~crshalizi/notebooks/probability.html)

I post it here so that I can find it again later, but I fear that this is not the answer to your question. These appear to be more probability theory than nuts-and-bolts statistical analysis [1], and they may require more math background than I have (and I managed to handwave my way to a solid state physics Ph.D.)

Try browsing around the MIT OpenCourseWare site. One of the great things about online courses, even if you don't _take_ the courses, is that they often publish syllabuses that contain a lot of good textbook references.

[EDIT: Ooh, look, if you follow links from the page I mentioned earlier you get here!

[http://www.cscs.umich.edu/~crshalizi/notebooks/teaching- stat...](http://www.cscs.umich.edu/~crshalizi/notebooks/teaching- statistics.html) ]

\---

[1] Not that _I_ can reliably tell the difference. I know just enough about probability and statistics to know that I should probably study them some more. ;)

</comment><comment><USER5> If you are starting from scratch without a very strong math background, I'd recommend:

1.) Head First Statistics -- Pretty good, but beware the section on Bayes Theorem which is a bit off. This is a quick and casual intro, but worthwhile. I used it to refresh me on my college stats course (which was a long time ago), and I like it. There's also Head First Data Analysis, which I haven't read but could be a reasonable companion. HF Data Analysis uses Excel and R.

2.) Using R for Introductory Statistics (Verzani) -- Good explanations and exercises, and you will also learn R. This second point is actually pretty important, because it's a very valuable tool. Whereas the Head First Stats book walks though pretty simple problems that you work out in pencil, the Verzani book has many real-world data sets to explore that would be impractical to do by hand. That said, I think it's valuable to work things out in pencil with the first book before you move on to this one.

After these books, Elements of Statistical Learning seems to be the current favorite.

</comment><comment><USER6> Unfortunately my favorites are a bit on the mathy side, so you may want wait for other commenters with better advice. But, there are two books I really feel should not be missed. For statistics: "Statistics, Third Edition" David Freedman, Robert Pisani, Roger Purves. For machine learning: "The Elements of Statistical Learning" Trevor Hastie, Robert Tibshirani and Jerome Friedman.

<comment><USER7> fyi, The Elements of Statistical Learning can be found here as a free pdf

<http://www-stat.stanford.edu/~tibs/ElemStatLearn/>

<comment><USER8> there's also an R package with the book's material: [http://cran.r-project.org/web/packages/ElemStatLearn/index.h...](http://cran.r-project.org/web/packages/ElemStatLearn/index.html)

</comment></comment></comment><comment><USER9> Video Lectures has a ton of DM and machine learning lectures up for free.

<http://videolectures.net/Top/Computer_Science/>

</comment><comment><USER10> The Strang book is pretty much indispensable, I think.

[http://www.amazon.com/Linear-Algebra-Applications-Gilbert- St...](http://www.amazon.com/Linear-Algebra-Applications-Gilbert- Strang/dp/0030105676/ref=sr_1_1?ie=UTF8&s=books&qid=1258826887&sr=8-1)

The course lectures are online, too.

[http://ocw.mit.edu/OcwWeb/Mathematics/18-06Spring-2005/Video...](http://ocw.mit.edu/OcwWeb/Mathematics/18-06Spring-2005/VideoLectures/index.htm)

I've been taking courses related to Machine Learning, and never having taken a Linear Algebra course, I find myself going to this text and watching these lectures a lot to bring myself up to speed.

</comment><comment><USER11> Statistics is a very interesting subject, and it is a distinct subject from mathematics proper. Here (in what is becoming a FAQ post for HN) are two favorite recommendations for free Web-based resources on what statistics is as a discipline, both of which recommend good textbooks for follow-up study:

"Advice to Mathematics Teachers on Evaluating Introductory Statistics Textbooks" by Robert W. Hayden

<http://statland.org/MyPapers/MAAFIXED.PDF>

"The Introductory Statistics Course: A Ptolemaic Curriculum?" by George W. Cobb

[http://repositories.cdlib.org/cgi/viewcontent.cgi?article=10...](http://repositories.cdlib.org/cgi/viewcontent.cgi?article=1002&context=uclastat/cts/tise)

Both are excellent introductions to what statistics is as a discipline and how it is related to, but distinct from, mathematics.

A very good list of statistics textbooks appears here:

[http://web.mac.com/mrmathman/MrMathMan/New_Teacher_Resources...](http://web.mac.com/mrmathman/MrMathMan/New_Teacher_Resources.html)

</comment><comment><USER12> For DM, try <http://cs345a.stanford.edu>, just browse through the slides they are pretty good. Also, if you can find - go for some video lectures about the topic. [try MIT, Berkeley ocw sites].

</comment><comment><USER13> It's not directly related, but I just ran into an interesting article on data analysis. It includes pointers to some good texts on topic, but also has some tips which aren't covered in the standard texts.

"Three Things Statistics Textbooks Don't Tell You" by Seth Roberts: [http://sethroberts.net/about/2005_Three_Things_Statistics_Te...](http://sethroberts.net/about/2005_Three_Things_Statistics_Textbooks_Don%27t_Tell_You%20_Dec_2005.pdf)

</comment><comment><USER14> Programming Collective Intelligence also has high marks from a lot of programmers whom I respect: [http://www.amazon.com/Programming-Collective- Intelligence-Bu...](http://www.amazon.com/Programming-Collective-Intelligence- Building- Applications/dp/0596529325/ref=sr_1_1?ie=UTF8&s=books&qid=1258827924&sr=8-1)

<comment><USER15> the 2 books on amazon's Bought This Item Also Bought" blurb are more rigorous, and quite useful books, also covering topics like Lucene/SOLR with full java code listsings:

Algorithms of the Intelligent Web by H. Marmanis

Collective Intelligence in Action by Satnam Alag

</comment></comment><comment><USER16> Stanford has a Machine Learning course available in iTunes U.

Direct link to course in iTunes: [http://deimos3.apple.com/WebObjects/Core.woa/Browse/itunes.s...](http://deimos3.apple.com/WebObjects/Core.woa/Browse/itunes.stanford.edu.1615003397)

Stanford iTunes U site: <http://itunes.stanford.edu/>

</comment><comment><USER17> Please search HN for this topic. I've probably answered it 10 times here at least, and other have answered it even more.

</comment><comment><USER18> Here is another free book, which teaches statistics through R:

Introduction to Statistical Thought <http://www.math.umass.edu/~lavine/Book/book.html>

I can also recommend the Statistical Aspects of Data Mining lectures by David Mease (on Google Video & stats202.com).

</comment><comment><USER19> "Machine Learning: An Algorithmic Perspective" by Stephen Marsland is very good if you want to see the ideas in code and if you're not mathematically ready for hastie or bishop.

<http://seat.massey.ac.nz/personal/s.r.marsland/MLBook.html>

</comment><comment><USER9> I recommend starting with weka and this great book: [http://www.amazon.com/Data-Mining-Practical-Techniques- Manag...](http://www.amazon.com/Data-Mining-Practical-Techniques- Management/dp/0120884070/ref=sr_1_3?ie=UTF8&s=books&qid=1258831010&sr=8-3)

</comment><comment><USER20> This site has a ton of references to books and papers, with some of them seeming fairly introductory:

<http://www.helixpartners.com/references/>

</comment><comment><USER21> If your math background is not strong enough I'd recommend you to spend a few weeks on algebra and calculus. It'll pay you back for sure.

To find research papers use scholar.google.com

</comment><comment><USER22> Thanks for these links everyone

<comment><AUTHOR> Thanks also from me. It'll take time to digest all the information but a thread like this is very valuable to distill from hundreds of sources.

</comment></comment><comment><USER23> Early in my career, I asked that question. Eventually I got some answers.

Broadly usually the 'big bucks' are in things that are new at least in some important sense. To find such things, one recommended approach is to stand "on the shoulders of giants". Then you may still do some original work, but you have one heck of a base. My view is that Moore's law, the Internet, etc. are not nearly fully exploited, that enormous value remains, and the best way to get some of this value is to find a big unsolved problem where a good solution will be very valuable, stand on the shoulders of some giants, do some original work to get some powerful 'secret sauce' for solving the problem (which appears not to be easy to solve otherwise), start a business, please customers, get revenue, grow the business and sell it.

Here, however, just what problem, giants, original work, secret sauce are NOT easy to see. Broadly we are trying to see, no, MAKE, the future, and usually that is not easy. While we can see what applications have been made of statistics in the past, what statistics will be applied where in the future is not easy to see.

Yes, broadly statistics is highly promising. Or, in 'information technology' we want valuable, new information, are awash in cycles, bytes, bandwidth, infrastructure software, and data, and want to use these to create the information. Here, on appropriate problems, statistics can commonly, easily, totally 'blow away' anything else. The marriage between (A) 'information technology' in business and (B) statistics has yet to be consummated, even to reach to 'going steady' stage, assuming the traditional order of events.

For 100 years or so, people have come to statistics from various areas of work. Usually they had some data and some questions. Some of the areas have been educational and psychological testing, experiments and testing in agriculture and medicine, industrial quality control, model building in economics, experimental work in physics and chemistry, investing, attempts to create mathematically based sciences in the social sciences, especially economics, psychology, sociology, and political science. In sociology, old examples would be James Coleman, Pete Rossi (professors of my wife when she got her Ph.D.), Leo Goodman.

Likely the best fast, practical path into statistics is via books, courses, etc. intended for students in the social sciences. These students commonly do not have good backgrounds in mathematics. For the mathematical prerequisites, generally can get by, as a start, with just high school first year algebra. With this path can learn about probability distributions, the central limit theorem, the law of large numbers, statistical estimation and confidence intervals, hypothesis testing, cross tabulation, analysis of variance, regression analysis, principle components analysis, and more.

For more, statistics is a rock solid part of mathematics, as solid as any part of pure mathematics, e.g., topology, geometry, analysis, algebra, etc. That is, statistics is based solidly on theorems and proofs, sometimes relatively deep ones.

Statistics as theorems and proofs is called 'mathematical statistics'. Long standard has been (with TeX markup):

Alexander M.\ Mood, Franklin A.\ Graybill, and Duane C.\ Boas, {\it Introduction to the Theory of Statistics, Third Edition,\/} McGraw-Hill, New York.\ \

The main prerequisite for this book is just a not very good course in calculus, and the book actually makes not much use of calculus. Mostly all a student will need from calculus is that it can find the area under a curve. Since the book has long been standard, can't really ignore it, but it's ugly. And often, with just calculus, the book doesn't really give solid proofs of the results. E.g., their treatment of sufficient statistics has some nice intuition, but their proof is junk. The subject cries out for a good book, but I'm not trying to write one or waiting for someone else to.

Can get some of the flavor of mathematical statistics done with high quality, as mathematics, in, say (with TeX markup):

Jean-Ren\'e Barra, {\it Mathematical Basis of Statistics,\/} ISBN 0-12-079240-0, Academic Press, New York.\ \

Robert J.\ Serfling, {\it Approximation Theorems of Mathematical Statistics,\/} ISBN 0-471-02403-1, John Wiley and Sons, New York.\ \

P.\ Billingsley, {\it Convergence of Probability Measures, 2\raise0.5ex\hbox{ed},\/} ISBN: 0-471-19745-9, John Wiley, New York.\ \

R.\ S.\ Lipster and A.\ N.\ Shiryayev, {\it Statistics of Random Processes I, II,} ISBN 0-387-90226-0, Springer-Verlag, New York.\ \

However, pursued mathematically, statistics has some relatively advanced prerequisites some of which curiously are not popular in US university mathematics departments.

For the prerequisites,

High School. Should have had high school first and second year algebra (reasonable facility with algebraic manipulations, the binomial theorem, complex numbers, both of which will see again in important ways), plane geometry (where nearly all the work was proofs -- first place to learn about proofs), trigonometry (usually assumed in calculus and important in, say, analysis of organ tone harmonics and, thus, the most important example of an infinite dimensional Hilbert space), analytic geometry (especially the conic sections, especially ellipses which definitely will see again), and, if can, solid geometry (for more intuition in three dimensions).

College. Need a standard calculus course, not necessarily a very comprehensive or difficult one because will do the subject all over again, maybe two or three times, and more later, WITH the proofs!

Then need linear algebra, that is, how to work with data of several dimensions, which is just crucial. The big result is the polar decomposition, and there get to think about ellipses and get to use complex numbers. Also the course is an introduction to functional analysis and Hilbert space. Use any popular book to get started but in the end cover the classic, Halmos, "Finite Dimensional Vector Spaces'. Halmos wrote this when he was an assistant to von Neumann and intended it to be a finite dimensional introduction to Hilbert space (which once von Neumann had to explain to Hilbert) which it is. It also has some multi-linear algebra, of interest to exterior algebra now popular in relativity, but likely for nearly any business applications of statistics for the next several decades can skip that chapter.

Then need some advanced calculus. That is a poorly organized, huge, catch-all subject beyond any one course. The usual start is 'baby' Rudin, 'Principles of Mathematical Analysis'. So, that's calculus with the proofs. Warning: The book is severe, succinct, with zero pictures. Have to draw your own pictures in your head. The book is packed solidly with powerful material just awash in important applications from statistics, economics, and engineering to physics, but there is hardly a hint of the applications in the book. I enjoyed the book, but few people will enjoy it or even get through it. Hint: Get a really good teacher! Then for more, popular is Spivak, 'Calculus on Manifolds', mostly because it is short. Actually, it's too short. I prefer Fleming, 'Functions of Several Variables' until get to the exterior algebra chapter at which time, if care, can now get the thin

Henri Cartan, {\it Differential Forms,\/} ISBN 0-486-45010-4, Dover, Mineola, NY, 2006.\ \

in English.

Between Halmos, baby Rudin, and Spivak, you will have covered Harvard's Math 55 with a colorful description at

[http://www.american.com/archive/2008/march-april-magazine- co...](http://www.american.com/archive/2008/march-april-magazine-contents/why- can2019t-a-woman-be-more-like-a-man/?searchterm=Sommers)

Harvard tries to cover these three for freshman, but in most math departments the material will take you through all or nearly all of a focused undergraduate pure math major.

If somewhere take a course in abstract algebra, e.g., with a little group theory, then that might help!

Graduate School. Might learn a little more about topology, say, from Simmons, 'Introduction to Topology and Modern Analysis'. So, get good with metric spaces and get started on duality.

The next big step is a course in measure theory and functional analysis. The Simmons work will help. Baby Rudin will be crucial; Halmos is recommended. So, with measure theory, do calculus over again and in a very different and much more powerful way and a way just crucial, even central, for mathematical approaches to statistics. The functional analysis will concentrate on representation theorems, the Radon-Nikodym theorem, and Hilbert and Banach spaces. Long popular, from Stanford and sometimes aimed at statistics students, is Royden, 'Real Analysis'. It's gorgeous. Should also read the real half of Rudin, 'Real and Complex Analysis'; it's a few steps up in difficulty from baby Rudin. Again, hint: Get a good course from a good teacher who can get you over the material without getting stuck. Then go back and study the material 2-3 more times, apply it, do some original research in it, and finally begin to understand it.

We're talking high, top, center crown jewels of civilization here; the stuff is of just awesome power; my view is that it is, for the rest of this century, one of the main pillars of increases in economic productivity via the exploitation of Moore's law; on 'what to program', computer science is stuck and this material is the most promising way forward; as a famous restaurant owner once said about some Morey St. Denis, "you won't find better".

Now are ready for probability. I recommend:

Leo Breiman, {\it Probability,\/} ISBN 0-89871-296-3, SIAM, Philadelphia.\ \

M.\ Lo\\`eve, {\it Probability Theory, I and II, 4th Edition,\/} Springer- Verlag, New York.\ \

Kai Lai Chung, {\it A Course in Probability Theory, Second Edition,\/} ISBN 0-12-174650-X, Academic Press, New York.\ \

Jacques Neveu, {\it Mathematical Foundations of the Calculus of Probability,\/} Holden-Day, San Francisco.\ \

Neveu is succinct, gorgeous, but not easy. This material is NOT popular in US departments of mathematics. At Princeton, see Cinlar.

Then should make some progress with stochastic processes: The big book is Gihman and and Skorohod, right, in three volumes, but mostly people settle for shorter treatments. Whatever, should learn about Poisson processes, Markov processes (discrete time, finite state space is enough to get started), Brownian motion, and martingales. Might also learn about second order stationary processes. A good course in stochastic processes is NOT easy to find, especially in mathematics departments.

Now are ready to attack statistics mathematically! I don't know of a good, single 'mathematical statistics' book at this level. Instead, there are many books -- I gave some above -- and then the journals. Thankfully, the field is relatively close to applications; so can take a practical problem and concentrate on what is relevant to it. One of my papers was some new work, at this level, in mathematical statistics for a problem in practical computing and computer science. The computer science community struggled terribly with the mathematics. So, it was some progress in computer science that community will have to struggle to understand.

One approach to work in computing is just to try things, that is, just to throw things against the wall and see if they appear to stick. Or, maybe the truth of the situation really is a simple statistical model. Likely that model will fit the data well. So, try many simple statistical models. We use these models mostly ignoring the mathematical assumptions; mostly we are proceeding 'heuristically', that is, with guessing. If any of the models fit well, then they can be considered candidates for the truth. So, are throwing things against the wall to see if they fit. This approach also called 'data mining'.

Problems:

(1) Will be quite limited in what statistical models can use. That is, will be drawing from a cookbook instead of being a real chef who can create good, new dishes appropriate for the available ingredients and customers!

(2) Don't have much, e.g., have not proceeded mathematically where from the deductive logic of assumptions and proofs actually know in advance some good things about the results. Something like breaking into a pharmacy, mixing up a lot of pills, taking them, and seeing if feel better! Uh, I'll pass and let you do that without me!

(3) May have gone through a lot of computer time in an 'exponential, combinatorial explosion' of efforts throwing against the wall.

(4) Have ignored a LOT in statistics that can add to what know about the results.

(5) Will be tempted to conclude have have found 'causality' but will likely not have.

(6) Will be tempted to conclude that have a model that predicts, but that is on shaky ground and risky and needs more work.

Applied to important problems, this approach can be dangerous.

There are not many healthy statistics departments. Much of the career interest is in biostatistics, especially related to FDA rules.

It appears that among the top statistics departments are Berkeley and UNC. Since Breiman and Brillinger are at Berkeley and since Stanford, long good in statistics, is not far away, if I were looking for a Ph.D. in statistics then I'd pick Berkeley.

There is a general problem getting a 'job' in a technical field and likely also with statistics. The assumption in US business is still as in factories 150 years ago: The supervisor knows more than the subordinate; the subordinate is supposed just to add common labor to what the supervisor says. In particular job descriptions are written by the supervisors, not the subordinates!

Well, there are nearly no supervisors in US business who have even as much as a weak little hollow hint of a tiny clue about the material described here. So, won't need that material to qualify for the job descriptions. Moreover, if actually know such material and let that fact leak out, then will likely not make it past the first level HR English major phone screen person who will tremble and conclude that you are not like the employees they have! If you do get hired and someone in your management chain discovers that you have used some mathematics they don't understand, you might be on the way out the door, especially is your work was valuable for the company!

Of course, the solution is to find a valuable application and start your own business. While maybe biomedical venture capital can understand crucial, core technical content, in information technology venture capital, likely you will be trying to explain this stuff to, say, history majors who worked in fund raising, marketing, general management, or financial analysis or have a background in just relatively elementary parts of computing. Just will NOT find more than six people, maybe not more than zero people, in US venture capital who can work the exercises in Royden or explain the strong law of large numbers. Sorry 'bout that! So, if you explain that the value of your venture is the powerful material in your 'secret sauce', then you will be regarded as a kook, far outside the mainstream of venture funded entrepreneurs, discarded, maybe even laughed at. As it is, some of the venture people are making money now, and the rest just want to be more like the ones who are making money. Looking for anything really new, powerful, and valuable is just NOT in the picture.

So, once you have some results in users, customers, revenue, etc., then maybe you can get some venture funding; just why at that point, owning 100% of the business, you would take venture funding, a Board that can fire you, etc. is less clear! Or venture funding is not for everyone! Or venture firms prefer to give money to people who don't need it!

For the real power of the 'secret sauce', you have just to keep that a secret!

Once mathematicians have yachts, at the venture firms math will be to info tech like biochem is to biotech. In the meanwhile note that a valuable application of statistics can put you on the Forbes 400 where there are not many people! Generally if you are making a valuable application of advanced or new statistics, then you will not know many people who understand what you are doing. Or, if lots of people understood it, then it wouldn't be valuable!

<comment><USER24> _statistics is a rock solid part of mathematics, as solid as any part of pure mathematics_

The impression left by reading Jaynes (The Logic of Science) was that a huge part of conventional statistics was a hodge-podge of ad hoc methods. The One True Way out of the mess being, of course, Bayesian statistics. What's your take? Do most of the books you advocate follow Bayes?

Also, thanks for the great post. I'd love to know where these statistics have taken _you_.

<comment><USER23> The materials I listed never or nearly never mention 'Bayesian statistics', 'subjective probabilities', or 'prior probabilities'.

For what has been done in statistics over the past 100 years or so, each research library has a large section of books and journals. Here my interest was to respond to the question about how to get started in statistics and to outline a future for statistics, especially for exploiting Moore's law for more in economic productivity.

For what I have done in statistics, my interests are in business, and there a good application mostly means starting a new business. I'm doing that, but I'm not supposed to describe the 'secret sauce' in public.

I can give an introduction that might be of some interest in computer science and practical computing.

Given a 9th grade math teacher and one class with 20 boys and another class with 18 girls, do the boys and girls do the same or is there a difference? Uh, maybe as in the URL I gave, some 'feminists' will be very picky about any claims of a difference!

Broadly what we do is make a 'hypothesis' that gives us enough in mathematical assumptions to do some probability calculations. This hypothesis is called the 'null' hypothesis apparently because we intend to reject it, that is, find it to be 'null'. Our intention is to conclude that the hypothesis leads to something of very low probability, so low we reject the hypothesis. Then we know something that appears to be false. Yes, this is less good and complete knowledge than we could want, but maybe this result is good considering how little in data and assumptions we used!

So, put all 20 + 18 scores in a pot, stir the pot, pull out 18 scores and average, average the remaining 20, take the difference in the averages, do this maybe 1 million times (thanking Moore's law), get the empirical distribution of the differences, pick a small number, say, 1%, for the really angry feminists, 0.1%, pick the region in the 'tails' with this fraction of the differences, get the difference for the real data before stirring, and see where it is. If that real difference is in the region, then have some bad news for the angry ones: Either boys and girls are not the same or they are the same and we have obsereved something rare, too rare to be believed. Amazing that anyone would suspect that by the 9th grade boys and girls were not the same!

Now, make some mathematics out of this!

When the hypothesis is true, we reject it, and make an error, 1% (or 0.1% or whatever) of the time. So, we have rejected the null hypothesis when it is true; this is called Type I error. Yup, the other possible error is to accept the null hypothesis when it is false, and that is Type II error. Semi-, pseudo-, quasi-amazing.

Hmm .... Here we have an introduction to distribution-free, that is, 'non- parametric', hypothesis testing based on ranks or permutations.

The stirring of the pot is called 're-sampling'. Actually, when we do the mathematics, we will likely want all the combinations of 38 things taken 18 at a time, and that might be 33,578,000,610. So, instead of straining Moore's law with all 33 billion, we just 'sample', in this case, 're-sample'.

So, we see that we get to select the probability of Type I error, that is, the 1%, in advance and get what we select. Progress.

Now suppose the class of 20 boys also takes English, general science, and history. Similarly for the class of 18 girls. So, now on each student we have 4 scores instead of just 1. How how to do the test? Hmm ...!

Or, suppose we are given a server farm and a network. We select a 'system' we want to monitor in real-time for health and wellness. Suppose that this system can report data on each of 12 relevant variables 100 times a second.

Our null hypothesis is that the system is healthy. Then, an instance of Type I error is a 'false alarm'. Suppose we want the false alarm rate to be, say, 1 a month.

Then an instance of Type II error is a missed detection of a real problem.

Then for tolerating that rate of false alarms, we want the lowest rate of missed detections we can get.

So, how do we construct our monitoring system?

We would like to use the classic Neyman-Pearson result. Here, however, we are asked for complete information on when our system is 'sick', and likely we don't have that.

Still we can select our rate of false alarms and do something smart with the 12 variables on problems never seen before, i.e., 'zero-day' problems.

So, we have obtained some automation of system monitoring with adjustable, known false alarm rate and, if we look a little, with some nice guarantees on detection rate. Progress in 'computer science'!

<comment><USER24> First, thanks for the long answer.

 _So, put all 20 + 18 scores in a pot, stir the pot, pull out 18 scores and average, average the remaining 20, take the difference in the averages, do this maybe 1 million times (thanking Moore's law)_

I can't believe there isn't a well-defined analytical expression for that... What you described sounds like the kind of inference I would tackle through Bayesian hypothesis testing, while you use... Monte Carlo?

I suspect I'm missing something. Anyway, it sounds like a very interesting problem and if I were around, I'd ask for an interview. Good luck.

<comment><USER23> "I can't believe there isn't a well-defined analytical expression for that..."

Well, yes, there is some solid mathematics behind the pot stirring!

Writing out some appropriate mathematics, will likely want all 33 billion combinations of 38 things taken 18 at a time.

With this math, there is no use of 'prior probabilities', and the Monte Carlo is just a fast way to replace finding all 33 billion combinations.

For more, can see (with TeX markup):

E.\ L.\ Lehmann, {\it Nonparametrics: Statistical Methods Based on Ranks,\/} ISBN 0-8162-4994-6, Holden-Day, San Francisco, 1975.\ \

Jaroslav H\'ajek and Zbyn\v ek \v Sid\'ak, {\it Theory of Rank Tests,\/} Academia, Prague, 1967.\ \

Sidney Siegel, {\it Nonparametric Statistics for the Behavioral Sciences,\/} McGraw-Hill, New York, 1956.\ \

So, it's old material. There are many such hypothesis tests.

But the old material essentially always has, for the student case, only one number on each student. The part about what to do when each student has 4 scores can take us into the journals and maybe start some more research. Similarly for the 12 numbers from the 'system' to be monitored.

<comment><USER24> _With this math, there is no use of 'prior probabilities'_

But the full hypothesis is "Given the data, are girls better than boys at this exam?" and clearly, the prior probability is relevant. Maybe in this case one might want to use a 50-50% prior, but in general, if the hypothesis was instead "Given the [same] data, can we conclude that this paranormal event really happened?" then a healthy skeptical prior would be in order.

Anyway, regardless of the "prior" issue, I've thought some more about your original problem, and I'm not so sure about your methodology. From my perspective, if you want to reach a "girls better than boys on this test in this class - true or false" conclusion, then individual variance is a crucial issue. Assuming that all girls and boys would _always_ get the exact same result were they to take the same test over and over again, then you have a variance of 0. And thus, one could simply check if (average of boys) < (average of girls) and conclude accordingly. At the other extreme, if students show huge individual variance (eg.: their score depends on whether they had breakfast that morning), then the test results are almost meaningless... So the outcome is crucially dependent on this variance, which your problem description makes no mention of, and which Monte Carlo methods really do nothing to recover. One would have to make assumptions about it.

Maybe a better example (closer to what your other "system health" problem) would be: a boy takes an exam 20 times, a girl takes the exam 20 times, they get such and such results (assuming they don't improve inbetween). Is the girl better than the boy? Then one could assume a Gaussian distribution of test results for both, estimate their average and variance, then check for overlap between the 2 gaussians and conclude accordingly.

Maybe your MC really does boil down to something similar, but I don't see it. And sadly, I can't quite construct an argument about what I think is problematic with it. It just doesn't feel right.

Thanks for the references, I might check them eventually, though they seem too specialized for my needs. Someone recently posted this book on HN, and people had good things to say about it:

The Elements of Statistical Learning - Data Mining, Inference, and Prediction

<http://www-stat.stanford.edu/~tibs/ElemStatLearn//>

It's next on my reading list, but was absent from yours. It's available as a PDF --- the determining factor for someone with no library access.

<comment><USER23> "But the full hypothesis is 'Given the data, are girls better than boys at this exam?' and clearly, the prior probability is relevant."

No, prior probabilities have nothing to do with it.

We state our 'null' hypothesis that the boys and girls do equally well. This hypothesis has nothing to do with a belief of prior probabilities or belief of any probabilities at all. Instead, we state this hypothesis as something that will give us some mathematical assumptions to do some calculations to reject it and, then, conclude that it was false.

Generally in hypothesis testing we don't believe the null hypothesis as prior probabilities; indeed, likely we don't believe it at all and are stating it to reject it and conclude it is false.

In more detail, we assume that 20 boys and 18 girls are 38 independent samples from some one distribution. It turns out, we don't need to say anything about that distribution because we are being 'distribution-free'. In particular, we get to ignore the Gaussian distribution. GOOD.

Independent? Okay: Suppose we DO give you the true distribution of the data and the first 37 scores. Now you get to guess score 38. Do the 37 scores help you beyond just the distribution? No. Same for any subset of the scores. Then, we have independence.

With this null hypothesis, the average of the scores of the 20 boys and the average of the scores of the 18 girls should be 'close'. How close? Well, under the null hypothesis and with the values we observed, we have a way to proceed: The distribution of the difference in the scores, with everything we do know given and fixed, we can find. For this distribution, basically we look at all the 33 billion or so differences obtained by taking all combinations of 38 things taken 18 at a time. Justification? If work at it mathematically, then under the null hypothesis can show that each of those 33 billion cases was equally probable.

Then we pick a small number, say, 1% for the size of our Type I error, that is, the probability of rejecting the null hypothesis when it is true.

Then we find the differences in the 1% tail of the 33 billion differences.

Then we look at the difference from our actual data. That difference will be one of the 33 billion. We see if that difference is in the 1% tail.

If the difference is in the 1% tail, then one of two things is true:

(A) The null hypothesis is true, the boys and girls are the same, that is, independent samples from the same distribution, and with our actual data the difference is relatively large, out in a tail, and we have observed something that happens only 1% of the time.

(B) The null hypothesis is false, that is, in some way the boys and girls are different. That is, we still believe the independence assumption, so what is false is just that the mean for the boys is different from the mean for the girls.

If the 1% is so small we don't believe (A), then we conclude (B).

Variance has nothing to do with it.

Welcome to distribution-free 'two sample' hypothesis testing 101.

<comment><USER24> I've been reading Jaynes again this week, and he's just very, very convincing. And so I'm trying to read everything you wrote through these Bayesian glasses, but sadly, I'm not successful. Jaynes is rather critical of Fisher's hypothesis testing, on the ground that you can't accept or reject an hypothesis on its own; you need an alternative to compare it to, and that alternative needs to make definite predictions. I don't see what the alternative to your null hypothesis is (the negation of the null hypothesis does not make definite predictions)</comment></comment></comment></comment></comment></comment></comment></comment>


Linux on Nintendo 64 - <AUTHOR> https://speakerdeck.com/murachue/nintendou64 <comment><AUTHOR> I tried translating the slides, but my Japanese is very bad.

[https://gist.github.com/huntiep/91d272e45fb4c87ed1e2bfc6e8d1...](https://gist.github.com/huntiep/91d272e45fb4c87ed1e2bfc6e8d1ddf8)

</comment><comment><USER2> I'd LOVE to see a video of this demo running linux.</comment>


Arduino.js - <AUTHOR> http://www.schillmania.com/projects/arduino-js/

<comment><USER2> Interesting demo, but I feel a bit duped since it's not a pure JS implementation. I expected something written to work with NodeJS, but instead I found a browser-based JS library that talks to an instance of Flash on the page.

Still, kudos to the author for coming up with this experiment.

<comment><USER3> That's kind of this guy's shtick - he's the mind behind the esteemed soundManager2 library (<http://www.schillmania.com/projects/soundmanager2/>) that makes managing audio in the browser easier with js+flash.

<comment><USER4> As well as a pretty sick CSS3 turntable hack he has demoed a few times. [http://www.flickr.com/photos/52201706@N04/5573063518/in/phot...](http://www.flickr.com/photos/52201706@N04/5573063518/in/photostream)

<comment><USER5> Thanks. I'm working on an article relating to that at the moment. Should be fun once it's out there (and on Github, etc.)

<comment><USER3> That's way cool! Is that rigged up to change the tempo for beatmatching?

<comment><USER5> That's in there also, yep. The prototype is aiming to recreate most of the usual tricks. It gets a bit hackish. Release TBD, hopefully soon. A teaser: <http://www.youtube.com/watch?v=6NiMMSyk9GU>

<comment><USER3> Jeru the Damaja!

</comment></comment></comment></comment></comment></comment></comment><comment><USER5> The next step would be to move to websockets on the client (and also node, yes), and eliminate the Flash dependency altogether.

I personally prefer to work in HTML/CSS/JS entirely, but there are some things that Flash does nicely - and it can talk to JS (and can be fast), so it makes for a way to extend functionality in some cases.</comment>


Transfinite Cardinal Arithmetic with Wolfram Alpha - <AUTHOR> http://blog.wolframalpha.com/2010/09/10/transfinite-cardinal-arithmetic-with-wolframalpha/ <comment><USER2> If my definition of infinity is the set of all integers, why isn't this true -

(2 * infinity) / (3 * infinity) = 2/3

Even if the definition is something else, it's the same set, and it should get canceled out - or don't even use it in like that, pretend it's a variable (x) - same effect.

> For Cantor, it simply must be accepted that the inner and outer circles have > the same number of points—our intuition that the outer circle has more > points, according to Cantor, is just wrong.

We know that the outer circle circumference is larger than the inner circle circumference. We can even find the ratio very easily.

How do you transition from that - to counting infinitely small dots on those circles and saying they have the same number of those dots? I don't see the connection from that exercise to the mathematical rules.

Normal logic dictates that there should be more real numbers between 1 and 10 than there are between 1 and 2. Because 1-10 contains all the numbers that are in 1-2, and then some!

But that's not true!?

<comment><USER3> > If my definition of infinity is the set of all integers, ...

The cardinality of the set of integers is aleph-0. Aleph-0 is one of many transfinite cardinals that measure the size of "infinite" sets.

> (2 * infinity) / (3 * infinity) = 2/3

This can be done with limits:

lim{x -> ∞} (2x)/(3x) = 2/3.

<comment><USER4> To anyone studing calculus:

lim{x -> ∞} (2x)/(3x) != (2 * infinity) / (3 * infinity)

In fact, the infinities we are speaking of are two completely different objects. The limit as x-> ∞ is explicitly defined seperatly from the limit as x->(Real Number). In fact this definition makes no further reference to ∞, and instead says that for any arbitrarily small, positive d, there is a real number m such that x>m implies (2x)/(3x) is within m of 2/3\. The infinity we are talking about when we say (2 * infinity) / (3 * infinity) = 2/3, (I assume) is the size of the set of integers, which bears no relation to the symbol ∞ used in limits.

Ultimately, the important thing to remember in situations like this is that if something has not been defined as true, or proven to be true, you cannot assume it. Many similarities (such as overloading operators) are chosen for convenience, not because they are the same.

The problem is that most of the time these similarities were made because there really is a strong parrallel, and you look like an idiot for wasting a math class to prove that 0x=0. (In my case, those two 0s weren't even the same object: the left one was a real number, the right one was a vector)

</comment></comment></comment><comment><USER5> On the other hand, Alpha does not seem to have the facilities currently for dealing with transfinite ordinal arithmetic, which is arguably much richer and more interesting (there's not much you can do with cardinals without CH).

</comment><comment><USER4> Regarding the ending point on Godel. If we have proven that it is impossible to disprove the continuum hypothesis, then would we be safe in assuming it to be true? If such an assumption were to lead to contradictions, we would have a proof by contradiction that the hypothesis is false. Of course, the same argument could be made for assuming the hypothesis is false. Therefore, doesn't this demonstrate that their are 2 constructions of math we can use: one where it is true, and the other where it is false. And that both of these constructions are internally consistent (assuming that the math with neither assumption is itself consistent). At this point it just becomes a question of which version is more useful/interesting.

</comment><comment><USER6> For a more intuitive explanation and history of transfinite cardinals and a cognitive analysis of their development, see writings by Rafael Nunez, including these two papers: [http://www.cogsci.ucsd.edu/~nunez/web/SingaporeF.pdf](http://www.cogsci.ucsd.edu/~nunez/web/SingaporeF.pdf) [http://www.cogsci.ucsd.edu/~nunez/web/TransfinitePrgmtcs.pdf](http://www.cogsci.ucsd.edu/~nunez/web/TransfinitePrgmtcs.pdf)

</comment><comment><USER7> "Pursuing that challenge, Kurt Gödel showed that the known axioms of mathematics are insufficient to disprove the hypothesis."

Is this related to Godel's Incompleteness Theorem? Are there any non- mathematician's explanations as to how Godel proved this?

<comment><USER8> There's no relation to his incompleteness theorems. Gödel proved that every universe V of sets that satisfies the (ZFC) axioms has a subuniverse L that satisfies both the axioms and the continuum hypothesis (CH). L is constructed by transfinite recursion. At each stage of the construction, the only new sets you admit are the definable subsets of the set of all sets you admitted in earlier stages.

[http://en.wikipedia.org/wiki/Godel%27s_constructible_univers...](http://en.wikipedia.org/wiki/Godel%27s_constructible_universe)

Unfortunately, proving that L satisfies ZFC and CH is a very delicate and technical matter. On the other hand, it can be fun because it's extremely meta. One of the steps of the argument is proving that L is the L of L. I learned the proof from a graduate textbook chapter titled "defining definability."

</comment></comment><comment><USER9> My first thought was -- what's the execution time on the algorithms?

Yeah, yeah -- I know it's symbolic. But still ... :D</comment>


Snapchat's 3D Bitmoji – Augmented-Reality Animations - <AUTHOR> https://www.nytimes.com/2018/06/14/movies/snapchat-3d-bitmoji-animation-augmented-reality.html <comment><USER2> Yeah, AR is going to find it's first killer application in the next year or two. But I doubt it will come from an established company like Snapchat unless there's some network effect that an AR-application needs to be successful.

<comment><USER3> >Yeah, AR is going to find it's first killer application in the next year or two.

My money is on something like Nexar [0]. AR enhanced driving is going to save commercial fleet operators incredible amounts of money while we're still reliant on human drivers. Real level 3+ autonomy is still years away, but AR HUDs are going to be huge very soon, and the mapping data they collect will build the basis for those autonomous systems. MapBox just released their new Vision SDK that does deep learning based AR locally on mobile devices that is going to have some amazing applications too [1].

[0] [https://www.getnexar.com](https://www.getnexar.com)

[1] [https://blog.mapbox.com/vision- sdk-d47486656821](https://blog.mapbox.com/vision-sdk-d47486656821)

</comment></comment><comment><USER4> Immediately closed the article after clicking the link. What on earth is NYTimes thinking embedding half a dozen auto-play videos with sound in this piece?

</comment><comment><USER5> Is there any library for detecting clear area and tracking movement like in Snapchat app?

<comment><USER6> Apple's ARKit</comment></comment>


Teachmate - new web educational project - <AUTHOR> http://teachmate.org What teachmate is all about?
TeachMate.org is a service that helps people who want to learn and people who want to teach find each other. It is a dating site, except that the aim here is education.
The idea behind this service is simple: whoever teaches — learns. There are few simple things we wish our user could find out:

    * You don't need to be a professional to teach. Instead, you have to teach to become a professional.
    * You don't need to pay money for learning or ask for money when you teach someone.
    * Learning is not about the degrees, it's about the process and what you can do with your knowledge.
 Have fun! <comment></comment>


Results from my Reddit advertising attempt - <AUTHOR> http://blog.ezliu.com/blog/reddit-advertising/ <comment><USER2> Also I don't think it's necessarily a fail.

You got 6 invitations to your service for a $150 spend. So that's a $25 cost per invite. What's the total lifetime value of a landlord to you? If it's more than $25, that's a win right? You have to discuss these results using math in order to say win/fail.

Also you're right that spending more and playing with the headlines day to day is a good idea. (I hear @patio11 in my head shouting "A/B testing!") The $25 acquisition cost could drop to $10 or $5 with the right message wording and trying different subreddits.

Lastly, I always shudder to hear about $100 FB ad campaigns people run and declare them to be a waste of time. As someone else said, it is a bit like golf. You have to play it a few times to get a sense of if you're good at it. The first time out is not a sufficient test.

<comment><AUTHOR> Agree. I don't actually consider it a fail. I generally treat forays like this as exploratory research, and like many other situations, you have to pay for information.

6 invitations != 6 conversions, but yes, I'd agree that 25/conversion would be pretty reasonable. I haven't done recent metrics on what my customer LTV is, but its at least in the ballpark of 25.

I entirely agree that this needs some split testing -- I didn't explore the reddit advertising tools enough to see if there's an easy way to split test, but it didn't jump out at me.

</comment></comment><comment><USER3> I think your big problem was the headline. If I'm on Reddit and I see that headline, I have no idea why you want to talk to my landlord and have no idea how much time it's going to take for me to connect you with them.

<comment> brown9-2 Exactly this. If you want me to give you someone else's contact info you better make the value proposition to me crystal clear.

The headline as written sounds somewhat spammy and confusing, and not many users are going to click thru to learn more and alleviate their confusion.

</comment></comment><comment><USER4> Was your campaign asking users to do something for you? Whats in it for them?

<comment><USER5> This is probably the most important part of a headline. Answering the question: What's in it for me? People don't care what you need or want, they only care about their needs and wants.

<comment><USER6> Underrated thread.

I read the landing page waiting for a "you get a $25 Starbucks card for each landlord signed up" or something, but there was nothing. Just "help me grow my business (that won't benefit you in the near term) for free please?"

</comment></comment></comment><comment><USER7> You spent $150 to learn the most valuable copywriting lesson to be learned. That the headline is the most important part of an advertisement. The job of the headline is to get people to keep reading. Your headline proved to be bad. Its ok, now go back, redo it and test the results.

Advertising on reddit is a bit of a puzzle. The community does not respond to very informercial sounding messages. They do respond to personal, first person point of view stuff.

You should try a variation of the following.

 _Reddit: Help me get in touch with your landlords. I want to talk to them about something that might benefit you._

 _Help! I might have a way to help your landlord (and thus, you) but I have no way of getting in touch with him/her._

 _Guys, this is my startup. It aims to help your landlord do some things better, and thus make your life easier. Help me spread the word._

And so on.

</comment><comment><USER8> A lot of people don't have great relationships with their landlords. In most cases it seems to be somewhere between neutral and active dislike.

In other words people contact them as little as possible unless there is something really wrong.

</comment><comment><USER9> One minor note: If you're using Google Analytics for your web analytics data, your time on site metric is somewhat meaningless. Bounces get counted as time on site = 0. That means that 37 seconds metric is weighted down by 89% of visits having a time on site of 0. Your non-bounces time on site number is probably more like 5:37.

<comment><AUTHOR> Thanks. I wasn't aware of this.

I generally just look at changes in my google analytics numbers rather than absolute values to see if I'm doing better or worse.

Admittedly not the most sophisticated GA approach.

<comment><USER9> That's probably the right way to think about GA data in general; the only reason I highlighted it is because the bounce rate so heavily skewed the metric that it made it almost meaningless.

</comment></comment></comment><comment><USER10> We've had nothing but great results from Reddit advertising. We average 0.18 CTR and 2.6 pages per visit with <70% bounce. The actual ad makes a huge difference, you really need to connect with the Reddit audience. We tried a couple of different ads before finding one that worked well for us.

<comment><USER11> Can you elaborate?

<comment><USER10> Redditors generally click on things that look more like content then an ad and stand out enough to catch their eye. If you try to just sell them something, they tune it out. But solve a problem for them in a relatable way and they will click. We paired an awkward penguin meme with a problem/solution statement to get our best results.

<comment><USER12> > We paired an awkward penguin meme...

Is this the post-internet equivalent of when your favorite bands from highschool and college start selling their popular songs for car and financial service ads for the middle aged? Of course today's redditors all think it will never happen to their generation.

</comment></comment></comment></comment><comment><USER13> Sorry this didn't work out for you. I make an effort to actually look at the ads on Reddit because I think those are some of the smarter advertisers. Ignoring that you actively advertised on r/realestate, here are some issues you probably ran into on the main page push (please correct me if I overlooked any of these):

Taking into account the median age range of the average Redditor, do they even have landlords or do they live in dorms or with their parents? Then if they do have a landlord, what's their relationship like?

Even as someone in my mid-20's, my relationship with my landlord isn't buddy- buddy because I have no intention of staying at this location and can't guarantee that she'll have a property in an area I look into in the future. We get along fine and she wants to be a connection on LinkedIn, but that doesn't make the scenario you're proposing anything more than an awkward, incentive- less sales pitch about a product I know nothing about. Additionally, another reason I went with my particularly property manager (and possibly why others have too) was because she is already using Appfolio (so everything is done online), which I'm going to assume has a somewhat similar feature to your product.

You also don't cite any trial time or discounts to property managers who want to compare and contrast before taking the plunge, which would at least give me an easier way to make the pitch. The sample is expecting me to fill out the form to see the rest of the experience, if there's even anything else there, whereas as a landlord I would be just as interested (possibly even moreso) in the admin interface and my customization options.

I'm also less likely to click on an ad with some random person's face. I don't know what it is, maybe the Jimmy Wales (et al) Wikipedia fundraiser effect, but I just ignore it because it's weird.

Lastly, does suggesting this service to her imply that I am unhappy with the one she currently uses or - worse - that I'm making a commission off of this sale unbeknownst to her? I just don't want that kind of baggage tied to the person who determines whether or not there's a roof over my head.

</comment><comment><USER14> I recently screwed up with Reddit advertising, but I think the mistake was the decision to do it, not how I did it - note that I'm talking about my particular case, not saying Reddit advertising is bad in general.

Promoting an event in Paris I targeted /r/Paris and /r/France, and achieved underwhelming $60 CPM and $10 CPC rates.

I sort of expected it and didn't spend too much money to mind, only really tried it because at the time I was pretty desperate to try any option possible (side note, a week later and all my marketing issues with this event are solved, hooray).

Really my issue was lack of targeting options, the only way I could limit the advert to relevant people was through subreddits, and as they were relatively small ones, it was never going to achieve great results. Hopefully some time in the future they'll offer better options, for example I would have loved to target French (or better Paris+10miles) readers of big subreddits like /r/gaming.

(I didn't actually bother with any AB testing for this as it didn't seem worth it, but hey, we got a pretty awesome CTR on the tiny number of impressions!)

edit: This is the only time I've ever used Reddit advertising, is it normal that total uniques aren't shown? I can see unique impressions and total impressions for each hour, but there's no total uniques over the entire campaign, which would annoy me if I'd spent more money and wanted to track roi.

<comment><USER15> Really interesting stuff. Since you were trying to market an event did you try using Facebook ads ? I read a post on HN a few weeks ago about how Facebook ads for a garage sale event were super successful!

Would love to know if you tried them and if you have any positive/negative experiences to share. Can never learn enough about marketing effectively!

Also, congrats on managing to market your event =)

<comment><USER14> I did, yes, and was pleased to get basically the results (in terms of CPM and CPC) that the Facebook platform had me expert before starting the campaign.

Again I didn't spend much (~$500) but got roughly $0.08 CPC (a little higher on CPM bidding) and overall I enjoyed both the flexibility of the platform in terms of demographic targeting, and the stats breakdown which let me compare how different adverts performed within the campaign.

(I don't have figures to hand for conversion from clicks to event registrations, they weren't fantastic, but that's largely because our landing page is much more targeted towards a specific audience who will go there not from adverts, and it's not great at converting people who clicked an advert and don't know much about what they are looking at.)

As a bit of context: I have no experience with advertising on Facebook, Reddit or similar platforms. I have a lot of experience in other digital advertising (on the selling end rather than buying - though not working in sales), but it's a very different game to social network advertising..

</comment></comment></comment><comment><USER11> Im not sure why a redditor would bother helping you out, you need to appeal to them with something, karma, fame, an iPad giveaway or something.

</comment><comment><USER16> Thanks for sharing your results. Advertising effectively is enormously difficult.

I've had results similar to yours in all areas of advertising. Someone asked me once, "So how has the ad we run on our blog for you been doing?" My reply, "Well, I think it's the least poorly performing ad we run, so maybe that's a win?"

<comment><USER17> Yes, me too. I've used google adwords quite a few times, and I always end up spending more than I make back from it. The problem is that it just costs too much to grab each new customer that it usually isn't worthwhile. Facebook is the only one that is actually useful, but it only works for certain types of campaigns.

<comment><USER18> It's all about having the right campaigns and optimizing for conversion. It isn't easy, but it is certainly doable for most people. Adwords is like golf - it's a simple game to play but the being good requires a mastery of nuance and fine details that takes lots of experience.

</comment></comment></comment><comment><USER19> It's all about the headline. I think it has to sound authentic and not like an advertisement at all. More like the start of a discussion.

And don't use memes. I've noticed that the advertisers that screwed up the meme, even it's a slight mistake, get roasted for it.

</comment><comment><USER20> Need better A/B testing of titles. Not a lot of data either.

Also as others have pointed out already, your incentives (get landlord leads) are not aligned with your viewers (uhm... feel good helping another Redditor?).

</comment><comment><USER21> I think your biggest problem was, you weren't offering anything to the user. Helping you is not an incentive for me or most other people. What can you do for me? You weren't doing anything for me.

</comment><comment><USER22> I loved 500px's terms and conditions page, but it's designed as a terms and conditions page. It's still enormously wordy. Additionally, all I saw in the ad or landing page were pleas for help - I was never sold the benefits.

I'm no copywriter/designer, but prehaps try rerunning the ad and A/B test some designs/headlines that sell the benefits ("no more real estate agents" or w/e the benefit is to redditors).

</comment><comment><USER23> I like where I live and I don't want my landlord to be thinking about how easy it is to replace me.

</comment><comment><USER24> Still better than Fb ads right now.

</comment><comment><USER25> Great post Ezl. Thanks for sharing

</comment><comment><USER5> I think your ad was a win, simply because you're learning by doing. This is very valuable.

</comment><comment><USER26> There's nothing in it for the viewer of this ad.

</comment><comment><USER27> Almost nobody knows who their landlords are -- almost all rentals are handled through agencies.

Plus: who wants to remind their landlords to come and do an inspection?

<comment><USER28> You and I live in very different markets. I have always known who my landlord was, and have asked my landlord to perform inspections.

Even if you only rent from property management companies, they are still the target demographic for the advertised SaaS.</comment></comment>


Ask HN: External Keyboard for MacBook Pro - <AUTHOR> The 2nd & 3rd generation Apple butterfly keyboards have a lot of problems [1], and I am one of the users that has to live with a broken keyboard.
Can anyone recommend a Bluetooth keyboard that can be put over the butterfly keyboard (13" MBP in my case)? Preferably mechanical, and access to the trackpad should not be blocked.
I use my own Ergodox for my desktop, but they are not portable enough. Something that can fit above the original keyboard will minimize the footprint.
[1] https://m.signalvnoise.com/the-macbook-keyboard-fiasco-is-surely-worse-than-apple-thinks/ <comment><USER2> Personally I use a bluetooth Happy Hacking Keyboard. I don't put it over the built-in keyboard though, I use a stand ([https://www.therooststand.com](https://www.therooststand.com)). This is (probably?) silly but it was just announced: [https://ergodox- ez.com/pages/planck](https://ergodox-ez.com/pages/planck)

</comment><comment><USER3> What's wrong with Apple's own Magic Keyboard?

<comment><AUTHOR> (a) At the price point ($100) I might consider a decent mechanical keyboard (b) it is humiliating to pay another $100 to Apple for their engineering problem</comment></comment>


How to never forget one's laptop power cord again - <AUTHOR> My laptop (HP EliteBook 8540p) has about 1.5 hours of battery life at best. So I have to plug it in all the time. (For work reasons, I need to use it at its highest performance, most power-hungry setting.)
In my work, I have to demonstrate lots of things on my laptop at places outside my office.
Once in a while, I'll leave my power cord behind at the previous place. Leads to all kinds of problems for me. (I'd rather not carry a spare cord because those things are heavy!)
So I googled "never forget laptop power cord". Sadly nothing came up --- a rare time the internet failed me :-(
Hacker sisters and brothers --- what are your homegrown solutions to avoid forgetting the digital umbilical cord at some inconvenient place?
Update: I now realize I should probably have posted this in the "Ask HN" section -- sorry about that, I'm new to this folks, please excuse. <comment><USER2> Buy a second power cord and keep it in your bag?

<comment><AUTHOR> That is a viable option, thanks for the reminder. I'd prefer not having to lug around the extra weight though :-)

<comment><USER3> Can you clarify? I do this. I have one cord at home and one in my backpack. This way I always have a cord with me if I go out. If you have to lug your power cord with you anyway, isn't that the same as you always carrying the same amount of weight? I don't really understand (unless you bring out your bag without taking your laptop with you in certain circumstances).

<comment><USER4> It sounds like the OP is thinking the GGP suggested carrying around two power cords so that if the OP forgets one, then they at least aren't stuck without a power cord (although they likely would want to retrieve the old cord at some point.)

</comment></comment></comment></comment><comment><USER5> Get one of these [http://www.amazon.com/Wireless-Anti-Lost-Tracker-Security- Re...](http://www.amazon.com/Wireless-Anti-Lost-Tracker-Security- Reminder/dp/B00A6092TY/ref=pd_cp_ba_2)

<comment><AUTHOR> Reasonable solution -- there are also some that can be connected to one's smartphone.

</comment></comment><comment><USER6> Put a little tag on the zipper of your laptop bag that says "Don't forget the power cord". Somewhere you can't avoid seeing it.

...or superglue/tape/tie the adapter to your laptop.

<comment><AUTHOR> Both are good ideas -- thanks!

</comment></comment><comment><USER7> do what i do, whereever I am. Be paranoid and look around and say "what have I left here, because I dont want to come back to get it."

<comment><AUTHOR> If only I could remember to be paranoid enough and paranoid always!

</comment></comment><comment><USER8> The problem approaches a solution asymptotically: Once I bought my third tablet charger, it was, in practice, impossible to leave behind.

<comment><AUTHOR> LOL -- I feel ya bud.

</comment></comment><comment><USER9> Get a Stick-N-Find and put it on your power cord. If you walk away from it your phone will remind you.

<comment><AUTHOR> Thanks for the suggestion -- I'm going to consider it. Here's the problem, I usually silence or turn-off my phone at these meetings, and so if I'm walking away, I may not hear the notifications. I usually only remember to un-silence or turn-on my phone much later.

</comment></comment><comment><USER10> Use fishing line or something to tether your cord to the bag or the laptop.

<comment><AUTHOR> This! (as old school as it may sound -- this sounds like the most doable -- I'm definitely going to do this).</comment></comment>


PirateBay is down - <AUTHOR> https://thepiratebay.sx/

<comment><USER2> If anyone is interested, I'm making an alternative to The Pirate Bay that doesn't depend on the BitTorrent protocol: [http://www.infinitecopies.com](http://www.infinitecopies.com).

email@example.com for feedback.</comment>


Any big surface becomes a touchscreen with Visualplanet's "touchfoil" - <AUTHOR> http://venturebeat.com/2011/09/06/any-big-surface-becomes-a-touchscreen-with-visualplanets-touchfoil/ <comment><USER2> <http://www.pranavmistry.com/projects/sixthsense/>

Very similar project</comment>


European parliament may propose search services split in threat to Google - <AUTHOR> http://www.reuters.com/article/2014/11/21/us-google-antitrust-idUSKCN0J525V20141121 <comment><USER2> Before people start suggesting this is some anti-American agenda from the EU, keep in mind there are many American companies that have been complaining to the EU such as Yelp and Expedia. They want to stop Google's tying practices such as using their monopoly in search (another thing to keep in mind is that their market share in the EU is something absurd like over 90%) to benefit other products and services. It's not really calling for breaking up the company. Also this is specifically about raising the temperature on the European Commission. Parliament itself doesn't do this.

This is a signal from politicians to the European Commission that they can go hard.

<comment><USER3> Ben Edelman, somewhat notorious for being paid by Microsoft, has done a really good excellent job at documenting problems with Google for many years. Irregardless of who paid for the time invested, these problems have been well researched and aren't simply American companies playing cry baby. I'm not a regulator, and am not suggesting a regulatory answer, but I think Peter Thiel has given a fairly honest explanation that Google's rationalizations are largely PR-fud.

There are very profound and far reaching issues here. Imagine if instead of Wikipedia we had some encyclopedia where half of page 1 was sponsored editorial and the rest of what made it on page 1 was done in a totally opaque environment by a company who had commercial interests on what was showing up. If your company operates in a fairly narrow market, has limited retention, and needs to use search traffic vs demographic traffic to build your audience, you are effectively operating day to day with a giant guillotine over your neck.

(Very happy to see DuckDuckGo get some meaningful traction, and for Google's sake with the regulators the better DuckDuckGo does the more room there is to let off some pressure.)

</comment></comment><comment><USER4> Google search competitors are not even close with the quality of Google search results. This year I started trying Bing, Yahoo and right now DuckDuckGo as an experiment and none of them has the same quality. Bing is the only one that came closer. That's why Google owns search. They may be using shady tactics, but the competitors still need to get better.

<comment><USER3> The last time someone told me this, I asked them when the last time they ran with adblocking was -- answer, years.

<comment><USER4> I do. I also use privacy badger and private/incognito browsing. Interpreting keywords is way ahead of the rest. One thing that's getting worst is geo targeting though. They are trying way to hard to show you local results for everything you search.

</comment></comment></comment><comment><USER5> Rivals should maybe ask themselves why Google has such an enormous market share in search.

Maybe a 90% market share originates simply from better quality?

<comment><USER6> Google is the only search engine that got their L10n right.

When I search non-local stuff in English I have the choice between at least Google, Yahoo!, Bing and DDG, which all give decent, comparable results. When I search for German content or local stuff there is really only Google.

German Google search results are terrible compared to what English speaking people are used to. For German queries it is the only option.

I guess for other languages spoken in Europe it's similar.

<comment><USER7> Fascinating. Is this likely to be because of a smaller corpus (seems unlikely - there must be millions if not billions of pages in most European languages)

Or is it that search algorithms don't cross languages easily - or that the core developers and testers use English as a lingua Franca anyway?

Still a fascinating point - can you put some kind of example or test up?

<comment><USER5> Reliable, reproducible testings are hard because every engine customizes results for every user, based on history...

</comment></comment></comment></comment><comment><USER8> Following this premise, I guess it's time to assess massive trade tariffs on all vehicles manufactured by German companies, either domestically, or imported. Their dominance in the luxury category is really just too much, it needs to be significantly regulated or reduced.

For Mercedes-Benz, BMW, and Porsche the US is their largest market. Between 20% and 25% of all sales typically come from the US for those automakers.

Jobs will be lost? Nah, auto supply will shift to domestics and or Asian manufacturers. The Germans will simply lose a big fat market, and it's unlikely all of Europe will agree to strike back against US automakers.

Economic war is the only thing that is likely to come out of the EU trying to split Google.

</comment><comment><AUTHOR> Scale has its advantage, products like Google Now or a Home Robot/Assistant is not possible unless there are a ton of data points available to be processed. I thinking breaking up is not a solution, but rather content(headlines included) and data about customer should be traded between companies using APIs. Like a data exchange, just like exchanges for electricity and other utilities. Imagine a time in future when you want a fancy Apple robot, but that robot is dumb unless it has access to all of your current and past data.

<comment><USER9> I think you are making two opposite statements: _" should be traded between companies using APIs"_ gives me he impression you want to give the companies control over your^W their data. At the moment there are laws prohibiting the former, IMO rightfully so.

 _" when you want a fancy Apple robot, but that robot is dumb unless it has access to all of your current and past data."_ makes it look as if you want humans to have control over who has access to their data. I think that should be the norm, and if laws are retired to get there, I think we should get them.

<comment><AUTHOR> ok, yep let me clear that. What I mean is

-Companies should be able to own and trade their users data with other companies. This should be like a stock or ad exchange. More pull requests for certain data, higher the data connection price.

-Also blogs should be able to charge $0 or some amount for search bots to crawl them. Its demand and supply if. NYT might be able to ask for some price while Buzzfeed can say .. eh we'll set that to free.

-Now for user, Example If I pay $200 for a robot - The company can say, hey it is $10 to use it monthly(because the robot company figures that is avg cost to fetch data, lets say from Goog FB AMZN etc). But some companies can say, the data of their users is owned by them and their user so if the user want to use his/her data then its free. But if the company wants to use to for some other means without user consent then its up for sale.

Its kind of like twitter's firehose, just more universal. Will encourage companies to develop APIs since now they can charge its use. I'd love a google now/robot to suggest me what netflix flick I should see based on my Bing search history.

</comment></comment></comment><comment><USER10> I am thinking of a tutorial on search result pages as a settlement.

</comment><comment><USER11> This is political posturing and grandstanding on steroids, whoever bought the EU parliament (evidently Germany publishers) did a good job.

They can't "break up" Google, it's an American company! it's not based there.

Also the previous competition chief ridiculed such notions when he was recently asked about it by Parliament citing the utility companies (German ones) that would be first in line for a breakup debate if that was a tool they wanted to use:

 _The decision to reopen settlement talks followed vigorous criticism from a widening range of politicians, including the economy ministers of France and Germany. The latter, Sigmar Gabriel, argued in May that a forced breakup of Google should be seriously considered because of its vast market power. Werner Langen, a European lawmaker representing German Chancellor Angela Merkel 's CDU party, echoed that suggestion, drawing parallels with U.S. efforts to break up monopolies in oil and other industries. "If we don't give them a bash we're not going to solve the problem," he said.

Mr. Almunia showed little sympathy for such demands. "I would tell you one thing, as a German friend," he said. "The day I [hear] that the railways will accept unbundling, electricity companies will accept unbundling, and we will discuss [unbundling] with telecom operators and others…let's discuss unbundling Google, but not before._

via [http://online.wsj.com/articles/google-must-improve-search- se...](http://online.wsj.com/articles/google-must-improve-search-settlement- or-face-charges-eus-almunia-says-1411462097)

The fact that they are drafting a motion targeting one company is so shamelessly political it's almost a public display of corruption.

Not to mention that the ludicrous levels of attention and weigh this issue is getting in the EU is unheard of and unwarranted:

 _One technology industry source with knowledge of the motion also called it a "politically-motivated campaign to do something that is a regulatory matter". He added: "These guys are calling for the break-up of Google. That is not in proportion to the degree of concern articulated by the commission during its investigation._

<comment><USER12> Google is not just an American company. As a big "fuck you" to Google: the majority of revenue runs through Ireland and the Netherlands. Nice tax plan, but it means they can't easily evade the European Commission.

I actually really applaud this step. Don't get me wrong, this should not be considered anti-American, but if you run a global company getting revenue from a lot of countries you should be prepared to follow their laws and pay taxes. It is that way for physical goods, and it's pure stupid arrogance to think that delivering online products exempts you from all of it.

<comment><USER8> Can you break out exactly how much revenue goes through Ireland and the Netherlands?

<comment><USER12> Everything that does not originate in the US is run through the tax construction they have setup going through a po-box firm in Amsterdam and Ireland. The same construction is used by many others, including Facebook, Apple and even U2.

Not sure whether they specify this number in their quarterly reports, but given that it's "the world" vs the US revenue it is definitely a majority.

Edit: 17 billion out of 40 billion (both in euros and 2013 figures) global revenue runs through Google Ireland Ltd alone and that's reported as ad sales from Europe only. I did not find a report for the Netherlands, but there is more revenue from outside Europe taking this route as well since it's the cheapest route for anything that is not coming from the US.

<comment><USER8> It looks like it's very unlikely that Europe is the majority of Google's revenue, and I doubt _all_ revenue globally outside the US runs through Ireland & Co (can you back that up with data?).

For the latest quarter: "Our revenues from outside of the United States totaled $9.55 billion, representing 58% of total revenues in the third quarter of 2014, compared to 58% in the second quarter of 2014 and 56% in the third quarter of 2013."

Unless the rest of the world outside of Europe only has ~8%, and or all global revenue actually does go through Ireland & Co (again, I'm skeptical it's 100%). That's possible, but it's seems unlikely.

<comment><USER12> There is a simple reason all revenue outside the US goes through Ireland and the Netherlands: [http://en.wikipedia.org/wiki/Double_Irish_arrangement](http://en.wikipedia.org/wiki/Double_Irish_arrangement)

From the Wikipedia page: "The offshore company continues to receive all of the profits from exploitation of the rights outside the US, but without paying US tax on the profits unless and until they are remitted to the US.[5]"

So yes: all of it goes through Europe.

</comment></comment></comment></comment></comment></comment><comment><USER13> Perhaps they should be forced to release keywords versus clicked links data to other companies like Duck Duck Go, just like Microsoft was forced to document things like SMB for Samba and the various Office formats.

</comment><comment><USER14> Google != AT&T

</comment><comment><USER15> Search is a public utility, just like the Internet itself.

<comment><USER16> How could search be considered a public utility? You can use any search engine you'd like. If you don't like Google, switch to Bing or something else. That's not the case with things like water or electricity (usually). Internet service as a utility? Maybe (in the US at least with limited options). But search?

Don't blame Google because they are really good at what they do. They aren't stopping you from using another service, nor are they stopping others from creating their own search engine.

<comment><USER17> for vast majority of people, what's the difference between 'the internet' and 'google homepage'? you can theoretically use something else than google, true, but if you bundle all search engines together and disable every single one of them, the internet for a great majority of the population that has access to it becomes completely useless.

<comment><USER16> For the vast majority of people Goolge doesn't control their web browser. For some, yes, but it's only a default for Android or Chromebooks.

Browsers can and do change their defaults whenever they'd like: see Firefox this past week. That's an issue for the browsers, not the search engine. I'm not aware of any browser (sans Chrome) where the default homepage is Google.com.</comment></comment></comment></comment>


Chicago police detain Americans at abuse-laden 'black site' - <AUTHOR> http://www.theguardian.com/us-news/2015/feb/24/chicago-police-detain-americans-black-site?CMP=share_btn_fb <comment><USER2> Among the alleged police practices at Homan Square, according to those familiar with the facility who spoke out to the Guardian after its investigation into Chicago police abuse:

\- Keeping arrestees out of official booking databases.

\- Beating by police, resulting in head wounds.

\- Shackling for prolonged periods.

\- Denying attorneys access to the "secure" facility.

\- Holding people without legal counsel for between 12 and 24 hours, including people as young as 15.

\- At least one man was found unresponsive in a Homan Square "interview room" and later pronounced dead.

<comment><USER3> We should torture in Quaker meeting halls: A modest proposal

We don't keep records. We speak when moved by a greater power. No one knows who "owns" our spaces. Very few attorneys in presence. Holding people for 12-24 hours is common. I think someone died during meeting last week and no one noticed.

</comment></comment><comment><USER2> Summary:

Located in a nondescript warehouse on Chicago's west side is where, according to the Guardian, one can find the domestic equivalent of a CIA "black site" \- an illegal, off-the-books interrogation compound used by Chicago special police units, one which renders "Americans unable to be found by family or attorneys while locked inside"; a place whose former occupants say is where you end up when you are "disappeared"; a place which confirms that when it comes to the eternal "who is better - us or them" debate, there really is no difference: "It brings to mind the interrogation facilities they use in the Middle East. The CIA calls them black sites. It's a domestic black site. When you go in, no one knows what's happened to you." It's a Guantánamo and Abu Ghraib rolled into one. In short: it is a place where the US constitution and basic human rights have absolutely no access.

According to an exclusive piece by the Guardian that is sure to send not only shivers down the spine of those who are still paying attention, but ripples across the "land of the free", not least because if there is one dark site on US soil, there are countless more - places where every single constitutional right of US citizens is trampled on - the secretive warehouse known as Homan Square is the latest example of Chicago police practices that echo the much- criticized detention abuses of the US war on terrorism. However, there is one huge difference: while those abuses impacted people overseas, Homan Square – said to house military-style vehicles, interrogation cells and even a cage – trains its focus on Americans, most often poor, black and brown.

Every former communist block country had them: hidden, dark places where the secret police could have their way with you, and even kill you if it so desired, and nobody would have any clue or recourse of action; something for which the "evil empire" was mocked by the "free western world."

And while the ubiquitous "terrorism" excuse for any and every extra- constitutional action could apply here as well, the reality is that Homan Square is hardly concerned exclusively with terrorism. Several special units operate outside of it, including the anti-gang and anti-drug forces. If police "want money, guns, drugs", or information on the flow of any of them onto Chicago's streets, "they bring them there and use it as a place of interrogation off the books."

<comment> guard-of-terra In fact communist states had it arguably better in the latter XX century - they didn't have war on dissidents with weapons and all, so it was usually enough to scare a dissenter using words - e.g. hint on losing a job or scholarship.

<comment><USER3> Jeśli mój przyjaciel we wrocławiu powiedział coś złego o Ameryce jego firma będzie upewnić się, że policja go uwolnić. Ponieważ jego firma jest amerykańska firma.

<comment> guard-of-terra How would he land in police in the first place? I understand most of the words but not the precise meaning :)

</comment></comment></comment></comment><comment><USER4> If this is so well known by the lawyers, how is it that nobody is taking the police to court? Are the lawyers afraid of the police?

<comment><USER5> Chicago is among the more notoriously extra-legal government zones in the US. I'd call it a municipality, but that doesn't do justice to the machine that is Chicago. It's essentially run by a powerful, unaccountable government- criminal-corporate organization structure, and has been for a century or more.

<comment><USER6> Look, Chicago has corruption problems, but calling it an "extra-legal government zone" is so hopelessly hyperbolic that it hurts any sort of real argument. The police have had literally thousands of lawsuits paying out hundreds of millions of dollars over just the last 5 years. Police officers, a police chief, mayors and even governors have been arrested and jailed over everything from bribery to brutality. If lawyers aren't suing on what looks like a slam-dunk case of civil rights violations, why have they sued (and won) so many other times?

</comment></comment></comment><comment><USER7> I wonder what the apologists will say, there certainly is no lack of imagination when it comes to defending torture and murder when done by the government.

<comment><USER5> They'll say what they always say: I have nothing to hide, I've done nothing wrong, why should I worry personally, I'm sure they're just targeting criminals.

They'll cower like they almost always cower, shrinking from a confrontation they're deathly afraid of: the one in which they're ultimately responsible for ensuring their own liberty, with the requirement being they have to stand-up to a vast and powerful authority. I'm not speaking of modern Americans, but of all people at nearly all points of history, whether Soviet Russia or modern Chicago.</comment></comment>


Twitter's Missing Manual (2016) - <AUTHOR> https://eev.ee/blog/2016/02/20/twitters-missing-manual/ <comment><USER2> > Replying to a tweet will also prefill the @handle of anyone mentioned in the > tweet. Replying to a retweet will additionally prefill the @handle of the > person who retweeted it. The original author's @handle always appears first. > In some cases, it's polite to remove some of these; you only need the > original author's @handle to make a reply. (It's not uncommon to accumulate > multiple mentions, then end up in an extended conversation with only one > other person, while constantly notifying several third parties. Or you may > want to remove the @handle of a locked account that retweeted a public > account, to protect their privacy.)

I wish they would fix this already. Twitter users are constantly embarrassing themselves by @mentioning the retweeter in their reply.

</comment><comment><USER3> Previous discussion:

[https://news.ycombinator.com/item?id=11171643](https://news.ycombinator.com/item?id=11171643) (71 comments)

</comment><comment><USER3> (2016)

<comment><USER4> Added. Thanks!</comment></comment>


Hacking away on travel - <AUTHOR> http://www.banjarey.com/what_travel_concern

<comment><USER2> Is this similar to Hoppr???

<comment><AUTHOR> I actually don't know what Hoppr is..I guess nobody does until they launch ! As far as I know they are converting big travel data into relevant data, and so are we planning to..though our approach is different (as far as I know)..</comment></comment>


One Dead Media (2008) - <AUTHOR> http://kk.org/thetechnium/one-dead-media/ <comment><USER2> I clicked through to the "Dead Media Project" looking for other kinds of dead media.

The dead media list is, appropriately, a 404: [https://www.deadmedia.org/mailman/listinfo/deadmedia](https://www.deadmedia.org/mailman/listinfo/deadmedia)

<comment><USER3> Wayback Machine has it.

</comment></comment><comment><USER4> Somewhat amusingly, the first comment under the article is from someone still using this exact form of technology right now, or at least was back in 2012.

So I guess this form of media can also be taken off the '100% dead, not available anywhere on Earth' list.

<comment><USER5> Found another example: Make, vol 3, p. 187 describes a way to make these (not form-factor compatible, but uses the same concept) with a hole punch, index cards, scissors, and a chopstick.

Edit: the url kk.org is not one that inspires a whole lot of confidence, either.

<comment><USER6> Or, perhaps, a two-letter domain means that _K_ evin _K_ elly, founding editor of Wired Magazine, founder of one of the first online communities called WELL, who's been working in tech longer than most of the people on this site have been alive, might know what he's talking about.</comment></comment></comment>