The Mathematical Hacker (2012)
The Mathematical Hacker – Evan Miller


By Evan Miller

November 16, 2012


(Corrections)


The programming profession is blessed with a number of gifted essayists. Today I will pick on three of my favorites — Eric Raymond, Paul Graham, and Steve Yegge — because they all seem to disagree about why (and whether) mathematics is relevant to the practicing programmer. Their attitudes might be summarized as:


    Eric Raymond: Mathematics is unnecessary except in specialized fields such as 3D graphics or scientific computing.

    Paul Graham: Mathematics is a sort of Zen garden from which to draw inspiration.

    Steve Yegge: Mathematics represents the foundational knowledge of computer science, and one can find a lot of interesting math articles on Wikipedia.


All of these views, I think, are myopic.

In a limited sense, each view is correct. If you are a systems and network programmer like Raymond, you can do your job just fine without anything more than multiplication and an occasional modulus. Graham is correct that mathematics can be a rich source of metaphors, and for his part Yegge is right that computation can be reduced to a mathematical formalism (a fact that Lisp hackers love to reiterate with frequency).

They seem to agree on one thing: from a workaday perspective, math is essentially useless. Lisp programmers (we are told) should be thankful that mathematics was used to work out the Lambda Calculus, but today mathematics is more a form of personal enlightenment than a tool for getting anything done.

This view is mistaken. It has prevailed because it is possible to be a productive and well-compensated programmer — even a first-rate hacker — without any knowledge of science or math. But I think that most programmers who are serious about what they do should know calculus (the real kind), linear algebra, and statistics. The reason has nothing to do with programming per se — compilers, data structures, and all that — but rather the role of programming in the economy.

One way to read the history of business in the twentieth century is a series of transformations whereby industries that “didn’t need math” suddenly found themselves critically depending on it. Statistical quality control reinvented manufacturing; agricultural economics transformed farming; the analysis of variance revolutionized the chemical and pharmaceutical industries; linear programming changed the face of supply-chain management and logistics; and the Black-Scholes equation created a market out of nothing. More recently, “Moneyball” techniques have taken over sports management. There are many other examples.

What is the relationship of these innovations to computer programming? Consider Eric Raymond’s definition of the word hack, the great desideratum and raison d'etre of any self-respecting hacker: An incredibly good, and perhaps very time-consuming, piece of work that produces exactly what is needed. In that sense, applied mathematicians have been “hacking” industry for over a hundred years.

The computer was invented, after all, to solve math problems, not to implement compilers or word processors. The computer-aided solutions to society’s problems — in engineering, mining, farming, agriculture, transportation, and defense — have provided a tremendous amount of value to society over the past seventy years. I would hazard a guess in the tens of trillions of dollars, and if you count the defeat of Germany among the achievements of computer-aided cryptanalysis1, and the surrender of Japan among the accomplishments of numerical computation, then the value has been immeasurable.

Yet these sorts of mathematical “hacks” are rarely mentioned by our great hacker essayists. The reason, I think, is that “hacker literature” tends to be dominated by Lisp programmers, and Lisp programmers tend to be ignorant of applied mathematics. While I am grateful that we have so many well-written essays by Lisp programmers, I think they tend to paint a skewed picture of what computer programming is and should be. One gets the impression reading Raymond, Graham, and Yegge (all self-styled Lisp hackers) that the ultimate goal of programming is to make a program that is more powerful than whatever program preceded it, usually by adding layers of abstraction. Call this the “Lisp school of programming.”

There is another school which has not been very well represented in the literature over the years, but which has undoubtedly produced a greater positive impact on the economy. Call it the “Fortran school of programming,” which I think is well summarized by Dr. Adam Rosenberg, the self-described last buffalo of industrial mathematics. Rather than viewing mathematics as an advanced tool reserved for extremely specialized computer applications, Fortran-school programmers view the computer as an advanced tool for doing mathematics. Historically, Fortran-school programmers have tended to work in industry or in the more technically inclined parts of the government (NASA, Defense, etc.). They are often mocked by Lisp programmers for their ignorance of recursion. (For a satirical essay, see Real Programmers Don’t Use PASCAL; if you think this satire is unfair, see Dr. Rosenberg’s Style Guide.)

The mockery, I think, ought to run with at least as much force in the opposite direction. In contrast to the Fortran tradition (which proudly “sent a man to the moon” and implemented critical infrastructure in banking, communications, and so on), the culture of Lisp is almost willfully ignorant of mathematics. This ignorance is disguised by all the talk of formalism and the instinctive genuflection before the Lambda Calculus, which, not unlike the Summa Theologica, is a closed computational universe that sheds little light on the observed world.

To appreciate how ignorance of mathematics is actively perpetuated by Lisp culture, consider the traditional introduction to recursion in any functional-programming text. It comes in a couple of variants: calculate a Fibonacci number, or calculate a factorial.

The usual textbook solution is recursive: one calculates the number of interest first by calculating the number before it, and the one before that, and so on, until one reaches the “base case” with a defined solution. It is the classic didactic programming example that demonstrates the supposed utility of recursive methods.

“Advanced” discussions might consider engineering considerations such as tail-call behavior, or the possibility of memoizing results.

But rarely in these discussions will you find relevant mathematical considerations. If the goal is to compute a Fibonacci number or a factorial, the proper solution is not a recursive function, but rather knowledge of mathematics.

A Fibonacci number calculator may be implement in C as:


long int fib(unsigned long int n) {
   return lround((pow(0.5 + 0.5 * sqrt(5.0), n) - 
                  pow(0.5 - 0.5 * sqrt(5.0), n)) / 
                 sqrt(5.0));
}


No recursion (or looping) is required because an analytic solution has been available since the 17th century. Similarly, if it is ever necessary to compute a factorial, a programmer should be taught to take advantage of the system’s log-gamma function, as the following C code does:


long int fac(unsigned long int n) {
    return lround(exp(lgamma(n+1)));
}


Again, no recursion is required as long as one knows that a factorial is actually a special case of the gamma function. (The implementation of log-gamma is usually a polynomial approximation which requires constant time to evaluate.)

Despite the aesthetic virtues ascribed to functional programming, I find the preceding solutions to be more beautiful than their recursive counterparts. They run in constant (rather than linear) time, and they are easily adapted to work with non-integer inputs. More importantly, they encourage the programmer to ask and investigate the questions: why does the Fibonacci equation involve the square root of five? What does it mean to take a factorial of a fraction? These questions have fascinating answers which are not easily admitted by the recurrence relation alone.

After coding his recursive solution, the Lisp hacker is more likely to ask the irrelevant question: how can I reduce these two functions down to one function?

The trouble with the Lisp-hacker tradition is that it is overly focused on the problem of programming — compilers, abstraction, editors, and so forth — rather than the problems outside the programmer’s cubicle. I conjecture that the Lisp-school essayists — Raymond, Graham, and Yegge — have not “needed mathematics” because they spend their time worrying about how to make code more abstract. This kind of thinking may lead to compact, powerful code bases, but in the language of economics, there is an opportunity cost. If you’re thinking about your code, you’re not thinking about the world outside, and the equations that might best describe it.

Although the early years in the twenty-first century seem to be favoring the Lisp-school philosophy, I predict the balance of the century will belong to the Fortran-school programmers who are able to successfully apply mathematics to practical problems. It is tempting to declare that most programming problems “don’t need math”, but this is only true in the same sense that manufacturing, or supply-chain management, or baseball, “doesn’t need math”: advanced mathematics seems completely unnecessary to existing practitioners, but only until someone figures out that a particular mathematical concept is the right way to think about the problem at hand. After that, it is vital.

There are two reasons I am optimistic about the future of mathematics in computer programming. The first has to do with the growth in the amount of data generated by web companies (“Big Data”). With more types of data at hand, there are more equations that might be applied with utility. There is a lot of interest in advanced machine-learning techniques for this reason, but even simple statistical techniques might have prove to have at least as many applications. Mathematics applied to business data will be yield better business insights, more efficient operations, better products (e.g. recommendations), and new products (e.g. prediction services).

The second reason I am optimistic about the place of mathematics in computer programming is related: the average consumer has more data than ever before, and mathematics can help to make sense of it, or at least make it more beautiful. Application areas that were traditionally considered to be “scientific computing” (for example, Geographic Information Systems or image-processing) are now of interest to regular people who own (say) a collection of geotagged digital photographs. Instagram, for example, was built on a few equations that operated on an image’s color channels. An understanding of mathematics can help the programmer solve practical problems for users as well as provide a more pleasing experience. (To this end, you might enjoy my previous essay, Winkel Tripel Warping Trouble.)

Mathematics, in the end, does not help you understand computer programming. It is not about finding metaphors, or understanding “fundamentals” that will never be applied. Rather, mathematics is a tool for understanding phenomena in the world: the motion of the planets, the patterns in data, the perception of color, or any of a myriad things in the world that might be understood better by manipulating equations. It is the hacker’s job to figure out how to encode the insight into a piece of code that will be used over and over.

Should we return to the good old days when men programmed Fortran and everything was an array? Hardly. What we need is an infusion of applied mathematics into hacker education. Hackers raised on Lisp-school essays have grown up with only one parent. (The other parent has apparently been too busy at work.) We need examples, tutorials, and war stories wherein non-trivial mathematics are applied with success in computer programs. Although braggadocio doesn’t come naturally to most computer programmers, we need hackers to toot the horn of triumph whenever a new and interesting application of mathematics is found. We need to celebrate the spirit of scientific curiosity.

Lastly, we need the next generation of aspiring hackers to incorporate mathematics into their program of self-study. We need college students to take classes in physics, engineering, linear algebra, statistics, calculus, and numerical computing, and we need them to educate their elders who grew up ignorant of these things. With the relentless proliferation of data, and the impending extinction of the Fortran-slinging old guard, there are vast opportunities for budding mathematical hackers to make a difference in the world simply by thinking about it in a more rigorous way.



Correction: A previous version of this essay made a reference to the use of linear programming in World War II. This was incorrect. But the general point about the importance of mathematical computation in the war still stands; in place of linear programming, I have substituted the more compelling (and historically accurate) example of computer-aided cryptanalysis, famously performed at Bletchley Park.




You’re reading evanmiller.org, a random collection of math, tech, and musings. If you liked this you might also enjoy:
    
        You Can’t Dig Upwards
        Don’t Kill Math: Comments on Bret Victor’s Scientific Agenda
    


Get new articles as they’re published, via LinkedIn, Twitter, or RSS.



Want to look for statistical patterns in your MySQL, PostgreSQL, or SQLite database? My desktop statistics software Wizard can help you analyze more data in less time and communicate discoveries visually without spending days struggling with pointless command syntax. Check it out!

WizardStatistics the Mac way





Back to Evan Miller’s home page 
– Subscribe to RSS
– LinkedIn
– Twitter 




