Too Much Calculus – Gilbert Strang (2001) [pdf] - <AUTHOR> http://www-math.mit.edu/~gs/papers/essay.pdf <comment><USER3> As a programmer, I've found far more uses for linear algebra than for calculus. It shows up in robotics, in machine learning, in game development, and in many other cool subfields.

I didn't get very much out of my university linear algebra course. (It was pure theory with nearly zero applications.) I don't think I really developed a good intuition for linear algebra—and why it's so useful—until I read Jim Hefferon's free textbook:

[http://joshua.smcvt.edu/linearalgebra/](http://joshua.smcvt.edu/linearalgebra/) (GFDL license, LaTeX source available)

(I get nothing out of recommending this open source textbook other than the joy of sharing a book I loved.)

<comment><USER4> As a programmer, I've used calculus and linear algebra exactly never in my professional (20+ years) career. It just doesn't really come up doing web apps, sysadmin or dba work.

<comment><USER5> A lot of this is whether or not you are looking for it, honestly. It is easy to stay away from it, though it is also easy to find if you look for it.

</comment></comment></comment><comment><USER6> (Speaking from my experience as a calc teacher at a big U.S. state U.; mileage may vary elsewhere) Most non-STEM students won't see the 'Calculus I, Calculus II, Calculus III' that starts the essay so dramatically. Increasingly, liberal arts students are able to get by with no more than a "math appreciation" course and maybe what amounts to high school algebra. The "math appreciation" course might actually touch on very neat topics for these students (mathematics of voting, etc.) so I'd say math curriculum has already been successfully reformed for them.

As for the average STEM student, in my experience, most freshmen wouldn't be ready for linear algebra (unless by "linear algebra" you mean "matrix arithmetic"). They have enough trouble understanding functions in Calculus I: linear algebra's "functions are basically matrix multiplication" feature would NOT help.

Another thing to keep in mind is calculus has had much more impact on history/society than linear algebra. The fundamental theorem of calculus is a legitimately earth-shattering breakthrough: none of the theorems in linear algebra come anywhere near it in cultural importance.

Pragmatically, calculus is the most suitable mathematics for the type of testing-oriented, uniformizable/transferable pedagogy that a university really (in cold practical terms) must depend on for first-year students. It would be extremely difficult to force the kind of uniformity onto linear algebra that would be necessary if you seriously want to have millions of transfer students moving around between thousands of universities every quarter, gossiping about which linear algebra teacher gives the easiest grades, etc.

<comment><USER7> I'm amazed that you say that calculus has had a much greater impact on society when virtually every database, graphics manipulation or economic analysis uses principles of linear algebra.

Can you substantiate "millions of transfer students between thousands of universities every quarter"? That seems like outlandish hyperbole. Irrespective of the precise number, a system with momentum is not sufficient justification for its continuation.

<comment><USER8> FWIW, I have a math degree and work in tech.

Both are incredibly important. I would say that linear algebra and discrete methods are more important for computing, but calculus has had a larger impact on science especially physics. Newtonian physics - a revolution in thinking and unification of mathematics and science - is intricately tied to calculus.

Even in the example of economics that you cite, calculus is used heavily for the theoretical underpinnings for finding equilibria and deriving min/max.

<comment><USER9> I agree that Calculus is a more readily applicable mathematics that easily applied to many things. It really bothers me when people downplay the usefulness of Calculus also the person that posted the fundamental concepts of Linear Algebra are generally a little beyond the Grasp of people that haven't even been through the Rigors of Calculus 1 - Calculus 3.

</comment></comment></comment></comment><comment><USER10> I think Strang has some good points, and his advice is true for the majority of students who won't go on to use substantial amounts of mathematics or need to learn much math in the future. However, if you _do_ want to learn more math beyond calculus and linear algebra, then a thorough understanding of calculus is absolutely a prerequisite and you honestly probably haven't learned it well enough taking the typical sequence of classes.

For example, I'm currently studying probability theory at a fairly rigorous level (building up from measure theory) and my relative lack of skill with calculus is the main stumbling block. And I was recently talking with one of my professors who said that the biggest obstacle he sees students facing in trying to understand advanced mathematics isn't the difficulty of the math itself, but in their lack of mastery of the fundamental calculations that are necessary to follow the ideas and to come up with them on your own. Fluency with calculus is an enormous part of this. That said, I agree with the point, which is that for most students linear algebra is more useful.

Actually, I might go a bit farther and say that most students don't need to learn much math at all, at least in the way it is taught. Math is pretty poorly taught for a subject that is actually quite interesting once you start to understand it better (part of the problem is that in order to get to the interesting parts you need to first master all the boring stuff), and most people won't use any nontrivial math in their future lives.

<comment><USER11> This may be true for mathematics in the analytical tradition (things like Topology, Measure Theory, Real/Complex Analysis), but Linear Algebra is far more important in, unsurprisingly, algebraic disciplines. These include Number Theory, Field Theory, and Mathematical Logic.

While studying mathematics in college, once I had finished my Real Analysis requirement, I jumped headfirst into the algebraic side of things and never found myself using any sort of calculus. Even in my Topology course, we focused much more on using techniques from Real Analysis than specifically calculus topics (the former just being a generalization of the latter).

Probability theory is somewhat deceptive in its classification, since much of it "feels" a lot like a discrete mathematics course; however, much of the concepts, like you say, are underpinned by measure-theoretic principles, which is heavily analytic. It makes sense that calculus would come in handy in a much deeper study of probability theory.

<comment><USER10> That's a good point, my own studies have been biased towards the analytical side of mathematics, but I can definitely see how calculus could be less useful in other fields, e.g. I don't remember ever needing to do an integral when learning algorithms. And the opinions of my professors are of course biased as well, since they've spent their entire careers on analysis.

<comment><USER12> When analyzing algorithms, I have used calculus as a shortcut to many calculations. I believe that learning calculus is not just about its direct applications. Calculus often comes up when just dealing with your set-up.

You might make a discrete model for a problem and end up using calculus for approximations, equivalent calculations, or reasoning.

</comment></comment></comment></comment><comment><USER13> I watched Prof. Strang's Linear Algebra courseware, great stuff. He is a wonderful teacher.

[http://ocw.mit.edu/courses/mathematics/18-06-linear- algebra-...](http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra- spring-2010/)

As someone who hadn't studied it for 20 years, it was invaluable to revisit the subject as an adult with the ability to reflect "ah, I can use that for... " instead of "Ok but when am I ever going to use this"

<comment><USER14> Those lectures are fantastic. Pro-tip: watch them at 2.5x speed (you'll keep up just fine, and you can always pause to play with the concepts in Sage or whatever).

Bonus: when you switch him back to normal speed, he sounds stoned out of his mind.

<comment><USER15> This is a great tip. I've been using it for a few years now to watch various lectures on youtube--especially mathematics. Professors lecturing mathematics tend to speak slowly since they don't want to say anything erroneous. Speeding them up is usually amazing. For youtube; open the javascript console in your browser and type '$('video').playbackRate = 2.5;'. I've found that each lecturer usually has their own magic number regarding speed--after watching someone for a few hours and varying the speed you can usually find it.

</comment></comment></comment><comment><USER16> My linear algebra professor insisted that we, as computer science graduates, know and can reproduce most if not all the proofs of the theorems presented in the class. He didn't have such demands for math majors. His point was that we as computer scientists / software engineers / programmers will always have to understand and construct all the minutiae of the programs we write (otherwise they won't work). In contrast, mathematicians could use a lot of it "just" as tools.

I also had to go through a lot of heavy calculus courses which were very proof-centric and built from first principles. Rarely we skipped a proof of a theorem and we were expected to be able to reproduce and understand all of them at exams. I hated it often back then. However, these days I can appreciate the rigor it has taught me and I would wish I could send some of the programmers I met to similar courses so that they would learn to think clearly and precisely and be able to spot unmet or missing assumptions.

Our teachers and the creators of the curriculum were not stupid or naive. They knew well how much of hands on calculus we would use in practice vs. algebra or linear algebra. One of their main reasons for including calculus was to teach us rigor. I know that once someone overheard two teachers talking - <USER2> calculus teacher was asking someone from a CS department what to teach the CS graduates and the reply was that he can teach us whatever he wants, he just should make sure to teach us to think. I think they were quite successful at it even if it wasn't a very pleasant experience for most.

That also reminds me of this very relevant and true comic: [http://savethepenguins.net/image/237344573](http://savethepenguins.net/image/237344573) ("Wtf, man. I just wanted to learn how to program video games.")

</comment><comment><USER17> I agree. As far as calculus goes, I am more enamored with books like Spivak's ([http://www.amazon.com/Calculus-4th-Michael- Spivak/dp/0914098...](http://www.amazon.com/Calculus-4th-Michael- Spivak/dp/0914098918)) that take a proof-centric approach to teach calculus from first principles.

Incidentally, for those who want to learn linear algebra for CS in a mooc setting there are 3 classes running at this very moment:

[https://www.edx.org/course/linear-algebra-foundations- fronti...](https://www.edx.org/course/linear-algebra-foundations-frontiers- utaustinx-ut-5-02x) (from UT Austin)

[https://www.edx.org/course/applications-linear-algebra- part-...](https://www.edx.org/course/applications-linear-algebra- part-1-davidsonx-d003x-1) (from Davidson)

[http://coursera.org/course/matrix](http://coursera.org/course/matrix) (from Brown)

The first 2 use matlab (and come with a free subscription to it for 6 months or so), the last python. One interesting part of the UT Austin class is that it teaches you an induction-tinged method for dealing with matrices that let you auto-generate code for manipulating them: [http://edx-org- utaustinx.s3.amazonaws.com/UT501x/Spark/index...](http://edx-org- utaustinx.s3.amazonaws.com/UT501x/Spark/index.html) .

And of course there are Strang's lectures too, but those are sufficiently linked to elsewhere.

<comment><USER18> My calc I course in university was applied calculus without a text. I had to go back and redo single variable by reading Spivak (and Polya's How to Solve It) to figure out the proofs in Concrete Math by D. Knuth, et. al.

</comment></comment><comment><USER19> You can probably do worse than to cut mathematics up into "algebra" (not just linear) and "analysis" (calculus). They're almost completely diametric points of view by the bulk modern mathematical canon [0].

There's a fair point to say that one shouldn't learn so much calculus as to lose your linear algebra, but the opposite is just as true. There are a lot of comments in this thread along the lines of saying that linear algebra is far more useful than calculus in CS. This statement feel not exactly unfair but certainly like it's a reactionary measure.

Whenever you have a notion of change you have some kind of calculus underlying it. Even if you end up having to view that notion of change algebraically. For instance, there's a famous functional programming result that there is a well- defined "derivative of an algebraic data type" which allows you to talk about changes in data "over time".

I'd also feel very bad for anyone who tried to learn too much probability without studying quite a bit of calculus first.

[0] Sure there's discrete calculus and algebra underlies the mechanism of most calculus computation. Likewise there are continuous groups and you can't avoid how topology mixes the two concepts.

</comment><comment><USER20> I really can't agree with this strongly enough. I had enough Calculus and Differential Equations courses to kill a horse in college (Chemistry major) and I've never used that knowledge once in my entire programming career. But I find uses for linear algebra and matrices in coding all the time. Thank god Strang's excellent course is available online for free.

<comment><USER21> If you were looking to become a programmer, maybe you shouldn't have majored in chemstry, where strong Calculus foundations make perfect sense :)

<comment><USER22> Chemist here, who also programs. Inorganic chemists do calculus all the time. Organic chemists do arithmetic (stoichiometry) and some discrete analysis (instrumentation, the computers do the heavy lifting), orgo is primarily a linguistic discipline, i.e. we are expected to parse sentences like "benzophenone in THF was refluxed in the presence of Pd black for 2 hours" and to know, without being told, that there's a workup before the next step in the reaction.

<comment><USER20> Exactly. I was an organic chemist. Never used a lick of Calculus. I didn't get into programming until I'd already been working as a chemist for about five years.

</comment></comment></comment></comment><comment><USER23> Perhaps amusingly, the Computer Science education at the University of Copenhagen includes no mandatory Calculus course, but it does include a Linear Algebra course. Even so, students have a hard time seeing what they'll use Linear Algebra for, before later on in their education.

<comment><USER24> Make them code simple wolfenstain 3d clone with opengl, directx or even unity3d.

It was a revelation for me when I've tried to get into game programming, and half of the math taught in primary and secondary school became immediately useful, also linear algebra (which was only started at the final year of high school). Made linear algebra courses on the university a lot more interesting. There's a difference if you look at a class "what I need to pass an exam", and "what I could use in my next game".

Also tinkering with 3d graphics makes thinking in vectors, planes, matrices, dot and cross products intuitive after a few weeks.

BTW I wonder hwo many kids learnt basic linear algebra from Denthor's Asphyxia tutorials :)

<comment><USER25> I can't see myself really needing to apply any Linear Algebra concepts writing a Wolfenstein clone in Unity3D. It does everything for you like that (shader graphics, collision detection, vector math, etc). All you'd have to do is code some simple AI for the enemy gameobjects and make the player gameobject in the standard FPS shooting format, which there are pre-built defaults for in the example projects. The majority of this project would be copy and paste with minor modifications.

<comment><USER24> Even "just" ai can teach you a lot. For example trying to calculate if enemy can see you, or which path he should take is interesting.

Of course you can go far with "turn towards player and shoot" ai, but where's the fun in that.

Actually it would be fun to give student's assigment like:

here's the AI interface, make a game using it, and write your bot. Then we will run all the bots in free for all mode on all the games (except your bot on your game), and we will grade by sum of the points.

<comment><USER25> >trying to calculate if enemy can see you

Send a raytrace from the bot to the player and see if it hits something other than the player, which is just one line

Pathtaking beyond simply moving towards the player is a lot harder, yeah.

<comment><USER24> If you just send 1 ray from eye towards the center of the enemy (or head, or whatever) - and this particular part of the body is occluded by a wall or a chair, but the rest isn't - the result will be wrong.

You can accept that, use many rays, or do proper view frustum/body model(simplified probably) intersection, after cutting the view frustum with the terrain.

You can also consider lighting (so enemy can't see you hiding in a dark corner), or even dynamic lighting (and give enemies light sources).

Everything becomes hard after you think about it enough :)

But I agree simple solutions + some tinkering with parameters works most of the time (and gamers probably won't notice).

</comment></comment></comment></comment></comment></comment><comment><USER26> Statistics are wildly important for both business and understanding the world around you, and without solid calculus skills you cannot truly work with statistics.

Its not something that comes up every day at work, but it is still pretty often. For example, I was on a large team developing a complicated distributed system. The performance requirements were uncertain (like 500000 clients +- 200000, sending 10 +- 30 messages per second, etc). The system itself was in early development, hadn't been tuned, and we had only rough performance numbers for it.

Given only that, I needed to purchase the correct amount of hardware for the test lab, keeping in mind that the lead time for hardware was several months. I needed both statistics and some basic calculus to build a model based on what we knew, calculating variances for each intermediate and output, and then getting 50%, 90%, 99% confidence level estimates.

In the end, we got pretty close to a bullseye. On the other hand, I learned the important lesson to hide all the math from the client, lest they get awfully confused.

</comment><comment><USER27> The Khan Academy lectures are a pretty way to start: [https://www.youtube.com/playlist?list=PL835D2252574274C5](https://www.youtube.com/playlist?list=PL835D2252574274C5)

</comment><comment><USER28> Possibly understated... not only is there the three course sequence + diff eq, there's usually also a two-sequence real analysis course (more or less "careful calculus") and probably either PDEs or complex analysis to boot.

I took the discrete emphasis when I studied and I think I actually ended up with fewer discrete classes (5, IIRC) than calculus classes (6).

<comment><USER29> Although it's not explicitly stated in the article, I think Strang's remarks are actually directed at non-math programs in STEM (I assume you are at least a minor in math because of 3 analysis courses). Math majors will get plenty of algebra either way, but it's the engineers and scientists who miss out from overemphasising calc over algebra.

</comment></comment><comment><USER30> I've never understood why Calculus is taught as a separate course. It seems so detached from practicality. I've always thought students should take perhaps 4-6 semesters of Physics -- and introduce derivatives, integrals, etc. as needed when they can be demonstrated to more easily solve the physics problems. If the physics content was more spread out, there would be ample time to introduce the necessary math concepts, instead of stuffing them into a separate course.

<comment><USER31> I remember, in my high school calculus class, the teacher said that there were six or seven applications of derivatives, and more like 40 applications of integrals. It's _way_ more than just physics.

</comment></comment><comment><USER32> What is the context of this short essay? Why did he feel the need to write it, at that time?

</comment><comment><USER33> I don't disagree that linear algebra isn't given enough emphasis. However, the first thing I thought of when I read this was something attributed to Richard Hamming, who is known for the transcript of his talk that pops up on here frequently, You and your Research [1]. I got this from a review of his book on Amazon [2]:

> Hamming, in discussion, concluded his life thinking "the best tool to teach > thinking was to teach the calculus."

I'm just thinking here that Strang is saying Linear Algebra is more useful for _applying_, where Hamming would say Calculus is more useful for reasons that go beyond applications (not that it doesn't also have applications).

[1] [http://www.cs.virginia.edu/~robins/YouAndYourResearch.html](http://www.cs.virginia.edu/~robins/YouAndYourResearch.html)

[2] [http://www.amazon.com/review/RWPF0EL3E7GJZ/ref=cm_cr_pr_perm...](http://www.amazon.com/review/RWPF0EL3E7GJZ/ref=cm_cr_pr_perm?ie=UTF8&ASIN=9056995014)

</comment><comment><USER34> I had the fortune of taking 18.06 (Linear Algebra) from Gilbert Strang around the time of this essay. And like other commenters, I will agree that he was a very good teacher. I could tell that he held a great emotional attachment to matrices; he'd often anthropomorphize them with genders and refer to them as "little guys".

</comment><comment><USER35> I think taking less calculus and more linear algebra is a mistake.

There is a really good book in two volumes that I feel shows how methods of linear algebra can be extended using calculus to solve a wide range of problems: A Course in Mathematics for Students of Physics:

[http://www.goodreads.com/book/show/2685577-a-course-in- mathe...](http://www.goodreads.com/book/show/2685577-a-course-in-mathematics- for-students-of-physics)

Volume 2 is more important. It starts demonstrating how methods of Linear Algebra can be used to solve circuit problem, then generalizes this to higher dimensions and from discrete to continuous spaces using the exterior derivative. Seeing this demonstrated gave me a much clearer, more intuitive understanding for both calculus and linear algebra.

</comment><comment><USER36> I wish Gilbert Strang would teach calculus and write a book as he did for linear algebra.

<comment><USER37> You are in luck!

Strang, Calculus, 2nd edition: [http://www.amazon.com/Calculus-Second-Edition- Gilbert-Strang...](http://www.amazon.com/Calculus-Second-Edition-Gilbert- Strang/dp/0980232740)

<comment><USER38> And the first edition is freely available at Open Courseware: [http://ocw.mit.edu/resources/res-18-001-calculus-online- text...](http://ocw.mit.edu/resources/res-18-001-calculus-online-textbook- spring-2005/textbook/)

<comment><USER36> I wish every day is like this!

</comment></comment></comment></comment><comment><USER39> I took Gilbert Strang's Mathematical Methods for Engineers course at MIT. Can't seem to find the link for it, but I would recommend the online version for anyone who wants to learn linear algebra.

</comment><comment><USER40> I don't think you can cleanly separate the topics. Most applications of linear algebra come from calculus since after all you use calculus to linearize something. In case it was already linear it's probably a linear differential equation -> calculus again. Complex analysis is also closely related to linear algebra via geometric algebra. Rather than teaching them separately and emphasizing linear algebra more, it would make more sense to approach calculus + linear algebra as an integrated subject.

</comment><comment><USER41> Anyone interested in getting started with linear algebra can check out this printable tutorial: Linear algebra in four pages: [http://minireference.com/blog/linear-algebra- tutorial/](http://minireference.com/blog/linear-algebra-tutorial/)

The page also links to another short tutorial which explains more of the geometric intuition behind linear transformations.

Neither tutorial can serve as a full course, but reading them will give you a good overview of what linear algebra is about.

</comment><comment><USER42> 14 years later, the popularity of machine learning and "big data", which requires quite a bit of linear algebra, makes this only truer.

<comment><USER19> It also requires quite a bit of calculus though!

<comment><USER42> Calculus is indeed a required tool if you want to design new machine learning algorithm. But without linear algebra, good luck implementing an algorithm or even using a library to apply it on a concrete case.

<comment><USER19> I don't see why this is an either/or decision?

Without a firm understanding of probability then good luck interpreting or fitting a model. Without a firm understanding of linear algebra good luck implementing or using an algorithm.

And hell, if you have a streaming algorithm you probably no longer need linear algebra but to grasp either the probabilistic or geometric interpretations you'll probably need calculus.

</comment></comment></comment></comment><comment><USER43> > We need to present the mathematics that is most useful to the most students.

I wish it weren't this way! Mathematics is a wide and fascinating field, and I think more students could find interest if our schooling reflected that. But moving some of the detail and drudgery that is the calculus sequence to subject matter that requires it, in order to make way for linear algebra, is a great start.

<comment><USER10> I agree completely, but calculus touches so much of advanced math that at some point you have to learn it in depth (assuming you actually want to learn advanced mathematics). I'd argue that linear algebra is a more interesting subject for most people, but calculus contains more important ideas that are crucial if you intend to continue studying mathematics.

<comment><USER43> I have a major in Mathematics, and I truly didn't need calculus after the multivariable/vector calculus course I had to take my Freshman fall. Why did I _have_ to do so much if it wasn't going to be necessary until graduate school at the earliest, or perhaps never depending on specialization? And why was it a prerequisite for my other math courses (in algebra, topology, real analysis, number theory, and combinatorics)?

<comment><USER44> How was topology motivated without talking about generalizations of limits?

(Honest question, no snark.)

<comment><USER43> It's a good question.

I think my first topology course, like in many intro analysis courses, shared concepts from calculus without requiring any previous expertise from our calculus courses. A rigorous approach to real analysis or point-set topology will be very set-theoretic, and you build the general definitions of continuity, compactness, etc from the ground up. So, I would have used concepts from my analysis and topology courses _in_ calculus, if only it had been taught in the other order.

I guess it feels like limits are a rigorous foundation for calculus, but they are not calculus. In any case, I didn't need anything from differential or integral calculus (or that entire _required_ course developing the theoretical basis for Stokes...), which is what I mean to say.

</comment></comment></comment></comment></comment><comment><USER45> I remember taking Calculus 1, 2, 3 but only having 1 linear algebra course and some engineering math. Although I did well in them, as a developer, I wish I payed more attention to linear algebra or at least wished that my uni offered more courses related to it, like discrete math and maybe even statistics. Good thing I still have my old books!

</comment><comment><USER46> To reinforce this point: here's just a fraction of what you can do with linear algebra [1]

[1]: [https://medium.com/@jeremyjkun/here-s-just-a-fraction-of- wha...](https://medium.com/@jeremyjkun/here-s-just-a-fraction-of-what-you-can- do-with-linear-algebra-633383d4153f)

<comment><USER47> To be fair, almost all that stuff uses calculus too.

</comment></comment><comment><USER48> heh, try teaching galois for a year, you'll understand why calculus is so popular.

<comment><USER49> it's a beautiful branch of mathematics though.

<comment><USER48> everybody thinks they know algebra if they study linear algebra. this is worse than not knowing about calculus i guess.

can't image teaching point-set topology, teaching uryshons proof to undergrads, gee.

</comment></comment></comment><comment><USER50> Another perspective: I am trying to teach myself physics from Morin's books, and for the first time I see how well structured was my lower division calc sequence d for this application.

</comment><comment><USER51> i actually have taught some linear algebra concepts on youtube and have gotten several hundred thousand views and some positive feedback. i think the key is trying to figure out a framework of intuition to help students through all the math but i liked the idea of four key equations we are trying to solve as a framework as well

</comment><comment><USER52> The way this was typeset...it's making my head hurt.

</comment><comment><USER53> > Too Much Calculus

Not for me! Heck, I never took freshman calculus or _pre-calculus_ and, instead read a book and started on sophomore calculus then continued with, right, ordinary differential equations, advanced calculus for applications, and advanced calculus, topology, modern analysis, measure theory, functional analysis, exterior algebra, etc. Not enough calculus for me!

Strang, of course, is emphasizing linear algebra. Great! It's terrific stuff. I had an abstract algebra course that started on vector spaces and matrix theory. Then I got famous book from Princeton on multi-variable calculus and read the first chapters on linear algebra, e.g., learned Gram-Schmidt. Then I got a linear algebra book and read a few chapters and applied them to some problems in classical mechanics. Then I wrote my math honors paper in group representation theory, right, more linear algebra. The out of school, I wanted to know linear algebra better and read a good book cover to cover, carefully. Then I got Halmos, _Finite Dimensional Vector Spaces_ and read it cover to cover. Wrote Halmos about his proof of the Hamelton-Cayley theorem and got back a nice answwer!

Then I got another famous text on multi-variable calculus and read it cover to cover, including the parts on exterior algebra. Then I did that again from Spivak's _Calculus on Manifolds_ \-- right, a lot of both calculus and linear algebra. Then I got Forsythe and Moler, _Computer Solution of Linear Algebraic Systems_ , read it cover to cover, directed a project in interval arithmetic for linear algebra, took a course in numerical analysis with a lot of linear algebra for solution of partial differential equations, e.g., iterative solutions (Gauss-Seidel) and wrote corresponding code.

I also used linear algebra in work with the fast Fourier transform, multi- variate statistics, and more.

For more on linear algebra, I read the first part, linear algebra, of von Neumann, _Quantum Mechanics_.

From a world class guy in linear algebra, I took an _advanced_ course in linear algebra -- by then the course was beneath me and I led the class by a wide margin. Favorite theorem -- the polar decomposition.

Read Coddington's book on differential equations -- gorgeous book.

The combination of linear algebra and calculus, with other math, continued.

At one time a little calculus did a lot: The initial value problem for the little first order, linear ordinary differential equation

y'(t) = k y(t) ( b - y(t) )

saved FedEx, that is, kept it from going out of business. So, _useful calculus_!

So, sure, I like linear algebra and like the combination!

I definitely do not see that I had too much calculus!

Here's what I saw, and still fume about: All I could get in chemistry was one too simple course in high school and one too simple course in college. All I could get in physics was one too simple course. I college, by the time I got to Maxwell's equations, I didn't know enough calculus to do really well. In college I never got math enough to do well with quantum mechanics.

But in grades 9-12 and then in the first two years of college, I was force fed like a goose with six years of English literature, with each year yet another play by Shakespeare. I agree that there was a good writer in England in the 1600s -- Newton! So it was also Chaucer, Milton, Wordsworth, ..., Dickens, _the great natural order_ (all corresponding theorems and proofs omitted!), etc. With good information about people? Nope.

Then there was history: It never got to the 20th century. It never touched on technology, economics, or any other _causes_. There was plenty of time in history courses; it's just that the courses didn't have much content.

If my startup works, then I'll get back to more in calculus, functional analysis, stochastic processes, and mathematical physics. "More calculus, Ma!".</comment>


Notes from the Collaboration Summit (Linux, IBM, Google, etc.) - <AUTHOR> http://lwn.net/Articles/383945/

<comment><AUTHOR> Related, quoted, links:

ELC: Android and the community - <http://lwn.net/Articles/383276/>

Contributing to Open Source projects - <http://brad.livejournal.com/2409049.html></comment>


Police Documents on License Plate Scanners Reveal Mass Tracking - <AUTHOR> http://www.aclu.org/blog/technology-and-liberty-national-security/police-documents-license-plate-scanners-reveal-mass <comment><USER2> The other problem is that state laws aren't keeping up with law enforcement technology. FOIA-similar laws in Minnesota meant that every ALPR-captured license plate was public information, which I requested to prove a point. I got back 2.1 million license plates (+ timestamp and coordinates) on a USB stick.

Without strong policies and laws, this data is ripe for abuse -- domestic abuse victims and stalking where someone eats, visits, and sleeps, or tracking regular routes of hazmat or cash trucks, etc.

[https://tonywebster.com/2012/12/minneapolis-police- license-p...](https://tonywebster.com/2012/12/minneapolis-police-license- plate-data/)

<comment><USER3> Yea, great so you complain about that and then they solve just that narrow case.

Doing so reduces some fringe harassment risks, but actually makes the power inversion worse since then only a privileged minority has access to the data.

In practice the stalking/harassment stuff isn't actually happening with any great frequency: The mentally ill people who would do this are generally not competent enough to go on a big data data mining exercise. Complaining about that just creates an excuse to remove access to the public to data they're paying to create while leaving access to it to authorities and commercial interests who might use it against them.

<comment><USER2> Despite being in public -- and on public streets -- I think anytime the government is tracking where I go, it should require probable cause and a judge's signature. I believe ALPR circumvents the judicial system.

I use my cellphone in public, but it's obvious that the government shouldn't be able to automatically log when and where I placed a call. I use my credit card in public, but the government shouldn't be able to automatically log every penny going into or out of my bank account. I use my car in public too, and I don't think the government should track everywhere I drive.

I don't think there's an acceptable amount of harassment that's tolerable. If the technology enables it to happen, it's worth challenging.

There are three issues for me with ALPR: (1) limiting the scope of persons who have access to the data, (2) ensuring that law enforcement agencies have strong (ab)use and access policies with checks and balances, and (3) ensuring that logs of where innocent individuals drive is never stored.

I think it's easier to progressively win a fight rather than going guns- blazing to a large issue that people don't understand. By first building the notion that 'yeah, this technology is actually somewhat scary' it's much easier to win a fight arguing that the police shouldn't be storing and accessing the data either.

In my situation in Minnesota, the public finding out about the police releasing 2.1 million license plate tracking points to the public was enough pressure to get the state to make the data non-public and to have police departments voluntarily create more restrictive policies. It was also enough to get a fabulous bill passed through the state House that prohibited police from storing any data on non-hit (non-wanted) vehicles. Unfortunately, it didn't get to the Senate floor in time before the session ended, but I was shocked at the number of legislators who spoke out against ALPR data storage. They weren't necessarily anti-ALPR (and honestly I'm not vehemently anti-ALPR either), but the prospect of storage is wildly unpopular among non-police communities.

There's another issue all together, which is privately owned and operated ALPR systems -- typically used by vehicle repossessors but it was also uncovered this week that a defense contractor had also been on the receiving end of Minneapolis Police ALPR data, which is somewhat unnerving.

<comment><USER4> If you're walking around in public, the government has the right to follow you around. In fact, everyone has the right to follow you around. Having cameras and drones around just means they got better at doing it.

<comment><USER5> Of course anyone can follow you in public, the alternative would be impossible to enforce.

The problem is that the _cost_ of getting that data could be made unacceptably low. For example, I think most people would object having their government uploaded their coordinates to a public website. The interesting question is what is the right tradeoff between transparency and cost.

</comment></comment></comment></comment></comment><comment><USER6> I have a personal experience with the harm that ubiquitous recording of location data can bring, even when you don't do anything wrong.

When I was in college, I was called to the police station and accused of breaking a glass door in the dorms. I knew NOTHING about this. I told the police I knew nothing about the event, but they continued to press me. They asked me details about where I was that night (many weeks prior), and I had difficulty remembering where I had been. To make matters worse, I originally thought the day in question was another day, so I ended up giving information about my location that was incorrect. If the police decided they didn't like me, they could have easily pressed charges for lying, even though I thought I was telling the truth. Luckily, they eventually believed me and I was let go.

How is this related to license plate readers? Well, the only reason they suspected I was the person who broke the door was because my ID card had been swiped at around the same time the door was broken. Simply because I was present near the time and location of a crime committed weeks prior, I was accused of a crime and nearly accidentally perjured myself. (Don't worry; I know now to never talk to the police!)

If we start tracking everyone's movements, this sort of thing will happen more and more often. You will EVENTUALLY have driven by a place where a crime was committed. You may be called to account for your being nearby, even weeks or months later. When the police ask you why you were there, will you remember? Will your answers satisfy them? Will your general unease at being questioned make you appear suspicious? Will you fumble, and say things that contradict yourself, making you appear even more suspicious?

<comment><USER7> At the point when they started asking about stuff you didn't know about I would have invoked my right to a lawyer.

<comment><USER8> ...which now means you need to spend money to protect yourself against a situation where you've done nothing wrong.

<comment><USER9> A lawyer might be provided pro bono -- but I completely agree with the point you're trying to make.

<comment><USER10> Public defenders are overworked and often unable to mount a sufficient defense.

<comment><USER9> Oh, I'm well aware.

</comment></comment></comment></comment></comment></comment><comment><USER11> From user qz10 (who seems to have been hellbanned absed on IP):

 _It should be no surprise that this is happening. Give anyone unlimited surveillance power and it will always be abused.

I live in a western EU country and have a friend in law enforcement. Last year, he brought me to see our city's traffic surveillance 'control room'. Almost every street corner and traffic junction here has a pole-mounted police camera transmitting video data back to servers accessible from this room and elsewhere.

To summarize the demo: He was able to type a vehicle registration plate into this system, and instantly see every journey the car has taken plotted on a map, as well as the thousands of raw video clips of this vehicle that were used to generate the map._

<comment><USER12> I think as a practical matter, expecting police to stay stuck in the 20th century is futile. The public will never strongly oppose that kind of technology when they themselves are wearing google glass everywhere.

The fear of this sort of thing depends on being suspicious and wary of the police, and people aren't. My wife thinks I'm crazy when I rant about cops, but as a practical matter she has no reason to be skeptical (as a young white woman). As long as most voters are either white, old, or female (groups cops tend to avoid harassing) you won't see the necessary skepticism to oppose this sort of thing.

Heck, over the last 20 years people happily handed the police enormous amounts of power via drug laws. The white suburban moms in my school in the 1990's shrieking about "just say no!" had no reason to be skeptical.

<comment><USER13> This is why I'm seriously of the opinion that surveillance data should be regulated somewhat along the lines of HIPPA or PCI rules.

Storing a weeks worth of traffic camera data and allowing to be misused should result in a demotion, a small fine, and a personal civil liability towards people who've been affected by the misuse. Signing off on storing years worth of ALPR/surveillance-camera records and having _that_ misused or exposed should be a career ending and company destroying event.

Collect that sort of data if you think it's of sufficient "value" \- but make sure everyone is aware of the risks and the magnitude of the penalties of being "careless" with other people's personally identifying data.

<comment><USER11> I agree, and think the US suffers from not having this codified in law as it is in the EU.

</comment></comment></comment></comment><comment><USER14> A few days ago I posted a document regarding NYC's relatively new booting program for cars which have accumulated too many tickets/fines. In it they point out that the system is backed by a team of vans which scour the city doing license plate recognition to find these cars. Obviously the consequence is that _everyone 's_ license plate gets photographed, but it's up to the NYPD what to do with that information. The ACLU doc has some stats, but if you want to see the document I'm talking about I've linked it below.

[https://news.ycombinator.com/item?id=6049276](https://news.ycombinator.com/item?id=6049276)

Though Bloomberg and the NYPD hardly have any trouble pushing these things through for the last decade, I imagine one impetus behind ramping these programs way up would be incidents such as the would be Times Square van bomber from a couple years ago. In that case however, the perpetrator had actually stolen license plates from another vehicle (which they learned after finding a VIN that wasn't scratched and the registration didn't match).

[http://en.m.wikipedia.org/wiki/2010_Times_Square_car_bombing...](http://en.m.wikipedia.org/wiki/2010_Times_Square_car_bombing_attempt)

</comment><comment><AUTHOR> I always wondered on the legality of private license plate scanning. With just a bunch of people cooperating one could pretty much map out all the troop movement throughout the city.

<comment><USER15> Well the argument usually heard for the government doing it is _" they've always been able to watch where you go in public"_. I suspect that somehow would not fly if it were reversed.

It's an interesting idea. Thanks to smartphones, the people have more cameras than their governments. The governments however have more targets.

<comment><USER16> An interesting app idea would be to have a distributed palantir-like system that specifically targets Ambulances, Police Cars, City vehicles, etc. OCR a license plate or other identifying tag on it, geo-tag, etc.

Given a decent proportion of people using the app, you could probably map out the location of most of the active government vehicles in a city. With the time-series data, you could add some analytics and probably predict where the cars were going to be at specific times and days. You could even gameify it, so catching a police car at a donut shop would win the user a little badge.

<comment><USER13> I wonder how close the russian dash-cam eco-system is to making that come true?

Inexpensive but ubiquitous and "always on" cameras, with a little bit of OpenCV smarts built in (or perhaps with a RaspberryPi-grade computer hanging off them) could probably manage a pretty good implementation of this.

If you could somehow roll this out as part of some innocuous looking project (off the cuff idea - approach a taxicab company, tell them you want to put camera's on each cab - so you feed each driver back information about nearby traffic density/speeds and approximate counts of people standing on sidewalks correlated with the number of other cabs around. Slip in ALPR without mentioning it, and a whitelist of marked and unmarked police car licence plates. Sell access to the real-time police location data to high-speed courier services, boy-racers, and drug cartels.)

</comment></comment></comment></comment><comment><USER17>
        CALIFORNIA CONSTITUTION
        ARTICLE 1  DECLARATION OF RIGHTS

        SECTION 1.  All people are by nature free and independent and have inalienable
        rights.  Among these are enjoying and defending life and liberty, acquiring,
        possessing, and protecting property, and pursuing and obtaining safety,
        happiness, and *privacy.*

I've often wondered if the right to privacy in California's constitution actually means anything. I guess it doesn't.

<comment><USER18> What's private about your public movements?

<comment><USER19> Anonymity != Privacy.

If you're in a crowd, by definition, you're not expected to have privacy, however you are allowed anonymity by virtue of being in a crowd. If you're in a cafe, you're naturally not afforded privacy, however you are still anonymous unless spotted by someone who recognizes you. If you lean over to tell something to this person, that conversation (despite being in public) is still private.

Your public information still afforded you anonymity in a sea of data however since we now know it can conveniently be assembled into a dossier, it's being used to destroy your anonymity which, believe it or not, used to be taken for granted.

<comment><USER18> > however you are allowed anonymity by virtue of being in a crowd.

That's not true at all. Anonymity is the state of personally identifiable information being unknown. That's _exactly_ what your license plate is, and it's displayed on the back (and front sometimes) of your car. You have _never_ been anonymous while driving.

<comment><USER19> Completely false.

Your license plate is the equivalent of a online pseudonym linked to a database. Common users aren't able to browse this database, however if someone's keeping a tally of the appearances of said license plate (the equivalent of a search engine), then they can infer a whole lot of information that goes beyond the scope of the purpose of license plates. _That_ is the real problem here. You _are_ anonymous in the crowd or should be as far as other ordinary people are concerned and that's clearly not what's happening.

Incidentally, you should stop using the same username for everything.

<comment><USER18> Your license plate is a unique identifier for your car, not you. It is not like an online pseudonym in that cars are often shared by others, whereas a pseudonym is a false _name_ , generally (with exceptions) reserved for individuals. To claim they're equivalent is therefore not true. You are _not_ anonymous in a crowd (I can take a picture of you and identify you later, your identity is tied to your _face_ ), nor _should_ you be (if people where anonymous all the time, personal responsibility would go out the window - there is a time and a place for anonymity and privacy, driving around on public roads is not one of them).

And why should I stop putting my name on things? I'm the one who did them, after all. Besides, if I were at all concerned that what I was saying would land me in trouble, I'd create a new face. I'm not against privacy, I just don't think you have any when you're in public.

</comment></comment></comment></comment></comment></comment><comment><USER20> [http://www.aclu.org/files/FilesPDFs/ALPR/kansas/alprpra_lene...](http://www.aclu.org/files/FilesPDFs/ALPR/kansas/alprpra_lenexapd_lenexaka.pdf) Page 364 mentions "PRISM".

<comment><USER21> PRISM is an acronym used all over the place, and may not be the NSA-specific program.

[http://waxy.org/2013/06/these_arent_the_prisms_youre_looking...](http://waxy.org/2013/06/these_arent_the_prisms_youre_looking_for/)

<comment><USER20> Yeah, seems like it's more likely to be referring to the US Dept of Transportation Performance and Registration Information Systems Management "PRISM" program.

[https://www.fmcsa.dot.gov/safety- security/prism/prism.aspx](https://www.fmcsa.dot.gov/safety- security/prism/prism.aspx)

</comment></comment></comment><comment><USER22> In case anybody's interested, these license plate scanners can be publicly found on the Internet:

[http://www.shodanhq.com/search?q=atz+executive](http://www.shodanhq.com/search?q=atz+executive)

And this is how the web interface for such a system looks like:

[http://money.cnn.com/gallery/technology/security/2013/05/01/...](http://money.cnn.com/gallery/technology/security/2013/05/01/shodan- most-dangerous-internet-searches/2.html)

<comment><USER9> That SHODAN search link doesn't return any results...

<comment><USER22> Ah crap, the above search requires you to be logged-in and have the "Telnet" add-on for your account. Here's a quick alternative that also returns some results, though they all require login over HTTP:

[http://www.shodanhq.com/search?q=title%3A%22PIPS+Technology%...](http://www.shodanhq.com/search?q=title%3A%22PIPS+Technology%22)

</comment></comment></comment><comment><USER23> It should be no surprise that this is happening. Give anyone unlimited surveillance power and it will always be abused.

I live in a western EU country and have a friend in law enforcement. Last year, he brought me to see our city's traffic surveillance 'control room'. Almost every street corner and traffic junction here has a pole-mounted police camera transmitting video data back to servers accessible from this room and elsewhere.

To summarize the demo: He was able to type a vehicle registration plate into this system, and instantly see every journey the car has taken plotted on a map, as well as the thousands of raw video clips of this vehicle that were used to generate the map.

</comment><comment><USER24> It is fascinating to observe the proliferation of technological misadventures federally funded in the past decade+ since 9/11\. There needs to be a real legal movement to place limits on these activities. The argument is already lost in many other countries (e.g. U.K.), but we have a fundamental legal doctrine, enshrined in the 4th Amendment, that should apply. That requires real congressional and judicial action to restrain these law enforcement and anti-terrorism activities. Unfortunately, since nobody can yet claim tangible harm from having their privacy invaded, the judicial route seems murky.

<comment><USER11> _we have a fundamental legal doctrine, enshrined in the 4th Amendment_

You need a constitutional amendment specifically for privacy and data retention. The 4th amendment does not do what you think it does.

Meanwhile, the argument is _not_ lost in countries like the UK, but very much alive in terms of people invoking EU law via the charter of fundamental rights.

</comment></comment><comment><USER25> One of the scariest bits is that most of the operators of these cameras pool information. The result is an ad-hoc nationwide surveillance system _operated by a private party_.

If you operate certain brands of cameras, you get access to the database: [http://nvls-lpr.com/nvls/](http://nvls-lpr.com/nvls/)

</comment><comment><USER26> I'm not a cryptography guy, but do you think it would be possible to come up with a technical solution which has these properties: \- License plate numbers are immediately encrypted at scan \- A judges possess some sort of encryption key \- Upon obtaining some sort of warrant, judge grants police specific encryption key \- Key can only be used to decrypt certain kinds of data - say, only decrypts license plates in a certain geographical area or around a certain time

</comment><comment><USER27> A little off topic, but it will get to the point:

Remember some of the biggest issues? Exploitation (wars, colonialism, imperialism, no labour laws), racism (slavery, Jim Crow laws), overeaching government (McCarthyism, Hoover, bureaucracy, protestors shot at, , surveillance, etc), rampant private interests (banana republics, RIAA, patent law, militias shooting strikers, corporate espionage, buying legislation), religious nuttery, etc?

Those things are not solved by technology. If you're lucky, you might find some technology that can nullify one of those issues.

But in general technology is an enabler. It can help those perpatrating those things to do them 1000 times more efficiently and far scarier. Science too, is neutral. Nazi germany was choke full or good scientists -- and they are behind all messed up inventions anyway too, not just the good ones.

This, among other things, is such as case, of technology being an enabler for bad things. To really effect change you must change how people think about things -- and then, the laws.

</comment><comment><USER28> Interestingly, my state Department of Transportation is the agency responsible for operating the ALPR systems, and according to my county Police Chief his department doesn't use ALPR technology, so I wonder how readily the various enforcemnt agencies have access to the the acquired database(s)?

</comment><comment><USER18> Can someone explain why your public movements should now be private?

And when someone brings up the "pseudo-anonymity of the crowd", can you accompany that with a legal precedent for such a claim? Who actually came up with the "I'm acting privately in public" thing in the first place?

<comment><USER29> Yeah. If you don't like the prospect that some random person can know the intimate details of your <travel|communications|financial transactions>, you might as well just abstain from the services and institutions that make anything remotely like a modern standard of living possible. It's a fair tradeoff.

<comment><USER18> If you don't like above said prospect, then maybe you should do those things in private instead of in public.

<comment><USER29> Ah, why didn't I think of that? We could all just be traveling entirely in private and this would be a non-issue.

<comment><USER18> I was speaking more to the "communications/personal finance" bits you mentioned.

You can't be wholly private. Merely by existing, people will be able to learn some things about you. Nothing you can do about it.

<comment><USER29> Sure, and by natural extension there are some people, whom you will never meet (or even be aware of their doing so), who will get to become intimately familiar with the aforementioned data, journalized and cross-referenced in handy tables and graphs. Indefinitely. Nothing you can do about it! Might as well just accept it. Really, all of this hubbub around privacy concerns is totally unfounded. It's the way things _have_ to be.

<comment><USER18> They have to be this way because of how we structured the English language. When you act in public, you are, definitionally, not acting privately. There is _literally_ nothing to be done about that short of changing what words we use.

Your movements in public are in absolutely no way equal to your conversations in private. You're trying to equate things which aren't equal, and you're taking an extremist, hard-line stance on what should and shouldn't be private.

It's utterly silly to think that a person can exist in society without anyone else knowing they exist. Soon you'll claim it's a violation of privacy to even look at other people.

<comment><USER29> > When you act in public, you are, definitionally, not acting privately. There > is literally nothing to be done about that

It seems that lately we're redefining what "public" is when it comes to the other privacy issues that are in the media spotlight recently.

> It's utterly silly to think that a person can exist in society without > anyone else knowing they exist

There's no disagreement there, but do note that in no way did I attempt to make that equivocation either.

There is a distinct difference between someone _knowing_ you exist or seeing you drive by, and someone having extensive, journalized, cross-referenced data, stored potentially indefinitely for perusal at their discretion. You will never know of this person or the fact they're studying your data.

Right now, there are only 3 states even pretending to place responsible restrictions on the usage of this particular data. That is the issue. Of course we will not be getting rid of ALPRs, but we can certainly make sure they're used for reasonable things. The potential for abuse is huge, and that's why there's outrage over the issue.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER30> I feel bad that I am not tracking anyone. It's so popular you see it every day now.

<comment><USER31> FOMO for 2013!

</comment></comment><comment><USER9> Ars Technica did a feature piece on ALPR systems about a year ago which was also discussed on here: [https://news.ycombinator.com/item?id=4387657](https://news.ycombinator.com/item?id=4387657)

</comment><comment><USER32> How do you feel about Onstar or Progressive insurance snapshot?

Progressive Snapshot: We look at your driving habits to see if you could be saving more. You can track your projected savings online. "We look, you track"

</comment><comment><USER33> Some comments about being a software engineer on an ALPR system.

[https://news.ycombinator.com/item?id=5962872](https://news.ycombinator.com/item?id=5962872)

</comment><comment><USER34> [http://intertraff.com/](http://intertraff.com/)

oh hey, at least they have olark :)

<comment><USER34> ah, it's actually a competitor..

</comment></comment><comment><USER35> It seems technology is always a double edged sword.

</comment><comment><USER36> thanks to those "red light" cameras...</comment>


Show HN: Edge, a Laravel Blade compatible template engine - <AUTHOR> https://github.com/ventoviro/windwalker-edge <comment><USER2> I wrote my own Blade compatible template engine many years ago as well.

[https://github.com/crodas/SimpleView](https://github.com/crodas/SimpleView)

It can compile the templates and generate a single PHP file, without needing any runtime at all.

I believe the simplicity of Blade is amazing.</comment>


Google AppEngine - A First Look - <AUTHOR> http://highscalability.com/google-appengine-first-look

<comment><USER2> I prefer the loose coupling of Amazon's web service components. But that said, this is certainly an interesting development with a lot to offer. Python wouldn't have been my language of choice but it's a logical choice for Google.

I giggle at the fact that your users will end up creating Google accounts. Very smart on their part, but feels more than slightly underhanded :). Then again, maybe I'm just bitter since I got waitlisted, ha!

<comment><USER3> Underhanded?

1) It helps me, because I no longer have to write auth/auth and registration code.

2) It helps users, because they no longer have to create yet another account to access some random app.

3) Yes, it helps Google, because it promotes other Google Apps by creating more Google accounts.

Seems like a pretty obvious strategy that works out well for all three parties.

<comment><USER4> Dare I say "Win-Win-Win!" The best choice for any negotiation. Quote Courtesy of Michael Scott.

</comment></comment></comment><comment><USER5> I'm confused about something. Mediawiki's PHP so, no luck?

</comment><comment><USER6> Good overview</comment>


How do you start a big project? - <AUTHOR>

I've got an idea for something I want to do for my family. That's it. That's all there is to it. I want to do a web app that can tell my family who is related to whom, how we are related (I have a really huge family), and allow us to share important announcements.
It's got a genealogy component; it's got a small social component.
My career so far has been 7 years of database guy, and 3 years of c# and python.
I've always worked on other people's stuff. Sure I have toy projects here and there.
My question is this: how do you start a new thing from scratch?
Everything is so important. There's login. There's security. There's the front end and how it looks, there's the backend. What's the best database structure. There is everything.
Where do you start? How do you get started?
I think I have all the skills, and I think I can do this. But I don't know where to start. <comment><USER2> Start small and grow it.

I don't have the link handy, (someone else, please post it!) but I've seen it on HN before. There is a site that has the screenshots of a lot of famous sites before they were famous. They're shit. Heck, I can remember when Facebook didn't use HTTPS and you could rip passwords out of the air of your local college student union.

Even if you don't intend for this to become famous, start somewhere!

Just start working on it, and aim to release something that a non-tech user can use every 1 or 2 weeks. Don't do a time period longer than that, or you'll get stuck back in analysis paralysis.

</comment><comment><USER3> Usually I start small and with only a golden rule: use the tools that will allow me to save time and money. This means using PaaS during the take-off and focusing on dev.

Since you already know python consider using a django boilerplate[1] for your webapp. You can deploy it for free with a nice plan (better than heroku) in GAE.

My 0.02

[0] [http://www.fullstackpython.com/django.html](http://www.fullstackpython.com/django.html)

[1] [http://appengine.beecoss.com](http://appengine.beecoss.com)</comment>


Show HN: Passmgr – Securely store passphrases and retrieve them via commandline - <AUTHOR> https://godoc.org/github.com/urld/passmgr/cmd/passmgr <comment><USER2> How does this compare to Pass [1]. I like its simplicity where all passwords are stored in regular files, makes it really easy to backup and move around. I have it synced between Android phone and laptop.

Only downside that I've seen users mention is that filenames are readable.

[1] [https://www.passwordstore.org/](https://www.passwordstore.org/)

<comment><AUTHOR> The main difference is, that passmgr uses authenticated symmetric encryption (aes-gcm) to store the contents in a single file, while pass as you already mentioned stores each secret in a separate file, using asymmetric encryption provided by gpg.

Storing secrets in separate files could leak metadata, especially when using a public git repository for synchronization between devices.

Other than that, you dont have to deal with gpg key management in passmgr.

<comment><USER3> gpg key management, especially hooking into gpg-agent, seems like a feature, really.

<comment><AUTHOR> I think there are many users or who dont want to setup gpg on multiple machines. I can also imagine scenarios where it is simply not possible to setup gpg on every machine. Maybe you could workaround such scenarios with portable versions of gpg or something like yubikey.

However i like the idea of having a single binary with a minmal set of dependencies, which can be moved around easily wherever i want to use it.

</comment></comment></comment></comment><comment><USER4> Whenever a product like this shows up here, I find myself waiting for the security guys to show up and explain why it's terrible.

<comment><AUTHOR> Lets wait together then. In the meantime i suggest this link: [https://team- sik.org/trent_portfolio/password-manager-apps/](https://team- sik.org/trent_portfolio/password-manager-apps/) Almost everything seems to be terrible.

<comment><USER4> Well, there's "has bugs" and there's "terrible and worse than useless", which is what seems to be the verdict, more often than not.

</comment></comment></comment><comment><USER5> Would need a whole bunch more features for me to want to switch from pass (passwordstore.org), which has already great git integration, command completion, stores arbitrary contents (not just username, password and url), a whole bunch of extensions and clients (I need an iOS client), can generate passwords, and more.

Also I think the 6 seconds for pasting the password before the clipboard is cleared is a bit too short, pass has 45 seconds which is a bit too much, maybe something between 15-30 seconds would be optimal, or better yet: Let the user configure it.

<comment><USER6> > I need an iOS client

Here you go: [https://github.com/mssun/passforios](https://github.com/mssun/passforios)

<comment><USER5> I meant that I need an iOS client as in pass has one (and I'm using it) but passmgr doesn't

</comment></comment></comment><comment><USER7> I've mused that a solid "delayed clipboard clear" feature would do some kind of atomic Compare-And-Swap, because otherwise the originating program could conceivably blow away something it didn't put there.

[https://en.wikipedia.org/wiki/Compare-and- swap](https://en.wikipedia.org/wiki/Compare-and-swap)

<comment><AUTHOR> Good point. Im going to implement this, although its probably not possible to make it an atomic operation.

<comment><USER8> On X11 you can do better: the clipboard isn't actually stored anywhere, the program requesting the paste gets the data directly from the program that claimed the clipboard.

The clipboard lib you're using just spawns xclip (or xsel) on Unix. xclip has a -loops argument that is the maximum number of pastes. So you could just spawn:

      xclip -loops 1 -in -selection clipboard

And the password would be available to paste exactly once. Unfortunately I don't think xsel has any such option.

There are clipboard manager apps that could grab the paste immediately and persist it, but I'm not sure how common they are or if they are default anywhere.

</comment></comment></comment><comment><AUTHOR> Thank you for all of your feedback.

I've already implemented some features based on your ideas:

\- configurable timeouts: [https://github.com/urld/passmgr/commit/069c209](https://github.com/urld/passmgr/commit/069c209)

\- basic filter functionality: [https://github.com/urld/passmgr/commit/3c4b1ed4](https://github.com/urld/passmgr/commit/3c4b1ed4)

But i think the UI needs improvement, and core features like master passphrase updates are still missing. Also the clipboard handling could be improved (depending on the platform).

Im not sure yet, if and how browser integration or iOS/Android clients could or should work with passmgr, but if i have time i will think about it.

</comment><comment><USER9> I wrote a bash script password manager[1]. It generates a random password, encrypts it with your gpg key, and hides it in a picture with steghide.

[1][https://github.com/jeremy21212121/bash-steganography- passwor...](https://github.com/jeremy21212121/bash-steganography-password- manager)

It was just an exercise to get better with bash, but I actually use it. It could be trivially modified to take a password instead of gpg.

</comment><comment><USER10> It sounds like a lot of us have been making password managers recently. Here's one I made a few days ago:

[https://github.com/ojensen5115/pw/](https://github.com/ojensen5115/pw/)

It uses a file in your private Keybase directory as its datastore (so obviously depends on Keybase). As such, it eschews the need for its own master password (or more accurately, it depends on your previous authentication with your Keybase master password), and thus can operate in an individual-run basis.

It could still use a decent amount of work (e.g. option to save a credential based on a randomly generated password, etc), but I've been using it myself recently and like it quite a lot.

<comment><USER11> > It sounds like a lot of us have been making password managers recently. > Here's one I made a few days ago:

> [https://github.com/ojensen5115/pw/](https://github.com/ojensen5115/pw/)

I made one as well. Same name, totally different philosophy. ;)

[https://github.com/gioele/pw](https://github.com/gioele/pw)

<comment><USER10> Hey, that looks really cool! Definitely something I'll want to look at in more detail when I get the chance :)

</comment></comment></comment><comment><USER12> I built a golang password manager a while ago, in the image of pass.

Shameless self pitch. It is a little slicker, not needing the master key to save sites.

[https://github.com/ejcx/passgo](https://github.com/ejcx/passgo)

</comment><comment><USER13> I'd need a good reason to use this over pass. Does it have something that pass doesn't?

<comment><AUTHOR> Featurewise, no (not yet). However i think my aproach, using a single encrypted file for all secrets is somewhat more secure as it does not leak information like what sites are managed by the password manager.

I guess you could also argue that by using AES, passmgr has the advantage of being post-quantum secure, if that is important to you.

</comment></comment><comment><USER14> Would it be illadvised to enable it to talk over a network to sync itself with hosts that you manually add? Or even an option to --sync to a remote git/GitHub repo?

<comment><AUTHOR> Personally i dont see the point in integrating synchronization features for now. There are already tools which are good at synchronizing files. And since everything is stored in a single file, it should be trivial to setup sync. E.g. you could do so with an alias like: alias passmgrgit='git pull; passmgr -file xxx; git commit -a -m "updated";git push'

</comment></comment><comment><USER15> c.f.: [https://github.com/rongarret/scache](https://github.com/rongarret/scache)</comment>


Bitcoin's Religion Problem - <AUTHOR> http://onemanmilitia.blogspot.com/2013/06/bitcoins-religion-problem.html

<comment><USER2> "...Bitcoin promises to be different: everyman is own bank, we can throw of the chains of Wall St. oppression, fees, financing of wars, bullshit financial advisors whom pimp you mutual funds into your strictly enforced 401k, and a litany of other advantages that only a technology, no a religion could provide, and most importantly, we can trade with one another like free people are supposed too, or at least like they did in the 1850's on Wall St."

You only notice your shackles when you've new room to move.</comment>


One guy, a film scanner, OCR and a lot of disks = 6 million pages of historical newspapers on line. - <AUTHOR> http://www.fultonhistory.com/Fulton.html

<comment><USER2> I very much hope that this site is not a joke, and that its hilariousness is purely unintentional. For the first time I'm glad that HN saves my upmods. It will take me repeated viewings to fully appreciate this webmaster's unique genius.

</comment><comment><USER3> I applaud the effort, however the scan quality is very low - it's barely readable, even to humans. Also he should highlight the search terms.

<comment><USER4> I agree. I would not read those scanned pages for very long, and I certainly can't visual scan the huge pages to find where my search term appears.

When I zoom in, it looks like the bitmap is over compressed with a DCT. There is quite a bit of edge ringing. Perhaps he scanned it better and just mutilated it in the PDF conversion.

</comment></comment><comment><USER5> Does anybody know what OCR software he used?

</comment><comment><USER6> it made my eyes hurt.</comment>


LXC Networking introduction - <AUTHOR> http://containerops.org/2013/11/19/lxc-networking/

<comment><USER2> LXC is interesting. It has been around probably for 5 or 7 years or so. At least I remember looking at it then. Then it just kind of made slow progress. It has been in the kernel since probably since 2.6.30, for example. But aside from kernel and virtualization forum discussions, it just didn't get much attention. That was kind of odd because it is a very cool piece of technology.

Then I guess something happened in the last year or so and all of the sudden it became very popular. dotCloud certainly has a lot to do with it. Was it better libvirt integration, too, perhaps: [http://libvirt.org/drvlxc.html](http://libvirt.org/drvlxc.html)

Now of course it cannot completely replace KVM because it is more of a container than a full virtualization. So running Windows VMs on will not work. For Linux one could probably have a farm of hosts based on various distros (with LXC enabled) and that would provide the ability to run various Linux OS guests, by picking the hosts that matches it.

<comment><USER3> I think the main reason it didn't take off until recently was because the kernel namespacing[0] work wasn't complete. Then, even when user namespacing was mostly complete, XFS still wasn't compatible until 3.12 [1].

[0] [http://lwn.net/Articles/531114/](http://lwn.net/Articles/531114/)

[1] [http://www.phoronix.com/scan.php?page=news_item&px=MTQ1Nzc](http://www.phoronix.com/scan.php?page=news_item&px=MTQ1Nzc)

<comment><USER4> Early implementations were quite buggy; I tried it in 2010 after a presentation at linux.conf.au and found that the tools simply segfaulted with monotonous regularity on a then up-to-date environment. It's made significant strides since then, obviously.

</comment></comment></comment><comment><USER5> Thanks for sharing this link.

LXC has gotten me very excited about testing new linux services on a home box again. I always worry about exposing a server to the internet with any new services. The idea that a compromise on the box could leak everything that's on the box usually leads me to avoid exploring new services.

LXCs seem to give me hope that I could experience a compromise, but not lose everything.

<comment><USER6> LXC is not secure at the moment. root in a LXC container can lead to root on the host. There is unfortunately no good summary of the problems - here is my list (take it with a grain of salt - a lot of these problems are mitigated in docker.io and with AppArmor in Ubuntu):

\- Without CONFIG_USER_NS and a newer kernel a lot of problematic things can happen. If /proc or /sysfs is mounted on the container DoS or escalation to root is possible: [http://blog.bofh.it/debian/id_413](http://blog.bofh.it/debian/id_413) \- At the moment no stock distro kernel has CONFIG_USER_NS enabled.

\- There are some issues related to remounting filesystems rw and altering files

\- Mounting cgroups in the container can also lead to problems - DoS and aquiring more ressources

\- Capabilities. You stock Linux distribution won't boot without CAP_SYS_ADMIN (see man 7 capabilities) - there are a lot of other capabilities that could be troublesome.

\- Not sure about this one: [http://seclists.org/oss- sec/2011/q3/385](http://seclists.org/oss-sec/2011/q3/385)

So for running services without CAP_SYS_ADMIN and with dropping a lot of other capabilities it can be considered somewhat safe. For everything else it's probably dangerous.

Not sure if all these issues are still a problem today but if you are running lxc on e.g. a current Debian Wheezy you have to know about all of them.

<comment><USER7> sure but you should not be mounting /sys and /proc in the container. just run one application. I do not understand the trend to run a whole Linux distro in a container. no one did that with chroots.

<comment><USER6> Why not? No virtualisation overhead, good io, memory and cpu and even network limits with cgroups and no extra committed ram for the vm. You just give team xyz a login and they run their favorite distribution and software without overhead. If you have a copy on write filesystem you even save more space and with lvm you have easy snapshots and backups. You can put a lot of users on a moderately fast machine this way. Thanks to lxc-attach it is also dead-easy to debug problems for them or install software. I'd love to have this possibility in the future.

<comment><USER7> Because 1\. right now it breaks security 2\. its a whole lot more to manage, and thats expensive.

I see the convenience argument, which is why people like docker, but basically adding a whole OS overhead to every process you want to run is basically insane in my view.

<comment><USER8> Docker doesn't require you to run an entire distro in your container. It's what a lot of people do out of the box because it's convenient and familiar. But as far as docker is concerned your container can be a single static binary in an otherwise empty directory.

There is a growing trend of people building micro-containers with just the bare minimum for their application. Docker is facilitating that trend, not preventing it. If only because it explicitly encourages thinking of containers as application-oriented, not machine oriented.

<comment><USER7> not saying Docker prevents it but it is hardly widespread. Unless you use Go or a few other languages your toolchsin won't even create static binaries for a start.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> A minor correction -- dotCloud the PaaS does not actually use Docker in production, as the article claims. It uses something pretty similar, which was the inspiration for the creation of Docker, but not Docker itself.

[Source: @solomonstre, in person at a Docker meetup a few weeks ago]

<comment><USER8> That's correct (I'm the @solomonstre in question :). We do our best to not imply the contrary by accident. Docker is a clean slate which incorporates all our operational learnings from dotcloud - but it is a full rewrite and this not yet production ready.

</comment></comment><comment><USER10> 10 pages to make one part of LXC understandable, is one of LXC's issues ;-)

<comment><USER11> ...plus now the enterprise needs to hire more expensive sysadmins to babysit all of it. That's exactly what enterprises are wanting to avoid.

Our sysadmin left after he deployed LXC "goodness" to make things "better" and we are still in a recovery mode from this.

<comment><USER10> thats an interesting point, but yes, LXC isn't all that simple/easy, it's the main issue. I don't know why they made it so complicated. The namespacing technology underneath is pretty straight forward.

I'm sure they'd argue for days how that's not true (plus some of the LXC folks actually implemented the namespacing) - but at the end of the day I make my "jail" with the "unshare" command and mount, much simpler..

</comment></comment></comment><comment><USER12> Great article! For me Docker has opened up what is great about the underlying LXC technology - a proper logical wall between apps on the same hardware. The networking was the hardest thing (for me) to grasp in Docker - this guide opens up a whole new level. It's cool however that if Docker won't do something networky I can drill down a layer to LXC instead.

</comment><comment> ezequiel-garzon Somewhat related, are there any cookbooks on how to "transform a heavy-duty server into several virtual machines"? By this I mean something that includes, among other things, mapping N available external IP addresses to N virtual machines. I realize a good prior understanding of networking, iptables, etc. would make sense. But... still... any well-detailed recipes out there? (I have tried to look them up, but everything seems to assume more knowledge than what I have in mind!)

</comment><comment><USER13> We are currently virtualizing our infrastructure's desktops with QVD and LXC. There have been a few challenges, but overall, it has been painless and exciting.

</comment><comment><USER14> A bit of a tangent, but I wonder what happened to lguest .. and how does (did) it compare to lxc?

Looked promising, but I never had success getting it going despite the promises on the webpage.

And it looks like little activity and still quite a low profile after all these years.

        http://lguest.ozlabs.org/
        http://en.wikipedia.org/wiki/Lguest

</comment><comment><USER15> I like LXC but for now all my virtualized stuff runs on KVM (if needed, like an old CentOS install with Oracle), or OpenVZ. I haven't sat down yet and compared all the features of LXC + Docker with OpenVZ yet, though.

</comment><comment><USER16> Fantastic article, super in-depth, and well written. One of the most in-depth guides to anything lxc related I've ever seen. Thanks for demystifying a little bit of lxc

</comment><comment><USER17> Are there nice GUI/web tools for configuring docker and LXC firewalling etc?</comment>