Math teachers should be more like football coaches - <AUTHOR> https://www.nytimes.com/2019/05/11/opinion/sunday/math-teaching-football.html <comment><USER3> I used to teach maths (undergraduate level) and I was also a snowboarding coach. These two roles aren't really comparable.

\- In my snowboarding groups, there were usually about five students, no more than ten. We would spend a full day together, and I had time to spend quite a lot of time with each student individually. In my math "exercise" classes, there were up to 25 students. I would see them 3 hours a week (they had other teachers as well).

\- Unlike math students, snowboarding students were all very motivated. Nobody forced them to be there.

\- Snowboarding students have usually quite similar level of abilities. Of course, some learn faster than others but nobody improves 10 times faster than someone else (which happens in maths). If a student is better, she/he can move to another group the following week (something impossible in my math class).

\- As a math teacher, I'm also the one who evaluates them, and who stands between them and their degree. As a snowboarding coach, I'm the one who helps them achieving their goal. Different dynamic.

\- I'm confident that anyone without disability can learn to be a decent snowboarder after a few weeks of training. But I think that a lot of students don't have what it takes to learn the mathematics curriculum they signed up for, no matter how you teach them.

<comment><USER4> > _nobody improves 10 times faster than someone else (which happens in > maths)._

This does not happen in math either. What does happen is some students have significantly more preparation than others (including, for example, introspecting more while doing their homework for several years, playing board games, solving logic puzzles, building with construction toys, ...) and do more or better focused practice during a course.

Better-prepared students already know a significant part of the material being presented in a course, have extensively thought about ideas which are similar, or have improved their generic problem-solving skills. They end up stuck behind fewer blocking misconceptions.

<comment><USER5> _> This does not happen in math either. What does happen is some students have significantly more preparation than others _

Are you saying people don't have inherent different abilities in mathematics? That's hard to believe based on real-world examples.

On the low end of abilities scale, you have people suffering from _dyscalculia_ [1]. You could repeatedly attempt to teach some of them algebra and calculus for _years_ and they'd never be able to master it. It doesn't mean they're stupid; It's just that math in particular is very difficult for them. There are 20-year olds in college that have been subjected to math education for _decades_ that still can't pass a remedial algebra class.

In contrast, on the high end, you have prodigies like Terrence Tao[2] that learned calculus at age 7. His brain literally burns less calories and requires less study time than most others to learn calculus.

[1] [https://en.wikipedia.org/wiki/Dyscalculia#Common_symptoms](https://en.wikipedia.org/wiki/Dyscalculia#Common_symptoms)

[2] [https://en.wikipedia.org/wiki/Terence_Tao](https://en.wikipedia.org/wiki/Terence_Tao)

<comment><USER6> I think the vast majority of math students don't fit the extremes you've mentioned. Personally I find it hard to blame anything but the system, at least in US schools. I began school in 2 Commonwealth countries and was a very average student, underperforming compared to my siblings. I moved to the US in 7th grade and was considered a genius, always getting in "gifted programs", etc. I was in possession of math books from my siblings high school and enjoyed them - ultimately finding that my bachelor's degree from an engineering school still didn't require math classes that got close to those books. So I have a hard time saying it's inherent ability holding math classes back.

<comment><USER7> What are some topics which were covered in your siblings' high school math textbooks which weren't required for your degree from a university? (You say "from an engineering school", but it's not clear if it was an engineering degree.)

<comment><USER6> Degree was computer engineering - less calculus than mechanical engineering, but all degrees from that department had a base standard of some 200-level courses from the math dept. for science & engineering majors. Linear algebra was probably most lacking - as it only covered system of equations and basic matrix operations and properties. Eigenvectors, subspaces, etc. I've had to learn all that myself. Calculus never actually exceeded AP calculus as taught in a US high school. Statistics included a pretty similar set of concepts, though the problems were much simpler (fewer layers of conditional probability) and didn't cover quite as many example of tests & methods, etc.

</comment></comment></comment></comment></comment></comment><comment><USER8> For those debating reading the piece -- do it! John Urschel wrote it, and he's probably the most qualified to do so.

I think his point is broadly correct: the way we teach football, and the way talented football players get mentored in high school, often puts mathematics to shame. Two responses come to mind:

1\. The incentive structures are different for high school math teachers and high school football coaches. If a high school football coach produces an elite college (or pro!) prospect then colleges will pay more attention to the coach. Get enough attention and the high school coach might become a college assistant, a college head coach, etc. There is a very real personal incentive for high school coaches to produce good players.

In contrast, a high school math teacher who produces great math undergrads will probably get student appreciation and little else. Maybe they'll get to work with the math olympiads or something, but it's not like a college will recruit them as tenure-track faculty. Most great high school math teachers are doing it for pretty pure reasons. Admirable, but hard to expect from rational people.

2\. Another big problem is that there are _very_ few high school math teachers who have a good idea of what mathematicians do, or have strong math backgrounds in general. Makes sense given that most people with strong math abilities can make much more money doing something that's not teaching. Plus, based on almost a decade around people who study, research, and teach math, it's a pretty small fraction of them who get excited about teaching math to people who, for the most part, aren't into it anyway. In contrast, outside of very small schools, most high school football coaches love football, even if they didn't necessarily play at a high level.

These factors combine to make math teachers not like football coaches.

I don't have a great solution to either problem. I do wish there was some better, more organized way for bored and retired math people to help teach middle and high schoolers. I think for many such people the desire is there, but they're not going to start a whole organization and overcome a bunch of administrative hurdles (which, apparently, are particularly offensive to many people who like math) to do it.

<comment><USER9> > or have strong math backgrounds in general. Makes sense given that most > people with strong math abilities can make much more money doing something > that's not teaching.

That argument supposes that high school teachers ended up with that career by looking at all job prospects and picking the one with the best economic prospects. Based on my experience, exactly none of them did that, in any field. My sophomore English teacher used to work at an aerospace company, and my biology teacher studied to be a physician, for example.

> Most great high school math teachers are doing it for pretty pure reasons. > Admirable, but hard to expect from rational people.

Exactly so. Until we can reliably identify success in a field (like football wins), and then make the economic incentives align to promote that success, we can't hope to increase the number of great people in a field by economic metrics.

<comment><USER10> My dream is to be independently wealthy and become a high school math teacher. I love math, I enjoy teaching, and I'm pretty good at it (I spend a good deal of time tutoring various youth in our church group).

Why am I not a math teacher? Because I make 5x what a teacher makes. And I don't want to raise three kids on a teacher's salary.

But I dream of being a teacher. Help kids out for nine months and spend three months hiking and camping with the family. But until then, I'll just make a very comfortable living doing quant marketing and building out Bayesian models...

I enjoy my work enough that the opportunity cost is not worth the forgone revenue to teach. Not yet, any way...

</comment></comment></comment><comment><USER11> The key difference that's being overlooked between math teachers and football coaches are the populations they are working with. Football teams consist of competitive kids that WANT to play the sport and perform well (win). On the other hand most math classrooms consist of kids that HAVE to be there against their will, regardless of their interest in math.

<comment><USER12> I have to confess I haven't read the article. My first thought though was this would work only if the participants in Maths lessons are as keen as football players. Football(rugby in my case) is fun, even now as a middle-aged man I have vivid memories of playing rugby but hardly any memories cracking hard maths problems. I was motivated to play sport, it was fun. Maths was fun but not to the same level as a sport.

<comment><USER13> Casual phone games have more mathematics involved than people appreciate. Even if that involves figuring out that you will die before completing the game unless you pay for it.

But, regardless of their merits, casual games are insanely popular for various reasons.

And not just casual games. All sorts of games involve mathematics.

That may not be your case, but a lot of people have fond memories playing SimCity or min-maxing their paladin.

<comment><USER14> I wonder what's the relationship with Simcity with Math? It's just a game with building keep growing and some random disaster.

<comment><USER13> Budgeting. Tax rates. Growth rates. Density. Efficiency. Geometry in the planning.

Most can intuitively guess and check to some success, but there is a ton of math involved even if most players don't bother to formalize it.

</comment></comment></comment></comment></comment><comment><USER15> Former math teacher here and basketball coach. I agree with the article. My training as a teacher emphasized fostering investment from students just as much as it did lesson planning. All of these things are are really important.

But one of the toughest problems in teaching math is that, unlike sports, a student doesn't get a whole lot of feedback on their success and improvement in the short-term, besides grades. And grades have some major cons when it comes to motivation. Also, most of us don't use the exact skills from math class in life outside of class. What we use is the mindset of finding first principles, of abstraction, of modeling, and of problem solving. It's quite difficult to connect these skills to the mentality of kids and adolescents.

This is where extracurriculars come in, in my opinion. They teach the meta- skills of improvement mindset, coachability, and work ethic. I saw first-hand how my players came to class with a different mindset when sports were in season.

I have to say, it sounds like Mr. Urschel benefited from having good sports coaches, while having math teachers that maybe weren't so good at the motivational part. A lot of folks have the opposite mix. So I wouldn't say there's necessarily an underlying thread that generalizes to all of American education.

</comment><comment><USER16> The workload of a football coach is probably much higher than that of a mathematics teacher. They are up and ready for morning practice early and stay late after school for more practice. They review videotapes of games, create strategy , and travel with students.

I doubt there are many mathematics teachers that have the same dedication for the success of their cohort as football occurs , on average.

<comment><USER17> Football coaches also don't have to deal with players that don't want to be in the team. It's much more rewarding. You can't compare The two.

<comment><USER18> They can also cut players.

The closest a math teacher can do is give the minimum necessary grade for the student to move on to the next class at the end of the year.

<comment><USER19> Failing a student _should_ be possible.

</comment></comment></comment></comment><comment><USER20> Math teams are a thing. They even have a coach just like football. Also football gets played in regular physical education also, with much less individual attention to students.

<comment><USER21> Yup, I was on the math team at my HS and we won state my senior year. Certainly not as rah-rah as football (which I was also in) but it was competitive and we all took it pretty seriously (obviously). Winning the state competition was just as satisfying than any playoff victory I had in football (perhaps more so).

</comment></comment><comment><USER22> Football coaches are helping students achieve their dreams.

Math teachers are living out hell - teaching a broken curriculum to students who don't understand why they have to suffer through it.

Name 5 superstar athletes. Name 5 superstar scientists. Fix that first, the rest will follow.

</comment><comment><USER23> You mean, they should only have to deal with people who are motivated enough to volunteer to spend extra, non-mandated time at school and who are supported enough that someone is paying extra money for them to participate, in an activity that while not mandatory produces broader social approbation for success than math?

Football coaches aren't generally better at motivating people than math teachers (heck, they are sometimes the exact same person.) They are just operating in a context that favors having more motivated people to start with and accepts them further weeding out the unmotivated.

</comment><comment><USER24> I agree with the general idea that coaches are able to provide a much more personal and thorough training than the traditional teacher, but that's not a good thing for students and society in general. The reason is because students should have a right and an expectation to equal access to education. The reason we see very little diversity in the history of mathematics and science is because of this unequal access to mentors in particular, and education in general. (Most of the outstanding non-white and/or non-male mathematicians had white male mentors or sponsors that helped them get into the Old Boys Club.) When you have a monoculture in a field, then people tend to follow societal and cultural norms. This tendency impedes progress at best, and can cause disastrous blindspots at worst. The current American educational systems is an imperfect attempt at providing all students with equal opportunites. If we'd like to improve upon this, then let us improve the teacher to student ratio, rather than leaving it up to new hybrid teacher-coaches to try to identify MVPs at a young age, potentially allowing late-bloomers and marginalized groups to slip through the cracks.

<comment><USER25> I'm a math professor, a white male, I was something of a child prodigy, and I was _enormously_ blessed with math coaches who went far out of their way to help me.

I agree wholeheartedly that opportunities need to be made more available to everyone:

\- Roughly 80-85% of those who get their Ph.D.s in math are male. That's a depressing statistic, and we need to do better.

\- Nearly all Americans who get their Ph.D.s in math are white. African- Americans and Latino/a Americans are underrepresented. Moreover, perhaps surprisingly, so are Asian-Americans. (There are many Asians in math, but usually they grew up in Asia and only came to the US for university or grad school.) Here, too, we must do better.

That said, your comment strikes me as rather nihilistic. Because opportunities aren't equally available to everyone, they shouldn't be made available to anyone?

Most people aren't like Urschel: most students won't go on to love math or excel in it, no matter how many opportunities they have. But some people will. Those people are spread across ethnicities, across genders, across socioeconomic groups, all over the world. We need to do a better job of finding and nurturing talent from diverse sources. And we need to make individual mentoring more widely available, not less.

Have I misunderstood you? Am I saying things you disagree with?

<comment><USER24> I really only disagree with your characterization of my point: "Because opportunities aren't equally available to everyone, they shouldn't be made available to anyone?"

I'm not saying opportunities shouldn't be made available to anyone, just that a stronger mentorship system than we already have in scientific higher- education is likely to reinforce the trend of white-male dominance that we already have, rather than temper it.

</comment></comment></comment><comment><USER26> I think the fundamental difference here is that football is inherently competitive while math is not. While it's true that football coaches have incentives to produce great players that will eventually play in college/professionally, the most immediate incentive is to have players that will win games. Math teachers have no similar competitive incentives. The "passion" that Urschel mentions here is probably largely a product of the desire football coaches have to win every Friday/Saturday night, and this is reflected in the way that they develop kids. Additionally, Math teachers have to both teach kids AND grade them. This limits how much a teacher can really be "on your team" because they need to retain the ability to give you a bad grade. This is true with football coaches as well, but with football there is always a bigger "bad guy" (other teams/players) so a good coach is always more aligned with and close to the players.

</comment><comment><USER27> "... it didn't stop my coaches from encouraging me to believe I could reach my goal, and preparing and pushing me to work for it. When they told me I had potential but would have to work hard, I listened. I heard their voices in my ear when I dragged myself out of bed for predawn weightlifting sessions ..."

How many adults can clearly communicate the incentives for predawn calculus sessions?

"You'll do better in college!"

<comment><USER28> "You'll have a chance at landing the dream job of the richest person on Earth"? (Bill Gates, member of the technical staff at Bell Labs in the late 1940s.)

<comment><USER27> "You might play in the NFL one day!"

</comment></comment></comment><comment><USER29> "Mr. Urschel is a Ph.D. candidate in mathematics and former professional football player".

That is, in essence, the only line you should read. I also have many opinions about how others should do their work. They are doing it all wrong.

</comment><comment><USER30> A plurality of the math teachers in my high school had degrees in PE, and two of them _were_ the football coaches, so, uh ....

</comment><comment><USER20> If advertisers can individualize teaching consumers about their products, maybe we just need an advertiser to start teaching math.

<comment><USER31> As someone who works in the field of digital advertising, please god no. Advertisers have zero idea what they're doing.

</comment></comment><comment><USER32> Like shouting and motivational.

Coach: you will put in 110%

Student: 100% is the max

Coach: you are cut off the team. Who else thinks 100% is the maximum?

</comment><comment><USER33> I'm sure math teachers would love to get a million dollar salary. Not quite sure whether they would be willing to leave their students with irreversible brain damage for the sake of their career, though.

[https://careertrend.com/much-average-division-football- coach...](https://careertrend.com/much-average-division-football-coach- earn-28837.html) [https://en.wikipedia.org/wiki/Chronic_traumatic_encephalopat...](https://en.wikipedia.org/wiki/Chronic_traumatic_encephalopathy_in_sports)

</comment><comment><USER34> Math teachers should be just like football coaches: well-paid, especially when they work at prestigious universities.

</comment><comment><USER35> Is this saying that they should be paid huge amounts of money?

</comment><comment><USER36> My high school calculus teacher was the football coach.

</comment><comment><USER37> Students should be more like jocks: giving a dam.

<comment><USER38> That probably has a lot to do with it. It's hard to maintain enthusiasm over the years if it isn't being returned. I think the lack of passion from the students is more about the voluntary nature of the football team than anything inherent to the jocks.

<comment><USER39> The rewards are different, too. There is no level of mathematical achievement that will get the kind of broad (but typically local) social status and public approval of being a starting high school quarterback. Math just doesn't tap into anything near as easy and primal.

<comment><USER38> That's true, but I've seen years where an enthusiastic professor gets a packed room for a normally undersubscribed elective course like computational complexity theory or advanced algorithms.

<comment><USER8> True, but there's a heavy selection going on there. A packed complexity class might mean 50? enthusiastic undergrads out of several hundred math/cs students at a university.

A dedicated and charismatic professor might be instrumental in turning an interested computer science student into a raging algorithmist (indeed, I actually think a lot of computer science grad students study what they study because somebody appropriately passionate introduced it to them), but converting a just-here-for-the-requirement high schooler is a different story.

<comment><USER38> The football team also has a heavy selection bias.

Personally, I got into CS because there was a required intro class at my undergrad. A dedicated and charismatic professor noticed my aptitude and encouraged me to take the next course and consider a CS major. This could easily have happened in high school if my high school had a required CS course.

</comment></comment></comment></comment></comment></comment><comment><USER40> Hell, my math teachers in high school were the coaches.

Geometry - <USER2>

Algebra II- Football

Trigonometry-Track

I wish I had a math teachers who taught math as a main function, not a secondary.

At least my daughters middle school math teacher has a BA in math.

</comment><comment><USER41> Perform brain imaging, split students into those with a strong math co- processor brain and those without. For those without, teach them math using logic rather than rote learning which is obvious to those with a strong math brain.

<comment><USER42> What an absurd, elitist, reductive, sci-fi response. Brains don't have ALUs in them, btw. Nor can we image for different skills. Nor should we subject people to this kind of dystopian hierarchy of faulty biological determinism.</comment></comment>


Django's tipping point - <AUTHOR> http://antoniocangiano.com/2008/03/20/djangos-tipping-point/

<comment><USER2> I'd love to use Django at my day job. Python makes me happy.

<comment><USER3> Where do you live? I know plenty of places in the UK (especially London) that are hiring...

<comment><USER2> San Diego, California. I'm moving back to Silicon Valley in a few months (my hometown).

<comment><USER4> <http://djangogigs.com/gigs/country/united-states/></comment></comment></comment></comment>


You Can Help Build the Future of Firefox with the New Test Pilot Program - <AUTHOR> https://blog.mozilla.org/blog/2016/05/10/you-can-help-build-the-future-of-firefox-with-the-new-test-pilot-program/ <comment><USER2> It's not explicitly stated, but it seems to work fine with Firefox Developer Edition as well as regular Firefox.

<comment><USER3> Yeah, we're mainly targeting release Firefox, and Dev Edition has a couple of rough edges (mostly cosmetic). But, feedback from all release channels will be useful & welcome.</comment></comment>


Windows 10 goes full malware - <AUTHOR> https://slashdot.org/submission/5878755/windows-10-goes-full-malware <comment><USER2> And everyone is happy in the world of "but I would be temporarily inconvenienced if I had to switch OS".</comment>


Another Reason to Use an Ad Blocker: Malvertising Has Tripled This Year - <AUTHOR> http://motherboard.vice.com/read/another-reason-to-use-an-ad-blocker-malvertising-has-tripled-this-year

<comment><AUTHOR> anybody tried to surf the web on an Android device WITHOUT ad blocker? It's a mess.

</comment><comment><USER2> Disturbing article.</comment>


Autobiographical memory is typically biased toward positive events - <AUTHOR> https://psycnet.apa.org/record/2019-31742-001 <comment><USER2> We can thank our lucky stars that the universe at least gave us that. Could you imagine the other way around?

<comment><USER3> > _Could you imagine the other way around?_

Many of us don't have to.

<comment><USER4> Couldn't have asked for a better answer to "is my brain doing something wonky or am I just an edgy slacker?"

</comment></comment></comment><comment><USER5> When I was a public school teacher in Baltimore (ten years later, I still start so many of my sentences this way), I would have days that were completely insane. One of my many responsibilities was to call parents to discuss behavior issues at the end of the day.

I realized at some point that I was literally forgetting what happened. I took anecdotal notes to be able to accurately describe behavioral situations, and rereading them mere hours later, I couldn't actually autibiographically recall many of these incidents.

I think it's what allowed me to have normalish evenings after insane work days. Otherwise, the stress would be completely overwhelming.

</comment><comment><USER6> Ted Chiang's short story 'The Truth of Fact, the Truth of Feeling' is very relevant.

[https://en.wikipedia.org/wiki/The_Truth_of_Fact,_the_Truth_o...](https://en.wikipedia.org/wiki/The_Truth_of_Fact,_the_Truth_of_Feeling)

</comment><comment><USER7> This can lead to bias about whole categories of experience, for instance, starting a business. If you only hear the success stories then you start to think it is easy. You might intellectually be aware that 90% of all businesses fail, but that anecdote doesn't carry as much emotional weight as hearing people's joyful stories of success. That's why I think it is important that we document some of the failures and we do it while the memories are still fresh and when we still have access to documents such as email and Slack messages that can offer the gritty details of what went wrong. That ideal if accuracy and specifics are very much what guided me when I wrote this:

[https://www.amazon.com/Destroy-Tech-Startup-Easy- Steps/dp/09...](https://www.amazon.com/Destroy-Tech-Startup-Easy- Steps/dp/0998997617/ref=as_sl_pc_tf_til?tag=smashcompany-20&linkCode=w00&linkId=2292ad04b2b6e409785d2e4cdd570c70&creativeASIN=0998997617)

</comment><comment><USER8> Of course it is. The people who write autobiography are the ones who are telling their own story. They are heavily invested in telling only the good parts.

Which is why I liked Warren Buffet's approach, let the writer write after talking with people. No interference. Of course, his is a biography.

<comment><USER9> This article is about autobiographical memory / recall, not autobiography as a genre of literature.

<comment><USER8> Isn't autobiography a written account of autobiographical memory?

<comment><USER10> No: one could easily imagine a world where you remember all of the bad stuff, but are still incentivized to not write it down; an autobiography is an edited work of text, while autobiographical memory is a capability of a human.

<comment><USER8> Yet autobiography is written by the person telling her own story.

You could remember bad stuff but you are not incentized to write it down.

Then how are they both different?

<comment><USER11> Dude, it's saying the things you remember about your history are biased positive. There is no writing down, there is no telling, there is no other person involved here. _You_ are remembering your own past. In this past story, you bias positive in the things you remember.

<comment><USER8> And when you remember it positively, would it not translate to written autobiographical accounts to be biased?

Won't there be a correlation?

Or am I missing something?

<comment><USER12> Your memories are a function of what actually happened. What you write is a function of what you remember. Two separate processes.

Bias on how you retell your own story is, as you say, a fairly obvious concern, so (for the purposes of this discussion) written autobiographies being biased is completely uninteresting.

Rather, it's the fact that your recollection of events biases towards the positive that's the novel, interesting information.

<comment><USER8> So you agree there is a correlation and yet you somehow disagree with me. Let's agree to disagree</comment></comment></comment></comment></comment></comment></comment></comment></comment>


Anti-science advocates are freaking out about Google truth rankings - <AUTHOR> http://www.salon.com/2015/03/06/anti_science_advocates_are_freaking_out_about_new_google_truth_rankings/

<comment><USER2> "Facts the web unanimously agrees on are considered a reasonable proxy for truth"

Mob rule has never been a proxy for 'truth'. I'm not an anti-science advocate, but I think this is a bad idea. Almost every day, I see pictures on Facebook re-posted as truth by hundreds (and sometimes thousands) of people, when a quick submit to Google images shows me it's complete BS.

Even the mainstream news pushes false and politicized stories as fact. Enough people see this as fact and will continue to believe so. Who decides what is the actual truth?

It's strange how open and free Google (and the tech community) claims to be, but will jump at the chance to block and silence dissenting opinions. It's actually not strange, it's scary.

"meaning anti-vaxx propaganda will not appear in the top results for a "measles" search, for instance"

I'm curious to see what kind of information will be seen as "propaganda", because this right here tells me this will be mostly politically based. Scientists also disagree on facts all the time. Which ones will be raised as "truth" and how can we stop corruption?

In other areas, how do we determine who is am "expert"?</comment>


Startup Therapy: Questions to ask yourself every month - <AUTHOR> http://blog.asmartbear.com/startup-business-plan.html

<comment><USER2> I can't believe how happy about getting virtually splashed in the face I am.

Great perspective reset questions to help us keep our eyes on the prize and ingrain solid startup habits.

<comment><USER3> Glad you liked it! :-)

The idea isn't to tell you what to do, but rather to make sure you're doing things on purpose rather than by accident. And sometimes we're so close to daily activities that we don't stop to ask the bigger, important questions.

</comment></comment><comment><USER4> _If someone handed you $100,000 today, how would you spend it to maximize future profits?_

I still find turning off the tight-fisted switch to make strategic investments difficult. Warren Buffet makes it look easy -- and the author's routine prompting is of course superb -- but they remain as poorly-reconciled opposites in my head. Until there's enough cash for delegation and perspective...

<comment><USER3> I completely understand your hesitation! I feel the same way. We used to say we straightened paperclips to save money.

I would say: Don't turn off the switch! Not ever.

Maybe a better way to think about it, especially at the beginning: Let's say you get a big order or have a big month and you have just an extra $1000. Then how to spend? Is there a particular ad campaign you'd like to try? Offload some crappy work for 1 month and see how much time it really frees up? Etc.

That is, make it $1k instead of $100k. Maybe more practical?

<comment><USER4> Indeed, it's very practical to open the spigot slowly as a way around the two ways of thinking, penny-pinching vs labor/resource abstraction.

I would quibble about turning off the switch. It's fine to hand that function to someone else (CFO-type), but when I've had nice budgets (e.g. 5mil) it's been necessary and efficient to ignore the spending noise to some extent and focus on the big picture. But that might be a personal fault.</comment></comment></comment>


How Should You Write a Fast Integer Overflow Check? - <AUTHOR> http://blog.regehr.org/archives/1139 <comment><USER3> We can take advantage of two's complement arithmetic, plus the knowledge that the overflow flag is the xor of the carry into the sign bit with the carry out of the sign bit, to devise a simple expression:

        int checked_add_n(int_t a, int_t b, int_t *rp) {
            uint_t s = (uint_t)a + (uint_t)b;
            *rp = s;
            return ((a ^ s)&(b ^ s)) >> (BITS - 1);
        }

This is essentially doing the same thing as checked_add_3. First, we extract the carry into the sign bit with

        ci = s ^ a ^ b; // upper bit of ci contains the relevant bit

Carry out can be found via the usual majority expression:

        co = ci&a ^ ci&b ^ a&b;

Xoring the two, and simplifying using De Morgan's laws and their relatives results in the expression above.

Although the metric is far from optimal, clang gives me 6 arithmetic instructions (minus the ret): [http://goo.gl/Amdo93](http://goo.gl/Amdo93)

<comment> torrent-of-ions You can't take advantage of two's complement arithmetic if you want portable C, though.

<comment><USER4> Are there any machines in widespread use today that _don 't_ use two's complement? I guess if there were, we'd hear a lot more about this issue... I know that some mainframes are an example, but they're somewhat of a niche.

<comment><USER5> Signed integer overflow is undefined in C, so optimizers can do really weird things to code that has overflows.

<comment><USER6> Yes, see: [http://people.csail.mit.edu/nickolai/papers/wang- undef.pdf](http://people.csail.mit.edu/nickolai/papers/wang-undef.pdf)

<comment><USER7> And see John Regehr's ( [http://blog.regehr.org/](http://blog.regehr.org/), of the headline article) other posts. He's written a great deal about undefined behaviour in C/C++ and how modern compilers handle it. Check the 'Compilers' and 'Software Correctness' categories.

</comment></comment></comment></comment></comment></comment><comment><USER8> I think "smallest number of instructions" is a pretty weird metric to use if you are going for "fast" solutions.

checked_add_3() has a lot of instructions that can be executed in parallel. checked_add_4() has a lot of conditional code which can be skipped. (Branching might be expensive in that case, though on the other hand overflow is rare, so these branches can probably be predicted accurately.)

I have little doubt that the assembly version is fastest in practice, but to compare the other versions, they should be benchmarked.

</comment><comment><USER9> EDIT 2: Editing...

EDIT 1: as spotted by @pbsd this was not working good with some values...

I have added a way of checking the overflow and I have tested it:

      #define BITS (8*sizeof(int))-1

      int checked_add(int a, int b, int *rp)
      {
          *rp = a+b;

          return (a^b) < 0 ? 0 : (a^(*rp)) < 0  ? 1 : 0) ;
      }

It is around 30% faster than the fastest in the blog and I believe the code is way cleaner. What we are doing is checking if all of the values has the same sign (the "^" is just a "xor"), if so, no overflow in other case, overflow.

This is the testing program (g++ compliant):

      #include <stdio.h>
      #include <time.h>
      #include <stdlib.h>

      #define BITS (8*sizeof(int))-1

      int checked_add2(int a, int b, int *rp) {
        uint ur = (uint)a + (uint)b;
        uint sr = ur >> (BITS-1);
        uint sa = (uint)a >> (BITS-1);
        uint sb = (uint)b >> (BITS-1);
        *rp = ur;
        return
          (sa && sb && !sr) ||
          (!sa && !sb && sr);
      }

      int checked_add(int a, int b, int *rp)
      {
          *rp = a+b;
          return ((a^b) | (a^(*rp)) < 0)  ? 1 : 0 ;
      }

      int main(int argc, char* argv[])
      {
          int a, b, c;
          long clong;
          srand(time(NULL));

          for(unsigned int i = 0; i < 50000000; ++i)
          {
              bool overflow = false;
              a = rand();
              b = rand();
              clong = (long)a + (long)b;

              overflow = checked_add(a,b,&c);

              if(clong != (long)c && !overflow)
                  printf("Overflow not detected %i + %i = %i \n", a, b, c);

          }

          return 0;
      }

Time for "checked_add": 2.694s

Time for "checked_add2": 3.200s

This is the dump (6 instructions so far): [http://goo.gl/cnBKdS](http://goo.gl/cnBKdS)

<comment><USER3> Your function is incorrect. Consider the case a = b = 0x80000000.

<comment><USER9> Ok, I think this should do the trick:

return((a ^ b) | (a ^ (*rp)) < 0) ? 1 : 0;

doesn't it?

<comment><USER3> Not yet. Consider a = -1, b = 1. Due to the OR, any a and b with different signs will return overflow.

<comment><USER9> Ok this is my dumbest time in life xD (I just directly didn't take into consideration that part because it never has overflow...)...

<comment><USER10> Most people don't appreciate how complicated C's mix of signed and unsigned is, especially when undefined behaviour of signed overflow is included.

<comment><USER9> Definitely.

Child, this is why I always add unit tests.

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> It has always made me wonder why there is no direct way for checking for overflow in most languages, since most CPUs _do_ have an overflow flag. Dedicating an operator or keyword is probably too much, but at least it can be a routine in stdlib that usually gets inlined.

I have also come to think "hey this perhaps could be a security hole" every time it comes to unchecked overflow. Does anyone know such exploits?

<comment><USER12> The most amazing achievement of the computer software industry is its continuing cancellation of the steady and staggering gains made by the computer hardware industry.

            — Henry Petroski

</comment></comment><comment><USER13> The assembly subroutine is not optimal because the compiler can not inline it. A ASM expression [1] is much better.

Also, it is weird to write the result to memory just to avoid to repeat the addition. A memory operation is quite complex, but there a few things simpler than adding two registers. A better abstraction would be to write a function that returns the overflow bit and nothing else. The addition itself is (almost) free anyway.

[1] [http://gcc.gnu.org/onlinedocs/gcc-3.4.6/gcc/Extended- Asm.htm...](http://gcc.gnu.org/onlinedocs/gcc-3.4.6/gcc/Extended-Asm.html)

</comment><comment><USER14> The assembly version looks simpler and more maintainable than any of the C versions. What's the overhead of using it? Having to rewrite for each architecture you target (likely to be only 2 or 3)? If this level of optimisation matters to you then I guess you'd be down in assembly anyway?

<comment><USER15> Even if you could write perfect assembly, sticking it in your program makes it more difficult for optimizations to flow across the assembly you have written because it is opaque to the compiler.

Better yet, imagine that the compiler has performed value range propagation [1] to determine that the range checks, after inlining, were redundant or simply elidable. This would be infeasible if it were written in assembly.

[1] [http://llvm.org/devmtg/2007-05/05-Lewycky- Predsimplify.pdf](http://llvm.org/devmtg/2007-05/05-Lewycky-Predsimplify.pdf)

<comment><USER4> If compilers don't understand inline Asm, maybe instead of avoiding it, why not try improving the compiler so it can understand? I know this can't be done if e.g. you use newer instructions that it doesn't know about and wouldn't emit anyway, but in the case of faster, shorter sequences using existing instructions and having an equivalent in C, I don't see why it can't "decompile" the instructions you write and use that information for optimisation. Even better if the compiler can _learn_ from it and use that faster/shorter instruction sequence when it sees that expression in the future.

 _Better yet, imagine that the compiler has performed value range propagation [...] This would be infeasible if it were written in assembly._

VRP doesn't depend on the instruction set; it could be LLVM IR or it could be x86 or ARM or 6502 or anything else. It's just looking at values and operations on them.

<comment><USER16> Huh? ... decompile the optimized instructions to what? There is no "equivalent C". That's the entire point. Otherwise someone would just write it in C and the codegen would emit the optimized instructions.

<comment><USER17> > Huh? ... decompile the optimized instructions to what?

To the compiler's own IR, whatever that is.

</comment></comment></comment></comment></comment><comment><USER18> The phrase "premature optimization" gets thrown around a lot, but it seems to me we'd avoid this whole discussion (and benchmarking, and testing) if we could safely and easily inline the assembly version of the function. After all, a machine instruction is just a library function burnt into hardware.

Maybe a better term is "premature nonportability".

</comment><comment><USER19> From `The CERT Oracle Secure Coding Standard for Java`

public static int safeAdd(int l, int r) throws OverflowException {

        if (r > 0
                ? l > Integer.MAX_VALUE - <USER2>
                : l < Integer.MIN_VALUE - r)
            throw new ArithmeticException(String.format(
                    "Integer overflow: %d + %d", l, r));
        return l + r;

}

[https://www.securecoding.cert.org/confluence/display/java/NU...](https://www.securecoding.cert.org/confluence/display/java/NUM00-J.+Detect+or+prevent+integer+overflow)

<comment><USER20> In java 8 you should probably use the Exact methods in java.lang.Math (In cases where the size is int or long and overflow errors need to be detected, the methods addExact, subtractExact, multiplyExact, and toIntExact throw an ArithmeticException when the results overflow)

</comment></comment><comment><USER21> As a similar but slightly different problem I wrote this article about testing whether a value can be converted from one type to another without loss: [http://blog.reverberate.org/2012/12/testing-for-integer- over...](http://blog.reverberate.org/2012/12/testing-for-integer-overflow-in- c-and-c.html?m=1)

</comment><comment><USER20> I would have expected to see the "jo" instructions in the assembly. Does anyone know why it was not used?

<comment><USER22> Because the function doesn't have anywhere to jump to. Instead it generates zero-or-one from the overflow flag (with seto) and the caller will then do something like test eax, eax/jnz handle_overflow.

In other words, the abstraction boundary introduced by the function call mechanism gets in the way of using the overflow flag directly with jo. Inlining would, presumably, re-expose the opportunity.

<comment><USER20> That is another part I missed, the ASM implementation seemed to optimise the function while potentially deoptimising the whole program.

For most code overflows are rare, jumping to deal with the rare cases but otherwise just doing the normal is often best for whole program performance.

But to be honest I am not current in assembly as I like managed languages to much.

</comment></comment></comment><comment><USER23> For the referenced libo, the file overflow.c ([https://github.com/xiw/libo/blob/master/overflow.c](https://github.com/xiw/libo/blob/master/overflow.c)) surprised me: very short, and with a curious #include as its last line.

Anybody know the logic behind this?

</comment><comment><USER24> Some compilers (MLton, fe) have static analysis in place to skip the check when no overflow can occur.

<comment><USER22> Note that MLton compiles a language which requires int arithmetic to raise an exception on overflow, the overhead of which provides a strong motivation for such analyses.

</comment></comment><comment><USER25> I saw Regehr and knew it had to be the U prof.

<comment><USER25> Lol, -3 for this? Must be some BYU fans up late...</comment></comment>


Registration for PyCon 2012 Opened, Tutorials Announced - <AUTHOR> http://pycon.blogspot.com/2011/12/registration-for-pycon-2012-opened.html

<comment><AUTHOR> If anyone has any PyCon related questions, I'm always around to answer them!</comment>


Y Combinator's Anyvite Takes On Evite, Keeps Things Simple - <AUTHOR> http://www.techcrunch.com/2008/07/01/y-combinators-anyvite-takes-on-evite-keeps-it-simple/

<comment><USER2> How are you guys planning to get traction? This is a _really_ crowded field, and adoption doesn't appear to be driven by rational behavior. For example, Invitastic (<http://invitastic.com>) is already a pretty darned good replacement for Evite, but nobody knows about it (no personal association; they just happen to be a local company in the space).

Also, a few seconds of googling turned up this laundry list of pretenders to the throne:

[http://www.scurrilous.com/blog/archives/2007/03/30/evite- alt...](http://www.scurrilous.com/blog/archives/2007/03/30/evite- alternatives/)

<comment><USER3> Specifically about invitastic -- they never made any serious attempt to go after evite. Jackson Fish Market is more of a marketing/consulting company. They developed invitastic for a client to promote a product (which is an interesting business model).

<comment><USER2> True, but it's not that they don't _want_ to take down evite -- they just realize the difficulty of the task, and prefer to take a few quick shots at the goal instead. Their subsidized development model is a really just a hedging strategy against the failure of any particular app.

I think the important point is that they've built a great evite-replacement product that nobody is using. There's clearly something interesting going on here, and I wonder if it's a case of evite being good enough? Geeks may complain about evite, but there are enough good replacements out there that the general public could have switched already.

</comment></comment></comment><comment><USER4> PLEASE crush Evite...they're my least favorite internet company that everybody uses. Good luck.

<comment><USER5> We would be more than happy to oblige! Whatever you can do to spread the word about Anyvite would be much appreciated ;)

</comment></comment><comment><USER6> Some minor suggestions/bugs:

1\. It would be nice to have a button which pops-up a small calendar (something like on kayak.com or any other flight site). The automatic parsing is nice, but for the less web-savvy, a button would be easier to grok.

2\. The time parsing is a little funky. If I type in 9:30, it auto-detects as PM. If I add a space after that, it suddenly switches to AM. This might catch people off guard.

3\. The auto date parsing is awesome (with the 'natural language' feature). However, I found that the word 'tonight' doesn't work...perhaps you want to add it to your dict?

<comment><USER5> Great suggestions! We're definitely going to add the pop up calendar ASAP and will look into the other bugs with the date and time stuff.

Thanks for the recommendations!

</comment></comment><comment><USER7> I like the concept, but the color and font of the logo makes me think AnyWine.

</comment><comment><USER8> Good luck on this but everyone I know just uses Facebook to coordinate events. Plus Facebook Mobile lets you keep updated even on the go.

<comment><USER5> We realize that Facebook is a dominant competitor in this market space and a lot of people use it for creating events, but there are still some people we've talked to who aren't entirely happy with it, as well as some market segments out there that aren't avid Facebook users.

<comment><USER9> How big is the addressable market? What you just described sounds pretty small.

"...still some people we've talked to who aren't entirely happy with it" This sounds like people that are slightly annoyed, and not really displeased enough to switch over to something else.

</comment></comment></comment><comment><USER10> I don't particular use Evite (I mean, I know what the site is, but that's about it.)

What are these problems that Evite supposedly has? From what I seen, people in general don't have a problem using them. Lay people, I mean.

<comment><USER11> Not much.

The biggest complaints has always been the placement of ads on the invitations and their insistence on drawing you to the site to look at any relevant information (which in turn increases the pageviews and expands their ad inventory). But that's their business model. Few other people have hooked on a more clever way of making money off invitations.

These toys that people make, and yes they are just toys, flame out quite quickly despite people clamoring that Evite's evil and so on. Here's a list of other similar services taking on Evite:

Mypunchbowl (complete event coordination)

Planypus (fuzzy arrangement)

Goovite (simplistic invitation system)

Invitastic (pretty invite templates)

Eventful

Imthere

Madeit

Crush3r (hipster/pretty invite templates)

Partee

Renkoo (friends getting together)

Skobee (dead)

Socializr

Upcoming

Zvents

Pownce can also be considered a similar service. Tack on the current trend of location-aware social apps like Loopt and you have another layer. Those apps are likely where these apps will evolve into. Plus the old standbys of email and phones.

In the end they all do the same thing: you type in info and it gets sent out, sometimes you get to gussy up the invites. The strategic advantage you have is simply size and inertia. If there's going to be anyone that's capable of crushing Evite it would only be Facebook.

<comment><USER12> Not Much? I think you may be in the minority of people who think that evite doesn't have many issues...

<http://en.wikipedia.org/wiki/Evite#Criticism>

[http://www.time.com/time/specials/2007/article/0,28804,16383...](http://www.time.com/time/specials/2007/article/0,28804,1638344_1638341_1638337,00.html)

[http://www.wired.com/culture/culturereviews/magazine/16-02/s...](http://www.wired.com/culture/culturereviews/magazine/16-02/su_evite)

[http://www.nytimes.com/2008/04/13/magazine/13wwln- medium-t.h...](http://www.nytimes.com/2008/04/13/magazine/13wwln- medium-t.html?_r=1&oref=slogin)

<http://www.evitesucks.com/>

<http://www.jabrams.com/thetruthaboutevite.html>

<http://venturebeat.com/2006/09/19/why-i-hate-evite/>

[http://valleywag.com/tech/modern-and-awkward/8-companies- we-...](http://valleywag.com/tech/modern-and-awkward/8-companies-we-all-hate- and-why-we-use-them-anyway-260371.php)

<comment><USER11> Yeah, I went through the same exercise with developing a product in this field and went through all those links and used it as justification of how an Evite killer is possible. All those links end up talking about the same 2 things over and over again:

\- ads on the invites

\- email bait you to visit the site

Great, but that's how they make money and they've been profitable with a small staff for a long long long long long long long long time.

Most of the other invitation sites focus on niches, Renkoo on the concept of friends getting together, Crush3r on hipster stylings, MyPunchbowl on coordinating the whole event, Socializr on being a social network, so on so forth. Besides Facebook, MyPunchbowl is the only other service getting quite a bit of traction. But it ends up being the same concept: single input info, multiple output info. I really believe this is a game of user conquest and attrition. Without the momentum and money to stay in the fight for the long haul, Evite (which has already been genericized) will just keep on chugging along.

Either way, good luck with the site and I'm sure you're much smarter than me at figuring out how to make this work.

</comment></comment></comment></comment><comment><USER13> What I love about anyvite... It lets me use my coordinatr login!

<comment><USER5> Yup! If you have an account on Coordinatr, then all of your data and past history should be available to you.

</comment></comment><comment><USER14> Same people as Coordinatr: <http://coordinatr.com/about> ? The sites seem very similar.

<comment><USER12> Yeah, it's the next iteration of Coordinatr. Coordinatr has been a great proof of concept for us and very useful for gathering user feedback. With our YC launch, we've added a ton of features and have really tried to improve the user experience.

<comment><USER15> Congrats! I wasn't that excited about YAEK until I realized it was you guys that made it. Coordinatr was the best user experience I've had with software because of how receptive and responsive you were to the feedback I gave. I wish you the best of luck and I _will_ be using AnyVite!

</comment></comment></comment><comment><USER16> I just started a party at my house in an hour!

<comment><USER17> how come I didn't get an SMS invite? I'm so bored at home...

</comment></comment><comment><USER18> No matter how good the service is, I think the design of the homepage could use a lot of work. Just my two cents.

</comment><comment><USER19> Havent you guys checked purpleTrail.com It has even more features; please check out

<comment><USER12> Yeah, we have seen purpletrail. As the article said, we're going for simple and straightforward. We're also pretty big believers that "more features" does not mean something is better.

<comment><USER20> Similarly, have y'all seen mobaganda.com? It is doing the simple == better thing, as well. I notice you guys seem (I haven't actually _used_ either site yet) to have a few more features. I'd be interested to hear your thoughts.

</comment></comment></comment><comment><USER21> <http://de.amiando.com/index.html?requestLanguage=EN>

The european competitor, as a I european I prefer them

</comment><comment><USER22> "Anyvite" is way to close to "an Evite."

Good luck not getting sued. Sincerely. :/

<comment><USER23> But Evite is way close to "invite" -- and Evite is going to have a pretty hard time showing that it has ownership over all derivations of the word.

</comment></comment><comment><USER24> didn't renkoo (adam rifkin's company) start out trying to do the same thing?</comment>


A Practical Guide To Varnish - Why Varnish Matters - <AUTHOR> http://highscalability.com/blog/2011/2/28/a-practical-guide-to-varnish-why-varnish-matters.html

<comment><USER2> Here's the thing - Varnish is cool. Well made, purpose built - a great tool.

But I can't help but think that in quite a few cases, people are jumping to it to compensate for a lack of proper engineering at the application level. Rails likes to add timestamps so things won't cache? WHY are you caching them then? Change the behaviour of rails for goodness sake.

This may be because Varnish gets deployed by a different person/team than those writing the apps - the web-turned-php developers can get sloppy and a decent frontend engineer can speed things up again by tossing varnish in front and fixing things up (which is a very common situation in my experience).

Certainly varnish can compensate for a wide variety of problems.. but it's also one more thing to go wrong.

<comment><USER3> Probably, but since there are similar commercial tools on the market (F5 WebAccelerator, for example), such a product is always welcomed.

</comment></comment><comment><USER4> Thank god, an app server with an imperative configuration file! Bliss! Declarative languages are not meant for programmers.

<comment><USER5> Upvoted assuming you're being sarcastic :)

Seriously, I'd love to hear how easy a varnish setup is to debug. Apache rewrite rules are imperative as well, just not obviously so.</comment></comment>


Ask HN: Do you use automated tools to create APIs, or do you code them manually? - <AUTHOR> Do you use some sort of framework/tool for creating the APIs needed for your product/service/application/ecc, for example (https://loopback.io/), or do you code by hand? <comment><USER2> APIs are interfaces (it's right in the name!) and should never be directly tied to implementation because:

1\. the interfaces must remain stable to the outside world that relies on them

2\. They select what underlying resources and functionality is accessible by outside users, and what is hidden. A lot of your internal implementation is either a mess, "temporary", insecure or intentionally internal.

3\. They control access to the internal application through authentication, authorization, security, and translating data in both directions.

4\. When the internal representation changes, they map the new implementation to the old interface to ensure the system remains reliable to API consumers.

5\. They offer migration paths when change is necessary

That being said...

Auto API generators are really useful for internal systems where you control the underlying system, the API, and all systems relying on the API.

They are also useful to build an initial API that you plan to fork.

<comment><USER3> Yeah I agree with your stance, but not the conclusion. The way that gRPC (and many other systems) handle this is beautiful and the way all APIs should be built: your API is a specification, not code, so you start with the spec (SDL), then generate the adapters your implementation needs to plug into it.

This helps elevate changes to the API itself; you can easily write automated systems which detect changes to the specific SDL files. Or, the way companies like Namely [1] do it, keep those SDLs inside a separate repo, then publish the adapter libraries on private npm/etc to be consumed by your implementation.

[1] [https://medium.com/namely-labs/how-we-build-grpc-services- at...](https://medium.com/namely-labs/how-we-build-grpc-services-at- namely-52a3ae9e7c35)

<comment><USER4> This has been around since the early 90s or previous. ONC-RPCs did this where you define the interface file and it generates the client and server stubs for you.

NFS is based on this, as with other services. Conceptually it's exactly the same, with some underlying differences.

<comment><USER5> Going even further back in history, ASN.1 is also like this. It's a description language for data structures, and there are separate representations that can be derived from them. It's sort of like JSON, JSON Schema and Protobuf in one.

TLS certs are encoded in ASN.1 DER, for instance, and LDAP messages are encoded in ASN.1 BER.

</comment></comment></comment></comment><comment><USER6> This is not an XOR question. Both are valid for different APIs. I start with problem and design the right solution to the problem.

Often I have a simple problem where I can write a simple clean API quickly by hand. Generation is a negative, generated APIs tend to be complex and hard for the user to read.

Sometimes my requirements need something that a tool does better. For example protobuf gives me an efficient over the wire API that can be used in multiple languages: I'll let protobuf generate those APIs as I can't do better by hand (though we can debate which tool is better for ages).

Sometimes I have a complex situation where I'll write my own generator. For example I once made a unit system generator for C++: it was able to multiply light-years by seconds and convert to miles/fortnight - no way would a handwritten API support all the code needed for that but with generation it was automatic (why you would want to do the above is an exercise for the reader). The API was easier to understand than boost's unit system (APIs are about compromises so I won't claim mine is better)

</comment><comment><USER7> In a few projects where we had a specific experiment we needed insights from we ran PostGrest[1]

Basically you create your tables and run PostGrest. Bam! You have an http interface / api for your database. We would then create light wrappers around those that took on specific responsibilities - security, audit etc. The wrapped apis is what we exposed publicly.

This may not sound all that helpful but it made the bit we implemented unbelievably tiny. As a plus, we found that a Java application that exposes an endpoint and calls an endpoint is fast to start / stop because it doesn't mess around with DB connection pools.

[1] [http://postgrest.org/en/v5.2/](http://postgrest.org/en/v5.2/)

<comment><USER8> How do you handle versioning? If you add a new required field to one of the tables (perhaps even a field that doesn't have a default value), how do you make sure the consumers of your old API keep working?

<comment> steve-chavez You can handle versioning with PostgreSQL schemas. You can have v1, v2, etc. These usually contain views and stored procedures.

</comment></comment></comment><comment><USER9> At work (enterprise stuff) we've grown tired of duplicating 1000's lines of boring CRUD stuff and turned to code generation. Which is so much better.

The workflow now is:

\- think really hard for 10 minutes about the business problem,

\- describe it into our meta language (typed structs, UML-like, really simple),

\- instantly click'n'build a whole set of API endpoints down to SQL create/alter/drop statements, along with full up to date documentation,

\- get excited to be able to deliver so much stuff to customer in no time,

\- aaand finally receive a requirement update ('the last one I promise') and send I-love-you letters back in time to our old-selves for such a nice malleable framework (which I dubbed The Platform).

</comment><comment><USER10> For the last years we've done everything by hand using Node or Go.

If I was to start a new API today I'd use Hasura. It automatically creates a GraphQL schema/API from a Postgres database. It's an amazing tool.

[https://hasura.io/](https://hasura.io/)

<comment><USER11> Can you do a full write-up about this tool?

Looks really interesting to easily layer a graphQL API on top of a Rails app with a few serverless functions...

<comment><USER10> A "full write up" seems a bit intimidating... :)

I'll expand a bit on my previous comment.

So the idea is that Hasura is a stateless layer on top of Postgres that generates all the necessary GraphQL schema/queries/mutations/real time subscriptions for doing CRUD based on the Postgres schema. If you change the tables (either via the Hasura admin or some migration system) it all adapts automatically as you'd expect. It can use a remote Postgres DB, you don't need to run the API and DB in the same machine.

Performance is fantastic. Hasura is very efficient in terms of speed and memory consumption. Even with a free Heroku dyno you should get thousands of reqs/s.

On top of direct data from tables you can also read Postgres views. Essentially you can read a custom SQL query from GraphQL.

Hasura can also integrate external GraphQL schemas via a mechanism it calls "stitching". The idea is that you can point remote GraphQL schemas to Hasura (on top of the current one from Postgres) and it will serve as a gateway of sorts between all your GraphQL clients and servers.

Hasura does not include authentication, but it's very easy to integrate with your current system or with services like Auth0 via JWT.

Hasura also includes a powerful fine grained role-based authorization system.

Whenever anything happens you can configure Hasura to call a URL (webhook) to do something. Maybe a REST endpoint or a cloud function. This is usually the way to integrate server side logic.

The only problem we've found is integrating Hasura with our current authorization system. Our users have multiple roles and we have no way of deciding which is the current role. Hasura requires a single role to be passed to its authorization system on the request headers. This is something that is being worked on AFAIK.

Their youtube channel has lots of little videos showcasing all the functionality.

[https://www.youtube.com/channel/UCZo1ciR8pZvdD3Wxp9aSNhQ/vid...](https://www.youtube.com/channel/UCZo1ciR8pZvdD3Wxp9aSNhQ/videos)

</comment></comment></comment><comment><USER12> We use OpenAPI to define APIs. There is an extension with new template generation, intellisense, snippets, etc for VS Code: [https://marketplace.visualstudio.com/items?itemName=42Crunch...](https://marketplace.visualstudio.com/items?itemName=42Crunch.vscode- openapi)

</comment><comment><USER13> Nobody remembers anymore SOAP it seems haha It's funny but all those new documentation and code generators for Rest were largely invented in SOAP messages before.

It doesn't make sense to send SOAP messages to browsers but I cringe every time I find myself with a vaguely documented Rest API when integrating systems.

<comment><USER14> I similarly cringe at vaguely documented APIs, but being a young developer, my experience with REST has been better. For instance, I've consumed a SOAP API where the WSDL specification was primarily a method named "Magic" that accepted a string "Method" and six string-typed parameters, "Parameter1" through "Parameter6".

I think the key is to pick a documentation tool that the team will actually use.

</comment></comment><comment><USER15> Disclaimer: This is not based on real world knowledge. (To be honest I have practically no "real world knowledge".)

That being said, I just finished a school project where we (our class) were divided into small teams and we had to implement small RESTful web apps. My team chose to kick it off by grabbing two people from the front- and backend team and writing an API specification by hand. It was a breeze and we were done in a few hours. After that front- and backend (almost) never had to interact with each other again until the end of the project where we had to stick the two things together.

This probably isn't applicable to real-world cases where the requirements are ever-changing and everyone's a full-stack dev (or you don't have a team at all) but I found this sort of separation quite useful for this project. (It kept team sizes managable, different kinds of devs were in seperate teams, we didn't have to wrestle with any tooling that would halt the whole project.)

I see no problem with generating client/server boilerplate from spec though (like Swagger does, I think).

<comment><USER15> This sort of philosophy could be useful when designing a public-facing API though. In that case you need a well-formed implementation-unaware API documentation and mapping it out upfront by hand could save you lots of trouble.

</comment></comment><comment><USER16> Use grpc. With one definition file you can generate:

1) Client code in various langs. 2) Server code in golang of python or nodejs. 3) Swagger. 4) Rest interface if you want to. 5) Gorm definition of you use golang gorm.

</comment><comment><USER17> We've been using gRPC and Protocol Buffers for the last couple of years. We write APIs using the Protobuf interface definition language, then generate client libraries and server side interfaces. Then it's a matter of implementing the server by filling in the blanks.

<comment><USER18> I love protobuf for this reason. Personally I've opted for Twirp instead of gRPC, as gRPC has a lot of baggage, and streaming is really not necessary for me.

We've had to drop-in-replace, or add a validation or access layer service for something, and using protobuf has made this super easy. Anything interacting with that service is none the wiser.

<comment><USER17> gRPC has been solid for us on the JVM, and streaming has been great when consuming from Apache Flink jobs, integrating with message queues, receiving push notifications and so on. For async work it's useful to have more than just request/response.

I've been playing the FoundationDB Record Layer for a personal project of mine, and with this setup I can generate not only the API implementation, but also the models used by the persistence layer:

Protobuf (Messages) -> gRPC -> Scala/Monix -> Protobuf (Models) -> FoundationDB

<comment><USER19> > Protobuf (Models)

Sounds really cool! Is this something that comes out of the box or generated by your own plugins?

<comment><USER17> FoundationDB Record Layer uses protocol buffers out of the box. They leverage the fact that you can evolve protobuf messages in a sane way. That's their equivalent of doing database schema migrations.

</comment></comment></comment></comment></comment><comment><USER20> (very very small team) We have some handmade scripts in place to generate basic crud endpoints, generated files are then adjusted to the specific needs, but it goes a long way in keeping things organized and consistent with very little effort.

</comment><comment><USER21> In my case I like the end product to be code. I use snippets/generators to create components (models/controllers/middleware) then modify as needed.

Having used loopback before, it's a quick way to get an api up and running, I personally struggle with injecting logic into endpoints/writing custom endpoints.

If the code's "all there", I know where to look. If I have to intercept hooks it adds an extra layer when searching.

Summary, loopback has been great for creating APIs where all I care about is crud, but for larger projects I stick with snippets/generators so I can extend easier later.

</comment><comment><USER22> Lately, I've been getting back into Spring Boot. Spring Data REST automates a lot of the CRUD endpoints, with easy enough configuration and customization. I've been declaratively securing it all with Spring Security.

<comment><USER23> I prefer to code one level deeper and I mostly use plain Spring MVC controllers. That way I can still have spring security for the endpoints but it keeps the endpoints more decoupled from the repositories.

I typically have a repository generated by Spring Data, a small service layer with business logic on top of those and then an MVC controller that only talks to the service layer, never the repositories.

Each controller also has its own DTO class(es) for request bodies and responses and a small converter between DTO and entity. Kotlin extension methods make it easy to add the toDto() method onto the entity so a typical controller will fetch the entity from the service and return entity.toDto().

Kotlin, Spring Boot and Spring Data are amazingly well suited for this.

<comment><USER22> I was doing things manually to, even security!

You don't really need DTOs because you can use projections and set a default projection to be used when that entity type is returned in a collection. Any entity fields that should never be exposed can be annotated with @JsonIgnore. And then if you need endpoints that aren't CRUD, you can build those the usual way.

<comment><USER23> I'll check out the projections as they seem interesting and I don't know them very well.

</comment></comment></comment></comment><comment><USER24> For personal projects I'll hand code them (usually) because I like thinking about API design and API UX.

For professional stuff... it really depends. I like GRPC but codegen needs team buy-in... It can quickly make a fast development loop hurt if done poorly. Doubly so if IDEs are involved for some users and the IDE is constantly updating it's caches of types and interfaces. I've just seen it turn into a hot frustrating mess very quickly.

</comment><comment><USER25> We tried writing OpenAPI docs to implement a contract-first development workflow, with the idea that backend & frontend/mobile engineers would agree on the API interface by discussing OpenAPI changes in a pull request, and only then start implementing it (on the backend side) and using it (on the client side).

This didn't pan out well, because it turns out OpenAPI isn't very easy to read, especially when you're reviewing a diff in a pull request. We didn't get the engagement we were looking for in pull requests.

We've since invested in building a simpler, human-friendly API description language based on TypeScript, which exports to OpenAPI 3. It's still early, but we've got a lot of positive feedback and quick adoption across the company (50 engineers).

You can check it out at [https://github.com/airtasker/spot](https://github.com/airtasker/spot). Feel free to send us feedback in GitHub issues or replying to this comment :)

</comment><comment><USER26> I prefer to write them by hand. Most APIs, to start, don't have a lot to them. They tend to grow in scope over time. So, it's pretty easy to just throw together your initial idea, and incrementally grow it from there.

I might think differently if confronted with a huge API surface area to build off the bat, but I haven't run into that yet.

</comment><comment><USER27> Manually. However, I have had the luxury of implementing relatively small APIs. If I was doing something like the Google APIs, I'd probably consider automation. That said, I'd probably want to write the automation, myself, as I'm an inveterate control freak.

</comment><comment><USER28> For HTTP APIs, I'm a full convert to OpenAPI - write your API document by hand, then code-gen the client/server stubs.

It requires a small investment upfront, but will pay huge dividends once your project is rolling. You have a single source of truth for publicly exposed endpoints and model descriptions (your API document), and you can instantly regenerate certain key components (e.g. model binding, new routes, etc) whenever that document changes.

I actually contributed the F#/Giraffe generator to the OpenAPI generator project, which you can find at [https://github.com/OpenAPITools/openapi- generator](https://github.com/OpenAPITools/openapi-generator)

</comment><comment><USER29> Yeah a couple years ago we switched from using vanilla Flask to Connexion which lets you describe your API through the an OpenAPI spec. Connexion handles routing and request validation and our developers can just import the yaml into Postman for testing as well as use Redoc for generating pretty documentation sites. Overall the biggest pain point as others have mentioned is writing and maintaining the spec. OpenApi's structure can take some time to get used to and maintaining the whole API in one file is a little tough, but it's not unmanageable with code folding and good schema definitions.

</comment><comment><USER30> Imho it matters less.

What's important is that you have rigorous testing around your API.

APIs are essentially external contracts people build against. You don't want to break this contract.

make sure it

\- never changes unless you know about it

\- updates the documentation whenever it changes

</comment><comment><USER31> I used Silex for a long time and when it got deprecated moved over to Symfony's MicroKernel ([https://symfony.com/doc/current/configuration/micro_kernel_t...](https://symfony.com/doc/current/configuration/micro_kernel_trait.html)). Tiny enough to get started in a matter of minutes and when your project grows bigger then you can easily refactor either the whole project or just parts of it to "standard" Symfony architecture.

</comment><comment><USER32> I've tried two approaches.

1) write code, generate Swagger/OpenAPI from it. Works pretty good with big frameworks like Spring for Java or Symfony for PHP. Drawback: it is too easy to change API, tends to broke BC too often.

2) write Swagger/OpenAPI, generate code stubs from it. Works good enough with Go and TypeScript. Tends to keep client-server contracts stable. Drawback: server code is overly complicated, needs extra layer of DTO to convert from domain terms to API models.

edit: 2nd approach also good for autotesting.

</comment><comment><USER33> I'm using a custom protocol on top of MQTT. I have a big CSV file with all the topics/payload types/etc. specified which is then use to generate a common library for our software services. Thanks to Rust's nice code generation capabilities, I have several types (many enums) which automatically serialize/deserialize from/to MQTT messages, checks included. Really cute.

<comment><USER34> Nice! Can you say more about how you're able to do the Rust aspects of code gen and checks?

</comment></comment><comment><USER35> It's interesting that no one mentioned CORBA... if anyone has success/horror stories to share about corba, i'd gladly listen.

</comment><comment><USER36> I write API code manually and use testing tool that also generates openapi file and save it in git. This makes API docs always up-to-date and history of changes in actual API via git. (stack: rails, rspec and some gem for openapi)

</comment><comment><USER37> I use Django REST Framework which may be or may be not an automated tool depending on the definition you are using - but DRF makes API's very declarative and I love it (batteries included).

</comment><comment><USER38> Okay, I understand some APIs:

(i) TCP/IP

(ii) HTTP

(iii) ASN.1

(iv) SQL

(v) The key-value session state store I wrote for my Web site (cheap, simple, quick, dirty version of Redis).

Etc.

Now, how can the design and programming of such APIs be "automated"????

</comment><comment><USER39> I use feathers [1] and like it a lot.

[1] [https://feathersjs.com/](https://feathersjs.com/)

</comment><comment><USER40> most languages have a library that takes a json structure from a file and creates an AP. for eg json-server on node.js, I just use that initially until the "need" for the db becomes clear ie what data do I need to interact with. After that it's custom all the way - it's more malleable I find

</comment><comment><USER41> Yes, I use the springfox impl for Java for both server api and client api (mainly for automated tests)

</comment><comment><USER42> What's an example of automating API endpoints in Node.js? I always just whip up an Express.js MVC by hand.

</comment><comment><USER11> I code them by hand, but I like to automate as much of the documentation generation as possible...

</comment><comment><USER43> it depends on the stage of the project i think.

I think early stage and MVP projects are almost always written by hand.

<comment><USER44> I feel the opposite: automated tools are really useful for smallish POC type things -- MVPs and early stage work, but fail when things reach a certain level of complexity.</comment></comment>


A new compression algorithm to make Google Chrome updates small - <AUTHOR> http://blog.chromium.org/2009/07/smaller-is-faster-and-safer-too.html <comment><USER2> The code has been available in their public SVN repository for quite some time:

<http://src.chromium.org/viewvc/chrome/trunk/src/courgette/>

BTW, I've read about 40% of the Chrome code and I have to say it is the single best open-source project I've come to study so far. Incredibly clean code, really shows how much code reviews done properly can benefit project in the long run.

<comment><USER3> Well, the "cleanliness" is probably because it hasn't been open-source for too long - most of the original work was done by a dedicated team of software engineers most likely operating under fairly strict code guidelines.

<comment><USER4> As someone who works on a project with a team of dedicated engineers under fairly strict code guidelines including code reviews, I can tell you from experience none of these factors have to do with "code cleanliness" (as ours is not, at least in my opinion).

</comment></comment></comment><comment><USER5> bsdiff was written by a HN regular, and I can't think of a better person to provide insight on this. (Hint.)

<comment><USER6> Did someone summon me?

I don't know all the details behind Courgette, but it sounds very similar to Exediff. In my thesis I show that bsdiff (well, a slightly improved version of bsdiff which I never got around to releasing publicly) performed on par with or slightly better than Exediff; so I'm surprised that Courgette is getting this far ahead of bsdiff.

Based on the numbers they've published (a 10 line source code patch to a 10MB executable resulting in a 700kB bsdiff patch), it sounds like there's something weird going on which is breaking bsdiff -- the FreeBSD kernel is roughly the same size as Chrome, but normally for a small (e.g., 10 line) source code patch I would see a bsdiff patch of between 50kB and 100kB.

Beyond that, I really can't offer any more insight -- unless someone wants to hire me to spend a week looking at the old and new binaries and figuring out where bsdiff is going wrong. :-)

</comment></comment><comment><USER7> Does this generalize to a moving "fixed offset" in the compressor? i.e. can't it be done without the disassembler? (Assuming they're parsing x86?)

x86 is variable length, so this isn't trivial, but surely it is possible. I guess I don't like the "assembler" step because, hmm, SSE7 might not look like x86.

Also, if you were willing to have the loader to do the work, it sounds like you could do a -fPIC sort of thing, and load most code as a DLL.

Very cool that they got it working, though.

</comment><comment><USER8> Without engaging conspiracy theories, I think the most interesting part is here:

The small size in combination with Google Chrome's silent update means we can update as often as necessary to keep users safe.

[An extra tidbit- courgette is a 'summer squash'. Silly Google...]

<comment><USER9> I personally found an order-of-magnitude increase in the compression ratio for their executables to be more interesting than that.

</comment></comment><comment><USER10> Technically, it's not a new compression algorithm. It's just a more efficient update scheme which uses a better suited binary diffing algorithm.

<comment><USER6> What you call "binary diffing" is called "delta compression" by people in the field -- and delta compression is considered to be a type of compression (just like image compression, audio compression, video compression, et cetera).

<comment><USER11> Perhaps what he meant is that it's not a new _entropy coding_ algorithm; it's a new prediction algorithm that feeds data to an old entropy coding algorithm. Overall, it's still a new compression scheme though.

</comment></comment></comment><comment><USER12> It's a very impressive achievment, but I'm not sure I buy their motivation for doing the work. 70k vs 700k? It's a blink of an eye on today's Internet. Sounds more like a very bright engineer wanted to work on a cool project, then reverse reasoned his way back to this "smaller is faster" idea.

<comment><USER13> "If the update is a tenth of the size, we can push ten times as many per unit of bandwidth. We have enough users that this means more users will be protected earlier."

<comment><USER12> Yes I read the article as well. The issue is that Google probably has as close as you can get to an infinite amount of bandwidth, which makes this sort of a low priority item. I don't want to trivialize the technology; it's obviously very cool stuff, but it's the solution to a problem I doubt they ever had.

"Hi everyone, welcome to the Monday morning Chrome staff meeting. Um... so we've gotten some push back from management. They -- guys you're not going to believe this -- they think we might be using too much bandwidth. I tried to make a snarky joke about YouTube, but they weren't having any of it. So, um... we're gonna go ahead and pull two or three guys from the [INSERT IMPORTANT BROWSER COMPONENT THAT WILL LEAD TO A BETTER BROWSER HERE] team and have them work on making the updates smaller."</comment></comment></comment>


Fix Shellshock Bash Vulnerability in OS X Bash - <AUTHOR> http://www.fizerkhan.com/blog/posts/Fix-Shellshock-Bash-Vulnerability-in-OSX-Bash.html <comment><USER2> This is basically a copy of my post at [http://alblue.bandlem.com](http://alblue.bandlem.com) and of my stack exchange answer.</comment>


Summer of Hacker League - <AUTHOR> http://masherydev.tumblr.com/post/93221419907/the-summer-of-hacker-league

<comment><USER2> Glad to see that Hacker League is getting some much needed development love. Keep up the great work!</comment>


Google Interferes with Its Search Algorithms and Changes Your Results - <AUTHOR> https://www.wsj.com/articles/how-google-interferes-with-its-search-algorithms-and-changes-your-results-11573823753?mod=rsswn <comment><USER2> Dont use google? [https://duckduckgo.com](https://duckduckgo.com)

</comment><comment><AUTHOR> No-paywall link: [https://www.msn.com/en-us/money/companies/how-google- interfe...](https://www.msn.com/en-us/money/companies/how-google-interferes- with-its-search-algorithms-and-changes-your-results/ar-BBWOCm2)</comment>