Kernighan's Law: You are not smart enough to debug it - <AUTHOR> https://github.com/dwmkerr/hacker-laws#kernighans-law <comment><USER2> Over 20 years ago, after I had devised a way to use Gray Code to embed a clock into a signal without increasing the signal's modulated bandwidth, one of my engineers said that someday there'd be a "Moyer's Law". I thought that was funny, so I created one myself:

Moyer's Law: "When we can finally print from any computer to any printer consistently, there will be no CS problems left to solve."

Since every good law has a corollary, I created one of those too (simply to be funnier):

The Corollary to Moyer's Law: "The printed dates will still be wrong."

While PDFs have brought us closer to "universal printing", I won't claim we're anywhere close to solving all CS problems. Sadly, date conversion and formatting continue to be problems (hint, consider UTC ISO-8601 or RFC3339 dates/times for the JSON representation).

P.S. I don't actually think I'm smart enough to have a law named after me ... nor have I really contributed enough to "our art".

\- [https://en.wikipedia.org/wiki/Gray_code](https://en.wikipedia.org/wiki/Gray_code)

\- [https://www.electronics- notes.com/articles/radio/modulation/...](https://www.electronics- notes.com/articles/radio/modulation/frequency-modulation-fm-sidebands- bandwidth.php)

<comment><USER3> Temporal literals really deserve dedicated syntax in JSON. I'm bored to parse yet another "18.2.2020".

<comment><USER2> I agree but I'd like the generic implementation of that. JSON schema never really took off and I believe that part of the reason is that there's not a way to indicate what type might be contained in a string or number (I'm okay with JSON booleans and null). As ugly as it could get, adding XML Schemas to XML documents did in fact help the parser.

The reason I stated I'd like the generic version is that there are other types that we use consistently. There's a very nice RFC available for telephone numbers that we've started following and we can marshal/unmarshal pretty easily from strongly typed languages (where we control the code), but wouldn't it be nice if there was a standard way (within the JSON to let systems know it was a telephone number?

\- [https://tools.ietf.org/html/rfc3966](https://tools.ietf.org/html/rfc3966)

<comment><USER4> I think it's time for a JSON version 2.

a) no mandatory quotes for dicts keys b) date and time intervals in iso 8601 format. c) optional type-specifiers for strings, so we can add e.g. ip4 and ip6 addresses. Eg. { remote: "127.0.0.1"#ip4 }

e.g. { guests: 42, arrival: @2020-02-17T17:22:45+00:00, duration: @@13:47:30 }

<comment><USER5> There's a corollary in there to Greenspun's tenth.

Any sufficiently complicated serialization technology contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of ASN.1.

</comment></comment></comment></comment></comment><comment><USER6> It reminds me of a typical code written by scientists.

IMHO there is the biggest gap between the raw intelligence and software engineering skills (including good practices). So, it means that most of the code ends up as code-golf-style contraptions, incomprehensible for anyone else (including their future themselves).

Full disclaimer: I used to be that guy who liked "clever hacks". Now I try to make code readable in the first place (largely thanks to Python philosophy).

<comment><USER7> I'll second this.

One grad student friend of mine traded me a case of beer for a Saturday of help on his code. It was some code to record spike timings in the olfactory cortex and also control valves (smelly research).

By the 12th nested for-loop, I gave him back the beer.

<comment><USER8> I thought this story was going to end with you consuming the beer on the spot.

On several occasions, and with lesser crimes, I've told the person something like "I think you are confusing yourself with your own code. I want you to change to meaningful variable names (stop recycling variables) and factor out a couple of child functions here, and here. If you still can't see the problem, come get me and we'll try this again."

<comment><USER7> > I thought this story was going to end with you consuming the beer on the > spot.

Haha, no way. Grad students are poor as is. Drinking a month's beer budget in front of the guy would be bad, leaving him with that code was cruel enough.

</comment></comment></comment></comment><comment><USER9> "The competent programmer is fully aware of the strictly limited size of his own skull; therefore he approaches the programming task in full humility, and among other things he avoids clever tricks like the plague. " \- EWD 340 [1]

[1] E.W. Dijkstra - The Humble Programmer, ACM Turing Lecture 1972 ([https://www.cs.utexas.edu/~EWD/transcriptions/EWD03xx/EWD340...](https://www.cs.utexas.edu/~EWD/transcriptions/EWD03xx/EWD340.html))

<comment><USER10> Really interesting to read this now, almost 50 years later. So much has changed — and so little has changed.

<comment><USER9> Absolutely. Many of Dijkstra's EWDs have aged remarkably well. [1]

[1] [https://www.cs.utexas.edu/users/EWD/](https://www.cs.utexas.edu/users/EWD/)

</comment></comment></comment><comment><USER11> There's a saying in my family, which is full of good cooks:

The sign of a great cook is not the ability to make great dishes -- it's being able to fix dishes that someone else screwed up!

<comment><USER12> Nice one. Added a paraphrased version to [https://github.com/globalcitizen/taoup](https://github.com/globalcitizen/taoup)

<comment><USER11> Wow! Interesting to see "firsthand" how the various bits of folk wisdom we come across make it into fortunes. Nice work on this project by the way!

</comment></comment></comment><comment><USER13> I've always found this one a bit peculiar - usually debugging is about looking at something on a simpler, lower level than implementation. Rather than looking at a whole set of logic, let's step through it piece by piece, look at each operation. When done slowly with care and understanding of the lower level components in the system, debugging code is simpler than learning and implementing most architectural patterns in all but the most extreme of circumstances (compiler bugs, hardware faults).

Or maybe I just spent too much time in OllyDbg at a young age to notice.

<comment><USER14> > usually debugging is about looking at something on a simpler, lower level > than implementation

the lower you go, the less context you have about the author's _intent_. with too-clever code it's easy to miss the forest for the trees.

e.g. it usually works fine to fix spelling errors at word level, but less well to restructure complex sentances without larger context.

<comment><USER15> Please tell me "sentances" was intentional.

<comment><USER16> Waht maeks yuo tihnk twas intantional?

</comment></comment></comment></comment><comment><USER17> I don't know, if I can debug code on my workstation, that does not seem extremely difficult. Time consuming, if anything, but I feel like anyone with sufficient grit could do that, regardless of education.

Tracking down a bug and fixing it with a one-liner does not feel like productive programming (though it brings value to the business).

If I can't debug code on my workstation, e.g. it's running on Kubernetes with service mesh and a cool feature must be added, that's when all hell breaks loose. For example, debugging why Kubernetes deployed on OpenStack will push an image to its internal repository but won't pull from it, that is hell and I'd much rather debug HTTPD than that.

<comment><USER18> For simple bugs yes. However if the bug is a one in a million race condition between threads where the lock is held/released at the right time except for one place where it isn't even obvious the other thread can write the data. This is particularly bad if there are performance considerations and the design ensures some places where it looks like a race are actually not and so a lock would be a bad thing. Without fully understanding the complex design you cannot add the needed lock in without adding in uneeded locks that kill performance.

<comment><USER19> > However if the bug is a one in a million

This is what RR is for: you only need to be able to reproduce the bug once inside RR.

> ensures some places where it looks like a race are actually not

Uh be careful, if you're writing in a high level language... such as C: there are no safe data races.

[https://software.intel.com/en-us/blogs/2013/01/06/benign- dat...](https://software.intel.com/en-us/blogs/2013/01/06/benign-data-races- what-could-possibly-go-wrong)

The compiler is free to reorder operations in ways that make what appears to be a safe data race unsafe.

<comment><USER18> There are limits you can place on the compiler. There are atomics in the compiler. There are times when you know you are safe because there is a future write that is memory barrier protected so even though there is a potential race, before the other threads can read it this thread will write something else. There are times where you know the other thread isn't started yet. There are platform specific ways to flush all the caches long after the race in question which can be tied to something else that is synchronized across threads...

<comment><USER20> > There are times when you know you are safe because there is a future write > that is memory barrier protected so even though there is a potential race, > before the other threads can read it this thread will write something else.

Often it is difficult to prove this to the compiler.

<comment><USER18> I don't need to prove it to the compiler. I can prove it by a higher level understanding of my own system. So long as the compiler emits the memory barrier when I need it to, and the other thread stays in the state where it won't be reading the memory in question it doesn't matter what the compiler knows.

</comment></comment></comment></comment></comment></comment><comment><USER21> You're ready for a promotion from software engineer to senior software engineer once you've spent hours debugging some piece of software, only to realize it's actually working as intended.

<comment><USER22> Is this implying the debugging was due to the code appearing incorrect or that the previously preceived erroneous behavior was actually intended?

<comment><USER21> The previously perceived erroneous behavior was actually intended. Just code that's extremely difficult to understand and debug.

</comment></comment></comment><comment><USER23> A possible corollary is that if you are smart enough to make the code simple, you are also smart enough to debug that code.

(Making simple code for complex problems is hard!)

<comment><USER24> I agree it's hard to build a simple solution to a complex problem, but not necessarily wrt what we normally talk about as problem solving skills in programming.

What I mean is the hard part is stepping back and actually solving the real problem in the best way. That might be utterly trivial code wise, it might be brute force vs optimal. It might be just solving a readily parallelisable problem on a single thread. It might be a unilanguage monolith instead of microservices in the 'right language for every task', etc.

The smartness involved in making code simple (and I mean simple, not elegant) is more about pragmatism, lateral thinking, discipline and focus on end goals than any sort of analytical smartness.

</comment></comment><comment><USER25> The intelligence required to write a piece of code =/= the intelligence required to understand it. Usually the second is a lot easier. Writing a fancy Haskell typeclass to structure a problem might require deep expertise or creativity. Working with such an existing solution isn't as hard.

Even in math or science it's harder to create something new than it is to merely imitate it.

<comment><USER26> It's easier to write code than to read. This isn't a fact but it is generally accepted by many. It doesn't seem obvious at first why but "reading code" isn't like "reading".

[https://jonathanfries.net/code-is-still-harder-to-read- than-...](https://jonathanfries.net/code-is-still-harder-to-read-than-write- part-1/)

<comment><USER27> It's also called the law of "Uphill Analysis, Downhill Invention" which states "It is much more difficult to guess the internal structure of an entity just by observing its behavior than it is to actually create the structure that leads to that behavior."

Been reusing and repeating this one quite a lot, since I saw it mentioned in one of this year's RubyConf keynotes: [https://blog.jessitron.com/2019/11/05/keynote-collective- pro...](https://blog.jessitron.com/2019/11/05/keynote-collective-problem- solving-in-music-science-art-and-software/)

(There is a blog-post version behind this link as well, in case anybody wanted to skim the contents rather than watch video of the entire talk to find out how it fits in.)

<comment><USER28> Uphill Analysis, Downhill Synthesis is a major point in the book _Vehicles_. [0]

Fascinating thought experiments.

Here's an introduction on YT [1], though it really only scratches the surface.

[0] [https://www.goodreads.com/book/show/483485.Vehicles](https://www.goodreads.com/book/show/483485.Vehicles)

[1] [https://m.youtube.com/watch?v=A-fxij3zM7g](https://m.youtube.com/watch?v=A-fxij3zM7g)

</comment></comment></comment></comment><comment><USER8> The trick, I believe, is in convincing yourself that finding a simpler way to achieve the same result is a more worthwhile form of cleverness.

You still get to be clever, but only you will ever know how clever.

</comment><comment><USER29> Kernighans law:

Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.

(Brian Kernighan)

The rest of the laws are interesting, so I'd recommend taking a look as well..

</comment><comment><USER30> I liked the optimistic twist on this, Kernighan's Lever: [http://www.linusakesson.net/programming/kernighans- lever/](http://www.linusakesson.net/programming/kernighans-lever/)

Kernighan's Law means that you're continually forcing yourself to level-up in order to debug your own code. It prevents stagnation.

Use caution in multi-developer environments. :-)

<comment><USER31> Or it could mean, be too stupid for complex tools, be dumb and choose [simplicity].

[simplicity]: [https://github.com/dosyago/dumbass](https://github.com/dosyago/dumbass)

Personally I find the original law inspiring.

<comment><USER32> A related interpretation: your debugging tools need to be _at least_ as good as your programming tools. Ideally, better.

Debugging a K8 cluster with print statements is hopeless, but if the cleverest code you can write in a dumb editor is going through a good test suite and profiler, you might be fine. How many as-clever-as-possible optimization tricks have been made viable by Valgrind?

</comment></comment></comment><comment><USER33> I write every piece of code as if someone else is going to have to be able to use it later, even if it's just for me. And I find remarkable overlap between the clarity that some random person would need and the clarity that future-me ends up needing.

</comment><comment><USER16> This law is actively biting me in the ass right now as we speak ;)

Gotta love T-SQL, where hard things are easy, easy things are hard, and one must perpetually choose between extreme cleverness and extreme verbosity because there is absolutely no middle ground.

</comment><comment><USER34> > Debugging is twice as hard as writing the code in the first place. > Therefore, if you write the code as cleverly as possible, you are, by > definition, not smart enough to debug it.

The expression "by definition" is misplaced here. The first proposition "Debugging is twice as hard as writing the code in the first place" is not a definition. At best, we can treat it as an axiom. In any case, the conclusion "if you write the code as cleverly as possible, you are not smart enough to debug it" does not follow by definition.

<comment><USER8> I promised myself I was going to stop getting sucked into pedantry.

I believe that the sense in which he meant 'by definition' is that it's tautological to say that more of something is more than less of something.

For n > 0, 2n > n

<comment><USER34> The meaning of what the quote is saying is clear. It is also clear how the logical reasoning is supposed to be. My - admittedly pedantic - point is that the choice of words is incorrect.

</comment></comment></comment><comment><USER35> I worked in a technical support department for a long time. I often was the guy between the customer (the technical customers) and the engineering teams.

After a while figured out pretty good ways to manage my interactions with the engineering teams, despite having no coding experience or really any visibility to the code.

How I worked with Development and Continuation teams was DRAMATICALLY different.

When working with "Development Engineering" (they wrote new code) I always asked them:

"What does X, Y, Z do?"

Effectively I always asked them what they think the code "should" do and what the results should be. There was no point in presenting them "OMG it doesn't do the thing" because they would just panic, get defensive / shutdown. Rather I understood they knew what their code does (well what they thought it did) and asking them that was the key to get them talking / sharing.

After I had their words and phrasing I could better present "Hey we see A, B, C under D conditions. I expected to see X, Y, Z." That would get a lot more buy in from those folks.

I also had to avoid some of the more obvious "hey it does G" where G would obviously break the feature entirely... they often didn't understand use cases (one guy I'm pretty sure didn't even know what the product did, but he could program an ASIC for sure..) so you had to be careful about spelling out customer experiences and rope in a program manager if you felt there was a fundamental problem.

When working with Continuation Engineering (bugfixes and etc):

These guys were much more receptive to the customer's story on on how the customer is using the code / equipment, and you could much more quickly present "Seeing A, B, C, under D conditions. So then I changed L and got M..." and so on. Continuation engineering grocked the meaning of why someone would change L... and so it was helpful. Continuation didn't often know what the code "should do" (or they weren't confidant in their understanding) so just saying "Saw A... that's not right" meant nothing to them. In contrast development engineering needed to talk about the happy path first.

Note that much of this occured after I gave them a good writeup / heads up of all the information I had (even if they didn't read it I never kept anyone in the dark).

It got to the point that if a panicked support call came in and it was end times and someone gave engineering a heads up, they would ask it be assigned to me if I was in the office. Technically I was far from the best tech, but I could talk to engineering.

It was telling how much a difference there was between debugging and initial development.

<comment><USER36> > Effectively I always asked them what they think the code "should" do and > what the results should be.

[https://rubberduckdebugging.com/](https://rubberduckdebugging.com/)

<comment><USER35> Pretty much, just warming them up to get them into debug mode ;)

For me I (and the customer) all thought we knew what it should do, but it's always good to spell it out for everyone so we all understand when / or even if we're seeing an exception.

Lots of "Woah hey is that what the protocol really says?" moments too.

</comment></comment></comment><comment><USER37> Hit a performance cliff and rewrote a simple divide-and-conquer multithreaded application into a two-thread solution using hand-rolled map and list structures using optimistic concurrency with atomics and paying attention to dining philosophers. Didn't work first time, funnily enough.

For about a day I was terrified that it would never work. Then I found my mistakes (two).

Except in emergencies like this, my time is much better spent writing "inefficient" yet easy to develop code that everyone can understand.

</comment><comment><USER38> This highlights the importance of writing clean, legible, and well-thought-out code but is also an excellent example of hyperbole and false statistics. I've debugged my code consistently for 15+ years (which if you were to take this as law, would imply I doubled my intelligence with a crazy-high compounding rate, and while I enjoy my ego, that's a bit too much for me).

</comment><comment><USER39> I'm tickled pink to see Hyrum's Law included here. Hyrum and I were grad school chums. Not too many brighter folks out there.

<comment><AUTHOR> That's awesome! Hyrum's Law is one of my favourites in the repo and one I share the most often :)

</comment></comment><comment><USER40> Writing code that can be debugged is a skill that can't be taught, only learned from experience. Sometimes painful experience.

</comment><comment><USER41> See also Mark Miller's adages: [https://web.archive.org/web/20090215132813/www.caplet.com/ad...](https://web.archive.org/web/20090215132813/www.caplet.com/adages.html)

</comment><comment><USER42> Such a good read, thanks for sharing this page! (And the authors of course)

</comment><comment><USER43> This is one of the things I learned to really appreciate about Go. Code readability and comprehensibility is way better than any other language I've ever used.

</comment><comment><USER44> The output of a machine learning algorithm is code that no one was clever enough to write. In light of Kernighan's law, that should give you pause.

</comment><comment><USER45> Although curiously it's a _lot_ more difficult to write simple code (if you want it both complete and correct).

</comment><comment><USER46> Eh... Not denying the law is correct, but this kind of thinking makes more sense when you're not doing some cloud SaaS solution that can have daily changes due to business needs and continuous deployment. Code can get complicated over time because business moves fast and there isn't time do everything perfectly. It isn't always people trying to be too clever by a half.

<comment><USER47> The business folks can be too clever too.

Dealing with that at work right now. We got super clever, changed a couple times, tried to be even more clever, and engineering gets to pick up the mess and try to make it all work.

</comment></comment><comment><USER48> I say: get clever, spend the inordinate amount of time debugging, then level up.

</comment><comment><USER49> Disappointed that there was no "Greenspun's tenth rule". It's HN, after all ;)

</comment><comment><USER14> title would be less vague as

/s/it/your cleverest code

<comment><USER50> > _title would be less vague as_

True but that's too many characters for HN's title field. Which might be why it was submitted with editorialised version.

> _/ s/it/your cleverest code_

massive nitpick but you should be suffixing rather than prefixing the regex with a slash:

        s/it/your cleverest code/

As I said, this is a massive nitpick; literally the only reason I pointed this out was because the submission was about debugging code and I liked the irony of debugging someone's posted about debugging code :)

(there is another law somewhere about people who nitpick are prone to making mistakes in their corrections thus getting nitpicked themselves -- I'm hoping I'm not exception hehe)

<comment><USER51> "there is another law somewhere about people who nitpick are prone to making mistakes in their corrections thus getting nitpicked themselves"

[https://en.wikipedia.org/wiki/Muphry%27s_law](https://en.wikipedia.org/wiki/Muphry%27s_law)

</comment></comment></comment><comment><USER52> Another way to think about this: The more steps that exist between the developer and the end product the less likely the developer is to care or identify whether their solution is clever.

</comment><comment><USER53> And if you write Laws thinking you're Moses, you're not smart enough to code.

<comment><USER54> You don't think _Brian Kernighan_ is smart enough to code?

</comment></comment><comment><USER55> Why do we pay attention to BS like this?

</comment><comment><USER56> Which brings us to the interesting question: Is God smart enough to debug it's own code?</comment>


How to Keep a Zibaldone, the 13th Century's Answer to Tumblr - <AUTHOR> http://www.atlasobscura.com/articles/how-to-keep-a-zibaldone-a-13thcentury-answer-to-tumblr <comment><USER2> I don't have a Zibaldone, but I do like my own wiki (vimwiki) which I love. It's a complete hodgepodge, a place to put links, snippets, stuff to do, whatever. And it's just markdown files under my own control. When I read books on the kindle, I extract any highlights and feed those in too, same for video's I've watched. I wrote a very simple android app (horrible horrible hack) with which to write notes / share links which feeds into the vimwiki. It's great having all the content go into one place, or it is for me anyway.

So yes, I reckon a personal wiki is a modern-day Zibaldone, and I recommend trying one out.

<comment><USER3> I have a friend who does a simple version of this on GitHub: [https://github.com/yoshuawuyts/knowledge](https://github.com/yoshuawuyts/knowledge)

I've been considering doing the same.

<comment><USER4> Why git? I can't think why I'd want to "roll back". Does he (assuming he by the name) feel the commit history and portability is worth the overhead?

<comment> qu4z-2 See, I can see why you'd put it in git (because it's easy) but not sure why you'd upload it to the cloud (ie github).

<comment><USER5> Git allows you to roll back in case you removed something you shouldn't have. The cloud allows it to be easily accessible on your phone, and easily able to be shared should one want to do that.

</comment></comment></comment></comment></comment><comment><USER6> I've held several jobs which required me to keep an Engineer's Notebook. It was the place were I jotted down all my ideas, thoughts and random pieces of info I needed or found useful. It served 2 purposes:

1\. I could go back in time and jump directly to the solution of something that I _knew_ I had solved but couldn't remember how.

2\. It was used by my employers to save in case a patent case arose and they had to find evidence pointing to engineers working on a particular idea. For that reason, the notebook had to have sequential numbered pages, entries had to be dated and binding had to be sewn (to decrease the likelyhood of tampering).

<comment><USER7> Regarding the first point, that can be done using any common note-taking software. I use a mix of Evernote and Zotero. I think both are good but each has its use.

I tried to use a simple Markdown-based folder structure, but it just was too unwieldy for my tastes.

</comment></comment><comment><USER8> AFter reading other comments here, I realized I've sort of been doing this at work for work related stuff in emacs with org-mode for years. I kind of like the idea of a paper-and-pen version that I carry with me. I mean, I'd rather type it, but not on a phone keyboard, and so the next best portable thing seems like paper and pen.

<comment><USER4> I've been messing a bit with Orgzly and it's cool, but a bit fiddly (having to press the plus sign to add an entry rather than just typing seems to defeat the spirit for me). I'll likely migrate back to paper/pen, but I'd like to limit the number of things I carry in pockets, and I know I'll have my phone.

What I'd like is some sort of Google Now integration with org mode so I can take notes verbally or quickly with my phone, then revisit them in org-mode later.

<comment><USER8> Agreed, Orgzly is close. The manual sync thing is weird too.

</comment></comment></comment><comment><USER9> I use e-mail for this. I have a label in Gmail called "saved", looks like it's got 514 threads dating back to 2004 (when I got a Gmail account). I e-mail the note/link/picture/screenshot/attachment to myself along with a few keywords so I can easily find it later, add the label, and archive the message. Then I get all the power of Gmail search to find it in the future.

</comment><comment><USER10> Curiously enough I've been using this thing I discovered here on HN a while back called Nuclino for this sort of stuff.

I've rather enjoyed it, though one downside is that it's only 80% portable for me. as in I can't really be bothered to use it with my phone and using it while hiking is a no-go for obvious reasons.

All well and good, that's why I keep Field Notes around for.

<comment><USER11> Their site is giving me a bad cert error</comment></comment>


How do I regenerate a microbiome? - <AUTHOR> in lieu of https://news.ycombinator.com/item?id=21892351, how does regen their gut bacteria ? <comment><USER2> Eat a wide variety of foods, especially fermented foods (like real yogurt) and foods high in fiber. Supplement with probiotics and inulin.

For lots more, Google this query: increase microbiome diversity inulin yogurt probiotics

If you want to see how diverse your microbiome is or becomes, buy a mail-in test from Viome (or if/when it relaunches, uBiome).</comment>


Ask HN: Best Read on HTTP Cookies? - <AUTHOR> Are there any articles or books that you guys recommend I go through to learn about implementation details of how cookies are done in the real world?<p>The only meaty material I found is the RFCs <comment><USER2> You can read the source code for the major open source browsers (Chromium, Firefox). I'm not sure if you mean the implementation of cookies or the application of them.</comment>


Verified by Twilio - <AUTHOR> https://www.twilio.com/blog/introducing-verified-twilio <comment><AUTHOR> I know it's overkill, but this is my current setup for my phone numbers. I treat phone numbers the same way I do my email address: I use different numbers for different purposes.

1\. I have a "clean" phone number I only share with friends/family. Certain prefixes and phone number actually receive next to zero spam calls. This number has received 2 calls in the years I've had it.

2\. I have a phone number I share with companies / loyalty programs / etc.

3\. I have a legacy phone number I've had for 10 years that receives multiple spam calls daily. For this number I've set up an incoming handler that uses Google Cloud to ask the caller a simple question (What is five plus two?), parses with the Speech Recognition API, and forwards calls with the correct answer. The rest go to voicemail. It's better to use the speech recognition API than it is to ask the caller to press a DTMF tone, in case they are in a car or are calling from a device without a telephone keypad. If I every need to, I'll add this protection to use numbers 1) and 2).

4\. I have a number that rings straight to voicemail.

All numbers use the Whitepages and Nomorobo APIs from Twilio. These score the "spamminess" of calls. Calls with sufficiently high values are forwarded to voicemail.

For incoming calls, I set the caller ID for the calls to my SIP clients to indicate which number is being called, and include the spam score.

<comment><USER3> Is this a uniquely American problem?

My Australian mobile phone receives maybe one spam call per year. It's my single phone and I give out the number to various businesses and people.

<comment><USER4> In Denmark, phone sales to consumers are generally illegal though strangely insurance and newspapers are excluded.

There's a national "do not call" list you can sign up to as well. This is amusing called the "Robinson List", presumably after Robinson Crusoe. The same concept is used in a dozen other countries.

I guess the penalties are severe enough that it generally only very shady people call, such as those pretending to be from Microsoft and calling to fix your computer.

<comment><USER5> Rather off-topic, I was surprised by how readable this was to someone who doesn't speak a lick of Danish (me):

> Robinsonlisten er opkaldt efter Robinson Crusoe, som er en fiktiv person, > der optræder i romanen Robinson Crusoe af Daniel Defoe. Han strander på en > øde ø og lever isoleret.

<comment><USER6> True but if a Danish person was to read this to you you would not even recognize the individual words...

[https://www.youtube.com/watch?v=s-mOy8VUEBk](https://www.youtube.com/watch?v=s-mOy8VUEBk)

EDIT: Just FYI this is a clip from a Norwegian comedy program. Danish and Norwegian have very similar written languages but sound very different when spoken. In reality with some practice Danes, Swedes and Norwegians can speak together and understand one another.

</comment></comment></comment></comment></comment><comment><USER7> Just what we need, another startup injecting themselves into our lives. And then for the right price, "acceptable robocalls" can be Verified by Twilio™!

SHAKEN/STIR[1] is a much more network-agnostic solution to the same problem, using essentially the same, well-established technology used by internet certificate authorities. As such, it has all the same basic problems that CAs do, but it's a hell of a lot better than tightly coupling yourself to Twilio.

[1] [https://en.m.wikipedia.org/wiki/Adblock_Plus#Controversies](https://en.m.wikipedia.org/wiki/Adblock_Plus#Controversies)

[2] [https://www.consumerreports.org/robocalls/spoofed- robocalls-...](https://www.consumerreports.org/robocalls/spoofed-robocalls- relief-is-on-the-way/)

<comment><USER8> Looks cool but wouldn't it be cheaper and easier to just outlaw robocalling like the rest of the world has?

<comment><USER9> The telephone network is international. The US is limited in its ability to enforce anti-spam laws against operations running out of other countries. (They also seem unwilling or unable to enforce these laws domestically, but that's a completely separate issue.)

<comment><USER7> In this case I think it's a case of being _unable_ to enforce the laws, because the technology for verifying callers is broken, making it easy to spoof phone numbers--even the US government doesn't know who is calling you illegally. SHAKEN/STIR is intended to close the security hole and make caller ID accurate again.

</comment></comment></comment></comment><comment><USER10> I'm skeptical for the following reasons:

1\. Companies like Twilio largely created the explosion in spam calls by providing extremely cheap, programmable calls, so I'm always a little skeptical when the company selling the poison is also selling the antidote. 2\. I really don't like the idea of a single company being responsible for a global verification system for something like phone calls. How are decisions made over what is verified (e.g. the example shows "call purpose" snippets on the screen - if I put something there saying "I have an important flight update", but then my call is "You're only getting free peanuts on your flight, but if you enroll in this MileagePlus card you'll get snack mix too!" who judges that?)

Overall I'd just say with all the consolidation of power with US tech giants I am extremely wary of giving them anything else that could consolidate that power further. I would much prefer an open, federated model.

<comment><USER11> Twilio is not responsible for much spam at all. Their prices are way too high to begin with. And you have no way to spoof your outgoing number.

<comment><USER12> One would just use a turn-and-burn via twilio. Get 10000s of cheap robot calls out, use their AVMD and have a ball

</comment></comment></comment><comment><USER13> > 70% of consumers don't answer a call if the caller's number is anonymous. > While that might cut down the number of unwanted robocalls, it also > potentially prevents consumers from receiving vital communications—calls > from the doctor, banks that are flagging issues, schools calling about > parents' children.

A message to my doctor, bank, kids' school, etc: if you call me on a private/unknown number and don't leave a voicemail (because I can assure you that I am in that 70%) I will assume that you are spam. You don't need Twilio for this, just some basic phone etiquette.

<comment><USER14> Patients would call my wife's office after hours and leave their number with the answering service for a call back. After hours calls were triaged into three groups: Emergency, tell patient to call an ambulance; Urgent, get patient a prescription or tell them to get to an after hours clinics; Non- urgent, patient can wait until the office is open and make an appointment.

Of course, when she made calls back with her desk or cell phone, Caller ID would display a number that was different from the main number of the clinic, and most patients would't answer, even though they were expecting a call they'd requested. I finally built her an app that would dial the phone and spoof the Caller ID with the clinic's main number. (This was also helpful because my wife was not the only on-call doctor, so a direct line to her was only occasionally the correct number to dial after-hours. The best move for patients was to always dial the main office number.)

</comment></comment><comment><USER15> I was just reviewing my call detail records on my "home" number yesterday, and realizing that the vast majority are spam - we pretty much don't answer that phone. This got me thinking about alternatives to even ringing for any unrecognised numbers, such as an IVR menu that said something like "Hi, we're screening spam, press 4 2 if you're a real person". I'm not sure how that'd be accepted by, for example, the dentist office though. Has anyone set something like that up?

I really hope Twilio is successful with this - it would definitely help, so long as there's a way to avoid abuse.

Just in case someone asks why I still have a "landline": it's VoIP, and I maintain it for basically two reasons:

* We use our VoIP number anytime we have to give a number out as part of signing up for something, rather than cell numbers

* It typically costs me under $25/year, which is actually mostly the 911 fee (oh, and it's an extra way to call 911, if we need to)

<comment><USER16> >Has anyone set something like that up?

Yes! I did "Captcha for phone calls" for a school project earlier this year. It would play a quick message along the lines of "Hey, to ring my phone just enter these numbers: XYZ", and redirect the call to my phone if the captcha was correct.

The challenge was redirecting calls to the challenge even if they were originally directed to my phone number - the solution was to write a custom call handler (Android) that rejected calls if they weren't in my phone book, and then set up call-forwarding on rejected calls to go to a Twilio number with the challenge. There was a nice edge case where if I actually missed the call, it would just redirect you to the challenge again... but it was good enough for the class.

While testing, I accidentally actually got someone to use it! A guy repairing my watch called me, got directed to the challenge, and actually followed directions! Which was surprising to me. I also caught some obvious spam callers (same first 6 digits), and unsure if there were any other real people who did call and got confused or just hung up.

I've been meaning to clean up the code and publish it, but haven't gotten around to it - there's lots of hardcoded numbers in there.

(Disclosure: currently employed by Twilio)

<comment><USER17> Off topic: what kind of watch?

<comment><USER16> Tisell Bauhaus, a very plain and simple design (though not immune to being dropped, which is why I needed the repair).

</comment></comment></comment></comment><comment><USER18> Well that's cool I guess. But:

1\. How are they partnering with all of the device companies to get this on at least most phones? That'd be interesting to know.

2\. I guess somebody has to type that intro text. As long as they're doing that, why bother with a normal call? How about just sending the text instead. Or send a text and let me call back for more details when it suits me, either right away or in 10 minutes. It sounds like whoever is sending these probably has a call center or somebody who's job is to sit around and answer the phone when it rings.

<comment><USER19> 1\. I'm really not sure yet. They haven't given any details publicly or privately. I'm most curious to see how this will be supported on iOS.

2\. The demo on the keynote gave a great use case where a school nurse called the speaker about their child, and she was very lucky to pick up and not let it go to voicemail.

There are still a lot of use cases for calls to hammer out important details, even if they are becoming rarer, like health or car maintenance, or places where sms is associated more with spam. This is a great way to get the best of both worlds, even if it's technically during the decline of calls.

<comment><USER13> > a great use case where a school nurse called the speaker about their child, > and she was very lucky to pick up and not let it go to voicemail.

a) if you're a sloppy enough parent that you don't even have your kids' school phone number saved as a contact, I suspect you're accustomed to relying on luck.

b) "Use our service or your children might die" is fairly over the top marketing, even for marketers.

<comment><USER20> My kids' school doesn't have _one_ number. It varies, and they also use emergency alert systems (presumably via something like Twilio) that don't always come from a predictable number either.

<comment><USER13> I don't know why I care about this - it's a service offered to business and if business wants to take it up that's their call. But if I were your kids school I would be looking at the far simpler solution of having one outgoing caller id regardless of the extension. I've had kids in school for 12 years now and have never received emergency communication from them. For the great balance of history, we've trusted schools to be able to make on the spot decisions on our behalf because we didn't have a phone in our pocket. And rightly so - <USER2> school has multiples of experience dealing with 8 year olds than I will ever have. What could they possibly need to consult me about in real time?

<comment><USER20> > What could they possibly need to consult me about in real time?

Your kid had an accident and needs new clothes. Your kid fell and is headed to the hospital by ambulance. Your kid has a headache, can we give them ibuprofen? etc.

<comment><USER13> 1\. Not an emergency, but I'll see what I can do (although the 3 schools my kids have been in have had spare clothes on hand for these situations)

2\. Good. I'll be there when I can being that you're not asking for my consent and are more than qualified to make that decision

3\. You already have a list of my child's allergies. Follow your policy which I have already agreed to, but thank you for the non-urgent notification

etc

</comment></comment></comment></comment></comment></comment></comment><comment><USER21> I will install whatever app I need to get this service if it works as advertised. Well done and congrats to the team(s) that worked on this project. It's a travesty that we need a private company to solve this problem, or even that the problem exists in the first place when we as a society could have legislated it away a while ago. However, a fix is a fix.

</comment><comment><USER22> > it also potentially prevents consumers from receiving vital > communications—calls from the doctor, banks that are flagging issues, > schools calling about parents' children.

That's bullshit. I have these important number in my address book. In the unlikely case the doctor wants to call me from a phone booth or from their friend's phone (why?), they'll leave a message and I'll call them back. As for the bank, they know my e-mail and I asked them not to use the phone for communication anyway.

Really, for many people there is no reason to answer unknown calls at all, especially if their experience is that most of these is someone wanting to sell something to them. Not to mention in our times asynchronous communication is very important for mental health.

<comment><USER23> Your response is the same bullshit you call on the message you're replying to.

\- I don't have the 10+ numbers that dial out from my doctor's office in my phone

\- I don't have the 5+ numbers that dial out from my child's school in my phone

\- I don't have the 10+ numbers that dial out from my child's doctor's office in my phone

\- Banks (god knows how many numbers each from multiple banks) \- etc

It's literally impossible to have all the numbers I want to allow to reach me in my phone.

The fact that _you_ happen to have a very limited set of numbers you need to be able to be called from does not invalidate the fact that many people have a vast amount of such numbers. Don't assume your anecdotal data speaks for everyone.

</comment></comment><comment><USER24> > Even more concern, 70% of consumers don't answer a call if the caller's > number is anonymous. While that might cut down the number of unwanted > robocalls, it also potentially prevents consumers from receiving vital > communications—calls from the doctor, banks that are flagging issues, > schools calling about parents' children.

I don't understand how this helps with anonymous callers?

<comment><USER25> Seriously?

Most corporations call from a featureless pile of numbers. Now you wont miss the important calls from numbers you'd otherwise consider anon spam.

<comment><USER26> Surely if you are verified by twilio, you aren't anonymous?

</comment></comment></comment><comment><USER27> My phone is always on silent and I don't pick up any calls except if its from my started contacts or if it's a pre-scheduled meeting. I do check my phone every hour to see if there are any important missed calls. This has helped me so much from distractions and overall well being. It took me a while to realize that there is nothing that requires me to respond immediately and in case of emergency I shouldn't be the one responding. There's 911 for that.

</comment><comment><USER28> With the new iOS beta allowing non recognized callers (not in address book) go straight to voice mail, I see little use for this service now than before.

<comment><USER8> It seems this service would allow calls from anonymous numbers to be identified by the receiver. Just routing unknown numbers to voice mail will lose you those calls. For example, my doctor's office always calls from "unknown number".

</comment></comment><comment><USER29> > Even more concern, 70% of consumers don't answer a call if the caller's > number is anonymous.

The concerning part is 30% still answer the calls.

<comment><USER30> Unfortunately sometimes important callers use anonymous numbers. I recently had to deal with this in the public healthcare system where I live (not in the US).

A certain person who needed to give me an appointment kept calling at unpredictable afternoon and evening times from an anonymous number, ignoring the voicemails I left with my availability info, and not proposing specific times in her voicemails to me.

I did get the appointment in the end, but only by happening to notice one of her calls in real time - still from an anonymous number - and answering.

I'm told similar important uses of anonymous numbers aren't too rare in this public healthcare system.

</comment></comment><comment><USER31> Interesting. Wonder if this is going to be a valuable public service in the long run. For sure the monetary incentive is for Twilio to operate this like an "acceptable ads" type shakedown [1], where companies pay Twilio to mark their calls as "valuable information" rather than spam.

Arguably this should be administered by some trusted third party, like EFF.

[1] [https://en.m.wikipedia.org/wiki/Adblock_Plus#Controversies](https://en.m.wikipedia.org/wiki/Adblock_Plus#Controversies)

</comment><comment><USER32> Huh? "We're partnering with leading call identification apps"

Are that those shady apps which require your address book access just to start, aren't they?

<comment><USER33> How do you think those companies identify numbers? I'm guessing they see if any of their other users have the number saved.

Of course they could still be doing shady things with people's contact lists beyond that...

<comment><USER32> They could do that silently or at least in some optional privacy-oriented way, e.g. like Telegram or Signal.

Instead they are actively implementing features to disclose everything to everyone.

</comment></comment></comment><comment><USER34> Received this unsollicited pearl from Twilio subsidiary Sendgrid today:

Hi aithrow,

One of my special interests is Whitebox machine learning, and has been for many years

With the dominance of Blackbox techniques causing lots of ethical questions to be raised on the uses of AI, I think it's time to revisit Whitebox techniques.

I've written a couple of non-technical articles on the subject on Medium:

Why, if you're planning to use AI you need a Whitebox system

Why are tech firms ignoring half of AI?

If you want to read about our Whitebox solutions please look at darl.ai

Whitebox machine learning is available (free) through our GraphQL API at darl.dev

Thanks for reading,

Andy Edmonds Doctor Andy's IP email@example.com

P.S. A glitch in our mail handling software meant that a small number of unsubscribes were missed on our last mailshot. If you were one of these my apologies. Just unsubscribe again, we won't miss you this time!

Sent by Dr Andy's Back Office AI system

If you'd like to unsubscribe and stop receiving these emails click here .

</comment><comment><USER35> I wonder how many of the spam calls are actually powered by Twilio? Is this an instance of a company trying to make money by getting paid by businesses on one hand to do spam calls, and getting paid by consumers on the other hand to block spam calls?

<comment><USER36> >I wonder how many of the spam calls are actually powered by Twilio?

Probably not very many. Think about it: the spammer has to pay for each minute of call time on Twilio. Let's say a spammer spams for 12 hours a day at $0.013 a minute. On average that's $9.36 a day. So if they want to do this with 24 "lines", that's $224.46 a day.

Now consider the the cost of (on-premise) phone lines provided by a company like Time Warner Cable: $25-$45 per line per month (the last time I checked). That's more like $36 a day. So with Twilio the spammer would be paying 6x the cost.

</comment></comment><comment><USER37> Android already does this, somewhat.

Whenever you receive a call you know beforehand if it's spam or not. Maybe a handful of times in the last year I've received a spam call that wasn't marked as spam. You can block callers and report as spam much like in Gmail.

When the call is not spam Android looks up on Google where the number is from and it shows the name of the business in the call screen.

This is working great for me. A year ago I received 2-3 spam calls per day which of course I've been blocking. Now I receive 1-2 spam calls per week at the most. Some weeks I don't receive any.

<comment><USER38> I've never seen a call marked as "spam" on Android. What kind of phone do you have, and which carrier?

Edit: My phone is from Samsung (who like to add a bunch of their own software to base Android) and my carrier is AT&T.

<comment><USER39> All the Pixels use Assistant to screen calls from numbers that aren't in your contacts. You watch a live transcript of the caller / bot and you can answer if it's legitimate or have it deliver a canned "stopping calling me and take me off your list" which also reports the caller[1]

1: [https://www.theverge.com/2018/10/9/17955274/google- pixel-3-s...](https://www.theverge.com/2018/10/9/17955274/google-pixel-3-spam- calls-assistant-screen-filter) (I have Pixel 2, so the feature isn't just > 3 at this point)

</comment></comment></comment><comment><USER40> Hate this is a problem that needs to be solved.

Love that Twilio is trying to solve it with a cool, seemingly useful approach.

Hate that Twilio will probably use something proprietary for this that locks people into their ecosystem even more.

</comment><comment><USER41> Been running iOS 13 with Silence all unknown calls..send them to voicemail option.

Spam calls are no longer a bother Nor are really seen and important messages from those not in my contacts leave voicemails.

</comment><comment><USER42> I can see few cases when I have not picked up calls but with this kind of verification I would pick up if they ID correctly. Calls coming from 1: hospital 2: library 3: my cable or phone company 4: my car or insurance company etc.

I don't have their corp numbers saved and if we can see who is calling then it could be helpful.

</comment><comment><USER43> Does anyone have a recommendation for a call filtering app that does not leak your contact list to the app developer or a third party?

Open source is preferred, but any app with a solid privacy policy would do. It would be nice to see suggestions for both Android and iOS.

<comment><USER44> iOS call blockers using the built-in API don't have this issue, it's like the content blocking for Safari but for calls. You enable them under Settings->Phone->Call blocking. See [https://news.ycombinator.com/item?id=15025709](https://news.ycombinator.com/item?id=15025709)

</comment></comment><comment><USER45> If this works as advertised, this is great. Because of all the spam calls people just don't bother answering the phone at all.

We just started testing texting people before we call them and getting consent on an opt in form to receive a text message.

Im still not sure on the "how"

</comment><comment><USER46> I have several pbx systems running, and granted this is a newer concept. Maybe this will help residential calls, business calls on the other hand will most likely allow the consumer to still take the path of least resistance.

</comment><comment><USER47> How does this compare to the MNOs joining together to create Project Verify.

[https://mobileauthtaskforce.com/](https://mobileauthtaskforce.com/)

</comment><comment><USER48> Is anything preventing debt collectors from being Verified by Twilio?

<comment><USER49> I would expect that legitimate debt collectors will be verified by Twilio.

> Verified By Twilio displays company branding and a reason for the call right > in front of the consumer

This should make the purpose of debt collection calls obvious, though I don't see how they plan to enforce accurate call descriptions.

<comment><USER20> > I don't see how they plan to enforce accurate call descriptions.

A reporting/flagging system, perhaps? "This call wasn't as described". After enough reports, they lose the verified status.

</comment></comment></comment><comment><USER50> _> Imagine a world where you receive a phone call and know exactly who it's coming from and what they're calling about before you press the answer button. Crazy, right?_

Now realize that it's not you who knows it, but rather a company that is tracking all your phonecalls and decides _for you_ whether you should answer or not.

 _> Through the programmability of the Twilio platform, businesses will also be able to assign a purpose for each call to give further context._

So a malicious actor can assign a legitimate purpose to his phone call ? No thanks

</comment><comment><USER51> Unrelated, but since this is a twilio thread ...

Could you please, for the love of God, give us an email verb in twiml ? Please ?

</comment><comment><USER52> Isn't robo calling about to be illegal?

<comment><USER53> It is illegal in many countries but it happens nonetheless.

<comment><USER8> Not in some countries, because they have actual enforcement of the law.

<comment><USER54> The spam operations are usually overseas, not US based. This makes enforcement harder and takes longer time. It still happens, because I see news about busts in the press from time to time.

<comment><USER55> I wish I could tell T-Mobile to never accept calls not based in the US

</comment></comment></comment></comment></comment><comment><USER56> Are there any good apps on iOS that help block unwanted calls, and identify callers?

</comment><comment><USER57> My final and unfortunate solution for this was to stop accepting phone calls altogether. Friends/relatives know to use Skype and IM, all important stuff goes to voicemail, everything else is on permanent mute and ignore. I wonder if it is the case for more and more people.

</comment><comment><USER58> Why won't spammers just spoof the verified by twilio data?

</comment><comment><USER59> Nothing to do with Twilio: If the recipient is a mobile number, can't just the caller text the recipient when they don't pick up? E.g. IM YOUR DOCTOR, CALL THIS NUMBER

</comment><comment><USER60> Sounds like CNIP from ISDN.

</comment><comment><USER61> FFS USA, not every problem is a problem to solve by "the free market". You can see the America-Is-Great Mindset in a lot of these comments: _Even_ the US has a problem with robocalls, so that _must_ mean it's an unsolved problem obviously. So we have to solve it with our superior technology!!

Just no. Almost nobody else in the western world has this problem. How? Regulation. In Germany, if you get reported for unsolicited calling, the Bundesnetzagentur will come to you and give you a very hefty fine, per violation. Problem solved. I remember exactly four unsolicited calls in my life, two of them on a number which was published in a phone book for 15 years.

Spam calling is not something were there is net value created for society so we need an elaborated discussion of what is ok and what is not (like e.g. paid universities). It just sucks. Make it illegal, and then actually enforce that. The idea of creating a million dollar industry to create a telemarket arms race is...not good.

<comment><USER62> I have lived in the U.K. for more than 8 years. I receive spam calls daily from a combination of London and blocked numbers. Everyone I know here also receives them frequently. I'm on the U.K. do not call list. It's endless PPI, "your recent accident", and just normal phishing attempts. When I'm in the Middle East it's endless spam texts. I guess Germany has it figured out but by no means is this an American only problem, or a solved problem in the Western world.

<comment><USER63> I'm in the UK and have had the same number for a decade or so, I probably get one a year? They are mainly from foreign companies. If said countries also had our regulations, and a sharing agreement in place, it wouldn't be an issue.

<comment><USER64> I am in the same situation yet I know people who are plagued by such calls. There must be something that triggers this off?

Having said that, these people will also give their phone number and email address to anyone who asks whereas I do not. If I have to enter a phone number on a form and it's isn't immediately obvious why I should do that then I'll make-up a number.

<comment><USER65> If you're not doing so already, please pick numbers that will never end up in use by someone else. E.g. for UK: [https://www.ofcom.org.uk/phones-telecoms- and-internet/inform...](https://www.ofcom.org.uk/phones-telecoms-and- internet/information-for-industry/numbering/numbers-for-drama) (expand the Notes section)

<comment><USER64> That's a useful list I never knew existed. Thanks.

</comment></comment></comment></comment></comment></comment><comment><USER66> This may be the most valuable innovation out of Silicon Valley in 10 years. Talk about solving a pain point!</comment>


Why You Shouldn't Trust App Store Reviews (and What to Trust Instead) - <AUTHOR> http://lifehacker.com/why-you-shouldnt-trust-app-store-reviews-and-what-to-1515379780 <comment><AUTHOR> Further down in the comments of this article is a comic from xkcd that just about sums up app store reviews:

[https://xkcd.com/937/](https://xkcd.com/937/)

;)

enjoy!</comment>


27 Hospital Workers Suspended for Security Violation - <AUTHOR> http://www.cnn.com/2007/SHOWBIZ/10/10/clooney.records/index.html How did they get caught?<p>If every viewing of data in the app is stamped, didn't the users realize what would happen?

<comment><AUTHOR> How did they get caught?

And if every viewing of data in the app is stamped, then didn't they realize what would happen?</comment>


The Lost 1984 Video: Young Steve Jobs Introduces the Macintosh - <AUTHOR> https://www.youtube.com/watch?v=2B-XwPjn9YY <comment><USER2> This is really great --- really loved the music. What is the name of the song ? Can't seem to recall it ... very majestic, beautiful

<comment><USER3> It is the "Chariots of Fire" theme song by Vangelis. It was part of a soundtrack for a movie of the same name that was popular around that time.

</comment></comment><comment><USER4> Did someone delete the video or is it not reachable from Sweden?</comment>


Recess PHP Framework Public Preview - <AUTHOR> http://www.recessframework.org/preview/

<comment><USER2> Thanks for the link. Recess is a full-stack PHP framework I've been working on the last 12 weeks or so. This preview release is intended to demonstrate core functionalities and help work out major issues. Over the next couple of weeks I'll be focusing on documentation, screen casts, and stabilization.

There are some interesting and novel techniques being used in Recess that you won't see in other popular PHP frameworks. It's a framework I believe will cater well to hackers and the HN crowd.

<comment><USER3> It will be great to see some comparison with popular frameworks - Cakephp, Zend, CodeIgniter, etc.

Please put some demo and allow downloading without the registration.

<comment><USER2> This is towards the top of list of documents to produce. More screen casts demonstrating other features of the framework are also on the way.

The overarching mission of Recess is to provide a full-stack framework with functionality along the lines of Cake but designed for PHP (as opposed to attempting to clone Rails which leads to poor design decisions as PHP != Ruby). This way Recess will achieve the performance characteristics of CodeIgniter without sacrificing higher-order functionality of modern full- stack frameworks like Django and Rails.

For links to download without registration: <http://news.ycombinator.com/item?id=410119>

</comment></comment></comment><comment><USER4> To: all software project websites

Please include code samples or screenshots (whichever is appropriate) on the first page of your site.

Thank you.

<comment><USER2> Will do, thanks.

</comment></comment><comment><USER5> I'll download and play with it the moment I don't have to register to do so...

<comment><USER2> mihasya, understand your concerns. I have high confidence in the HN crowd so take your pick :)

Zip: [http://www.recessframework.org/preview/content/files/recess-...](http://www.recessframework.org/preview/content/files/recess-v0.10pp.zip)

Tarball: [http://www.recessframework.org/preview/content/files/recess-...](http://www.recessframework.org/preview/content/files/recess-v0.10pp.tar.gz)

Registering at this stage is required as solid documentation/tutorials are still on their way (now the top priority) so having a simple, single place to go for support/discussion is important to help bridge the gap between now and then. Download away but reconsider registering if you run into issues or questions.

<comment><USER6> Everyone should take note of this.

Even exceedingly tech-savvy users will avoid using things that require registration. By all means - encourage registration, but if you _require_ it when it isn't strictly necessary, you'll run off most of your potential users.

</comment></comment></comment><comment><USER7> looks interesting. I like the way it promotes good coding practices and your idea of relative routes is cool. Don't know about including routing info in controllers though. I wasn't able to tell what the benefit of doing it this way is over having a routes config file. Seems that the routes would get hard to manage in a large app (even with the route browser tool).

+1 for not having to register to DL

<comment><USER2> Thanks for the feedback. The belief is that though this practice may feel uncomfortable at first it should make things easier to mange in large applications. The premise is routes don't exist without a controller and method to route to - so why duplicate that information in another place? With routes stored somewhere else if you want to change a method name in a big app (especially one you're not the sole author of) there is an extra step of finding all the places that method name was potentially referenced. Routes in controllers is an attempt at being more DRY than routes.rb and less raw than Django's list of regexps.

For what its worth Rails actually generates code with comments that give examples of routes in its controllers: <http://tinyurl.com/5jstgu>

Regarding not having to register, here are links directly to the bits: <http://news.ycombinator.com/item?id=410119></comment></comment>


Recreating Lost Colors - <AUTHOR> https://www.nytimes.com/2018/09/05/t-magazine/pedro-da-costa-felgueiras-recreating-lost-colors.html <comment><USER2> Really interesting, and what a cool niche to work in (although I also think it would be ... frustrating).

This part didn't parse for me:

 _[...] racks of wooden-handled brushes with bristles made from rat's tail fur and the hair of pearl divers._

The hair of what? Can "pearl diver" mean something else than a human diver, hunting for pearls? Do they really donate their hair to make brushes? Why them? What ... ?

<comment><USER3> From what I've discovered, it must be a Japanese Lacquer Brush. Traditionally these are made from the hair of pearl divers, "selected as the finest and densest fibre."[1] Another reference I found quotes that "They say that the salt strengthens the hair"[2]

[1] [http://spitalfieldslife.com/2012/01/12/pedro-da-costa- felgue...](http://spitalfieldslife.com/2012/01/12/pedro-da-costa-felgueiras- lacquer-paint-specialist-japanner/)

[2] [https://www.nippon.com/en/views/b02309/](https://www.nippon.com/en/views/b02309/)

<comment><USER2> Thanks! I guess that makes sense, but I didn't think it through. I wonder what the original author thinks of that, do they expect their audience to be aware of this? Weird.

<comment><USER4> Ha, ha, author wants the reader to perk up and do some googling... ;-)

</comment></comment></comment></comment><comment><USER5> Having built my own brush out of a (deceased) fox tail hair made me appreciate the skill and patience it takes to produce an usable, precise tool</comment>


Migration Could Triple Populations in Some Wealthy Nations  - <AUTHOR> http://www.gallup.com/poll/142364/Migration-Triple-Populations-Wealthy-Nations.aspx

<comment><USER2> Great post!- Given the high debt-to-GDP ratio in U.S. and the upcoming onslaught of new taxes, is the high Canadian number reflective of interest of potential American expatriates?</comment>


MIT Economist Has a New Theory of Finance - <AUTHOR> https://www.bloomberg.com/news/articles/2017-10-16/mit-economist-andrew-lo-has-a-new-theory-of-finance <comment><USER2> > The efficient-markets hypothesis says markets are rational and self- > regulating, but it doesn't account for crashes and crises

It's also a computationally insane assertion, if it were true then markets are capable of computing NP in P (which even with the human element would still be an impressive find; especially considering the `n` it's commonly proposed over in "formal" economic settings is in the area of tens of thousands). It's like people who claim they made a perpetual energy machine are claiming things that are physically insane. Except it's not just some cranks, it's entire fields that haven't caught up to the ideas computer science has discovered.

The adaptive market hypothesis appears to resolve this by tying the efficient market hypothesis to the biology of the computational unit. Since biological systems (appear to) obey the principles of computation, the issue is fixed. Overall it seems a better way forward by connecting pseduo-sciencey intuitive aspects of economics (that violate what we now know about the properties of information) to real science.

<comment><USER3> > It's also a computationally insane assertion, if it were true then markets > are capable of computing NP in P

Unless I missed it, they don't assert that markets are _optimal_ , which seems to be what you're saying, but merely _efficient_ and thus, it would be hard (but possible) to do better.

<comment><USER4> Well, actually, they kinda do. Also, [https://arxiv.org/abs/1002.2284](https://arxiv.org/abs/1002.2284)

<comment><USER3> From your link:

> Financially, the "economic calculation problem" of von Mises (1920) and > Hayek (1935) suggests, among other things, that, even if a free market is > not perfectly efficient, it will certainly be more efficient than a > regulatory or government alternative. In other words, even if mispricings > occasionally occur, most of the time they are smaller than any other > alternative system

So the paper possibly disproves a particular kind of efficiency (weak form efficiency), but that's not the colloquial meaning to which I was referring.

</comment></comment></comment></comment><comment><USER5> Andrew Lo's test for long memory/fractal order of integration is standard in econometrics packages such as RATS, Eviews, Stata, etc.

It's a technical clarification of vaguer ideas put forward by Mandelbrot, Hurst, etc. but it still means the man's no Taleb, he's the real stuff.

<comment><USER6> What's wrong with Taleb? Not arguing, just curious.

<comment><USER7> Not OP, but survivorship bias, mostly. He's maybe not wrong, but he's also not very rigorous in his arguments.

<comment><USER8> This.

Also, the unacknowledged inconsistency of his arguments is ridiculous: ~"Unpredictable events happen more frequently than people expect. I'm 100% right about this prediction of the future and you are an idiot if you disagree".

<comment><USER9> I think he meant rather:

\- it's hard (impossible) to predict population statistics from a sample with size 1, ceteris paribus. Over the short term, you have sample size 1 for rare events, over the long term ceteris paribus doesn't hold.

\- people "coerce" everything into a normal distribution in order to get a handle on things which is of course wrong if the thing you look at is not normally distributed. Finance comes to mind.

<comment><USER8> Sorry the two sentences in the quasi-quote are separate things. The first sentence is his book 'Black Swans' and the second is his Twitter feed. I should have made that more clear.

</comment></comment></comment></comment></comment></comment><comment><USER10> Has anyone read Adaptive Markets? If so would you recommend it? I'm concerned about this line from the article:

> Lo's next step is to develop mathematical models to test the ability of his > hypothesis to explain market behavior.

Is there any experimental data to back up his theoretical framework? It's a very tempting idea to buy into (people in general work things out, on an individual level they make a lot of mistakes). I worry about his arguments following logically but falling apart once his theories have been tested.

<comment><USER11> If he is the Lo I think he is then his textbook on econometrics (Mckinsey & Lo?) is one of the best. I would guess econometric studies support his insights already but he tries to formulate experimental results in a consistent model that aligns with his theory.

<comment><USER12> Lo & MacKinlay. Anytime I hear Andrew Lo I think of MacKinlay as I heard their names used together 1,000 times during undergrad.

<comment><USER11> Thank you. I knew I probably had the name wrong. It's been a few years since my undergrad years and that's a painful reminder.

</comment></comment></comment></comment><comment><USER13> Since undergrad while taking economics classes, I thought this was entirely obvious. And honestly, I still do. To assume everyone is a rational actor is to assume everyone is perfect: they don't make mistakes, they don't have asymmetric information, etc.

You can already see the changes that this articles suggests should happen in practice with high speed trading. I'd love a more detailed article, because this describes something obvious and vague.

<comment><USER14> You may be misinterpreting things. Relevant from Wikipedia: "Lo argues that much of what behaviorists cite as counterexamples to economic rationality—loss aversion, overconfidence, overreaction, and other behavioral biases—are, in fact, _consistent_ with an evolutionary model of individuals adapting to a changing environment using simple heuristics."

And as an aside, I am pretty tired of people saying things like this, without fail, in every thread about economics. A simplified model is better than no model at all. People have been trying to account for irrationality for ages. It's not particularly astute to recognize that individuals are not perfectly rational.

<comment><USER15> > A simplified model is better than no model at all.

This is true in physics where the system being modelled is governed by stable laws. You can confirm the model's predictive usefulness by experiment. If you test the model and find its limitations, you expect them to hold in 50 years' time.

Economic models, however, are themselves part of the economy. The ideas they are built on and their predictions become part of wider knowledge and influence the very thing the model is trying to predict or explain, possibly reinforcing or weakening certain behaviours. If a simplified model becomes widespread it can lead to blind spots in our understanding which cause systemic issues.

Of course that doesn't mean the model was bad or unsuitable for its original intent. But a simplified model can, in the wrong hands, be a net negative.

<comment><USER14> That is an interesting point, that people have bought too much into this simplified model s.t. reality bends towards the model. I don't know if I blame the model for that - like you say in your last sentence.

But what's the alternative besides starting with simplifications and iterating?

</comment></comment></comment></comment><comment><USER16> [https://en.wikipedia.org/wiki/Adaptive_market_hypothesis](https://en.wikipedia.org/wiki/Adaptive_market_hypothesis)

</comment><comment><USER17> I'm sure I'll like his book, but this article is shallow to the point of being trite. Any recommendation for a better intro?

<comment><USER18> Lo's investment management firm, AlphaSimplex Group, has posted a decent intro here, and a link to a more comprehensive (15pg) essay (2004): [https://www.alphasimplex.com/insight/the-adaptive-markets- hy...](https://www.alphasimplex.com/insight/the-adaptive-markets-hypothesis- market-efficiency-from-an-evolutionary-perspective/)

<comment><USER17> Thanks so much. I would probably have overlooked that thinking it was out of date.

</comment></comment></comment><comment><AUTHOR> You can use this ([https://github.com/njuljsong/wsjUnblock](https://github.com/njuljsong/wsjUnblock)) to bypass the paywall.

<comment><USER19> What is the legality of making something to help people not pay for stuff? Not trying to be confrontational, I sincerely don't know + don't feel sorry for them.

<comment><USER20> I haven't tried it either but I'm pretty sure there is no legality issues outside of the US, probably dubious legality inside the States but I wouldn't want to go up against their lawyers to prove it..

</comment></comment></comment><comment><USER21> Let me guess: "A optimal market configuration is when power and money aggregate at".. placeResearchDonatorHere.." ."

</comment><comment><USER22> If Andrew Lo's theory of adaptive markets is any good, why isn't he rich? This looks like dressed up technical analysis.

<comment><USER23> Keep in mind the following:

1\. Economic theories can be empirically descriptive without being constructively prescriptive (much like other theoretical disciplines, such as math or physics),

2\. You're reading about his work from what is effectively a watered down blurb on a Bloomberg webpage,

3\. He actually is the founder and chief investment officer of a fund with over $5B AUM, so yes, he likely is "rich".

</comment></comment><comment><USER24> On the path of a new revolutionary attempt to predict the past!

</comment><comment><USER25> #harambe

</comment><comment><USER26> Doesn't seem like a new theory so much as a description of a possible lemma (...real soon now) for behavioral economics.</comment>


Bullet-proof Node.js coding - <AUTHOR> http://stella.laurenzo.org/2011/03/bulletproof-node-js-coding/

<comment><USER2> While this is a good article, things like this make me wonder if Node's asynchronous programming model is a good one. Certainly I don't think it's worth the mental overhead when you aren't dealing with highly concurrent IO situtations.

I constantly feel like I have to jump through hoops when programming against async APIs. None of the code in the article feels "elegant" to me.

Of course the popular alternative, preemptive multithreading with shared mutable state, is probably worse.

<comment><USER3> I've criticized Node.js in the past on this site but I actually only recently realized what is really _wrong_ with it in a strongly-grounded computer science way rather than an intuitive way. Or rather, wrong with the asynchronous event-based programming style in general that Node.js adopted, rather than Node.js in particular (which is a fine implementation of the bad idea).

Does anyone remember the structured programming arguments that occurred, oh, ten years before I was born? There was a lot of arguing back and forth, but one of the arguments made was that when your program was structured, your position in the program means something. Just making up some quick psuedocode:

      def x(y: int, z: int):
          for i in 1 to 10:
      -->      print i * y + z

      def a(b: int):
          for i in 1 to 5:
               print i
               x(b, 3)

At the line that I marked, the very fact the program counter is pointing there, and by extension the stack trace up to the point given, means certain things. We know we have an x and a y which aren't just "undef", we know we're in a call from a (in this case), and therefore, any preconditions or invariants that are provided by those functions are also in effect at this point in time. One of several reasons spaghetti code is bad is that the program counter means much less; we don't know how we got there, we don't know what invariants are in effect, in the old school assembler case we know almost nothing at all, for all we know we just lept here with a goto.

As is almost always the case, this is a small trivial example, but when you start layering many things on top of each other, layering in functions that provide safe file handling or other resource management or state machines or any of the other tools we've built on structured programming or its smarter child Object Orientation, it becomes difficult to in some cases impossible to follow all this in your head. Or deal with the work to make this stuff compose together properly without layers actively stomping on each other. After spending some time with the modern functional world and their increasing focus on composability, working with async event programming feels like stepping back in time 20 years.

Asynchronous event-based code is not as bad as old-school assembler, because the event handlers themselves are still using the ideas of structured programming internally. But the code as a whole is spaghetti code. I'm not the first to say that, but it turns out upon reflection it's not a slur or a metaphor, it is actually descriptive and fair. Async event code actually does share many of the critical properties of spaghetti code, as the term was first used. You don't have a call stack, you don't have the invariants, you're just adrift in the code.

Of course with massive amounts of discipline you can function anyhow. You can program structured code in assembler with massive amounts of discipline. But A: you are spending valuable developer mindpower maintaining that discipline which is better done by a language/runtime/VM, no matter how smart you are you're still better off spending your smart on something other than raw plumbing and B: it's actually harder than you think. We've so thoroughly, utterly internalized structured programming since even before I was born that we can't hardly even see what we're getting out of it, and consequently we don't easily realize what we're giving up when we adopt this style. We aren't any better _people_ than our assembler ancestors, we're not particularly more disciplined than them, and they switched to structured programming for a reason.

I'm this critical because I'm actually trapped in this style at work, fortunately just in one of several subprograms but it's still annoying as hell and the one that always takes far longer to work with than I'd like. It's an excess of experience, not a deficit, causing me to be this critical. And I've really come to loath this style. Compiling is for computers, not humans.

That's what Erlang and similar languages that can take care of the asynchronousness at the VM/language level bring to you; they bring you _at least_ back up to structured programming in power and safety, and possibly beyond. And the arguments about how wonderful async event based programming is and how you've got it all under control and how it's performant and not a problem sounds to me like an absolute repeat of assembler programmers ranting against structured programming back in the day, to an almost scary degree... and every bit as correct and likely to win the future.

<comment><USER4> Very good points. I wrote the article and have struggled a lot with how _right_ node is as a general purpose tool. My interim conclusion, for many of the reasons you point out, is that to the extent that it can be made relatively safe, it is a good tool for those situations where my other choice would have been to write a massively scalable, non-blocking C program. This class of problems does not come up every day, but it is coming up more and more as the web and mobile arena move to more of a fully connected mesh of clients and the switchboards that arbitrate their exchanges need to become more flexible and efficient.

On the flip side, building massively scalable systems based on blocking IO is rife with problems and I've found that enforcing architectures that make it manageable have a way of obscuring the code and making it difficult for people to intuitively get right as well. I personally find the explicit functional style to be just as intuitive as structured and/or OO and therefore find that when I need it, a platform like node that makes the knife's edge that is asynchronous programming explicit and manageable with a good functional style is a good trade-off in the world where I am trying to find less bad options to a hard problem.

Sometimes you have to step back 20 years in order to break through the layers of assumptions that were added in the interim. I doubt node is the last word on the topic, but it is a refreshing interlude to what was becoming an unwieldy calcification of the theory of how to program computers. The water will find the right course eventually as we experiment with the different styles.

<comment><USER3> "it is a good tool for those situations where my other choice would have been to write a massively scalable, non-blocking C program."

Agreed. I'd take a Node.js implementation over straight C any day.

"On the flip side, building massively scalable systems based on blocking IO"

Please remember that's an implicit false dichotomy. "Blocking" has to do with the runtime and the VM, _not_ the visual appearance of the programming language syntax. Erlang is _non-blocking_ , but it is _also_ structured (and functional). In fact, by my personal standards Node.js is _still_ a blocking language; you have to jump through hoops to get non-blocking behavior and you only get it when you ask for it. In Erlang or Haskell, you simply _get_ it. Go ahead and do a lengthy math computation if you'd like. Take several minutes. You won't block anything else in the meantime. And you don't have to manually break the computation up yourself. Just do it.

I say Node is stepping back 20 years not because it feels primitive compared to C#, but because it feels primitive compared to the "really existing, I use it in production code" Erlang runtime. Also Haskell, except I can't say I use that in production code. And probably Go (still waiting for someone to confirm), and Stackless Python, and several other things.

The real reason I speak up so often on Node.js articles is not that I hate Node.js, it is that I hate the _hype_ because it is shot through with falsehoods, which I'm rapidly upgrading to "lies" as the same blatant falsehoods continue to get around without the community fixing them. Your alternative is not async event based or synchronous; there's a third option where the async is managed by the compiler and runtime and it _works_. In fact it's Node.js that has to catch up to even the previously existing async frameworks, and Erlang is simply miles beyond Node.js in every way, except it isn't Javascript. (Which I freely admit is a problem. Erlang-the-language could stand to be improved, even as it is hard to borderline impossible to match the total environment.)

(If you know the choice exists and you don't choose it for some reason, hey, great. Like I said in my first message above, I've got my own async-event based program I have to maintain, and I'm the original author, it was my choice, because when I took all the issues into account, it was the right choice. But you ought to choose with an understanding of the full picture and all the relevant choices, and understand all the tradeoffs, not because you think that async event-based programming is better than everything else at everything. It's got some really serious drawbacks, and there are really- existing, production-quality, "web-scale" things that can be used that do not have those drawbacks.)

<comment><USER4> I can certainly see your point, and I am aware that I am playing a little fast and loose with some of the terminology around blocking vs non-blocking. To be precise, in node's case, it presents a consistent, callback based API to non- blocking IO. Given that that API is reflected exclusively in JavaScript the language, this has impacts the style and appearance of Node programs. The resultant style is imprecisely referred to as asynchronous, non-blocking, etc.

Using Erlang vs Node on my current project was a serious consideration. It is well thought out and its message passing and lightweight process based design is more evolved than anything node has or likely will have. It's also Erlang and I have found productively programming in it to be incomprehensible. Maybe its a personal problem, but it is what it is. Ditto for Haskell. I was surprised to find that my brain didn't bend that way.

<comment><USER5> There's always Scala/Java with Akka as a way to get both the actor model and a "more c-style" language. (<http://akka.io/>)

</comment></comment></comment></comment></comment></comment><comment><USER6> Im curious on node fibers. They seem like a second class citizen in node, any chance they will become mainstream or will it always be like Twisted is to Python.

<comment><USER2> Most of the Node core team is vehemently opposed to anything they consider a language extension, which fibers arguably are, so I doubt you'll be seeing them baked into Node any time soon.

See: [https://groups.google.com/d/msg/nodejs/GDqkQzmnwHM/FKETaPivX...](https://groups.google.com/d/msg/nodejs/GDqkQzmnwHM/FKETaPivXH4J)

</comment></comment><comment><USER7> Cache link: [http://webcache.googleusercontent.com/search?q=cache:http://...](http://webcache.googleusercontent.com/search?q=cache:http://stella.laurenzo.org/2011/03/bulletproof- node-js-coding/)

</comment><comment><USER8> This arrived just in time since I deployed my first Node app on Joyent last night! Great examples--they're all pretty new to me, but I'm planning to revisit your tips when I get further along.

</comment><comment><USER9> I'm experiencing "node anxiety", let me explain:

There really are two kinds of functions in a node program.

Synchronous functions and asynchronous functions.

This has major consequences during refactoring when what used to be a synchronous function now needs to become an asynchronous function => all synchronous functions that used to call the synchronous function needs to be turned into asynchronous functions themselves.

Sometimes the ramifications go way beyond first expected.

Sometimes the ramifications turned to be massive.

This becomes worse when one realizes that synchronous function are much more readable, half the code and about 5 times faster than asynchronous ones (see <http://jsperf.com/asynch-cost>)

Eventually the idea of refactoring a synchronous function into an asynchronous function becomes a source of worries.

And that, my friends, is not something easy to figure out and is what I call "node anxiety"

</comment><comment><USER10> Excellent, meaty article with a lot of practical examples and hard-won knowledge.

</comment><comment><USER11> For #1, why not suppy a success and failure callback function to your doSomeAsyncCall function? There is no reason to muddle those two lines of logic together (which is the real reason for the described error).

And for #2, Relying on javascript hoisting functions isnt a smart way to keep your code organized.

</comment><comment><USER12> Fairly amusing that a page with this title returns a 404 at this moment (and I tried reloading twice more).

</comment><comment><USER13> Anyone used node-fibers? What are the gotcha's?

<comment><USER14> Main gotcha is listed in the "Garbage Collection" section of the readme:

<https://github.com/laverdet/node-fibers>

</comment></comment><comment><USER15> seems like laurenzos bulletproof node.js code just got caught in the eventloop

<comment><USER16> The site is running on Wordpress. Incidentally, one of the major reasons for Wordpress blogs to go down under heavy traffic is excessive keep-alive times causing too many Apache threads to be open. It's exactly this lots-of- connections problem that node specializes in handling gracefully.

<comment><USER4> That was embarrassing. I do know how to craft a good apache config, and am just going to claim a case of being stupid.

It is ironic that node is in fact designed to handle this type of memory/connection scalability problem reliably, but in this case it was pure administrative (me) error.

<comment><USER17> Throw nginx in front :-)

[http://blog.rassemblr.com/2011/01/wordpress-need-for- speed-o...](http://blog.rassemblr.com/2011/01/wordpress-need-for-speed- optimization-with-nginx-caching/)</comment></comment></comment></comment>