Exploring constraint programming with CL's Screamer - <AUTHOR> https://unwindprotect.com/constraint-programming <comment><USER2> Beautiful stuff. Brian, thanks for writing that up - I don't think I have used Screamer in 20 years, but I will put playing with it on my TODO list.

I have been using Common Lisp for about 38 years for research and building commercial products, and it is amazing how portable code is through the decades.

<comment><AUTHOR> Thanks! I'm glad you enjoyed it.

</comment></comment><comment><USER3> Exploring yes. Extremely nice lib and syntax.

But for real work you should use a proper OR system and solver.

google ortools or COIN-OR for the C/C++ folks and horrible syntax, or any Prolog with solver backend or Picat are much better. Screamer with a proper solver backend would be really nice. Prolog solved it this way.

</comment><comment><USER4> Oh wow, Screamer. I used it to write a Sudoku solver 14 years ago, and blogged about it. [http://i-need-closures.blogspot.com/2006/03/](http://i-need- closures.blogspot.com/2006/03/) Looking at the code now, well, it's like looking at any code you wrote a decade and a half ago. Common Lisp was educational but now I just use Python and Javascript to get things done, like everyone else does. I suppose I use the lessons learned from then, though.

<comment><AUTHOR> Very nice. I too initially had a problem handling a lot of variables, so I used a macro at first to do the typing for me, but with some help from the maintainer I used cl:apply rather than screamer:apply-nondeterministic which worked.

</comment></comment><comment><USER5> It feels odd to see libraries pushing thirty years of age getting covered. In a good way.

I'm curious what forms of bit rot this has to love with. And if not that much, why? Stability of what it was built on?

<comment><USER6> common lisp the language hasn't changed since it was standardized, so code written to the strengths of the standard has not experienced bit rot. screamer, series, cmu's ai code archive, old common lisp-based books (like norwig's paradigms of ai) will all work out of the box. that is not to say that the language runtimes are not improving, instead they focus on improvements that complement the standard, but you can grab any old algorithm, and it'll work in recent sbcl.

there are underspecified parts of the standard, notably the protocol around the object oriented system (common lisp object system meta-object protocol), and then there are parts that are not specified at all: networking, interactions with C libraries, unicode, threading. each runtime implements it in its own particular way. that means that the core of something like CLX, which is the network interface to X11, could've stayed exactly the same, but the networking outer layer had to be updated with the details of specific runtimes. in case of X11 that support goes all the way back to lisp machine Genera. on the other hand something like SSL support which will both rely on external C library and concern itself with network, will usually bit rot very rapidly.

<comment><USER5> It just strikes me as different from many newer projects that don't have this level of stability anywhere.

We literally embraced refactoring as a thing you have to constantly do across the entire codebase.

<comment><USER6> there's definitely sub-groups in the computer world who argue for stability. i've first encountered that in common lisp, but i think a recent interest in Ada is also driven by similar concerns. don knuth with TeX and literate programming also comes to mind. for example i wrote my billing code almost 15 years ago, and it's common lisp code that spits out plain TeX file. i've added features to it, but if i were to attempt to run the first version, it'll work without any changes. honestly i personally think it's aesthetically pleasing to work with old code, with changelog spanning a decade, because of the richness of thought that went into it. like picking up a woodworking tool at a yard sale, and noticing all the different hints of care that went into it.

arguing for stability in the face of other tradeoffs is an unpopular position. there are people who do it, but simply linking them from here will probably result in downvotes. i think arguments against systemd or in support of X11 come from people that want to escape the constant marginal benefit refactoring cycle.</comment></comment></comment></comment>


Excel and Machine Learning - <AUTHOR> https://www.predillo.com <comment><USER2> It used to be the case that Excel couldn't even correctly calculate a standard deviation [1], I don't know if that was ever fixed.

[1] [http://eprintspublications.npl.co.uk/1793/](http://eprintspublications.npl.co.uk/1793/)

</comment><comment><AUTHOR> What people think about this? Good to have feedback</comment>


Capturing bacteria that eat and breathe electricity - <AUTHOR> https://news.wsu.edu/2019/03/05/capturing-bacteria-eat-breathe-electricity/ <comment><USER2> So...in the future, when we can harness bacteria for our mobile/fuel cells, will we simply call these devices BACtteries?

(You see what i did there? And, yes, I'm a father, so am compelled to pump out silly jokes. ;-)

<comment><USER3> As a father, I approve. Have an upvote.

</comment></comment><comment><USER4> The author did an incorrect job of dumbing down the subject matter, but still, seems like neat research.

I assume they are talking about geobacter, but they didn't say so.

<comment><USER5> This is a blog post about research that was done, if you look at the actual journal article ([https://www.sciencedirect.com/science/article/pii/S037877531...](https://www.sciencedirect.com/science/article/pii/S0378775319300291)) it has actual information.

</comment></comment><comment><USER6> Only tangentially related:

Does anyone know if any labs are attempting to produce new antibiotics by exposing antibiotic resistant bacteria to fungi, or other bacteria?

<comment> xkcd-sucks Yes this process is called "drug discovery"/"development" and more specifically "drug screening". Typically new antibiotics are developed vs resistant bacteria because that's clinically relevant.

"In the pipeline" blog [https://blogs.sciencemag.org](https://blogs.sciencemag.org) is a really good intro to the field

<comment><USER7> The "in the pipeline" blog also contains the "things I won't work with" section ([https://blogs.sciencemag.org/pipeline/archives/category/thin...](https://blogs.sciencemag.org/pipeline/archives/category/things- i-wont-work-with)) which is one of the most awesome reads ever, at least for people with some practical experience of an organic chemistry lab.

<comment> xkcd-sucks Working as a chemist for a while, one tends to develop an explosion sense. Because it is never _your_ stuff blowing up, but rather the postdoc behind you dumped his peroxide waste into the organic waste bottle or whatever and you're not looking at it.

There's a bunch of very subtle hisses,"stretches", gentle little clinks, "gas expansion into a closed container" that all seem to indicate an explosion is in progress.

Personally, I will _pre-consciously teleport_ out the room when this happens. After leaving the field for more lucrative sofware stuff, it has embarrassed me a few times and saved my skin a few times (shoddy LP cylinder igniting, low pressure plastics getting attached to a soda stream carbonator etc)</comment></comment></comment></comment>


Technology Bookshelf - <AUTHOR> http://www.gutenberg.org/wiki/Technology_(Bookshelf) <comment><USER2> Sorry for a long citation (this is from "Practical Mechanics for Boys"), couldn't help myself:

"The American method of teaching the mechanical arts has some disadvantages, as compared with the apprentice system followed in England, and very largely on the continent.

It is too often the case that here a boy or a young man begins work in a machine shop, not for the avowed purpose of learning the trade, but simply as a helper, with no other object in view than to get his weekly wages.

Abroad, the plan is one which, for various reasons, could not be tolerated here. There he is bound for a certain term of years, and with the prime object of teaching him to become an artisan. More often than otherwise he pays for this privilege, and he knows it is incumbent on him "to make good" right from the start.

He labors under the disadvantage, however, that he has a certain tenure, and in that course he is not pushed forward from one step to the next on account of any merit of his own. His advancement is fixed by the time he has put in at eachp. 2 part of the work, and thus no note is taken of his individuality.

Here the boy rises step after step by virtue of his own qualifications, and we recognize that one boy has the capacity to learn faster than another. If he can learn in one year what it requires three in another to acquire, in order to do it as perfectly, it is an injury to the apt workman to be held back and deterred from making his way upwardly.

It may be urged that the apprentice system instills thoroughness. This may be true; but it also does another thing: It makes the man a mere machine. The true workman is a thinker. He is ever on the alert to find easier, quicker and more efficient means for doing certain work."

<comment><USER3> I'm not sure if the message has become garbled, but I can't seem to get the conclusion there. That the American system instills more innovative thinking at the expense of thoroughness?

The original English/European Guild/Apprentice system roughly went Apprentice -> Journeyman -> Master. An Apprentice would learn from a Master and become competent in basic techniques, no innovation was expected at that stage. A Journeyman would travel around and work for different Masters, learning more and different techniques and transferring them between Masters. Finally a Journeyman would produce a "Masterpiece" to become a Master and setup shop somewhere. Innovation was really only expected by Journeymen and Masters.

<comment><USER4> The idea of "making good" right from the start is the thoroughness part. Whatever task the apprentice is set, they think of it as part of their career path and do it absolutely to the best of their ability. That's the idea, anyway. It's like the situation with sushi chefs: they spend years doing little more than washing rice, but it supposedly instills an attitude that even the most trivial task is beautiful and worthy of care -- because they have had time to realise that while focusing on it.

That's just my interpretation. The experience I've had in various semi- vocational educational contexts in the UK is that part of the message is that preparatory/ancillary tasks are just as important as the skillful work. Focusing on them is arguably the only way to acquire the attitude needed to do the skilled work.

</comment></comment></comment><comment><USER5> Love it! I thought this would be another collection of books about programming languages and data structures and stuff. I was very pleasantly surprised!

</comment><comment><USER6> There is a much larger and more relevant collection at [http://archive.org](http://archive.org)

My favorites:

[https://archive.org/details/pdfy- MgN0H1joIoDVoIC7](https://archive.org/details/pdfy-MgN0H1joIoDVoIC7)

[https://archive.org/details/CProgrammingLanguage2ndEditionBy...](https://archive.org/details/CProgrammingLanguage2ndEditionByBrianW.KernighanDennisM.Ritchie)

[https://archive.org/details/cprogramminglang00kern](https://archive.org/details/cprogramminglang00kern)

<comment><USER7> You might be missing the point.

<comment><USER6> Well, if you really want mechanical engineering texts, here's 11,803 results:

[https://archive.org/search.php?query=mechanical%20engineerin...](https://archive.org/search.php?query=mechanical%20engineering)

The Gutenberg collection is quite small.

</comment></comment></comment><comment><USER8> Not on the same level - but may I highlight the Fun Press Bookshelf [1] (for Free Functional Programmings Book) in TXT and the Yuki & Moto Bookshelf [2] (for Free Ruby & Friends Programming Books) in TXT (using the Manuscripts format) [3]. [1]: [http://funpress.github.io](http://funpress.github.io) [2]: [http://yukimotopress.github.io](http://yukimotopress.github.io) [3]: [http://manuscripts.github.io](http://manuscripts.github.io)

</comment><comment><USER9> Are project Gutenberg books still relevant? I am not someone who knows a lot about technology. Therefore, I am not sure how to gauge the relevance of these 50 to 100-year old texts.

<comment><USER10> Yes.

</comment></comment><comment><USER11> Note to self. Remember to bring this if you ever travel in time to the past.</comment>


Clean up Python source code with PythonTidy, available as Sublime Text 2 plugin - <AUTHOR> http://opensourcehacker.com/2012/11/29/clean-up-python-source-code-with-pythontidy-available-as-sublime-text-2-plugin-too/

<comment><AUTHOR> I think this seems somewhat interesting - but it does defy this part of PEP8 a bit.

[http://www.peps.io/8/#a-foolish-consistency-is-the- hobgoblin...](http://www.peps.io/8/#a-foolish-consistency-is-the-hobgoblin-of- little-minds)

tl;dr - You shouldn't comply to PEP8 blindly.

<comment><USER2> I mention some situations at the beginning of the blog post when tthere is a use case for PythonTidy - one should indeed not blindly use it</comment></comment>


Javascript and DOM Load Time Tester with jQuery - <AUTHOR> http://www.reynoldsftw.com/2009/08/javascript-jquery-and-dom-load-time-tester/

<comment><USER2> Can someone explain why this is better than a real profiler, or something like YSlow?

I'm not criticizing - just trying to figure out what I'm missing.

<comment><USER3> If you're using an analytics package that let's you send custom events, you could use something like this to find out how your page performs in the wild. With things like YSlow, you're only getting data for your location and your browser.</comment></comment>


Do compilers take inline as a hint? - <AUTHOR> https://blog.tartanllama.xyz/inline-hints/ <comment><USER2> "Do The Right Thing And Measure It": words to live by.

This article has also reminded me of just how painful the main GCC codebase is. If you have a choice: Use LLVM for your compiler-hacking.

<comment><USER3> Well, I guess you could have tested whether or not the inline hint is used without looking at any source code at all: just find a suitable example program and see whether the executable changes after you compiled the program with or without any inline hints.

However, this bears the risk of being non the wiser in case your chosen program doesn't produce any differences in the compiled output: you wouldn't know whether it's because the compiler doesn't respect _inline_ or whether the particular program you tried was for some reason not a good choice.

<comment><USER4> Depending on the simplicity of the program, the compiler may go ahead and inline it anyway!

<comment><USER5> My takeaway is the keyword's really just a hint anyhow. And that's borne out by the code dive.

<comment><USER6> I use it to show intent, as functions I mark as inline are those that, pre-C99, would have most likely been a macro. I prefer the inline function over a macro as it prevents a certain class of errors and you get some additional type-checking that macros lack.

</comment></comment></comment></comment></comment><comment><USER7> Nice article! All the hours of arguing, testing or experimenting you can save by simply reading the only true source of information, the code. Wish more people would actually do this. I've seen people spending days trying to understand if a feature is supposed to be on by reading requirements or issue trackers, when a quick look at the code would tell you the answer in 30 seconds.

</comment><comment><USER8> In the LLVM case you can see that PGO overrides the inline hint. And if you're not using PGO you don't care about performance that much anyway.

The best way to find out if your function was inlined is to build it as if for release and then disassemble and read it.

<comment><USER9> or you could just dump information from LLVM opt's inliner pass.

</comment></comment><comment><USER10> The inline keyword does not tell the compiler to copy and paste the function body into the caller. This is a misconception. It has semantic differences with where functions can be defined without causing duplicate symbol definitions. That is all.

Forced inline, on the other hand, is telling the compiler to copy and paste the function body. This will not always improve performance because code has interactions with the processor and memory/cache which make things more complicated.

<comment><USER11> I assume adding "inline" in front of a function declaration without actually defining it in the same compilation unit doesn't result in inlining. If this is true the article is wrong and the only important aspect about inline is how it affects the linker.

<comment><USER6> It may very well depend upon the tool chain. The Solaris C compiler/linker can inline external functions (a previous developer at my company relied upon that behavior when writing code), but that's the only one I'm familiar with that behavior.

</comment></comment></comment><comment><USER12> I have lots of forceinline and always_inline in my code, but you can do too much of it, so always benchmark to be sure.

</comment><comment><USER13> "The source for GCC is a fair bit less accessible than that of LLVM,... " \- understatement of the weekend!

</comment><comment><USER14> For more, there's Torvalds on GCC Inlining: [http://yarchive.net/comp/linux/gcc_inline.html](http://yarchive.net/comp/linux/gcc_inline.html)

<comment><USER15> This post is from 2000 and describes the old "gnu89" behavior of GCC which doesn't conform to C99.

<comment><USER16> Most of the mails are from 2008-2009

</comment></comment></comment><comment><USER17> Warning: heavy lags on mobile while scrolling.

<comment><USER18> To add one data point concerning an "old" phone: smooth scrolling on Samsung Galaxy S4. At least with ads blocked.</comment></comment>


China Blocks Wired.com With 'Great Firewall' - <AUTHOR> http://www.wired.com/threatlevel/2010/01/china-wired-censorship/ <comment><USER2> imdb was just blocked as well. I don't know of anyone who can figure out their logic for how they determine why and what sites get blocked and when. It's a fool's errand to try. [http://www.hollywoodreporter.com/hr/content_display/technolo...](http://www.hollywoodreporter.com/hr/content_display/technology/news/e3i3a9920d504eb96547cc1febea85d451e)

That they blocked Wired though (irrespective of people's views of the magazine), is doubly silly for what people can learn and speaks to the ever present threat censorship plays in innovation/web development in China.

<comment><USER3> > I don't know of anyone who can figure out their logic for how they determine > why and what sites get blocked and when.

It's totally blackboxed, pretty typical like the rest of China's fuzzy politics.

Most of the ban were from random Internet user's anonymous report to net.china.cn, or from Party leader's direct orders.

IMDb was blocked because a banned movie's poster, rumor says.

PS to all YC: If you have a website accidentally blocked by the GFW, call 12300 in Beijing. This phone number is kinda official but unofficial GFW customer service.</comment></comment>


Exclusive interview: Apple's Phil Schiller on how the iPhone X seemed impossible - <AUTHOR> https://www.t3.com/features/phil-schiller-interview <comment><USER2> At least he didn't use the word "courage" again...</comment>


How to take adavantage of crypto losses on your taxes before the new year - <AUTHOR> https://www.cryptotrader.tax/blog/how-to-handle-your-bitcoin-and-crypto-losses-for-tax-purposes <comment><USER2> This is completely generic information on capital gains/losses (it applies to any capital asset, like stocks or ETFs).

The whole point of this post is to advertise the crytotrader.tax service, which seems useless: any generic tax prep software (TurboTax, etc.) can generate 8949 forms and Schedule D automatically (as well as the rest of your tax return), so why would you need any extra software to handle cryptocurrency transactions for you? TurboTax can even import data directly from financial institutions, and presumably the crypto exchanges provide a standard file format that's importable directly into TurboTax.

> _This way you can skip the upcharge of an expensive "crypto accountant"._

Why would you need an expensive accountant? Trading cryptocurrencies generates simple capital gains/losses just like selling any other asset. Any tax software can handle this. Is there even such a thing as a "crypto accountant"?</comment>


Burnout, How to Recover - <AUTHOR> http://sentientmachine.blogspot.com/2010/07/burnout-how-to-recover.html

<comment><AUTHOR> I'm not sure if this would have helped me when I was going through this process. But, it certainly is a very accurate description of burning out and offers up practical, (and in my case useful), solutions.

</comment><comment><USER2> The content was interesting, if you can look past the repetive "you are pushed to burnout" comments. It really detracts from the message and feels a lot like a powerpoint presentation</comment>


Building Great User Experiences with Concurrent Mode and Suspense - <AUTHOR> https://reactjs.org/blog/2019/11/06/building-great-user-experiences-with-concurrent-mode-and-suspense.html <comment><USER2> I really like the idea of React, but I can't help but feel it will always be slow to due to DOM manipulations. It would be nice if there was a first class mechanism to pre-transform some of the structures server-side that will always be transformed.</comment>


Pandora (And Other Internet Radio) Has Officially Been Saved - <AUTHOR> http://www.techcrunch.com/2009/07/07/pandora-and-other-internet-radio-has-officially-been-saved/ <comment><USER2> Pandora was nice, I've moved on to other services I listen to thesixtyone most of the time because a large amount of it is generally good music and if not I skip it, for more mainstream I can find songs on grooveshark (don't know if that will get shut down or not).

If I was ever going to pay for a radio like service it would have to be something like di.fm where pandora has variety and choice di.fm still has had enough quality that variety didn't matter to me.

<comment><USER3> Grooveshark is an awesome service. Especially now that Youtube muted all the copyrighted songs.

Its a good place to go when you want to find that one song or artist.

<comment><AUTHOR> the autoplay feature sorta works like streaming radio.

<comment><USER4> Disclaimer: I'm working on Autoplay at Grooveshark.

I'm interested in hearing any feedback you might have about this. Please e-mail me or just respond to this comment with your thoughts.

</comment></comment></comment></comment><comment><AUTHOR> Good deal for Pandora, but I think they'll still lose users.

Pandora One is $36/yr. Under this payment plan, they pretty much get most of their users that they'll keep onto a $12/yr plan. Because 40 hours of radio is nothing.

But since most people hate paying, even if its $.99/mo, I think that'll cost them a ton of users. A free alternative will spring up quickly enough.

Good opportunity for pirate bay here, Pirate Radio

<comment><USER5> Pandora isn't really about the music. It's about the song selection engine, which is amazing.

Pirate radio could never come close.

<comment><USER3> >Pirate radio could never come close.

On hackernews, I believe that's a challenge.

</comment></comment></comment><comment><USER6> Good that legislation is moving towards the direction of dealing with Internet services in a reasonable way (since Internet as main distro media pipeline cannot be stopped).

Also good that they've figured out rates. Although, they're still higher than radio rates. Little steps...</comment>


Mattermark (YC S12) to shut down after selling to FullContact - <AUTHOR> https://techcrunch.com/2017/12/21/mattermark-to-shut-down-after-selling-to-full-contact/ <comment><USER2> The CEO of Mattermark spent a lot of time and energy in the early days of the company establishing a public presence as a brash, highly opinionated thought leader in the "how to run a start-up" intellectual space, which makes the failure of the company a little ironic and is almost certainly contributing to the Schadenfreud I've read on Twitter.

Take, for instance, this post, in which she seemingly boasts of the company's massive burn rate: [https://medium.com/@DanielleMorrill/is-my-startup-burn- rate-...](https://medium.com/@DanielleMorrill/is-my-startup-burn-rate- normal-882b2bd20f02)

Or this post, in which she shames a number of "Zombie Unicorns" (including WeWork): [http://www.businessinsider.com/mattermark-names-troubled- uni...](http://www.businessinsider.com/mattermark-names-troubled- unicorns-2015-10)

Why be so publicly hostile and truculent?

Also, it appears that the CEO lacked focus: according to her Twitter profile, she's a VC and a very prolific angel investor. I'd be upset as an employee of the company receiving $0 for my common stock that the CEO was effectively hedging her Mattermark bet with a number of side gigs. The CEO's job is to be singularly focused on the success of the company.

<comment><AUTHOR> Notably, there's been complaints that the TechCrunch article is _too negative_ ([https://twitter.com/sm/status/944087102982995969](https://twitter.com/sm/status/944087102982995969)), which has the inverse problem.

<comment><USER3> Yeah, unreal. Hey @sm give me $17m and I'll give you $1m back and we'll gauge your positivity/negativity.

This is the stuff that gives SV a bad reputation- the sense of entitlement. That of course we deserve "others people's money" to do with what we please and should be praised for it.

This is real money @sm !!

</comment></comment></comment><comment><USER4> You can't make this up:

        Dear Mattermark Common Shareholders,

        I'm reaching out to share some great news: Mattermark
        is being acquired by FullContact!

        ... lorem ipsum ...

        Common stockholders will not be receiving anything in
        this deal (cash or stock).

It's a letter to common stockholders saying they get zilch and calling this a great news. I sure hope it's just a quirky inner joke of some kind, because otherwise it reads like a spit in the face of aforementioned stockholders.

<comment><USER5> Your 'lorem ipsum' actually wipes out another gem:

"We are happy to have found an exit for our shareholders"

But not _all_ shareholders.

</comment></comment><comment><USER6> I signed up for a trial of Mattermark a month ago. Within several minutes I learned that:

1\. Their database is terribly outdated. A company that was funded by X _two years ago_ did not show up in a search for companies funded by X.

2\. Their "show contact info" feature wasn't working. That's not an easter-egg feature; that's the feature they're selling!

After several frustrating minutes I thought, "People pay for this?!" Evidently not.

To be clear, I don't wish this on anyone. I have friends who've had to sell their company for peanuts, and I had a pleasant chat with the Mattermark CEO just recently, but what do you expect when you enter a crowded market with a product that isn't delivering on its basic promise?

<comment><AUTHOR> It's worth nothing that employees have been leaving Mattermark gradually over the past year, which could explain an out-of-date database/features if there isn't anyone to update it.

<comment><USER7> Yes, exactly. I see a lot of these comments on HN made by folks from-the- outside-looking-in.

"I tried out this product. It had this bug I noticed that affected me a lot. By extension, this entire company will fail."

That's like saying, I went to a restaurant. I noticed the silverware is a bit dirty, thus the shop will close down.

Usually the commenter is someone who works in a specialized day job. Like an engineer :) Or silverware washer at a restaurant. Someone who sees and handles a narrow part of the picture.

There are many possible reasons why Mattermark closed down. I am sad to see it. But it is probably not due to a specific feature or bug. There are probably reasons beyond the description in the article that led to the disappointing outcome. I hope Techcrunch will be able to find the reasons behind this so other startups can learn and avoid future tragedies :(

<comment><USER6> No, this is more like a pizza place that can't make good pizza. Oh and there are reports that they weren't able to sell their pizza:

> We're told that it was hard to convince people to pay for the business intel > in a competitive landscape that includes Crunchbase, PitchBook and CB > Insights.

<comment><USER7> Yes, so the problem is not that the pizza quality is not good enough in some way. It's way bigger than that.

It's that it's hard to convince people to even buy pizza. That's a much bigger-picture issue than saying "pizza not fresh, so they will close down".

Put another way, making the pizza fresher, is not going to substantially improve the situation. Fixing the issues you saw (e.g. update the database to be super fresh, fix the contact feature so it is 100% working) will not get Mattermark out of it's current situation (unfortunately)

</comment></comment></comment></comment></comment><comment><USER8> Danielle (if you read this), I was rooting for you to succeed (as a fellow entrepreneur in the same space). I hope you would write soon on what worked and what didn't.

Data business is hard for a few reasons:

1\. Data is a commodity until it is not. People's contact information is a 'done to death' problem but not solved yet (and probably not solvable completely)

2\. Data is capital intensive

3\. Data businesses (and the modern ones at that) are more annuity businesses and less like SaaS. Monthly recurring patterns take time to emerge and you need investors that understand this dynamic. They will if you prove that your core hypothesis works and customers are buying more (if not following a strict monthly pattern).

4\. You need to have enough money and execution speed to prove your hypothesis. Unlike software where customer development and execution discipline can help you launch a good product, in the data business you need to stitch partnerships early on with data sources. That's not easy.

You tried. Again, I'd love to read your reflections on what you could've done differently.

<comment><USER7> +1 I too was also rooting for you, Danielle, since the early days when you started Referly before pivoting to Mattermark. Back then there was not a lot of YC companies, and even fewer female YC founders to root for.

+1 to a reflections blog post! Hope you succeed in the future--the best is yet to come.

<comment><USER9> +1. You had an impact.

</comment></comment></comment><comment><USER10> That sucks. I used to work in investment banking where we used a tool called Capital IQ for our research, despite the fact that our focus was on venture- backed startups and Capital IQ's coverage of startup companies was really crappy. When I left banking one of my top 2 or 3 ideas was to start a company that would essentially be Capital IQ for startups. The idea being that I would sell the overall platform to VC firms and investment banks, gain access to their data, and use it to build predictive models of non-venture-backed startups based on publicly-available information. I didn't end up pursuing that idea, but I was aware of Mattermark and thought that it was a service that should exist, would be a no-brainer for people targeting that market. To be honest, I think they did a lot better of a job of it than I would have.

Back in April when I was looking for jobs I did get the chance to interview at Mattermark, though. Kevin Morrill and the rest of the engineers that I met all seemed very friendly and bright (they ultimately rejected me, which is good because [1] I would have accepted their offer, and [2] I ended up joining Flexport, which I love). Even at the time I got the impression that they may have a tough road ahead, but I'm sorry to see that things turned out this way.

</comment><comment><USER11> I started getting worried when their pricing was no longer public and their blog posts didn't make it on here any more. Enjoyed their insights on fundraising and the progress of their company. I imagine it's harder to be open when things are not going as expected.

<comment><USER12> > I imagine it's harder to be open when things are not going as expected.

For a number of reasons, I believe this.

* You don't get the rush of external validation

* You are heads down trying to make things better at the company and blog posts don't help (enough)

* It's hard enough to share bad newest with employees and investors, who wants to share it with the world

* You may fear scaring off customers ("I was going to mattermark but then they seemed like they were going out of business so I used competing project")

</comment></comment><comment><USER13> Brutal result after years of work - "a less than $500,000 cash consideration from FullContact, which will be used to facilitate shutdown."

<comment><USER14> It wasn't an all-cash deal:

> _The deal additionally included a stock transaction_

<comment><USER12> Yes, 500k of stock (at present value). So $1M.

Tough outcome.

<comment><USER14> Sounds like you've got inside info. In situations like these does that go 100% to investors, or do founders/early employees take even a small piece?

<comment><USER3> The "inside info" of actually reading the article. Hahah.

<comment><USER14> Ah, I see the article has been updated. The details on the stock side of the transaction were not in there when I read the article yesterday.

</comment></comment></comment></comment></comment></comment><comment><USER15> I just read through their website and I can see why it was tough.

There's something especially difficult about a business like this. It doesn't feel like a finished product on offer, but more of a raw material for making of it what you will.

Because when you look at their home page as a potential end customer, there isn't much in the way of real benefits on display. In fact, the main "features" are:

- _Automatically enrich leads in Salesforce_

- _Customize your experience with API_

- _Export Mattermark data to spreadsheets_

- _Right Data. Right Format. Right now._

3 of those 4 are purely meta-features around accessing the data (which is presumably the core feature), but there's virtually nothing about why you'd want to or how to make it easily actionable. They're saying "here's a bunch of data and a bunch of ways to access it. Go make it do something for you."

That's a tough sell because it doesn't answer an immediate need wherein you simply sign up and a problem is solved. Even if you are able to connect the dots WRT how you _might_ make use of the data, you're now left with the task of building something on top of it.

I'm not sure how many customers can do that. Seems the only way to make it work is to pull in very tight partnerships to build finished apps in specific verticals, through which you license the data/API.

</comment><comment><USER16> Anyone reading this thread that said "disgusting!" reveals to everyone else that you've never raised money. It is brutal and you have to be 100% prepared to tell your investors you lost everything. Even if you are one of the few who doesn't actually lose everything, you'll have multiple moments in your entrepreneurial path where you go over that conversation in your head, like an anti-Oscar speech, as you face the death of your company (and more importantly, your identity as one of the brilliant).

</comment><comment><USER17> Yes, we should respect founders who try and fail but the amount of praise for a company with such little value doesn't make sense. This praise for the failing founder would make a lot more sense if the company was chartering new and interesting ground.

</comment><comment><USER18> Great news everyone!

Ps. Not you.

</comment><comment><USER19> Sorry to hear it didn't pan out well for Danielle. I was rooting for them to try and find a business model that worked for them.

</comment><comment><USER20> I don't understand why they pivoted to this contacts thing. I thought they had a solid idea previously. I guess CB Insights came along and took the ball the last 20 yards.

<comment><USER21> I agree! I receive newsletters from both the services and I prefer reading the CB Insights for its casual style and lots of graphics.

</comment></comment><comment><USER22> Looks like FullContact got data/systems/source code, a customer list and 6 employees for $1m. Bargain!

<comment><USER7> yes. Would have cost Full Contact way more to do it themselves from scratch (collect all the company data, get the customer list, the source code, and recruit 6 trained engineers who are ramped up on the source code)

</comment></comment><comment><USER23> I wasn't using their product (found the data wasn't that great), but I really like their daily newsletter - I hope it'll find a new home.

</comment><comment><USER24> Hoping they will honor my year subscription somehow...

<comment><USER25> Depending on when you charged it, you should probably contact your credit card company to charge it back if they're unable to provide the service going forward.

<comment><USER24> Ohh. News you can use. Thank you!

</comment></comment></comment><comment><USER26> > "I'm reaching out to share some great news"

This is disgusting. Remember that this letter is:

1\. for common shareholders who will get absolutely nothing out of this deal.

2\. yet it's asking them to sign so that they can go ahead with the deal.

3\. And to top it off, "I'm reaching out to share some great news"? Are you kidding me?

To be clear, I support all entrepreneurs regardless of what ridiculous business model they're trying to pull off. I even think Magic Leap should be given a benefit of the doubt and they may be able to pull it off.

Also I think it's great that silicon valley celebrates failure. Without this type of environment it's hard for founders to shoot for the stars.

That said, people have taken this too far, and it's become ridiculous. It's no longer about it being ok to fail, but it's about "I failed, so what? I'm the genius and I totally deserve it and you just paid to tag along."

I totally despise bullshit like this. If you lost, just say you lost. And say sorry. Don't bullshit people about how "it's a great news that I'm asking you to sign this letter so that other people--not you--will get the money you actually deserve".

<comment><USER27> This was an internal email to investors. You have no idea what convos she's had with them previously about the situation but you're very quick to assume she's some self-important Jobsian figure.

Be human, chill on the assumptions. Startups are hard and you don't have all the data so Danielle deserves the benefit of the doubt, not some vaguely related ranting about responses to failure.

<comment><USER26> I normally never criticize founders when startups go down because I have been there too and understand how hard it is for the founders first hand.

But please enlighten me. What kind of conversations could have possibly happened before this letter that could justify saying something like "You invested money (or time) and we've just lost all of it. You will be getting $0. But I need you to sign this so that other people CAN get some money. Oh by the way, it's a great news!"

Most common stock holders are probably former employees (whereas preferred stock holders are VCs). At this point I'm sure they didn't expect much out of this anyway, but they were once people who truly believed in the vision. If I were one of those people I wouldn't be happy to hear this fake facade about how this is a "great news". Phrases like this you should only use it for PR you send to Techcrunch, saying you had an incredible journey. But not to former employees who made a lot of sacrifices. To those people you be honest.

<comment><USER28> > But please enlighten me. What kind of conversations could have possibly > happened before this letter that could justify saying something like "You > invested money (or time) and we've just lost all of it. You will be getting > $0. But I need you to sign this so that other people CAN get some money. Oh > by the way, it's a great news!"

This reminds me of when my father's employer went out in the wake of the 1999 bubble popping. A couple of years later, the CEO sent him a very nice email saying "hey, mind signing over your stock please" and he was about to do it. I thought it was fishy that the ex-CEO want something worthless back, so I called my uncle who invests in SMEs on the side of his job and he agreed that we should ask for something back in return.

So we asked, and got, a few thousand for his stock, without a whisper of complaint from the ex-CEO (guess we asked for too little).

<comment><USER15> I'm reasonably certain that any of the common stockholders here can decline to sign and prevent the deal from going through.

No what they can leverage from doing that is a different story--blood from a turnip and all that. But, there's nothing that says they must take the deal as offered.

<comment><USER5> Drag along clauses exist but typically require that the stock gets paid for, if you're going to play tricks with multiple stock classes you can find yourself in awkward positions when you wish to screw the 'lower classes'.

<comment><USER15> Right. So, we may be able to infer that a drag-along doesn't exist or that the majority doesn't want to activate it, given that they are trying to offer zero to common, whereas a drag-along would _typically_ require that all holders are treated evenly.

Also, the claim that the amount paid would go towards company wind-down makes you think that plan would be negated through a drag-along, as some of the cash would "leak" to common holders.

In fact, it's actually curious that they are the acquiring the company in earnest vs. just acquiring its assets. Not sure why they would acquire the liabilities to the tune of $500K just to shut-down. Just pay for the assets and let the company use the funds to shut itself down. But, who knows? Maybe there's something (like paying customers) that they are finding difficult to transfer as an asset for some reason.

<comment><USER5> All agreed. So, if you're holding common stock sit tight and don't sign anything, a better offer is likely just around the corner.</comment></comment></comment></comment></comment></comment></comment></comment>


Show HN: I'm working on a Python front end for the USDA nutritional database - <AUTHOR> http://thiscurrentproject.wordpress.com/2013/08/22/perfect-meal/

<comment><USER2> This is excellent, great job.

I'm currently doing an internship with USDA-Agricultural Research Service(ARS), and was pleasantly surprised at the amount of free meta-data they offer to the public. For instance, I'm currently doing some work with GIS data/display and the USDA makes available soil data and GIS data files of the entire US as well as some other places like Puerto Rico for the public through their data gateway website[0].

[0] [http://datagateway.nrcs.usda.gov/](http://datagateway.nrcs.usda.gov/)

<comment><AUTHOR> Thank you!

I've been playing around with the soil data map, looks like a huge undertaking. About two minutes in I started to realize just how shallow my geological knowledge really is. It's really cool though, reminded me a little bit of my Geotech days.

Thanks again for checking the program out, feel free to request features or bug-fixes if anything occurs to you.

<comment><USER2> Yeah that soil data mart is a whole other animal. Luckily I'm mostly on the agronomy side of things right now, focused on Nitrogen, so we just kinda skim the data for soil properties (such as hydrology) that we need and display the GIS plots as a visual aid for the user. Then it's just manipulating that data to show trends on the GIS plots that the user is interested in. Having that soil data sitting there in a free and consistent format makes my job a hell of a lot easier, as I'm sure you found with the nutrient db.

I've recently started playing with Json (well, Google's Gson anyway) myself for a mobile application I'm working on so this may be a nice learning experience for me, but I'll dive into your source a little more this weekend and maybe try to knock an issue or 2 off your repo's list :)

<comment><AUTHOR> I've actually started a second json database for the program for different nutrient profiles and various data that needs to be stored. I'll probably push it later today once it's working. I'm planning on asking a lot more from json, and now that I'm doing it I'm not sure why I didn't do it sooner.

Python's great for working with json because a json file is actually a valid list of dictionaries in python. I've taken to validating it with a python interpreter, since it'll find any mismatched braces or extra/missing commas.

The only issues I have listed on the repo have to do with adding some minor functionality to the GUI layer, which would entail learning wxpython and dealing with the somewhat disorganized wx code I've written. I'm never sure what the best way to clean up the wx code is, but it's always on my mind.</comment></comment></comment></comment>


This Is the Biggest Threat Facing Google Right Now - <AUTHOR> http://time.com/money/3093863/google-android-biggst-threat-aosp/

<comment><USER2> without $open-source project donated by some company, $whatever wouldn't exist right now.. as i like to say to my friends..apache is probably the world's largest software vendor..</comment>


Authentication service in microservices by jwt in django - <AUTHOR> Can anyone give me a example how to implement authentication service by django using jwt. I googled but not find out any appropriate result <comment><USER2> [https://krzysztofzuraw.com/blog/2016/jwt-in-django- applicati...](https://krzysztofzuraw.com/blog/2016/jwt-in-django-application- part-one.html)</comment>


WebSub: Open protocol for distributed pub–sub communication on the internet - <AUTHOR> https://www.w3.org/TR/websub/ <comment><USER2> "Hub deliver content to each subscriber with a POST"

It's quite common on some social networks to have several thousands/millions subscribers for some pages/communities/accounts.

Is it really wise to build a publish-subscribe delivery system on top of HTTP? This seems to be a huge overhead.

In the meantime XMPP is already offering similar features (XEP-0060: Publish- Subscribe [https://xmpp.org/extensions/xep-0060.html](https://xmpp.org/extensions/xep-0060.html)) for more than 10 years. It's implemented in several servers and can handle huge loads without problems (everything is handled in real-time through encrypted TCP sockets accross the network).

We are building social networks on top of XMPP for several years now, you can check Movim ([https://movim.eu](https://movim.eu)) and Salut à Toi ([https://salut-a-toi.org/](https://salut-a-toi.org/)) :)

<comment><USER3> What's the overhead of HTTP vs XMPP? A few headers? Doesn't seem that different.

<comment><USER4> "a few headers" if 50% of your payload is headers, thats a big deal.

Mind you XMPP isn't all that efficient either. as its all based on XML.

<comment><USER2> I'm inviting you to read this page [https://xmpp.org/about/myths.html](https://xmpp.org/about/myths.html) :)

<comment><USER4> I didn't say slow, I said efficient. Its hilariously verbose compared to a decent binary protocol. Failing that something based on protobuffers.

In the embedded world a JSON/XML parser eats a tonne of resources.

One could of course use it over satcomm as it says, but its hilariously expensive when you are paying by the byte. But, compared to a massive JSON goop with embedded pictures that twitter uses, its a paragon of speed.

</comment></comment></comment></comment></comment><comment><USER5> XMPP has already been mentioned (thanks edhelas). What I didn't see mentioned was ActivityPub ([https://www.w3.org/TR/activitypub/](https://www.w3.org/TR/activitypub/)), or some of the advanced messaging features users have come to expect (e.g., store and forward, forms, automated route handlers).

What I am curious about are the following questions?

1) What differentiates WebSub from XMPP?

2) What differentiates WebSub from ActivityPub?

3) How are you handling the N-squared delivery problem, if you are delivering content directly to each subscriber with HTTP POSTs?

4) Does WebSub currently support store and forward? If not, is that on a roadmap for a future version?

5) Same as 4, except for support for forms and form responses? Examples are a builtin Yes or No reply, or a builtin poll vote.

6) Same as 4, for automated message routing.

7) Why not have it be transport-agnostic, instead of mandating HTTP? And why HTTP? The growing trend is towards more decentralized.

8) How does this compare to Sir Tim Berners-Lee's SOLID ([https://solid.mit.edu/](https://solid.mit.edu/))?

</comment><comment><USER6> So I read the comments but I still don't understand what problem this specification solves. Does it enable new use cases? Does it enable new trust models? Since it's only a server protocol, how do final users actually read the content?

Edit : I don't understand the downvote. I use RSS daily to fetch news and I don't have problem with it. It's simple and deliver news to the edge (my mobile phone). I'm not saying WebSub is useless, I would like to understand what a 3 entities model brings to the table compared to a simple server-client delivery. What's more, the Subscriber entity cannot be a mobile device with the current network, because mobile internet providers block incoming GET requests. Therefore to fetch news, it has to be a pull model.

Why not add a simple rational in the header of the spec, explaining the problem, the existing solutions, and why this new solution? Does it solve a security issue, a scalability issue or a trust issue?

<comment><USER7> WebSub is the successor to PubSubHubBub, which was designed to make distribution of RSS items more efficient (ie receive items as they are published on the host):

\- publisher would send updates, instead of having everyone poll \- publisher would be protected from thundering herd if a content suddenly becomes popular \- publisher and subscribers wouldn't need to exchange a full "page" of items when only one is needed

<comment><USER8> This sounds great in principle, but this is 2018... virtually none of the use cases for RSS can be replaced by something which requires the consumer to be online and capable of receiving an HTTP POST (which isn't possible on the web platform at all, so this shouldn't be called WebSub). To the extent to which a "thundering herd" ever existed for RSS, it was from large numbers end users with normal computers behind firewalls constantly checking the feed, which is an environment where this protocol has no hope of ever being applicable.

So far, no one on this entire thread has descried a single use case where WebSub actually provides value. Someone mentioned that theoretically a service like Facebook could use this for Facebook, but then they themselves linked to a page on Quora with an explanation from someone at Facebook for why they tested out PubSubHubbub and then gave up on it which stated that he "think[s] the benefits of adopting PubSubHubbub are less clear" for this very issue (that it is inherently a server-to-server protocol, where there wasn't really a problem in the first place).

<comment><USER7> Are you saying that webhooks have no purpose in 2018 ? Because while WebSub was created as an effort to help RSS get more realtime, it's a more generic architecture for publishers who have content and need to notify interested subscribers.

<comment><USER6> What we are saying is : In the attempt at understanding the use cases of this proposal, people said in the thread webhooks were created to replace RSS+GET pull, as an "enhancement".

As we are trying to explain : this proposal doesn't cover all use cases of RSS+GET pull, so it should not be considered a replacement of RSS nor an enhancement of it.

If WebSub has a purpose (if any), many people fail to explain a valid one. Does it worth wasting the resources of the W3C for such proposal?

The "world wide web" is wide, and doesn't only include servers

<comment><USER7> > people said in the thread webhooks were created to replace RSS+GET pull

That's the root of our misunderstanding then: never was PubSubHubbub _replacing_ RSS+GET pull, it was supposed to _help_ it for people who want notifications, on top of the existing system.

<comment><USER6> The root of the misunderstandong is that their is no rational in the proposal. How are people supposed to understand any proposal without a rational?

</comment></comment></comment></comment></comment></comment></comment><comment> friend-monoid 1\. Why not a SUB HTTP request? And a PUB http request. The response URL could be a required header.

2\. We have HEAD, can we do service discovery using HEAD?

3\. Why not let a topic be a HTTP URL? "PUB /user/john/position HTTP/1.1\r\ndata...".

4\. Subscription expiration as a way to force subscribers to renew and upon renew get redirected to other servers is pretty cool. NATS has a special message (the INFO message) to do the same, but you might be in the middle of an important request-reply session you don't want to abort.

5\. The authors could have made this protocol very "non-http-ish" by implementing what amounts to Redis but in HTTP. I'm glad they didn't. This still feels like HTTP, which is great.

<comment><USER9> Middleboxes and bad HTTP servers/clients would explode, I guarantee it.

<comment> friend-monoid But we have the PATCH method, which is... Non-standard, I think? And it works fine.

<comment><USER3> It's been a standard for a few years: [https://tools.ietf.org/html/rfc5789](https://tools.ietf.org/html/rfc5789)

</comment></comment></comment></comment><comment><USER10> Superfeedr[1] (acquired by Medium in 2016[2]) was probably the biggest company yet, that based its entire business model on WebSub / PubSubHubbub protocol.

[1] [https://en.wikipedia.org/wiki/Superfeedr](https://en.wikipedia.org/wiki/Superfeedr)

[2] [https://techcrunch.com/2016/06/02/super-to- medium/](https://techcrunch.com/2016/06/02/super-to-medium/)

</comment><comment><USER11> Just thinking beyond a replacement for RSS, this could be used for large-scale persistent virtual worlds. WebXR version of Second Life. GLTF as a content transmission format. As the Hub updates with a POST request, Subscribers callback could return local state. Hub becomes global state of truth. And there is no need to manage peer net or continuous WebSocket connections.

</comment><comment><USER12> This has come along way from "WebSub was previously known as PubSubHubbub"

<comment><USER13> The former is a much better name IMHO!

</comment></comment><comment><USER14> So it's an alternative to something like WAMP? (Web Application Messaging Protocol on top of websockets - [https://wamp-proto.org/](https://wamp- proto.org/))

<comment> friend-monoid Except you don't have an open connection where you receive messages on. Instead, you register a callback url. It's more like webhooks, but it's it's own thing, and not a setting on a page.

</comment></comment><comment><USER15> Just bring back Fidonet

</comment><comment><USER16> How does this compare to ActivityPub used by Mastodon et al?

<comment><USER17> See it more as an alternative to RSS. Aggregators subscribe and feed information to their users. The polling step is just switched to pushing.

<comment><USER18> Not really an alternative, more of an addition to. The most common use case for Pubsubhubbub/WebSub is in combination with RSS/Atom feeds, informing subscribers about updates to those.

</comment></comment></comment><comment><USER19> I wish there was a pull aspect to it, like NEWNEWS with NNTP. If the subscriber is offline it seems that there doesn't seem to be a way to retrieve missing items except through the publisher.

</comment><comment><USER20> > The subscriber must be directly network-accessible and is identified by its > Subscriber Callback URL

Does this mean the subscriber needs to have a forwarded port open to the internet for this to work? Without IPv6, users behind NAT (and specifically behind CGNAT) wouldn't be able to use it.

</comment><comment> Muhammad-saleh I think Websub is enhancement to XMPP. Both are asynchronous. Which means it should be faster than HTTP. Actually it's a push-based protocol where server sends single POST to the hub, the hub then sends that notification to all subscribers.

</comment><comment><USER21> So as with webhooks, this will require you to have a permanently reachable server sitting on the internet. Couldn't they at least have defined an alternative transport using websockets or SSE?

<comment><USER3> WebSub is essentially just the transport; the payload is usually Atom or RSS, but it's not specified. Had they included a websockets version, it would essentially be something almost completely different with the same name.

</comment></comment><comment><USER22> What I don't get is what the motivation is for running the hub, other than "get bought by Medium?"

</comment><comment><USER23> Is this compatible with new distributed crypto projects like scuttlebutt or dat, and if not, why not?

<comment><USER3> WebSub is just the standardization of PubSubhubbub, which has existed (and been used by publishers like Feedburner) for longer than those projects.

<comment><USER24> So essentially there's no difference right? When I was reading the spec I couldn't tell of any notable differences between the two texts.</comment></comment></comment>


A Proof That P Is Not Equal To NP? - <AUTHOR> http://rjlipton.wordpress.com/2010/08/08/a-proof-that-p-is-not-equal-to-np/

<comment><USER2> Edit: I see I'm in the wrong thread. Everyone else is over here: <http://news.ycombinator.com/item?id=1585850>

Here's what Lance Fortnow just tweeted: "Much ado about an unverified proof. The word 'must' is troubling. I'll let others check it carefully."

Fortnow wrote this popular article in the Communications of the ACM on the status of the P=NP problem: <http://www.cs.uchicago.edu/~fortnow/papers/pnp- cacm.pdf>

</comment><comment><USER3> this has to be the world record for most dupes in a single day :)</comment>


LifeLock CEO Had His Own Identity Stolen Numerous Times, Suit Alleges - <AUTHOR> http://www.dvorak.org/blog/?p=17930 <comment><USER2> I don't see what the big deal is. Lifelock explains what they do and have a guarantee.

<http://www.lifelock.com/lifelock-for-people>

Here they explain that everything they do can be done by you for free. They describe what they do as changing the oil in your car. You could do it yourself, but it's just convenient to have the professionals do it.

</comment><comment><USER3> The fault lies in the system. Nowhere else in the world it's possible to create an official document (like a driver licence) or a bank account without proper authentication.

<comment><USER4> Am I understanding this correctly? In the US, you can create driver's licenses and bank accounts with only a social security number?

<comment><USER5> In Missouri at least, to get a driver's license you need your SSN, your birth certificate, and some form of identification like a passport or a bill in your name at your current address.

<comment><USER6> as if it's so hard to give cynthia at AT&T a fake name and address.

<comment><USER7> it's much harder to get the birth certificate. The utility bill is only used to establish address, and making sure that everyone's address is up to date is not easy or worth too much effort.

</comment></comment></comment></comment></comment><comment><USER8> Social Security # are security through obscurity. We need a national ID system that provided security by design. And, don't bother arguing that we shouldn't have a national ID system because Social Security #s are already a national ID system. If we're going to have a national ID system, it might as well be a good one.

</comment><comment><USER9> I'm so glad this was posted on YC. Thank you! I'm glad this place is featuring the exact same links as reddit, digg, slashdot, etc. Woohoo copycats!

</comment><comment><USER10> A suit alleges to have had his identity stolen... I, for one, am not surprised.</comment>


Apple's iPad Seen Cannibalizing Netbook Sales - <AUTHOR> http://www.macrumors.com/2010/05/06/apples-ipad-seen-cannibalizing-netbook-sales/ <comment><USER2> Apple doesn't sell netbooks, so the iPad isn't cannibalizing netbook sales. If iPad sales decreased sales of, say macbooks, then it would be appropriate to cry cannibal.

Cannibalization from Wikipedia:

"In marketing and strategy, cannibalization refers to a reduction in the sales volume, sales revenue, or market share of one product as a result of the introduction of a new product _by the same producer._ " [emphasis mine]

<http://en.wikipedia.org/wiki/Cannibalization>

</comment><comment><USER3> It's just about a month into it. There's no way a valid correlation can be made. This is just linkbait.

<comment><USER4> You're right: the attempt at making a correlation with the slowing of netbook growth deceleration is dubious. That's only half of the article, though. The other half is data from survey respondents:

 _a Morgan Stanley/Alphawise survey conducted in March that found that 44% of U.S. consumers who were planning to buy an iPad said that they were buying it instead of a netbook or notebook computer._</comment></comment>


Android 3.0 and Motorola tablet make surprise appearance at All Things D - <AUTHOR> http://techcrunch.com/2010/12/07/video-and-screenshots-of-android-3-0s-surprise-appearance/

<comment><USER2> Avoid blogspam, direct link to interview snippet; [http://video.allthingsd.com/video/google-andy-rubin- highligh...](http://video.allthingsd.com/video/google-andy-rubin-highlights- at-dive-into-mobile/D2214853-E3EB-44B0-B641-47DB98994533)

I have to say the women is very annoying in every single interview I have seen her in. Why does she feel like its OK to interrupt people when they are talking? Its not even in the form of follow-up, relevant good question. Just interruption for the sake of interruption to say something. arghhh</comment>


Simplify: move code into database functions - <AUTHOR> https://sivers.org/pg <comment><USER2> Stored procedures, functions, triggers, etc, are very useful tools. They are used regularly in Microsoft SQL Server and Oracle systems.

But, one has to be very careful to use them when warranted. The out-of-the-box tooling for debugging, maintaining, versioning, and testing code that lives in your database is not nearly as robust as what you have for most other development environments.

The key here is to use the tool when it is most useful.

SPROC's are good when:

1\. You need extreme data security and you are able to pass authentication tokens from application to database. You can apply authorization rules when you read/write the data, rather than transporting more than you need over the network. This eliminates a number of potential attack vectors. I have worked on HIPAA compliant systems for government agencies that held sensitive data that required all data access to be through stored procedures with proper access permissions, in addition to an authorization tier in the application logic. Very common use case.

2\. You need to churn through a bunch of data and need very close data locality to pull that off and meet your performance requirements.

2a. Your database nodes are network IO bound and this is due to applications requesting more data than it needs, and that logic can be performed in the database.

3\. You have a very strong DBA/Developer that can manage your data access layer that your application uses as well as code the database. This can abstract the "how" of data storage from the application developer, which can be very useful in a data-centric large application.

However, SPROCs are bad if:

1\. You have to switch DB providers due to scalability, or cost issues, etc. The switching cost can be huge if there is a ton of logic buried in your database.

2\. Your database nodes are CPU bound. In this case you'd want to do as little logic as possible in your DB. This is more rare nowadays, but used to happen.

3\. SQL (or whatever-sproc-language) is not a core language you want your team to have to become experts in.

4\. You don't like adding more code. You'll end up generating and writing yet another layer of code, this time for stuff that lives inside your database. This code has to be maintained, versioned, etc.

<comment><USER3> _The out-of-the-box tooling for debugging, maintaining, versioning, and testing code that lives in your database is not nearly as robust as what you have for most other development environments._

This is the key point for me. I've seen SQL databases with 60k+ LOC in stored procedures, and functions (thankfully they avoided triggers). This code wasn't versioned, tested, or commented. While it's technically true that you can test and version SQL code that lives inside a database, I've never seen it done.

This is mostly a tools issue, although its also partly a cultural issue, where 'stuff that goes in the database' isn't seen to be subject to the same laws of entropy and software development that all other code is.

<comment><USER4> > I've never seen it done.

See my comment[1]. We're doing it and it works beautifully.

[1]: [https://news.ycombinator.com/item?id=9485502](https://news.ycombinator.com/item?id=9485502)

<comment><USER3> Awesome, I'd love to try working in an environment like this to see how it works and experience the tradeoffs.

</comment></comment></comment></comment><comment><USER5> The reasoning always offered for treating the database as a dumb store with no intelligence has been about portability. Which is fine if you are selling third party software meant to be installed onto an existing customer's database. But for all other use cases, not using the features of the platform is a mistake.

PostgreSQL offers so much!

        * Stored procedures in Java, Python, Lua, Perl, JavaScript etc [1]
        * Multicorn [2], query external data sources
        * Common Table Expressions [3]

[1] [https://wiki.postgresql.org/wiki/PL_Matrix](https://wiki.postgresql.org/wiki/PL_Matrix)

[2] [http://multicorn.org/](http://multicorn.org/)

[3] [http://en.wikipedia.org/wiki/Hierarchical_and_recursive_quer...](http://en.wikipedia.org/wiki/Hierarchical_and_recursive_queries_in_SQL#Common_table_expression)

<comment><USER6> Yea, I've never really understood the portability argument. How often do people switch between sql dbs?

<comment><USER7> One of the advantages is the ability to run a different database during local development / testing from your production systems (say H2 locally vs. DB2 running in production). Personally, I view this to be an anti-pattern, but some people have to work with systems like this.

<comment><USER8> Yes, this is a very bad idea. Due to the way that query planners work, it's not even a good idea to vary the shape of the dataset between testing and production. If you have a different (bigger, higher stddev, etc) distribution of data in column x, the planner will often complete a query differently

<comment><USER9> Agree, unless your need for a DB is quite trivial, then perhaps something like SQLLite for the devs and whatever else for production. However this breaks down quickly once you grow beyond needing very simple INSERT and SELECT type operations.

<comment><USER5> At which point one should just run SQLite everywhere, It is often excellent for datasets up to about a gig.

</comment></comment></comment></comment></comment></comment><comment><USER10> It takes guts to write like this. And I fully agree with Derek.

The sheer momentum created by the pontiffs, academics and pundits on this subject who exhorted the virtues of layering architectures moved us into a blind alley. The OO crowd (barbarians) were at the gates first. After two solid decades of this slow exodus of logic from RDBM systems to Java, .NET and whatever other middle-tier architecture we dreamed up, we were inevitably back to the same problems we thought we were avoiding: remove complexity, improve maintainability and portability. And we gave up a lot to get that: we gave up reliability, we gave up performance, we gave up security and we gave up simplicity. Ah, but no worries, we thought, we could write faster iterating loops, introduce threads, write custom access control and authorisation, custom caching solutions ... yeah, we will do that but do everything to eschew the simplest of set-based operations on SQL databases! :-(

But there was no lying about how we missed the database in our business tier. Oh we missed it. We missed the database so much outside the RDBMS that we REcreated it! And we christened this compromise of an ingrown toe-nail an "ORM". Two decades have come and gone and we are still agonising this unwinnable war that Ted Neward infamously called the "Vietnam of Computer Science".

So, while we jettisoned the relational database onto the scrap heap of software development tools, ironically the makers of the RDBMS software (both open source and commercial) continued to advance and mature it. Patiently. We have CTEs now. We have parameterised tables.

It takes all of 5 minutes to show a developer who has never seen the impact of doing a set-based business rule implementation vs one in Java etc that we are too far gone in our craft!

Derek is spot on. Respect.

</comment><comment><USER11> It's incredible to me that he watched Simple Made Easy (which is an incredible talk), and his mind went to "stick more stuff in the database". Not because that approach wouldn't work, but to me a clear sign of complection would be conflating the medium of data storage (a database) and the business rules that govern the data itself. But I guess that's the the sign of a good talk. It's open to interpretation.

It also complicates your scaling story, your caching story, your ability to change databases and a great number of other things.

If you're building an app that has no chance of ever possibly needing to scale, or use a different database for caching, searching, or analytic reasons, then this approach may be optimal, but if any of those levers need to move at some point, I think you'd be in for a world of hurt.

<comment><USER12> I think we're talking relational DBs here, right? In fairness you already have business rules in the DB, implicitly, just from the layout of your tables. You probably also have foreign keys. At that point, a stored proc that translates business needs like "at customer location X, stop service A on this date and replace it with service B" to the multiple table reads and inserts required, seems like a good idea. It's actually not that weird to keep most of your devs out of the database entirely, in which case they probably appreciate being able to just call a stored proc to get something done.

</comment></comment><comment><USER13> A serious problem with this approach is scaling.

When your code is in Ruby/JS/whatever on separate servers, and your traffic increases, you can easily scale up by adding more stateless servers.

Going from 1 to 2 DB servers is much harder, and can easily mean a major rewrite, during which your site may not really work.

<comment><USER14> > A serious problem with this approach is scaling.

Stack overflow runs off one SqlServer. Do you think your app is going beyond that? If yes, maybe this approach will not work for you. If no, you are in the 99,99999% of all other sites/apps that are not as big as SO, FB or google and you can use this simple way to make your site or app.

<comment><USER15> StackOverflow's database doesn't need to work that hard because the vast majority of traffic is read requests for public, essentially static webpages that get re-rendered and updated a handful of times in their lifecycle. I'd guess 90% or more could be served by a reverse proxy without ever touching SQL Server or even the app tier.

It would be easy to exceed its DB load with an application that serves unique content to each user, is write-heavy, etc.

<comment><USER14> > It would be easy to exceed its DB load with an application that serves > unique content to each user, is write-heavy, etc.

Just curious: did YOU ever reach that point? I mean where a well optimized db engine reached it's limit? Can I ask you on what website?

</comment></comment></comment></comment><comment><USER16> In most of the example he gives, these solutions are ensuring the integrity of the data stored, which I personally have no problem with; that is the job of the database in my view. If I try and store crap the database should refuse, be that via constraints, triggers or stored procedures.

As for returning the data in JSON, again, I see no problem with that; you were going to get the data out in rows and columns, what difference does JSON make, especially if it simplifies things, as he is arguing.

My main concern is business logic; should business logic be stored in the database. At this point things become a little grey, primarily due to the difficulty in maintaining and managing SQL code. I suppose that PostGres is better that most of the DBs I have had to deal with in the past in this respect in that it supports a wider range of languages for this purpose, but I'd be interested to hear others opinions and experience in this regard.

<comment><USER8> Sadly, to some extent the data integrity is part of the business logic. For example, if products can or cannot be sourced from multiple suppliers

</comment></comment><comment><USER17> I find myself a bit thrown by encrypting passwords with a stored database function... which involves communicating with the database server with the unencrypted password.

The Microsoft SQL Server and Database Management Certifications and associated course work strongly discourage sending a password in that manner -- no matter how secure the connection is.

I'm unable to agree with the "simplify" aspect of the article title. My own personal experience tells me data should be validated where it is accepted rather than sent on to the next step -- the contrary may raise unintended additional security concerns or exploits.

Granted, my view stems from the article author's directive given towards "web or api developers." There are many situations where software makes a connection directly to a database rather than indirectly.

</comment><comment><USER18> Historyically, I've been against putting logic inside (MySQL) database since it's a bitch to version and maintain (especially cross-database-user)

Maybe Postgres is a lot better, but for MySQL I'm staying the hell away from it

<comment><USER19> It does mean you need to treat your data schema and functions as code.

My short checklist for doing database centric development (postgresql centered):

* Database schema is kept under version control ( essentially all the DDL for a given database is kept in one place and managed as a unit ). You can use `pg_dump -s` to extract the schema from an extant database.

* data types and functions are kept in separate files under version control

* the database is built and filled automatically for tests, for migrations and for upgrades. Shell script, Makefile, Chef; the tool doesn't matter so long as the process is repeatable and automatic.

* data fixtures for testing are kept separate from the code ( own repo usually, since you want to be able to add edge cases and errors to the test database ).

* migrations on the production database are run immediately after a backup.

Nothing too fancy, just basic hygiene. But staying on top of that means you're ready to do things like build your application onto a new set of machines; or make drastic changes to how your data is structured.

There are a number of tools that do much of the scutwork for you ( South for Django, ActiveRecord::Migration for Rails, etc.) but the tools are not a substitute for understanding what your database is doing.

<comment><USER20> > pg_dump -s

I wholehartedly agree that the schema should be versioned. But the schema dump is imho unreadable. I edit it manually instead, which has its own drawbacks.

Since I referr to the schema a lot, I think it is worth optimizing for readability.

</comment></comment></comment><comment><USER14> It's 2004 all over again: [https://web.archive.org/web/20060525094651/http://www.oracle...](https://web.archive.org/web/20060525094651/http://www.oracle.com/technology/pub/articles/odtug_award.pdf)

I love this approach and I used it whenever I could in the last 10 years. It's just so much easier when all you have is a DB and a JS/HTML/CSS frontend.

</comment><comment><USER21> Have we learned nothing in the last 10 years? Aside from the siren song of better performance, there are few other reasons to model your domain in the database.

Throwing business logic into stored procedures wily-nilly is a terrible idea for a lot of reasons. First, there is no standard method for testing stored procedures functions; you have to put a lot more effort into devising a test strategy for your database functions than your controller functions. In addition, you've now coupled the business logic with the database, and you cannot move your data around easily. And, lastly, if you have a large database, adding new functions and triggers will involve some downtime, too.

To sum up, don't replace your Rails app with a database unless you really know what you're doing.

<comment><USER9> Most SQL databases that support stored procedures have some kind of unit testing framework available these days, in fact have had for many years. Possibly as a third-party product or project, but the tools exist.

If you design your stored procedures right, they can be an abstraction over your data model. You then can change the data model without the stored procedure interface necessarily having to change (though in most cases, changes to the data model are usually done to support new requirements, which typically involve new or changing business logic as well).

<comment><USER21> That's good to know. The question then becomes whether it makes sense to support yet another testing framework. The pros of performance gains have to outweigh the cons of additional complexity by a considerable margin to make database functions a viable choice over application level functions.

</comment></comment></comment><comment><USER22> The article makes some valid points but anyone who thinks databases should be used for everything and the kitchen sink should come fill out an application where I work. They could then spend the next 10 years debugging hard to troubleshoot database issues instead of easy to troubleshoot code issues. Jamming too much logic in the database is a recipe for disaster in my experience.

</comment><comment><USER23> I've gone down this road before, and one of my biggest complaints was around managing and deploying the code built this way.

I just found plain server side code easier to reason about.

</comment><comment><USER24> Having maintained a giant codebase in Oracle PL/SQL, I'm going to have to respectfully disagree and run screaming in the other direction.

</comment><comment><USER25> I'm currently reading Martin Fowler's "Patterns of Enterprise Application Architecture"[0] and I think that one of the main points that stuck with me is his recommendation for separation between the various layers and functions of a software system.

Basically the whole system should be a layer cake of smaller modules/systems, as "opaque" as needed to one another, but interacting with each other solely through a well defined "internal API".

I think the approach proposed in the article is good when starting or prototyping projects, but the object oriented and layering approach, even if it's a bit more challenging to visualize correctly at the start of the development of an app, is worth its weight in gold as features start piling up.

[0] [http://www.amazon.com/Enterprise-Application-Architecture- Ad...](http://www.amazon.com/Enterprise-Application-Architecture-Addison- Wesley-Signature- ebook/dp/B008OHVDFM/ref=tmm_kin_title_0?_encoding=UTF8&sr=8-1&qid=1424636059)

later edit: for clarity.

<comment><USER5> Putting more intelligence in the data layer doesn't break the modularity of the system, it also doesn't obviate the need for middle layers. The biggest thing it does enforce is a semantically clean gateway to the underlying data store. Databases can do a whole lot declaratively in way that is correct by inspection. Six lines of stored procedure can save hundreds in the middle tier while allowing other tools to access the database directly, w/o forcing everyone through the middle tiers.

</comment></comment><comment><USER9> I like most of this thinking, and it's the way I've built systems for years. Data model and an API for it in the database. Apps built to use that API, never access tables directly (or at least, that is the ideal).

This lets me build systems using different front-end platforms and languages as needed and appropriate for the task, calling one tested API to interact with the data.

One caution I would have is with triggers. Triggers are a way to make things happen as if "by magic" and I avoid them. I prefer to keep things more obvious and easier to reason about. If inserting a row into some table should have a side-effect, I code that explicitly and don't make it happen via a trigger. One exception might be to auto-generate IDs if the database doesn't have a type that does that for you.

</comment><comment><USER26> > "In 1997, I started in Perl. In 1998, I switched to PHP. In 2004, a rewrite > in Rails. In 2007, back to PHP. In 2009, minimalist Ruby. In 2012, client- > side JavaScript. Each time I'd have to re-write all of the logic around the > database: how to add a new person into the database, how to verify an > invoice is correct, how to mark an order as paid, etc. But that whole time, > my trusty PostgreSQL database stayed the same."

And what if during that time instead of switching programming languages, it was the storage system that had to be switched? You could easily invert the argument "Don't keep the logic in the DB! I had to switch from Postgres to MSSQL which meant I painstakingly had to convert all my db-logic, I wish it had been in the code instead!"

<comment><USER27> Switching programming languages is far more common than switching databases.

<comment><USER26> *Citation needed

You wouldn't even need to switch the database, just add a cache between the storage an the app, or add/switch ORM etc.

There is a huge impedance mismatch between table storage and code, but I don't see how it gets more responsive to changed requirements by using logic in the DB.

And don't even get me started on the whole debugging/unit testing issue with logic-in-databases.

</comment></comment></comment><comment><USER28> I think it can be a good idea for certain very predictable applications - predictable in terms of scaling and future feature enhancements.

But beyond that this strikes me as a bit too complicated. In particular, coupling business logic with database choice is risky. Martin Fowler, Bob Martin, and others underscore that the real cost of development is not the inability to predict the future, but in how difficult or time-consuming it is to change, when reality changes unexpectedly.

As soon as something happens that would bring repository choice into question, or would suggest the need to split parts of the business logic up into different locations, the choice to couple business logic with database implementation would start to feel really limiting.

<comment><USER14> > As soon as something happens that would bring repository choice into > question

I don't know. Once a project is started with postgres/mysql/orcale, how often do you change the DB?

We have some projects that started with struts, then we changed the fe to jsp, the asp.net, then HTML/JS + MVC.net. But the DB was always Postgres. It never changed. In my experience what changes often is the frontend, not the stuff that keeps your data.

I would optimize for fe/middleware changes, not db changes.

<comment><USER28> Plenty of examples, although they all tend to be higher-scale considerations that might not apply for your small business or side project:

1) You have "friends" or "friends of friends" modeled in Postgres, but then you realize that you want/need to do more social graph stuff in particular, so it makes more sense to use a graph database like Neo4J

2) You're saving a ton of activity data in postgres but your site is slowing down, and you realize you don't need the archived data, so it makes more sense to use something like Redis for that chunk of data

3) You're saving a ton of log data but the storage of it is going way up, and you're only querying it for big data / data warehouse purposes, so it makes more sense to use something like Cassandra

4) You're wanting to write a new feature set that relies on existing normalized data but would require new complicated queries and joins and business logic to deal with it, plus you're running into scaling considerations, so it makes more sense to convert to a reactive event-based system, such as micro services consuming a kafka topic, where each micro service has its own dedicated repository

In each of those cases, if you have used Postgres combined with sprocs, it's much harder to transition than if you had coded to interfaces and relied on your db in a dumb sense.

</comment></comment></comment><comment><USER29> The biggest reason to use stored procedures to me is that db developers tend to stick around around a business lot longer than web developers, so putting most of the business logic into the db is better for the business in the long run.

Like the many developers, I generally develop custom apps specifically for a particular business. When I develop new apps I start by by putting all the intelligence in the serer-side code, so it's easier to change things without needing to go through someone else.

Once the apps stable then much of the business intelligence gets moved into stored procedures in the db level.

</comment><comment><USER30> It depends what you do...

If you need to manipulate a lot of data that ends up travelling on the wire, you can save a lot of time by running it on the server.

On the other hand, the language on the DB servers are often a bit harder to reason with.

<comment><USER31> _> the language on the DB servers are often a bit harder to reason with_

But can't we say that about every other language? I've been writing a lot of PL/pgSQL lately and I've enjoyed every bit of it. For PostgreSQL there's also PL/Python, PL/V8, etc. Those are packages for writing the server code in python and JavaScript respectively and out the window goes the obscure language excuse. But that's if you're on Postgres.

</comment></comment><comment><USER32> I always enjoy reading about new ways of doing things, so thanks! How would you test your logic in isolation with unit tests using an architecture like this?

<comment><USER33> The same way you test any other piece of code.

A stored procedure is just an encapsulation of logic, with inputs and possibly state changes and/or outputs.

Expected/actual asserts can be coded in SQL.

The code tends to be verbose, but some RDBMS have evolved to include more powerful language features. It's probably possible in almost all RDBMS, but certainly easier in some than others.

</comment></comment><comment><USER34> Before the days of GUIs when regular office workers used the command line, the database was an all-in-one application. Stored procedures were created by the DB admin so that the non-techies could run reports with simple commands.

I've found it interesting that all of this functionality has remained in databases, but is almost never used in modern web applications.

</comment><comment><USER35> I wrote a gem to help test that unique indices match uniqueness validations in rails [https://github.com/emma- borhanian/schema_expectations](https://github.com/emma- borhanian/schema_expectations)

If you only do rails-side validation, you have an (exploitable) race condition.

</comment><comment><USER36> Is anyone working on better tooling for debugging, maintaining, versioning and testing code that lives in the database?

</comment><comment><USER37> So, at one end, we're moving stuff out of middleware and into the database, and at the other end we're moving all views into the client.

Imagine if SQL returned javascript readable sets of tabulated data... If it did, then you could completely remove all processing on the web-server.

</comment><comment><USER38> And now you have two problems.

</comment><comment><USER39> been there done, killed productivity of a team and then stopped doing it :)

<comment><USER39> to expand, DB functions are usually:

* hard to test

* code is harder to maintain

* vendor specific

* often written in very weird programming languages</comment></comment>


Google Squared - the Cuilest search app ever - <AUTHOR> http://www.theregister.co.uk/2009/06/08/dziuba_google_squared/ <comment><USER2> While I usually dislike rants, I played with google squared and agree. It's pretty much useless for everything I tried.

Does anyone have _any_ searches it's good for? I'd love to be proven wrong about it's utility.

</comment><comment><USER3> Yeah, it kind of seems like the "example searches" they have up are the only searches that actually produce anything comprehensible...Cool concept, needs more development before it will be ready for primetime.</comment>