Unit testing is not (generally) useful - <AUTHOR> http://prehacked.com/60388.html

<comment><USER3> The author rejects unit tests because they don't help find bugs. I don't think his reasoning adds up to much, more like rationalizations for already-held beliefs.

But if you want to take the argument seriously, I would say that "finding bugs" is not the primary reason I write unit tests anyway. It's to prevent regressions, and make me feel safe that I haven't broken anything after a refactoring. If not for my unit tests, I'd be scared to change much of anything, for fear of stirring up a whole bunch of trouble that I'll have to fix later on.

<comment><USER4> "preventing regressions" is a subset of "finding bugs", and I've found the other methods I listed to be more effective (from an ROI perspective) at doing that too.

<comment><USER3> Apparently you work at Justin.TV. I don't know how big your individual projects are, but personally, I didn't get unit-testing religion until I started working on _really big_ stuff, close to the limits of what I'm capable of. I worked for a year on a 60,000 line program, and it started getting painful. I worked for another year on a 100,000 line program, and that's when I finally had to relent and start writing unit tests.

Here's a common scenario that made me want to write tests. I want to implement feature A, but to make it work, I'm first going to have to write library B and C. So I write library B, but there is no way to test it in the program yet, because feature A doesn't exist. Right after I've finished the code, I have a pretty good idea of where things might go wrong, and what corner cases I want to exercise. But by the time I finish library C and then get around to implementing feature A, I've forgotten that stuff. Pain points go unaddressed.

In summary, unit tests really start to show their value when you're working on a program that's too big to fit into your head all at once.

<comment><USER4> Couldn't you have tried to break that 100,000 line program up into ten 10,000 line programs? You might have found the additional benefit that each of the ten programs could be useful on its own. That's how I would usually attack this kind of problem, rather than throwing unit tests at it.

<comment><USER5> I think it would be preferable to do both. Have a few really solid and well contained modules that are well tested.

Testing is only a piece.

</comment></comment></comment></comment></comment><comment><USER6> Judging by his analogy and depth of insight, I'm guessing he's never even tried to write unit tests on a daily basis and certainly never done TDD. It's the same old argument that everyone makes when they've never actually learned to do something properly.

<comment><USER4> Actually I wrote unit tests daily at my last company, for about two years, and in the last six months of that I gave TDD a try.

My claim isn't really that they don't work, it's that (compared to the other things I listed) they have a _tiny_ ROI.

<comment><USER6> Unit tests have nothing to do with, and aren't in any way mutually exclusive to load testing and logging and user reports. Frankly, it just sounds naive to say that you replaced unit testing with logging, or load testing, or letting your users find the bugs for you.

People that do unit tests do all that stuff too!

Your lack of specs also has nothing to do with unit testing. It might dictate the difficultly in writing acceptance tests, but that's a completely different subject.

I don't feel the need to proselytize unit tests since you've already written them off, but what's the point to posting about why you don't like something only to back it up with some sort of rambling, incoherent analogy that _you_ probably don't even believe?

<comment><USER4> I certainly do believe the analogy.

The point was that the ROI on unit tests is almost always very much lower than the ROI of the others. So when time gets tight (as it always is in a startup), you should generally put effort into all the other things and _not_ into unit tests.

</comment></comment></comment></comment><comment><USER7> I'm currently writing a personal project for university, and I've made sure to test it all the way through.

When I was part-way into coding this thing, I took a day to go through all the code and write tests for anything I could. I found a few bugs, so I could at least say that it helped somehow. But the tests have helped a lot more when I added to it - it feels -good- to see a big green bar light up whenever I add something. It's -nice- to run all tests, leave it a minute, and confirm that it's not going to blow up before I have to demonstrate to anybody. Of course, finding bugs is nice, and I set up a better testing rig to make sure it doesn't fail on anything I haven't thought of, but the benefits of unit testing go further than bug detection.

When you're all crestfallen with no desire to code, finding bugs is only secondary.

<comment><AUTHOR> It's a false sense of security though. It's like putting a big badge on your website saying it 'validates' - even if it's totally broken in real world browsers.

In my experience, the more common bugs are integration bugs, scaling bugs, weird memory usage issues, not to mention interfacing to external systems - for example some weird browser comes along and does idiotic requests you'd never expect a sane person to do.

Those things are very hard to test for, and don't really produce a good ROI as abstractbill says... Best just to log things, graph things where possible, spot weirdness, investigate and fix.

<comment><USER8> Everything you mentioned I count as part of TDD or BDD or what ever automated testing acronym you like.

What I suspect is that all of us have basically the same idea in mind, and what we're disagreeing over is mostly semantics and context.

Btw, all of my unit tests include memory usage validation. It's easy in c and c++, but I'm not sure how it would be done in a web development environment.

</comment></comment></comment><comment><USER9> To me Unit Testing is more about testing the expected outcome and making sure of the status quo rather then finding bugs and testing all the extreme small edge cases. Then when an abnormal and unexpected situation occurs, writing a test that induces it and fixing the code so that all the related tests pass.

To use the doctor analogy, I would say unit testing is more about testing patient outcomes when they are put through varying courses of drugs or treatments (fixing bugs and adding new features) rather then trying to make them die before declaring them healthy. Is the patients blood pressure normal? Check. Now the patient wants to start some treatment for something else. Oh wait, I'm getting an abnormal blood pressure reading.

</comment><comment><USER10> I didn't appreciate unit testing until I started doing development test-first in Smalltalk. You just sketch out the skeleton of your classes with instance variables and accessors. Then you write tests the way you want your objects to work (design your API by pretending to use it). Then run your tests, and marvel as the debugger stops on missing methods and lets you create them at that point and continue the test. It's a terrific way to get your code right the first time.

I don't know what he's using there at Justin.tv, but most languages that are more rigid (than Smalltalk or Ruby for example) are certainly not quite as nice for unit testing.

<comment><USER4> Sounds great! But in the meantime, I've put a couple of prototypes online, got a bunch of feedback, iterated, and figured out that _nobody wanted the thing we designed in the first place_. I've moved on to an evolution of the original idea, while you're still focused on getting the first idea to work "perfectly".

My point is that unit tests aren't cost-free, and their ROI is awful for most things.

<comment><USER11> Unit tests don't take that long. If it takes you that much longer to write your unit tests, you must be doing something fundamentally wrong - <USER2> perhaps testing the wrong thing or focusing on the wrong kind of testing.

<comment><USER12> It is hard to make inline unit tests when the inputs and outputs are really complicated, like the request/reply data structures in a web framework.

<comment><USER11> Totally agree, that's why you need good MVC separation and you test those layers separately.

The models testing is obvious. What you're referring to there is controller testing. In Rails/Rspec, the way you do that is by mocking/stubbing the models (so you're only testing the controller logic). Then you pass in the parameters that would come in from the web, and check that the controller code (which should be fairly straightforward) behaves appropriately.

This article explains this approach quite clearly: [http://www.patmaddox.com/blog/2007/9/15/easy-controller- test...](http://www.patmaddox.com/blog/2007/9/15/easy-controller-tests-and- expressing-intent-through-expectations)

</comment></comment></comment></comment></comment><comment><USER13> Whenever someone comments on the usefulness of unit testing (or in the extreme, testing in general), I point them to Feynman's contribution to the Challenger shuttle report:

Appendix F - Personal Observations on Reliability of Shuttle <http://www.ralentz.com/old/space/feynman-report.html>

A good summary is at [http://duartes.org/gustavo/blog/post/richard-feynman- challen...](http://duartes.org/gustavo/blog/post/richard-feynman-challenger- disaster-software-engineering)

<comment><AUTHOR> How many people are writing mission critical software? In startups, hardly any.

<comment><USER13> If you rely on your software to be up and running for your livelihood (aka any web startup), it is mission critical.

<comment><AUTHOR> I disagree completely. A failure might mean you loose a bit of ad revenue, or maybe a customer.

The consequences of a bug are usually very small. It's also very obvious if a bug is causing you pain.

</comment></comment></comment></comment><comment><USER14> For me the primary use of unit testing is in the prototype stage. I write the tests instead of the application code to make sure everything is sane. I find lots of architectural/assumption problems this way.

After you've got real app code the tests serve more or less as regression tests to make sure I haven't broken anything major deep in the guts of my models and such.

In short, like some of the other comments here: Unit tests aren't for finding bugs, they're for helping you not write the bugs in the first place.

</comment><comment><USER15> It all depends on how your development works.

Let's look at a scenario where you are using an outside framework.

Not writing tests for your code? Framework v2.0 comes out and you want to use a great new feature in that version. Will your app work if you upgrade? Sure you'll have to test/QA your app regardless, but with tests you might have a better idea of where to look for problems.

This is what I'm facing at the moment as a single developer. The app has no test coverage so updating is a laborious process of testing everything by hand.

But getting into testing has it's own problems. The biggest I have with testing is when I read discussions about bugs within the test framework. How much time will you spend working around bugs in the testing framework?

</comment><comment><USER16> The article has a rather narrow perspective on unit testing, given his product is not mission critical. He's basically treating live production as a giant integration test.

Some of my perspectives:

class/function level unit testing is only needed for common library code.

component level unit testing is critical if you want to do any significant refactoring.

non trivial integration tests (especially timing sensitive multithreaded code) can find many subtle bugs that would be too late to find in production.

I've written a few tests (for a database engine) that paid themselves many times over.

<comment><USER4> _The article has a rather narrow perspective on unit testing, given his product is not mission critical._

I do have a habit of assuming every other hacker is doing some kind of a startup, so this is a fair point.

</comment></comment><comment><USER17> I'd say that it's also important once the team gets beyond a certain size; codebase and team size tend to scale together, of course, but not always. The value of tests in catching regressions in code you didn't write, didn't realize was there, and don't understand is immense. It's one kind of difficult to not introduce regressions in 100k lines of code that you were deeply involved with; it's a whole different deal to walk into 100k lines of someone else's code and try to change it without breaking anything.

Unit testing is like vaccination: it's good for _me_ that I'm vaccinated against smallpox, but it's also really good for _you_ that I won't be getting sick and giving it to you, and your ability to avoid vaccination (or testing) and still function is often directly correlated to how well everyone else is vaccinated (or how well they've tested their code so you know when you're breaking it).

Those don't always have to be _unit_ tests in the strictest sense, so you could perhaps argue that a test that breaks as a result of unrelated changes isn't really a "unit" test because it wasn't properly isolated, but I really hate those sorts of language arguments. I'll call something a "unit" test if it's not an "end-to-end" test, even if I didn't mock or stub out every possible dependency. And it certainly sounds like the author of this post is arguing against unit tests in that sense rather than in the stricter sense.

</comment><comment><USER18> It's a money make. We needed some python work done. It was a small project 42 hours 6 hours was unit testing. Unit testing increased the invoice by almost 17%. I can see why consultants like it.

<comment><USER11> I know many people who do unit testing (myself included), some of them consultants. Not a single one of them does it "to buff up the invoice". In fact, every single one of them will argue (sincerely) that good unit testing practices help _reduce_ the time they need to deliver code and increase the quality of that code. They'd also argue that good unit testing practices make it easier to then hand over the code to someone else.

</comment></comment><comment><USER19> This is another case of overgeneralizing based on limited experience. Automated unit testing may well be of limited value for building the applications that run a video sharing web site (where failures are no big deal). But now try building a complex library used as a core component of several packaged applications. The library developer doesn't really have a "program" to put through user testing. The only solution to keeping the library stable while refactoring and adding new features is to build a comprehensive suite of unit tests that exercise the library's public API.

</comment><comment><USER20> If your unit testing isn't useful, then find a way to make it useful. Author misses the point completely.

Not unit testing is like building a car door and not check that it fits until all parts are to be assembled, and then reviewing what's wrong. Dumb.

</comment><comment><USER8> When (or should I say if) JustinTV grows much bigger and hires a lot more programmers this whole unit tests are not useful thing will come back to bite them.

But I can see how right now it seems their time is better spent on other things.

<comment><USER4> Out of interest, are you speaking from personal experience?

<comment><USER8> Yep, I got TDD religion when I started working for a huge company.

I did some testing, but not much when I worked for a startup.

<comment><AUTHOR> For a huge company, I'd say TDD/pair programming etc is also to ensure _some_ productivity from bad programmers.

If one bad programmer writes the unit tests, and another bad programmer writes the code, then you might end up with a mediocre outcome, instead of a bad outcome. Same with pair programming.

Likewise, if you get a _good_ programmer to write the unit tests, then give the programming work to a terrible programmer, given enough time, he'll write a working solution.

<comment><USER8> I'd have to say that two bad programmers are not any better then one bad programmer even with TDD.

But TDD benefits even the best programmers when they are dealing with a huge, ugly, old code base.

When on the other hand you're user testing every day or multiple times a day, like you might with a web startup, then TDD may only slow you down.

<comment><USER4> _When on the other hand you're user testing every day or multiple times a day, like you might with a web startup, then TDD may only slow you down._

These days I'm at a point where I can usually push 4 or 5 new releases into production every day. It's pretty awesome :-)

</comment></comment></comment></comment></comment></comment><comment><USER21> Really it's like that bridge in 'Apocalypse Now'. The sergeant is telling Martin Sheen how the bridge is always bombed and they have to come in and re- build it, "just so the generals can say the bridge is open." Well, I have done a lot of unit testing (CPPUnit), and it is like that. We do unit testing so management can say the product has been unit tested. Coals to Newcastle, to be sure, but hey, it's a job!</comment>


SpaceX Hid a Second, Hidden Payload Aboard Falcon Heavy - <AUTHOR> https://www.sciencealert.com/spacex-hid-a-second-hidden-payload-aboard-falcon-heavy-and-it-sounds-amazing <comment><USER2> They didn't 'hide' anything, this was publicly revealed when the launch happened in numerous articles.

</comment><comment><USER3> 360 terabytes potential storage and they just sent 3 books? It seems such a wasted opportunity. They could have loaded it up with _everything_.

Arch reported on this when it happened: [https://medium.com/arch-mission- foundation/arch-mission-foun...](https://medium.com/arch-mission- foundation/arch-mission-foundation-announces-our-payload-on-spacex-falcon- heavy-c4c9908d5dd1)

<comment><USER4> _Theoretically_ 360 terabytes potential storage, so they could _theoretically_ have loaded it up with a bunch of stuff but in _reality_ they have only ever made them ~3 MB in size specifically for these PR pieces.

<comment><USER5> Without having dived into the specifics, I imagine the hard part is writing the data into the crystal (high detail 3D laser etching?), and the crystal is just a high-purity medium.

</comment></comment></comment><comment><USER6> if we can make these devices and they can store 360 tb then a high priority for something like nasa should be to put all the digitized books we have currently available in outer space on maybe 100 or 1000 such devices.</comment>


Google flagged our site as malware on our prelaunch announcement day - <AUTHOR> http://wp-abtesting.com/worst-timing-google-flagged-our-site-as-malware-on-our-prelaunch-announcement/ <comment><USER2> The domain that was blacklisted was not demos.shapingrain.com (I checked), shapingrain.com itself was blacklisted as you can see from the Google Safe Browsing report here: [http://safebrowsing.clients.google.com/safebrowsing/diagnost...](http://safebrowsing.clients.google.com/safebrowsing/diagnostic?site=shapingrain.com)

The theme currently in use on [http://wp-abtesting.com/](http://wp- abtesting.com/) has a main stylesheet called style.css which contains the URL [http://www.shapingrain.com](http://www.shapingrain.com) in its comments in the header.

It looks like shapingrain.com itself was infected on 2013-08-19 but cleaned by 2013-08-20. It was likely infected with a JavaScript injection vulnerability linking to the site lartedio.com which served the actual payload (likely something trying to self-install, break out of the box, etc.).

After shapingrain.com was infected and flagged by Google Safe Browsing, wp- abtesting.com would then have been flagged when Google analyzed the CSS file and saw what appeared to be a resource link to an infected site. This would appear to be a limitation via the scanner which is scanning CSS comments and treating them as valid code, though this is not without precedent and certain browsers will evaluate what is contained in comments under certain circumstances (see IE conditional comments).

So, in the end, it looks like shapingrain.com was infected yesterday and Google blacklisted that site as well as any sites pulling resources from the infected site, erring on the side of caution (possibly) and interpreting URLs within comments in CSS as possible resource links.

<comment><AUTHOR> Hi John,

Thanks a lot for your detailed response. Really appreciated.

Probably both sites were infected at some point. Again, I never saw the malware message myself but the guy that alerted us first copied the message he got and it was explicitly mentioning the demo site: "Content from demos.shapingrain.com, a known malware distributor, has been inserted into this web page. Visiting this page now is very likely to infect your Mac with malware."

And now, we're going to immediately clean the CSS since this is something that had not occurred to us could be the cause of the problem. Let's make sure we are not blacklisted again!

<comment><USER3> You shouldn't pull static assets from 3rd party sites like that. Looks like you've fixed it. You should also use W3 TotalCache, WP Minify, or a similar plugin to minify and combine your CSS and JS. You have a lot of files loading which slows the site down.

<comment><USER2> They apparently weren't pulling static assets from a 3rd party site. They were using a theme developed by a 3rd party. That theme is hosted on their own server but has the developer's name and URL listed in the comments of the CSS (which is fairly common practice). No assets were being loaded from an external site, but it looks like that URL in the comments triggered Google's alerts. That said, combining and minimizing CSS and JS would have eliminated the comments and prevented the issue with Google's scanner.

</comment></comment></comment></comment><comment><USER4> I can't be the only one here that think Google has way too much power for anyone's good.

This being said, I have no idea what could be done, if anything, to avoid being in a situation where Google can, mistakenly, blacklist your from the Internet.

This is why I stopped using Chrome, and learned to love Firefox, again.

<comment><USER5> We could require that Google, Microsoft, Mozilla et al use only third-party open blacklists (and contribute to those projects with their blacklist data if they feel they know more than the project does, but where the project has final say in what does or doesn't go in.) This decouples the incentive structure.

<comment><USER6> My experience is that third party blacklists (especially for e-mail spam) have a much worse false positive problem. As I said in another comment, economics is working against you. There is relatively little or no cost to a blacklist provider if there is a false positive. And there are a lot of people who care very much about spam, or malware, who put a lot of pressure on blacklist providers to be as comprehensive and to react as quickly as positive.

Heck, if you are looking at this from a larger social good perspective, it might even be better for society at large to have the blacklist provider be much more aggressive about blacklisting sites quickly. What's the cost of a malware compromising someone's machine, and requiring someone to take their desktop off-line for a day or more while they reinstall everything from scratch (and find out that they no longer can find their MS Office reinstall disks, and the new MS Office requires them to relearn where all controls are on the reorganized toolbar)? Versus the economic cost of some minor web site getting blocked for a few days?

In any case, given that you as the minor web site won't be providing any payments to the blacklist provider, why do you assume the incentive structure will be any better with third-party blacklists?

<comment><USER5> I think the interesting thing about decoupling the incentive structure isn't so much how the blacklist operator would react (more liberal blacklists), but how the _browser maker_ would react.

Since the browser maker will no longer control the blacklist, they will now have users telling them that sites are broken (because they've been blacklisted), and they won't be able to do anything about it on the blacklist side. So, what they _will_ be incentivized to do, is to make _whitelisting_ a blacklisted site (especially those that only get loaded through invisible iframes etc.) have a much simpler/easier/clearer UX, so that their complaints go down. This is good for everyone, but it's not something they'll do when they still have the option "just remove X from the blacklist."

<comment><USER6> But the sites aren't really _broken_. There will be a warning displayed to the user, but the user can always say, "give me the site anyway". And users do hate it when they hit a web site which trashes their machine with malware. So they might be in favor of more stringent blacklists as well.

It might be a bad assumption that users will demand a more liberal blacklist. That's certainly not how e-mail blacklists have worked out. Sometimes the people most in favor of the blacklists that hit all sorts of innocent mail senders are the users sick and tired of spam.

<comment><USER7> Have you seen the "warning" lately? It's a pretty dire message. The only button visible is "Go Back." You have to click a tiny "advanced" text link to even see that there's an option to go to the site anyway. Try for yourself in Chrome:

malware.testing.google.test/testing/malware/

</comment></comment></comment></comment></comment></comment><comment><USER8> We had the same thing happen to us yesterday (our launch day) by Facebook.

We launched our application, announced it to the world, got a flood of users that was apparently "abnormal" and were flagged by their ban bot.

Our app was summarily deleted (without warning or notification). All links to the application were flagged as "abusive". And all data published by users of our application was deleted.

The only reason we discovered this was because we were alerted by our users that the application had disappeared from their bookmarks and that they were unable to access it.

Of course, when we brought this to Facebook's attention they restored the application within a couple of hours. Unfortunately, significant damage had already been done: everything published by our users was (apparently) permanently deleted, our open graph stories and actions were completely deleted, and our subscriptions to the payments apis were deleted.

The particularly insidious one is the deletion of the payments subscription. For the last 12 hours, anyone who has tried to make an in-app purchase has been charged by Facebook but not had their purchase relayed to us for fulfillment.

<comment><AUTHOR> Reading your experience I think I should be happy about what happened to us today. Could have been much worse :-)

</comment></comment><comment><USER9> When I saw the headline, I immediately thought, "they must be using WordPress". WP is a giant exploitable target, and I've personally told Matt this. Automattic saw an opportunity long ago and started VaultPress for WP security. He argued it's not a WP problem, and I frankly disagree, but he obviously understands the situation better than anyone. WP is free, but security is not because self hosted WP is so exploitable.

A launch for a client also went through the same problem in 2010, and that was after 5 years of managing other WP installs (including 2 VIP WP sites). I've seen it happen too many times for it not to be Automattic's problem to address more so than they're doing now.

Stay away from self hosted WP unless your install is absolutely _bullet proof_ , and cross linking, especially to resource files from other WP sites is the last thing you should ever do because you do not control their security which can directly affect yours, or at least your black list vulnerability due to associated content.

Our office used to be above Automattic's in SF, and I love those guys, and what Matt has done for the web, but with great power comes with great responsibility.

<comment><USER10> I'm a great promoter of Drupal, including in those cases where it competes with WordPress. However in all honesty I can't really play the security card against WP. I think WP itself meets generally accepted security standards; the 3d party code loaded into it sometimes does not, but that's the same case with any framework that allows modules.

WP probably gets a bit of a bad rap because the types of sites made with it often don't have the budget to bring high quality development. When you serve 20% of the web, and people choose you precisely because they can get cheap developers, there will be some problem sites out there running WP.

In this particular case, it seems to me that they would have been flagged if they had been running anything, Drupal or Jekyll or a static site - they had an external theme provider who referred to a domain in CSS comments that was listed by google.

The problem seems to be the accuracy of Google's flagging, not WordPress.

</comment></comment><comment><USER11> I think the situation is unfortunate because of their new launch, but the real problem is, as the author mentioned it, that Google webmaster tools did not alert them of the problem, so they were not able to address it on time. I also think their vendor should have alerted them (their clients) about Google flagging them and that perhaps this should be considered immediately, which would have saved them some headaches. Therefore, if the two previous problems has been taken care of, then I think we would mostly all agree it is good to be alerted of potential malware, which noone wants on their machines.

</comment><comment><USER12> Happened to me as well, developed a website for a client and they started to get that malware warning on a specific page.

there was absolutely nothing special about that page except the content, I noticed that the content included words like "Visa, passport, license etc .." so Google classified it as a probable scam.

after a few submissions, I think the page got whitelisted.

its very annoying indeed, especially that this functionality is now built into the primary browser.

</comment><comment><USER13> I feel like I should share a story here. The fight against spam is becoming a serious problem for legitimate businesses.

I've suffered from the same malware issue earlier. Since we use ad networks for advertising it works as follows. 1 ad network has about 3000 different ads running in different locations. If any of those domains get compromised and blacklisted and Google notices that you served something from that domain - BOOM! You're on stopbadware.org and need to get your website reviewed. You block the ad, and apply for a review. To their credit, it takes about 24 hours for the whole process (review happens, they delist you, Google caches update, etc.). However, for 24 hours anyone who comes to your site or clicks on a google result for you, or clicks anywhere to come to you - Gets a massive warning. The cost in terms of lost revenue and reputation damage are almost incalculable. This has happened to me in the past, several times. I have severed relationships with several ad networks and yet this keeps happening - even with the most reputable networks and you try to stay ahead of the curve but if you fall behind even a bit, you may get blacklisted again. All this for a site that makes maybe $200/month.

Another problem I faced was with SURBL. It's a spam blacklist that works on a bizarre system. Basically if they find your domain name beind spammed around the internet (it could be anyone else doing it) they will blacklist you. What's worse is that providers like bitly, facebook, etc. use the SURBL blacklist. So what happens ? Well, one day someone goes and spams your domain on internet forums, etc. SURBL picks up on it. Then, suddenly all your facebook links, bitly links shared on twitter, etc. start showing warning pages. Basically someone clicks on your link on facebook and gets a page saying "This site may harm your computer". Ditto for Bitly as well.

I tried to get delisted but no one at SURBL would respond. I kept trying to get in touch through their online form but no one responded for 2-3 weeks. Finally, I did a whois on the domain, found the admin contact and emailed him. I also sent him a text on his phone. At last after about 4 weeks of being on SURBL I managed to get delisted.

That. Was. An. Ordeal.

This in my opinion is absolutely unacceptable. Spam blacklists do have a responsibility to be correct in their assessments. And if they do have a false positive for any reason, they should have a streamlined resolution process. Unfortunately, the internet is the wild west and shooting before asking questions appears to be quite acceptable in these parts. I recognize that a lot of this is an attempt to protect users but when I open my mom's PC, I still see a bunch of browser toolbars, bookmarking widgets, etc. etc. (malware, right ?) Folks are still getting phished. This fight needs to be rethought.

<comment><USER14> I'm on the other side of this debate. Two years ago I was innocently using Chrome to read reddit when I suddenly got a virus in my computer that took me four hours to clean up from. Turns out it came from an ad on reddit. With things like ad networks distributing malware, Google and others _should_ shoot first and ask questions later.

[http://www.reddit.com/r/announcements/comments/e7988/a_numbe...](http://www.reddit.com/r/announcements/comments/e7988/a_number_of_reddit_users_have_reported_finding/)

<comment><USER15> The unfortunate thing is that more often than not some ad networks aren't punished but rather the downstream ad-space seller.

It comes down to browsers half of the time having bad security principles. Anything can change window.location in an any iframe (on any domain) and the new sandboxing for iframes is HORRIBLE (not to mention the support of sandboxing itself is barely there). Ad networks work by passing the user around between networks until an ad is found but this removes any chance of being able to find who is responsible for a given bad ad.

The entire thing is a security nightmare and the only "fix" Google can provide consists of blacklisting downstream websites that may indirectly link to a bad ad via ad networks. I don't disagree with Google's "fix" but I do think they should approach the problem on Chrome and show what good security means when it comes to ads for the other browser vendors.

<comment><USER16> It's complicated. Modern frame navigation evolved a few years ago out of this work: [http://www.adambarth.com/papers/2009/barth-jackson- mitchell-...](http://www.adambarth.com/papers/2009/barth-jackson-mitchell- cacm.pdf)

Those changes represented a pretty dramatic improvement and all modern browsers are now in alignment. But the resulting behavior does make numerous concessions for compatibility with existing content. Take your complaint about any child frame being able to navigate top. That behavior was retained for both content compatibility and because it was a necessary security measure.

To explain the security concern, standardized frame communication predates the broad adoption of X-Frame-Options. So, the only reliable, cross-browser click- jacking defense at the time was frame-busting via top-navigation. And because we're talking about the Web, you have to consider a transition path of many years for existing content.

It may not seem like a great explanation, but the fact is that most of the confusing security behaviors on the Web really are the result of being boxed in by the need to keep existing content working. And, unfortunately, it takes many years to get improved mechanisms widely supported among browsers and to then migrate the majority of the Web onto them.

<comment><USER15> I appreciate the complexity of the subject a lot. I understand we can't just break iframes. The frustrating thing is that sandboxed iframes left out one essential option: allow-plugins. Yes, allow-plugins would be a security concern and it goes against sandboxing but so does allow-top-navigation. We can go back and forth forever on that subject but the fact is that ads use flash and limiting the use of iframe sandboxing sucks (which makes me say it is horrible).

Fundamentally, something better should exist and websites get the flak of bad iframe security. Annoyingly, any site with an iframe on it can't determine what locations are being accessed within the iframe to be able to accurately let there be a "report ad" feature that grabs the sites involved. Sometimes ad network iframes are nested 3-4 deep (disgusting but true) and it is impossible to read the location (yet these iframes can call window.top.location with ease).

I don't know what needs to exist but trying to work against these bad actors is nearly impossible for web devs (yet the burden is placed on the site).

Thanks for taking the time to comment - I understand it's very complex but I feel like this important subject hasn't seen a lot of real world use (we've decided against using sandboxing because it doesn't work for ads).

<comment><USER16> I appreciate your frustration, but I think you may be misunderstanding the security impact of browsers allowing top navigation. Either way, an exploit can still be launched from within the nested browsing context (be it a plugin or HTML frame). So, the risk here is that some of the content (no matter how deeply nested) is being served from the same location as known malware. And the risk is the same regardless of whether or not the nested context can navigate top.

<comment><USER15> Given that most successful malware infected via a good browser (like Chrome) is obtained due by downloading (rather than an exploit) I'd say allowing top navigation is a pretty big concern. But if there is an exploit that targets a specific browser there is very little that anyone can do in advance if it is being served on an ad network.

So, if as a normal user (someone that doesn't understand malware versus OS- level dialog) sees a big warning in their trusted browser (or a popup) without having clicked on anything at all they might be compelled to download some malware. So yes, getting sandboxed iframes right would have been amazing but in my mind it was a failure.

<comment><USER15> Just to clarify "sees a big warning" \- I had meant the fake warnings that a lot of malware display: "1000 INFECTIONS FOUND, DOWNLOAD THIS NOW!!!"

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> Is it possible that false positives like this open Google to defamation lawsuits? They're making an assertion that is false (and that their webmaster tools show they know to be false), and broadcasting that in a way that definitely has a financial impact on the businesses involved.

<comment><AUTHOR> Good point. I don't have the knowledge to answer this but what is true beyond any doubt is that a false statement like this causes financial damage to the site

</comment></comment><comment><AUTHOR> And if you still see the malware warning yet, well, join me in our appreaciation to Google efforts for protecting internet users against very evil business like mine :-(

(I guess they think they only ones that follow the "Don't be evil" motto)

</comment><comment><USER18> The domain name looks like that of a spam website. You should avoid hyphens when possible.

<comment><AUTHOR> A slightly related problem is that we cannot use WordPress as part of the domain name which would probably make easier to distinguish ourselves (and all other WordPress related products) from other kinds of sites

<comment><USER19> Also (slightly OT):

* Link to homepage is on the right-side!?

* Somehow, content text feels too thin/dull. Little bit difficult to read :)

<comment><AUTHOR> Thanks for the input. We´ll look into it (the link to the homepage is just because the menu starts from the right and now there is only one menu item)</comment></comment></comment></comment>


Ask HN: Do you have an eye for UI? Help us select our App new UI. - <AUTHOR>

http://bit.ly/xYxxTY Click the thumbs to see before and 3 new themes. Then vote. Any feedback appreciated. <comment><USER2> Nice UI designs. I really like the Win 7 theme. What development environment?

<comment><AUTHOR> We are developing in Adobe Flex.

</comment></comment><comment><AUTHOR> <http://bit.ly/xYxxTY></comment>


Clojure-conj: Composable abstractions - <AUTHOR> http://tech.puredanger.com/2010/10/25/composable-abstractions/ <comment><USER2> Hi, I was lucky enough to be attend both days of the event and thought the talks about NOT using macros were some of the most interesting.

As a total Clojure newbie with minimal Lisp chops, I liked hearing the "good vs bad" stories from Christophe Grand and Stuart Sierra with their tangles with macro-ology.

One of the aspects of these talks that really resonated with me boils down to something I read in On Lisp: "By default we should use functions: it is inelegant to use a macro where a function would do. We should use macros only where they bring us some specific advantage."

<comment><AUTHOR> Exactly. Macros are the icing, not the cake.</comment></comment>


The Oatmeal fights back, snaps photo of cash, sends money to charity - <AUTHOR> http://arstechnica.com/tech-policy/2012/07/the-oatmeal-fights-backs-snaps-photo-of-cash-sends-money-to-charity/ <comment><USER2> Carreon's wife sounds even crazier than Carreon himself, apparently saying that there is a conspiracy of actual grammar Nazis killing Americans and that the Oatmeal is part of it.

You can't make this stuff up, folks.

<comment><USER3> Right? Reading the second response by his wife was particularly disturbing.

"...Just look at their cartoons. They like to kill. They think it's fun to kill. They think their friends will admire them if they kill. Without mercy!

...They are stupid, silent people, who are absolutely worthless to our society. Really, what good is their life, but a burden to the planet?"

She is accusing "these people" of being murderers? Honestly, from the following comments, I'm quite a bit more concerned that this whole story will end with her going on a killing spree.

I'm slowly becoming more and more convinced that this is one majorly elaborate practical joke, concocted by the oatmeal and funnyjunk to raise tons of money for charity. Because I'd rather believe that than believe that people this dense actually exist.

<comment><USER4> As I recall, the same thing happened with a lawyer named Jack Thompson. I think it's just something about Law as a discipline that it attracts people who can be very literal-minded and who at least in their own mind are moral crusaders. (Disclaimer: I don't know whether Carreon's wife actually does anything legal herself or whether she just married a lawyer.)

I think when you mix that sort of mentality with modern entertainment, you get quotes like these. Sure, we say, "any normal person" understands the difference between cartoons and real life. Perhaps Mrs. Carreon would even agree that _she_ understands the difference. But she is somehow not convinced that _we_ understand the difference. Jack Thomson was the same way; he referred to games as "murder simulators" and assumed that anyone attracted to them was merely 'in the closet' about actions that they secretly intensely wanted to do. Since indulging a normal addiction usually makes it worse and not better, it's easy from this screwed-up perspective to imagine that The Youth will fail to be satisfied one day with the latest violent media -- and will turn to violence itself in their craving.

There is another aspect here which is: the things that disturb us about others are usually things which disturb us about ourselves. It's about resonance. Normally if someone is being stupid people say "to hell with him, he's stupid." But let us all endeavor, when we find ourselves outraged, to ask not only "what is wrong with THEM?!" but also "why does this bug me so much? What resonates here?" If we apply that to Mrs. Carreon, I think we'll all see that we've all been on the wrong side of a dispute, that we've all been wronged before and wanted to "get back," etc.

<comment><USER5> _I think it's just something about Law as a discipline that it attracts people who can be very literal-minded and who at least in their own mind are moral crusaders._

Since this whole case is, um, horribly embarrassing to my profession, I'll just respond here and move on. The huge majority of lawyers I know are much more like the EFF lawyers here than like Carreon. They're smart, creative, resilient, flexible, and more often than not they're honest and respectful. Some of them do it for moral reasons and some are just doing a job; some of them make the world a lot better in my opinion and some of them end up screwing it up. But there's not one lawyer I know who I can imagine acting anything like Carreon, let alone his wife. I mean, even the few jerks would at least make more effective choices.

In short: don't think of this guy as a typical lawyer. He's not. Think of him as a typical guy-screwing-up-his-career-with-a-weird,-paranoid-flameout,-in- an-unusually-public-way-thanks-to-the-internet. It's just too bad he happens to have a career that will let him waste a lot of other people's time in the process ...

<comment><USER4> Oh! I'm sorry, it occurs to me that my comment can indeed be read that way, and that's not what I intended.

I mean it as "they attract these people", not "they are mostly these people:" the tiny percentage of society which is A is much more likely to be L, but that does not mean that most L's are A. So for example I would guess that most programmers are pretty sociable people, but we have a nasty reputation as highly antisocial because we attract people who prefer programming to partying. The Catholic church can tell you all about their difficulties with this type of thing.

<comment><USER5> Hey, thanks for the clarification. That's a totally fair point. I was probably a little over-eager to reply here since this story so much invites the lawyers-are-evil thing and I've been looking for a chance to put it in context. Like, "yes OK lawyers are evil sometimes but this isn't that." :)

After thinking more about it, the literal-minded moral crusade is something I've definitely seen in the legal system, not from lawyers but from pro se parties -- non-lawyers who are representing themselves. There's a group of people who are absolutely sure that the world has wronged them and that they'll eventually be able to make everyone understand, but are unable to parse or accept the reality checks they're getting back from the system, so they go through a series of lawyers before striking out on their own. I bet any clerk's office you walked into, they could name a handful of people like that who they recognize by sight when they come in to file their next complaint (and then a complaint against the judge who handled the last complaint, and so on). They're sad situations.

I don't really know anything about Carreon or the rest of this mess, but assuming that's what's happening, maybe there's some observer bias here -- the reason we're hearing about this at all is that he _is_ a lawyer, so he's able to navigate the system well enough to cause real trouble, at least for a while.

</comment></comment></comment></comment></comment></comment><comment><USER6> It is really hard to understand how Carreon and his wife don't realise that they are digging themselves into their own hole. They even went out and bought the shovels themselves. Getting involved in a battle which isn't yours and then stretching it this far is beyond belief.

It's as if they have never heard of the internet before, and expect _everyone_ to side with them. It's pathological ignorance.

<comment><USER7> Shooting off your mouth about an ongoing case is just asking for trouble. All of this could be brought into the courtroom, with Tara being an associate of Charles. It wouldn't be the first time a judge has ruled on a case partially due to professional behavior outside the courtroom.

<comment><USER8> On the evening of June 14th, Carreon told Forbes that he didn't know what was wrong but he'd find _something_. "California code is just so long, but there's something in there about this." That evening around 9pm he donated $10, and then filed his suit the very next day claiming he had been misled.

<comment><USER9> Sadly, he's probably right. Many estimates suggest that the average American is unknowingly committing multiple felonies every day due to vague criminal statutes.

<comment><USER10> Try getting a definitive list of state or federal laws. No such list for the Feds and most states.

<comment><USER11> I have the list:

      1) Don't do anything an agent of the government doesn't want you to do

      2) Don't do anything a future agent of the government won't want you to do

And they say the legal system is complex!

</comment></comment></comment></comment></comment></comment><comment><USER12> I didn't even realize until this article that Carreon had actually donated to the campaign, apparently expressly for the purpose of then claiming that Inman was going to use the funds in a different way than promised? Except I have no idea how he can possibly argue that taking the picture of the funds is somehow wrong, when that was kind of the main objective of the fundraising (with the secondary objective being to then distribute the funds to the 2 charities).

<comment><USER13> > different way than promised

Once the originally-planned $20k had been exceeded, Inman suggested that he would find some more charities that might benefit from the cash. This thought was abandoned shortly after Carreon cried foul about the "different way than promised". So, good-good for the bears, they'll get a bigger money pit to roll around in.

<comment><USER12> I'm not sure what bearing that has on taking a picture of the cash. Nor do I see how this could be related to the current lawsuit if Inman abandoned that idea.

</comment></comment></comment><comment><USER14> "One man's vulgarity is another's lyric." -Inman's lawyer

This is going into my quotes file.

<comment><USER15> Inman's lawyer is quoting Justice Harlan in Cohen v. California:

<http://en.wikipedia.org/wiki/Cohen_v._California>

</comment></comment><comment><USER16> This is starting to feel like one of those fake hip-hop artists fights (recognizing that The Oatmeal is clearly in the right imho).

I wonder if FunnyJunk is now just continuing this for the publicity.

<comment><USER17> It left FJs hands a while ago now, this is all on Carreon. Unsurprisingly FJ haven't mentioned anything for a while.

<comment><USER8> Given his previous attempt to drum up public outrage over Mattel's failure to take him up on his offer to represent them over another trademark dispute, there's a good chance that this whole thing was entirely Carreon's idea. "Hey, you should get a trademark, and then let me file a contingency suit against that guy who is more successful than you."

</comment></comment></comment><comment><USER18> Wait so where are the photos?

<comment><USER19> Seems like the Ars headline is inaccurate. Inman posts in his blog that the money is still with IndieGoGo pending a restraining order to have it transferred.

"Once the money is moved, I still plan on withdrawing $211k in cash and taking a photo to send to Charles Carreon and FunnyJunk, along with the drawing of Funnyjunk's mother."

<http://theoatmeal.com/blog/fundraiser_update>

<comment><USER8> Yesterday, IndieGoGo filed documents, revealing that

1\. the money donated by PayPal was _already_ in Inman's hands.

2\. the money donated via credit card was already sent to the charities.

Oddly, Inman's blog isn't as up-to-date as the court filings. Inman's filing is here: [https://www.eff.org/sites/default/files/filenode/OatmealOppT...](https://www.eff.org/sites/default/files/filenode/OatmealOppTRO.pdf)

<comment><USER20> It could be that Inman was unsure what he could legally say, and didn't want to make things worse. He'll get his 15 minutes and rushing things won't make any difference.

</comment></comment></comment></comment><comment><USER21> Let me quote Humphrey Bogart for you: "The only reason to have money is to tell any SOB in the world to go to hell."

</comment><comment><USER22> Am I the only one who thinks Carreon wants to be the new Jack Thompson?

<comment><USER23> That's exactly what I was thinking. How long before they disbar this clown for abusing the legal system with frivolous, vindictive lawsuits?

</comment></comment><comment><USER24> Tara and Charles are most clearly mentally ill. At least they have each other.

</comment><comment><USER25> It's rather difficult for most of us to accept criticism but at some point most sane people would try to take a step back and conclude that if thousands upon thousands of people think you are behaving poorly it might be prudent to at least _consider_ their viewpoint is valid.

</comment><comment><USER26> don't wanna be a 9gag supporter but even 9gag gives some source/credit

</comment><comment><USER27> About that hate speech...

      Tara Carreon responded:

      "There is now plenty of proof that Matt Inman is one
      of a gang of people who promote the same type of ideas
      that inspired Jared Loughner to try and kill Gabrielle
      Giffords," she wrote. "Mark Potok of the Southern Poverty
      Law Center, who studies hate groups and hate speech,
      examined Loughner's sites and concluded that his material
      on grammar, in particular, likely came from the writings of
      far right activist David Wynn Miller. If you Google Matt
      Inman + grammar, you will find a similar obsession. And
      similar hate."

Citing SPLC suggests Tara Carreon is left leaning. But her misunderstanding / misuse of "hate speech" is very right wing.

Kind of like how trogs don't grok that Colbert is making fun of them.

Hate speech is wishing harm on your opponents for political reasons. Calling people names is not hate speech. Neither is making fun of people. That's just being a dick.

Alas, wingers don't get that distinction. I suspect the root cause (mental failing) also explains trogs rampant false equivalency (e.g. Clinton got a blow job == Bush allowing 9/11, illegally invading a foreign country, killing 100,000s of its citizens, making refugees out of a few million, and using depleted uranium to convert much of the formerly inhabited areas into Superfund sites).

I once had a winger tell me that the Rodney King riots in LA were an example of left wing hate speech. The stupid, it hurts. I honestly can't decide if trogs say ridiculous non sequiturs because they're thick or if it's merely to distract and enrage (a la Limbaugh, Beck, Coulter). I've decided to simply judge people by their actions, vs their intent.

The grandma having sex with a bear is obviously funny. To you and me. But the trogs can't see it that way. Any mocking is a direct personal attack demanding a defensive response.

Inman didn't know the Carreon's were mentally impaired. But for future, just know that trogs will misunderstand, misinterpret, twist, assume the worst, distort, etc. No one can control the audience's reaction. But in my mocking, I do try to avoid borderline humor which can be labeled "hate speech". No sense giving the trogs another chew toy. And there's no shortage of funny stuff to say.

<comment><USER28> You come close to smearing roughly half of _all_ people with the comical stupidity of the worst within that half. All of what you say seems true, but even describing reality along those lines (even without your epithets) arouses passion+stupidity.

<comment><USER27> Just half?

I had to read Tara Carreon's quote for myself. The "hate speech" reference jumped out. No one else had commented on it. Probably because few here are as political as me, so wouldn't know that it's a politically charged phrase.

Oh, I forgot to mention something.

Carreon's equating Inman to Jared Loughner is just sick. As in reprehensible. No different than invoking Hitler (Godwin's Law). And is a pretty good example of hateful (vs hate) speech.</comment></comment></comment>


Italo Calvino, The Art of Fiction No. 130 (1992) - <AUTHOR> https://www.theparisreview.org/interviews/2027/italo-calvino-the-art-of-fiction-no-130-italo-calvino <comment><USER2> One of my favorite passages of prose is from Calvino's "Invisible Cities"; I can't pass up the opportunity to share it on HN:

"When a man rides a long time through wild regions he feels the desire for a city. Finally he comes to Isidora, a city where the buildings have spiral staircases encrusted with spiral seashells, where perfect telescopes and violins are made, where the foreigner hesitating between two women always encounters a third, where cockfights degenerate into bloody brawls among the bettors. He was thinking of all these things when he desired a city. Isidora, therefore, is the city of his dreams: with one difference. The dreamed-of city contained him as a young man; he arrives at Isidora in his old age. In the square there is the wall where the old men sit and watch the young go by; he is seated in a row with them. Desires are already memories."

<comment><USER3> Invisible cities was too visual for me (I have a weak visual imagination, so I can't appreciate books that rely too heavily on images), but I really liked If On A Winter's Night a Traveler, and this quote reminds me that I want to read more Calvino.

Would you (or anyone else here) recommend another book of his, given those preferences/weaknesses of mine?

<comment><USER4> Cosmicomics has long been my single favorite collection of written work. Calvino's amazing.

<comment><USER5> Yes that's the best I think, and very suitable for HN audience

</comment></comment></comment></comment><comment><USER6> A friend lent me a copy of Calvino's "If on a winter's night a traveler" which I would strongly recommend if you're looking for something a bit different. It's written in the second person (the protagonist is you, the reader) and is alarmingly postmodern.

<comment><USER3> I think it's even enjoyable by people who would be put off by the phrase 'alarmingly postmodern'.

It's playful and weird and meta, sure, but it's not _difficult_ (or purely an excuse for Calvino to show off), and there is sincerity not far below the surface.

<comment><USER6> Perhaps I chose my words poorly; I very much enjoyed reading it, and would strongly recommend it to anyone.

<comment><USER3> Sorry, I didn't mean that as a criticism, and your message was pretty clear. I just thought that some people (possibly me, if I hadn't already read the book) might be unnecessarily put off by that final phrase.

</comment></comment></comment></comment><comment><USER7> calvino was a fascinating person and a powerful author. it's a shame he isn't more recognized in the mainstream of western literature.

he strikes me as the kind of person who is both mysterious and familiar at the same time -- someone who is wise and accessible, yet avuncular and rare to interact with. and, of course, he's deeply italian.

<comment><USER8> In general Italian Fiction, I think receives much less attention than French or Russian, at least till the end of last century, it is not easy to find translations of the Italian greats like Manzoni, Tasso, even Petrarch these days.

</comment></comment><comment><USER9> [http://archive.is/wHKO2](http://archive.is/wHKO2)

</comment><comment><USER10> For those not interested in a tremendous amount of meaningless piffle about the article author, skip twenty paragraphs in to get the actual interview with Calvino.</comment>


Using JAX, numpy, and optimization techniques to improve separable image filters - <AUTHOR> https://bartwronski.com/2020/03/15/using-jax-numpy-and-optimization-techniques-to-improve-separable-image-filters/ <comment><USER2> Is it fair to say JAX is what Google made when they looked at Pytorch/Autograd and thought, "oh damn, that's what we should have done?".

If so, is this the beginning of the end of Tensorflow? I know Tensorflow is still top for production, but it is certainly rapidly losing followings in the research field, and Pytorch and now starting to focus on deployment as they know this is their weakness.

<comment><USER3> It sounds like JAX is necessarily storing your whole calculation in memory, so it will necessarily use more memory for automatic differentiation of heavily iterative calculations, while other implementations of backward-mode automatic differentiation can instead restart your calculation from checkpoints to avoid storing the whole thing. This could be an advantage of several orders of magnitude for some calculations: using twice the CPU or GPU time in exchange for one thousandth or one ten-thousandth of the memory.

<comment><USER4> Rematerialization in autodiff is super interesting! XLA does rematerialization optimizations, so you get those automatically under jax.jit. There's also the jax.checkpoint decorator ([https://github.com/google/jax/pull/1749](https://github.com/google/jax/pull/1749)) which lets you control reverse-mode checkpointing yourself; you can use it recursively to implement sophisticated checkpointing strategies (see Example 5 in that PR, which is the classic strategy for getting memory cost to scale like log(N) for iteration count N but requiring log(N) times as much computational work). It'd be interesting to experiment with heuristics for deploying those strategies automatically (e.g. given a program in JAX's jaxpr IR) but one of JAX's core philosophies is to keep things explicit and give users control through composable APIs. Automatic heuristics can be built on top.

Another goal is to make JAX a great system for playing with things like this!

<comment><USER3> Thank you for the correction! I should have checked out the software before posting my incorrect surmises from the blog post. It sounds awesome!

<comment><USER4> No worries! I didn't mean it as a correction so much as just a discussion; I'm sure it's true that other autodiff systems have very sophisticated automatic remat (like [https://openreview.net/forum?id=BkYYXJ9i-](https://openreview.net/forum?id=BkYYXJ9i-)). I'm hoping as users push JAX on new applications, especially in simulation and scientific computing, we'll learn a lot and be able to improve!

There's also "cross-country optimization" ([https://www- sop.inria.fr/tropics/slides/EdfCea05.pdf](https://www- sop.inria.fr/tropics/slides/EdfCea05.pdf)) for mixing some forward-mode into reverse-mode to improve memory efficiency. Analogously to jax.checkpoint, we've only experimented with exposing that manually (in jax.jarrett, named because of [https://arxiv.org/abs/1810.08297](https://arxiv.org/abs/1810.08297)), and even then only for a special case. There's a lot to learn about, experiment with, and build!

</comment></comment></comment></comment></comment><comment><USER3> This is a very interesting article, with much better illustrations and deeper investigation than the note on separable image filters I wrote in Dercuano. And I didn't know about JAX, and it's very valuable to know about it now. But the article does have some errors.

The article says:

> _Optimization of arbitrary functions is generally a NP-hard problem (there > are no solutions other than exploring every possible value, which is > impossible in the case of continuous functions)_

It is true that optimization of arbitrary functions, or even many interesting classes of functions, is NP-hard. However, the definition given of NP-hard is incorrect, and in fact, on modern hardware, existing SMT solvers such as Z3 can solve substantial instances of many interesting NP-hard optimization problems, precisely because they do _not_ explore every possible value. Moreover, it is in general possible (but again NP-hard) to use interval arithmetic to rigorously optimize functions on continuous domains (which seems to be what is meant), as long as they are not too _dis_ continuous; the answer you get is only an approximation of the true optimum, but you can calculate it to any desired precision.

One particularly interesting class of optimization problems — because they are _not_ NP-hard — are continuous _linear_ optimization problems, which can be solved in guaranteed polynomial time using interior-point methods or usually in polynomial time using the "simplex method". Contrary to what you'd think from the quote from the article, going from continuous to discrete makes the problem NP-hard again. There is also a note in Dercuano surveying the landscape of existing software and methods for solving linear optimization problems; there's a lot of very powerful stuff out there.

It turns out that you can efficiently solve an enormous range of practical optimization problems by introducing a small number of discrete variables into a linear optimization problem, thus gaining most of the performance benefit of using a linear optimizer. I don't know if there's a way to get a reasonable perceptual result in a case like this with a linear optimizer, though.

> _This is where various auto-differentiation libraries can help us. Given > some function, we can compute its gradient / derivative with regards to some > variables completely automatically! This can be achieved either > symbolically, or in some cases even numerically if closed-form gradient > would be impossible to compute._

Automatic differentiation is a specific approach to differentiation which is an _alternative_ to symbolic differentiation and the older kind of numerical differentiation. What JAX does is automatic differentiation.</comment>


I've Noticed a Major Flaw with Viral Coming Soon Pages - <AUTHOR> http://takara.posterous.com/a-major-flaw-with-viral-coming-soon-pages-83204

<comment><USER2> I certainly think less of companies that require me to tweet about their product before I've used it, just so I can jump up higher in the queue.

On top of that, I've had it happen twice that I did what I was supposed to, and I did -not- get access.

So I've really, really soured on the whole thing. At this point, I only pay attention to people who have an actual product that I can actually use. Every else gets ignored.

<comment><AUTHOR> I wish I could take that approach... I'm a beta launch addict.

</comment></comment><comment><USER3> I've also never liked those coming soon pages. They usually give no information, only a vague promise and signup box. Example of that is LaunchRock. My sense it doesn't work, if anyone has any statistics ... I'd guess that less than 2-3% subscribed share their link to get to the priority list.

My suggestion for new startups is to put an information-rich page with screenshots and explanations how that idea came to life, how it will change your life, etc - of course if your goal is to get people interested and to signup :)

<comment><AUTHOR> I agree. I'd totally do-over my coming soon page now. Lesson learned.

</comment></comment><comment><USER4> This is a great post with great comments as well. Currently, we are using launchrock as well and we've stagnated at 36 sign ups as we are trying to get to 100. We haven't spent a penny on marketing but I feel we have hit a wall. I know that the soft launch is supposed to test and validate our business idea but I feel it isn't doing it justice, not by far. Especially with the limited amount of words you have to use. On the otherhand, I also understand the overall goal, it's to test without spending too much on programmers and developers and build something that no one wants to use. We are conducting surveys as well, which I feel are a better means of gauging if you have the right audience and validating your assumptions. However, I'm torn. I can see both sides of the argument. We started the soft launch march and it's about 5 months and running. They say the idea validation phase typically is about 5 to 7 months. But I feel strongly that even though we have stagnated in signups it doesn't mean that our startup idea is going to flop. In fact I feel very strongly in it's success. Half because entrepreneurs are just optimistic and half because the data proves it on our surveys. Any suggestions? Thanks!

<comment><USER5> Well your first round of 36 is your early adopter group which you survey, and prioritize problems/solutions. You then iterate your product with what you just learned. The feedback loop. Infact, Ries and Ash say you could do it with just 5 ppl. So you should be testing a new landing page or product by now, not waiting for another 64 ppl to 'signup'.

</comment></comment><comment><USER6> Totally agree with you.

To be honest, Launchrock are just being ridiculous at this stage. They actually seem to be going backwards. I signed up for their beta over a year ago, still nothing except several "coming soon emails".

</comment><comment><USER7> If you don't want to share with your friends, then don't.

But if you are a startup founder, remember the reason you see these viral sign up forms everywhere is because they work. You'd be silly not to offer the opportunity w/ your visitors because a small percentage will indeed share with their friends. Of course, that percentage is going down as people get fatigued or get burned by other startups that don't follow through.

</comment><comment><USER8> > If there's a long line of people waiting to get in already. A long visible > line coming out of the door, makes more people want to know about it and in > some cases stand in line for it.

I think this is the main point, if you have a way to let people know how long the queue is and where is your position in the queue, it might actually work.

<comment><USER9> I doubt a simple number will be effective, because you lose the human element. The physical presence turns a waiting list into a social event. Maybe if you could somehow interact with the people already in line.

<comment><USER10> So, a mini-forum on the page?

</comment></comment></comment><comment><USER11> The more interesting hidden nugget in this post was the analogy to Krispy Kreme. Krispy Kreme primes their line up with free donuts and a visual display of how they make the donuts.

The lesson for startups is that put a teaser on the sign-up form so people know what they are getting excited about.

</comment><comment><USER12> TL;DR: "I waited in a long line for a donut. Startups!"

</comment><comment><USER13> Key quote: "Why would I invite my friends to sign-up, when I haven't even tested it out first?"

</comment><comment><USER14> Thanks for the article. I do agree that these viral launch pages may only work if there's a special set of conditions met. I am actually right now writing a comprehensive article about it (I hope it will be at least). So, thanks for the opinion.

<comment><AUTHOR> Sure, send me a link when you're finished. I'd love to hear more about this.

<comment><USER15> My idea is beyond this, but the basic gist is to create several stages. 'Alpha' 'Beta' 'Live' '10' '#'. So if a person is very interested or wants the finished product, etc, then there's a variety of matching options.

However it'd really depends on what the offer is. This is an indirect suggestion to do some marketing 101 ;)

<comment><USER14> I recently blogged an interesting approach by an app (called livepath) which let people buy into a beta account. Whenever the beta was a bit further developped he dropped the pricing a bit. Here's the link: [http://blog.thegodfounder.com/post/5894377639/lifepaths- sign...](http://blog.thegodfounder.com/post/5894377639/lifepaths-signup-page- has-an-interesting-take-on)

<comment><USER15> Minecraft creator took the inverse, charge more as software entered alpha to beta to gold. I suppose the high initial cost until normal cost makes sense in a reverse dutch auction if you got the hype.

<comment><USER14> Oh yeah, did not know that. Thanks for the info.

</comment></comment></comment></comment></comment></comment><comment><USER16> this is a terrific idea. if you don't mind, we may revamp our own landing pages to reflect this. hope this works for you!

</comment><comment><USER17> So here's one theory I believe: Facebook and other large social media companies today will evolve into universal login systems eventually. We've already seen it with "Connect with Facebook" or "Connect with Twitter" buttons, but I believe this will become more and more prominent as the year progresses.

Why? Like the article states, people don't like giving out their email address. Why reinvent the login interface millions of times when you can just use one. Google, Facebook, Twitter (etc etc) already provide those interfaces, so use them!! Hell, I've turned away from betas just because their sign up form is too annoying.

<comment><USER15> What is it about sign up forms that are too annoying? Just the email to put in?

<comment><USER18> Pressing a button feels easy. I do not know if this is a big difference; may be. Time will show.</comment></comment></comment>


China squandered its chance to build a home-grown semiconductor industry - <AUTHOR> https://www.scmp.com/tech/big-tech/article/3024687/how-china-still-paying-price-squandering-its-chance-build-home-grown <comment><USER2> Yeah lets see again in 2025~2030, that is when China plans to be the top of these sectors with the Chinese Made in China 2025 plan.

Semis are already feeling Huawei presence in the world, huawei designs its own soc and some of its components. The bigger Huawei market share the smaller amount of US components need to be exported.

</comment><comment><USER3> I do not think China ever prioritized home grown semis, since its viewed as "plumbing" for more lucrative things built on top of chips ( software, etc).

Why focus on it when South Korea, Taiwan was already doing it ?

As the article states, it's a 500 billion dollar industry.

Compared to energy, defense it's much less important ( but critical ).

Now that the US is threatening to shut them off, they are being forced to invest more heavily.

I saw a graph the other day, within 2030 or something, china was poised to hold a monopsony position in semis, something like 80% of global semi consumption is going to come from China.

It's just surprising that the US would kick their biggest semiconductor customer in the head.

<comment><USER4> Is that consumption measured by units or total value? I'm sure China could compete for cheap smartphone chips and microcontrollers but it would take them a while to make something like a Xeon.

<comment><USER5> You are making assumption, that they will stick to x86 architecture. What if they don't want this old thing with all the microcode and hacks inside? What if they start on clean page with RISC-V or alike and software for it? Or China will just buy company building competitive chips: [https://www.tomshardware.com/news/qualcomm-server-chip- exit-...](https://www.tomshardware.com/news/qualcomm-server-chip-exit-china- centriq-2400,38223.html)

<comment><USER4> I just meant a high-quality vectorized server CPU by referring to the Xeon, didn't mean to imply any architecture

<comment><USER6> Posted a few months ago. Admittedly it's not a Xeon but these things are becoming commodities.

> Ara runs at 1.2 GHz in the typical corner (TT/0.80 V/25 oC), achieving a > performance up to 34 DP-GFLOPS. In terms of energy efficiency, Ara achieves > up to 67 DP-GFLOPS/W under the same conditions, which is 56% higher than > similar vector processors found in literature.

[https://news.ycombinator.com/item?id=20126280](https://news.ycombinator.com/item?id=20126280)

</comment></comment></comment></comment></comment><comment><USER7> A quick note --just because China is not a major player in the semiconductor industry today, doesn't the CCP is not placing its state resources (both state financial backing [1] and the industrial espionage kind [2][3][4][5]) to try and incubate a semiconductor industry

[1][https://outline.com/j35sNG](https://outline.com/j35sNG)

[2][https://www.scmp.com/news/china/diplomacy/article/3005738/ch...](https://www.scmp.com/news/china/diplomacy/article/3005738/chinese- spies-stole-dutch-chip-machinery-giants-secrets)

[3][https://www.wired.com/story/us-accuses-chinese-stealing- micr...](https://www.wired.com/story/us-accuses-chinese-stealing-micron-trade- secrets/)

[4][https://www.justice.gov/opa/pr/prc-state-owned-company- taiwa...](https://www.justice.gov/opa/pr/prc-state-owned-company-taiwan- company-and-three-individuals-charged-economic-espionage)

[5][https://www.wsj.com/articles/taiwans-technology-secrets- come...](https://www.wsj.com/articles/taiwans-technology-secrets-come-under- assault-from-china-1530468440)

What I've read about made in 2025 in the past few years, the motto of CCP can be distilled down to:

1) Whatever CCP can't build, it will try to buy

2) Whatever CCP can't buy, it will try to steal

<comment><USER8> > just because China is not a major player in the semiconductor industry > today, doesn't the CCP is not placing its state resources (both state > financial backing [1] and the industrial espionage kind [2][3][4][5]) to try > and incubate a semiconductor industry

Did anyone ever think that?

</comment></comment><comment><USER9> China's major problem is that it's not the USA. The USA has lots of capital to throw around, it can shift manufacturing to other countries. The USA is also the leader in style, culture, and innovation - China just plays catchup. With a centralized government, you'll always be playing catchup.

China may led the world in cheap electronic components, but there is no reason that India couldn't do the same if the USA trade war with China keeps escalating. Italy also has many small workshop that can crank out small precision parts. Sooner or latter Eastern Europe will get back into the game.

<comment><USER10> > China just plays catchup. With a centralized government, you'll always be > playing catchup.

Japan/S Korea/Taiwan beg to differ. Those are under democratic governance, but 'centralized' never the less. Japan has lion share of Semiconductor equipment, while South Korea and Taiwan are leading the most advanced foundry business.

<comment><USER11> Centralized in what sense? Those are all free market economies. On Heritage Foundation's economic freedom index, Taiwan ranks in the top 10, ahead of the US, while South Korea and Japan rank around Germany or Norway, and ahead of Spain, Italy, or France: [https://www.heritage.org/index/ranking](https://www.heritage.org/index/ranking).

</comment></comment></comment><comment><USER12> Where does ARM fit into this picture - it's clearly hugely significant, but doesn't feature in any of the tables shown...

Also it's noticeable that Japan doesn't feature, why's that and why are the Japanese cool about that?

<comment><USER13> The Japanese (Softbank) own ARM.

</comment></comment><comment><USER14> It's not like it's too late, especially with the US pursuing 19th century solutions to 21st century problems. The level of understanding possessed by the US presidential candidates for the 2020 elections is a little worrying. For starters, we have a number of folks who want to break up big tech without a corresponding plan to break up the oligarchies held by the big telcos.

The article alludes to a major Chinese shortcoming: that it thought it could own semiconductor manufacturing by owning the equipment. A lot of US nationalism is centered on how the US leads in terms of culture. It would be precarious to assume that we should really be so advantaged, or more broadly, _to pin your success on the failure of your competitor._

<comment><USER15> > It's not like it's too late, especially with the US pursuing 19th century > solutions to 21st century problems.

Of course not. It just requires China actually do the research for the tech instead of stealing it.

> The level of understanding possessed by the US presidential candidates for > the 2020 elections is a little worrying. For starters, we have a number of > folks who want to break up big tech without a corresponding plan to break up > the oligarchies held by the big telcos.

Another set of 19th century solutions.

> The article alludes to a major Chinese shortcoming: that it thought it could > own semiconductor manufacturing by owning the equipment.

IP theft isn't the same as IP research.

> A lot of US nationalism is centered on how the US leads in terms of culture. > It would be precarious to assume that we should really be so advantaged, or > more broadly, to pin your success on the failure of your competitor.

It's more than just culture. We have a set of law that protects our research efforts. In China, what protection is there from competition or even the government?</comment></comment>