Somali Pirate: I made $2.4M From Ransoms in 2010 - <AUTHOR> http://www.foxbusiness.com/markets/2011/03/11/somali-pirate-24m-ransoms-2010/

<comment><USER3> I never guessed there would be pirate VC, I suppose money follows any profit.

</comment><comment><USER4> This link sparked one hell of a conversation in #startups @ freenode. We mainly talked about prevention, and it turns out that Xe (formerly Blackwater) has tried a Honey Pot tactic. Anyway, the guys in the channel basically came t the conclusion that there aren't strong enough economic incentives for mercenaries to deal with the pirates.

<comment><USER5> Blackwater/mercenaries don't go into the situations where they can really be hurt. One thing is to shoot unarmed civilians and the other - pirates whose "berserk" state is fueled by millions of dollars and the fear/risk brain center is dumped by khat (google khat)

</comment></comment><comment><USER6> He should enjoy the money while he's still breathing, which won't last as long as he thinks it will.

<comment><USER7> I doubt that's true, as he didn't participate in most of the hijackings. If he keeps that up, at worst he'll lose money, and the only risk is getting at the wrong end of a gun in a dispute over money.

<comment><USER6> Some countries have special forces operating in Somalia hunting these guys down. Parading and bragging about how much money he's earned makes him a target.

</comment></comment></comment><comment><USER5> the guy participated only in one hijacking, in all the other cases he was an investor.

<comment><USER8> So he's an angel?

</comment></comment><comment><USER9>
        "The South Korean and Russian rescue operations did not affect us,
        but ... we must take revenge," Yare said. "We shall be killing Russian
        and South Korean crew until their navies stop attacks against us."

Anyone want to bet on how long it will be between the time they kill a Russian or South Korean crew, and the world gets serious about stopping the pirates?

<comment><USER5> i wouldn't be optimistic here. 3 weeks ago there were killed 4 Americans. Nobody would care about crew of 8-10 Russians or Koreans ... even Russians and, i would venture to guess, Koreans as well.

Stopping such a massive piracy (i.e. decimating it to rare occurrences) is easy. But like with simple plastic credit cards vs. complicated microchip, etc... - the current losses is less than the cost of changing the situation.

In case of piracy there is also additional issue of separation of who is paying or to pay - piracy losses are carried by private business, where is any action against the piracy would involve weapons - thus it is to be carried and paid by the government(s).

</comment></comment><comment><USER10> Constitutional method to fix this is "letters of marque and reprisal" - <USER2> used against the Barbary Pirates.

</comment><comment><USER11> when I first read the headline I thought it was going to be a "How-To" blog for all you would-be pirates

</comment><comment><USER12> how long before copycat operations start?

</comment><comment><USER13> With all this pirate VC money going around, I sense a pirate bubble.

<comment><USER5> "bubble" - when an investor willingly drops stable 300% profitable business (his drug trade in this case, which in Somali is as conservative business as Safeway here) to invest into that new opportunity.</comment></comment>


Ask YC: Anyone in SF have a spare couch this weekend for a 2-guy "startup"? - <AUTHOR>

A friend and I are pitching in mountain view this weekend (13th to 17th) and are looking for like-minded people in the SF area that have a spare couch or floor.  Hotels seem to be expensive and don't provide the hacker atmosphere we're looking for.   We're going to be busy practicing our demo, so we will try to stay out of your hair.  We'd gladly compensate you for any expenses.
We're both male and in our early 20s.  We're also not crazy, if that matters.
I'm posting this here because there's probably several other startups in town this weekend that might be interested in the same thing.
My email: sam -@- odio.cøm <comment><USER2> I thought that was a crazy weird post... Then I realized it was from somebody I know! I hung out with Sam at Startup School 2007, and he did not rape me. And his college friends were cool and regular posters here, so I assume the same again. If I was in California, I would let Sam stay over, why not you? Thanks.

We even stopped by Adam Smith's place, where Drew Houston was also... and look where they are now with Xobni and DropBox!

Five stars, recommended.

(Sam didn't ask me to post this, I just saw this.)

By the way, what type of startup is this? Good luck!

<comment><USER3> > He did not rape me.

That's the most glowing endorsement if I ever did see one.

<comment><USER2> I wrote it because it made me laugh, every single time I read my testimonial.

</comment></comment></comment><comment><AUTHOR> Thanks for everyone that sent me an email or posted here. I'm replying to the emails & making arrangements with my 'partner' (we're strictly professional, of course).

Also thanks for the YCers who vouched for us and/or promised we wouldn't rape you. That was pleasantly unexpected.

Anyone else who's looking for a place to stay should post as well, btw. I don't want us to be the only ones to benefit from this thread.

</comment><comment><USER4> 2 guys, 1 couch?

<comment><USER5> San Francisco

</comment></comment><comment><USER6> We're interviewing too, I'm located in San Jose (close to the San Jose airport, if you're flying in there). I have three extra couches and even more floor space... would be pleased to put you up (or anyone else coming in for the weekend)! firstname.lastname@example.com

</comment><comment><USER7> Here you go: <http://sfbay.craigslist.org/pen/fuo/914889246.html> :-p

</comment><comment><USER8> Sam ... just left you a vm message, I think I can help you out. I have a couch AND an air mattress and I'm just a block away from the MUNI.

I owe ya from your great hospitality back during the Wamily days, anyway. Give me a call. ~Nate

</comment><comment><USER9> Yes, I have a place, email me at firstname.lastname@example.com.

</comment><comment><USER10> Try <http://www.airbedandbreakfast.com/>

I think the founders of the site have a room themselves...for $50 per night.

<comment><USER11> yes we encourage you to use the site. My listing is the first one in SF.

</comment></comment><comment><USER12> I just wanted to add that I've never been raped by these two startup founders either.

</comment><comment><USER13> Hey I'd love to have you but I actually don't have a couch. In fact I don't have any furniture except a desk and mattress. Good luck!

</comment><comment><USER14> Correction: you are a little crazy if you're a startup founder.

Good luck ;-)

</comment><comment><USER15> You guys could've crashed at my place, but I'm moving. Bummer, good luck! In the future, my couch will be open.

Btw, is this a big week for pitches. i've heard a few other friends are doing the the same thing.

</comment><comment><USER16> you should try couchsurfering.com i have friends that it has worked well for them. there are also some entrepreneurial clubs on there.

</comment><comment><USER17> Good Luck!</comment>


COVFEFE Act would make social media a presidential record - <AUTHOR> http://thehill.com/policy/cybersecurity/337416-covefe-act-would-make-social-media-a-presidential-record <comment><USER2> > "President Trump's frequent, unfiltered use of his personal Twitter account > as a means of official communication is unprecedented. If the President is > going to take to social media to make sudden public policy proclamations, we > must ensure that these statements are documented and preserved for future > reference. Tweets are powerful, and the President must be held accountable > for every post,"

Archiving tweets != holding people accountable

There were countless screenshots on archived websites across the web of the tweet immediately after it went out.

This is merely political theater to create more layers of control with little real ROI of the intended problem. What matters most is the _feeling_ of controlling something which you don't control. The type of policy politicians love!

I'm sure if this passes it will result in tens of millions of dollars in consulting fees to what largely could be solved with a server or two hitting the twitter API to capture each tweet and stored securely. Or a single person hired to document tweets manually in a database.

I'm curious what exactly Trump isn't being held 'accountable' for here because every major US and international news organization documents each of his tweets with whole articles - and clearly including ones he deletes.

I'm sure they could legislate processes to eliminate mistaken tweets and the risks of human error here, but lets be honest it's not going to make Trump stop making bold statements at will - which is what they really have a problem with regarding his Twitter. He does the same on every platform he uses directly. Are they going to screen his calls to journalists and other people he talks to as well? Or his aides who leak to journalists version of his day- to-day interactions without the usual PR filters? If that's not the target goal then what is really being accomplished here? Eliminating typos and draft tweets from being accidentally sent out?

<comment><USER3> Hilariously, inside the article it is clearly stated, by somebody who speaks for the institution which would be responsible for this archiving, that the Tweets are already considered Presidential Records.

> _In January, National Archives spokesperson Miriam Kleiman told the > Associated Press that social media posts would qualify as presidential > records, but that statement is not explicitly spelled out in the law._

So not only is the private sector adequately serving the purpose of archiving the president's Tweets, but the National Archives probably are as well.

I really hope that the people who voted for Mike Quigley are embarrassed that this is the pathetic, childish level that an Illinois 5 representative operates at.

</comment></comment><comment><USER4> Ugh, this is a real issue (twitter, or other "social" platforms) are all presidential communication platforms -- legislation shouldn't (i know, i know, ideal world, old man shouts at sky, etc) be special cased. It should be "all communication on any public platform should be archived, no matter the nature or location of the platform". Naming it the COVFEFE act trivializes what is a real issue (presidents should not have the ability to retroactively remove records of public statements).

</comment><comment><USER5> Dumb. It's not like deleting tweets makes them go away. If this is what Democrats are bringing to the game it's gonna be a long eight years.

</comment><comment><USER6> To be honest, this sounds like a terrible idea, but then again something really _should_ be done about the level of obnoxious nonsense on display.

<comment><USER2> > but then again something really should be done about the level of obnoxious > nonsense on display.

If only people asked whether something could be meaningfully accomplished before demanding something be done then we'd have far less bad laws.</comment></comment>


What Are Visual OKRs - <AUTHOR> https://www.quora.com/unanswered/What-are-Visual-OKRs?share=1 <comment><USER2> I honestly don't know why we need a new name for dashboards.</comment>


Low-cost Deterministic C++ Exceptions for Embedded Systems (2019) [pdf] - <AUTHOR> https://www.research.ed.ac.uk/portal/files/78829292/low_cost_deterministic_C_exceptions_for_embedded_systems.pdf <comment><USER3> So. The proposed mechanism is:

1\. Every function now receives an additional parameter that is basically a pointer to a std::optional<exception_t> (from now on, "exception holder"; exception_t can hold any value whatsoever).

2\. "try" is translated into allocating a (new) exception holder on the stack and passing pointer to it in all function calls inside of the try block. "catch" is translated into checking this exception holder for holding an (appropriate) exception. If "yes", handle it, if "no", go on.

3\. Throwing an exception is done by putting the constructed exception into the exception holder passed to you, and returning as usual.

4\. Each function call is now followed by checking whether the exception holder holds an exception. If "yes", return immediately, if "no", keep executing.

Example:

        void C(std::optional<exception_t>* exc_holder) {
            Bar bar;
            exc_holder->emplace(0);
        }

        void B(std::optional<exception_t>* exc_holder) {
            Foo foo;
            C(exc_holder);
            if (*exc_holder) { return; }
        }

        void A(std::optional<exception_t>* exc_holder) {
            std::optional<exception_t> new_exc_holder;
            B(&new_exc_holder);
            if (new_exc_holder)
                if (new_exc_holder.value().is<int&>()) {
                    int& p = new_exc_holder.value().as<int&>();
                    // catch body...
                } else {
                    *exc_holder = std::move(new_exc_holder);
                    return;
                }
            }
        }

So, it's basically "if err != nil { return <default_value>, err }" but with slightly less amount of copying error values. Somehow, it ends up with the same performance despite all of this constant branching error-checking and better performance when re-throwing exceptions.

<comment><USER4> Is that an ABI break due to the extra parameter? I skimmed the paper, but it only seemed to talk about the standard table-based unwind in passing, with no mention of compatibility. Did I miss something?

If it is in fact an ABI break, how much of an obstacle would that be to getting this through the committee? I was also under the impression that up- to-date compilers/runtimes are not something one can take for granted in the embedded world, so recompiling everything is a shaky proposition.

<comment><USER5> Is there an example where changing the exception handling system _does not_ change the ABI?

<comment><USER4> I'm not familiar enough with the darker corners of C++ implementations to say, but I'd guess not. From what the paper describes it seems table-based exception handling is deeply ingrained into the Itanium ABI so any significant change would be an ABI break, which would make said change more difficult to spread.

<comment><USER6> I am not sure if all implementers made the same choices, but for example I believe Microsoft concluded that table-based EH is always better and made not only their IA64 ABI but also their amd64 ABI use it. They keep 32-bit x86 at an older implementation for compatibility reasons, but any ABI introduced later gets the new thing.

<comment><USER4> Wait, is the Itanium ABI not the amd64 ABI? I thought that the 64-bit x86 ABI was called the Itanium ABI despite Itanium having gone kaput.

Did Microsoft release a paper with that conclusion? I'd be curious to read more about their reasoning.

<comment><USER6> Hmm, I missed the C++ism for this term. Googling around:

> Although it was initially developed for the Itanium architecture, it is not > platform-specific and can be layered portably on top of an arbitrary C ABI.

My mistake, my comment is mostly rubbish then.

<comment><USER4> I didn't realize it was a C++-specific thing, to be honest. I should probably try for more generic wording in the future.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> Seems to be better exceptions than what is already there. And fully compliant as far as I can tell. Someone just had to go out there and do it. From doing my own benchmarks on exceptions, I already know you should basically never use them. The bloat and unexplained slowdowns in real programs is just a headache you don't want.

In the end, I still use exceptions, so this gives me hope. I like C++ exceptions as a feature, but I'm not happy about how it's implemented right now.

<comment><USER8> > I like C++ exceptions as a feature

In your opinion, what are situations/examples that justify the use of exceptions?

I haven't used a language with exceptions since 3 years (i.e: since I switched to Go), and I haven't yet faced a situation where I felt a need for them.

Also, I started to learn C++ as a hobby a few months ago, and I have to say that I still don't understand when I should use exceptions or not, things seem fine so far by just returning some error values.

<comment><USER9> Exceptions are ideal for happy path programming. You keep state in the stack and build your software so things get thrown away. For some types of programming this is not that important but for OLTP it is very handy.

[https://ninenines.eu/articles/dont-let-it- crash/](https://ninenines.eu/articles/dont-let-it-crash/)

Basically this approach is handle errors if you know what to do with them, otherwise leave the error for something else.

In erlang, where you have one process per transaction, and nothing handles the error, the process crashes.

In C++, you could just have an exception handler around a transaction, and then when an exception reaches that point you throw away the transaction, but log that it failed, you could have exception handlers below this to try and do rollbacks etc.

This is overly simplified but it gives a basic idea.

<comment><USER8> Thanks for your answer. What is OLTP in that context? Is it [https://en.wikipedia.org/wiki/Online_transaction_processing](https://en.wikipedia.org/wiki/Online_transaction_processing)? First time I see that acronym.

<comment><USER9> OLTP in this context is indeed online transaction processing.

This general approach of «let it "crash"[1]» or happy path programming is what I used when I wrote OTLP systems for mobile networks which were performing financial transactions. The exception handling would allow you to write fairly complex business logic and relying on exception handling to recover to a stable state.

[1]: Crashing in Erlang terms is not at all what people generally mean with crashing. It is a very unfortunate word because in part many of the problems in node.js comes from joyent not understanding what "let it crash" meant - <USER2> did not mean let the unix process crash. Erlang does one process per transaction and they are incredibly lightweight. Some details from joyent can be found here: [https://www.joyent.com/node- js/production/design/errors](https://www.joyent.com/node- js/production/design/errors)

</comment></comment></comment></comment></comment><comment><USER10> This is still kinda bad. What you really want is multiple return addresses. This is liked "double-barelled continuations", but the two continuation closures share the same activation record (and inductively, the same stack)

See [http://www.ccs.neu.edu/home/shivers/papers/mrlc- jfp.pdf](http://www.ccs.neu.edu/home/shivers/papers/mrlc-jfp.pdf)

</comment><comment><USER11> This paper is truly interesting. People have been fighting the performance cost between exceptions and returning error codes for years, So the paper just designs an instruction level mapping of generating code as if exceptions would be returning error codes.

The approach is somewhat simple and really not that clever, yet the outcome looks impressive enough. This surprises me as it's just a mechanism of what people have been arguing over years.

Nicely done!

</comment><comment><USER12> They propose a major language change: all functions would be noexcept by default, and only those explicitly marked with "throws" would be allowed to throw exceptions.

Interesting paper, but it's not gonna fly in practice.

</comment><comment><USER13> This proposal is sort of based on herbceptions [1]:

> Sutter [19] has only very recently proposed re-using the existing function > return mechanism in place of the traditional stack unwinding approaches, > requiring no additional data or instructions to be stored, and little to no > overhead in unwinding the stack. Furthermore, by removing stack unwinding's > runtime reliance on tables encoded in the program binary itself, the issue > of time and spatial determinism is solved. As it is possible to determine > the worst-case execution times for programs not using exceptions, it follows > that exception implementations making use of the same return mechanism must > also be deterministic in stack unwinding.

> Given these clear advantages, we have based our implementation on this > design, with a core difference being the replacement of their use of > registers with function parameters, allowing for much easier > interoperability with C code, which can simply provide the parameter as > necessary.

> A limitation with [19] is that they require all exceptions be of the same > type, leaving much of the standard exception-handling functionality up to > the user. Our novel approach includes a method of throwing and catching > exceptions of arbitrary types (as with existing exception handling), without > imposing any meaningful execution-time penalties when exceptions do not > occur.

At least, it uses a comparable implementation strategy. Herbceptions used a different surface syntax, adding a new kind of exception and a new way to declare them.

I rather like that herbceptions are more restricted in type. Restricting the type means that a programmer catching an exception has more things they can do with it. It's currently impossible to sensibly handle exceptions of unknown type in C++. The change is an improvement, not a limitation.

The point about interoperation with C is interesting. How would that work? If you wanted to call a C++ function from C, would you have to kludge the prototype to add the exception pointer?

The herbceptions paper proposed trying to get the C ABI changed to support the same exception mechanism:

> 4.6.11 Wouldn't it be good to coordinate this ABI extension with C (WG14)?

> Yes, and that is in progress. This paper proposes extending the C++ ABI with > essentially an extended calling convention. Review feedback has pointed out > that we have an even broader opportunity here to do something that helps C > callers of C++ code, and helps C/C++ compatibility, if C were to pursue a > compatible extension. One result would be the ability to throw exceptions > from C++→C→C++ while being type-accurate (the exception's type is preserved) > and correct (C code can respond correctly because it understands it is an > error, even if it may not understand the error's type).

[1] [http://www.open- std.org/jtc1/sc22/wg21/docs/papers/2018/p070...](http://www.open- std.org/jtc1/sc22/wg21/docs/papers/2018/p0709r1.pdf)

</comment><comment><USER14> Interesting, thanks.</comment>


Singapore Govt commits $450m to Public Private Co-innovation Programme - <AUTHOR> http://www.channelnewsasia.com/stories/singaporebusinessnews/view/1095577/1/.html Ideas looking for implementers at their website<p>http://www.coinnovation.gov.sg/co_innovation_projects.html <comment><AUTHOR> Ideas looking for solutions in their official website <http://www.coinnovation.gov.sg/co_innovation_projects.html></comment>


NILFS: A File System to Make SSDs Scream - <AUTHOR> http://www.linux-mag.com/id/7345

<comment><USER2> Note that NILFS makes _old, crappy_ SSDs scream. Also, the article makes NILFS look good by comparing it to old filesystems, not COW-tree filesystems that are the current state of the art. NILFS may have merit, but this article doesn't demonstrate that in an honest way.

<comment><USER3> What's a COW-tree? And which filesystems are COW-tree filesystems?

<comment><USER2> Copy-on-write tree. See WAFL, ZFS, and Btrfs.

<comment><USER3> There's one comparison to ZFS:

 _One of the most noticeable features of NILFS is that it can "continuously and automatically save instantaneous states of the file system without interrupting service". NILFS refers to these as checkpoints. In contrast, other file systems such as ZFS, can provide snapshots but they have to suspend operation to perform the snapshot operation. NILFS doesn't have to do this. The snapshots (checkpoints) are part of the file system design itself._

<comment><USER2> AFAIK ZFS never "suspends operation", whatever that means.

<comment><USER4> Pretty much doesn't as a COW fs does not need to.

</comment></comment></comment></comment></comment></comment><comment><USER5> Pardon my ignorance of SSDs and how they work, but aren't seek times supposed to be low or negligible? Isn't a log-structured FS optimised for a disk with high seek time?

I guess my questions boil down to:

1\. Why is this approach fast on an SSD? Wouldn't the performance boost be more noticeable on a spinning disk?

2\. Shouldn't someone try to make an FS optimised for SSD characteristics? Have someone already done this?

<comment><USER2> _Isn't a log-structured FS optimised for a disk with high seek time?_

Sort of. It makes writes sequential (and thus fast) and reads nearly random (and thus slow).

 _Why is this approach fast on an SSD? Wouldn't the performance boost be more noticeable on a spinning disk?_

Old, crappy SSDs have very slow random writes (which is like seeking), so log structuring increases write performance by orders of magnitude. Since random and sequential reads are equally fast on SSDs, the randomized layout doesn't hurt read performance as it does on disks.

 _Shouldn't someone try to make an FS optimised for SSD characteristics?_

If the characteristics are different for every SSD model, no. Recent SSDs are becoming insensitive to access pattern, so the correct "optimization" is probably to perform no optimizations other than trim.

<comment><USER6> Trim?

<comment><USER2> Trim is a command that punches a hole in the SSD (much like a sparse file) so that the SSD doesn't waste effort managing unallocated space.

</comment></comment></comment></comment><comment><USER7> While a speed boost is great and all, don't you have the added concern of making sure your blocks don't wear out on an SSD? Because log structured file systems commit metadata and data in large sequential writes to the hard drive, they have background processes that constantly move data around to prevent massive fragmentation from files that are later deleted. It seems like there would just be too much writing going on if what you want is a long-lasting SSD.

They do mention in the article that it would be a bad idea to use NILFS on your root partition because of heavy traffic, so lets say I decide to use it on a volume where I store all my media (where I pretty much write once, read a lot, and never delete). Now, a) why would I use an SSD for this given that they're so much more expensive and b) wouldn't I get comparable performance using an HDD here since my files will be laid out pretty much sequentially given my workload? Doesn't the same apply to companies storing mounds of data? Where is the middle ground where I would want to use NILFS on an SSD? One where I a) won't be sending enough write traffic to my drive to wear out my SSD too quickly and b) will be reading a lot of random blocks?

What am I missing? I see the merits in using NILFS, but I just don't see the purpose in comparing its performance against other filesystems on SSD's.

</comment><comment><USER8> Ah. Garbage collection for file systems. With screaming, to boot.

Wasn't some work done on log-structured file systems done by DEC some time ago? I can't find a reference to it.

<comment><USER9> > Wasn't some work done on log-structured file systems done by DEC some time > ago? I can't find a reference to it.

Didn't Ousterhout (who, amongst other things, worked on a log based file system) work there on sabbatical?

<http://en.wikipedia.org/wiki/John_Ousterhout>

<comment><USER10> He's mentioned in the article along with another guy.

> The concept was developed by John Ousterhout of TCL fame and Fred Douglis.

</comment></comment></comment><comment><USER11> How would you implement a swap on such a filesystem? You couldn't, could you?

Oh, and the jokes about having _NILFS_ on your _hard drive_ are going to be so lame...

<comment><USER12> "How would you implement a swap on such a filesystem? You couldn't, could you?"

The days of "swap" are numbered. The future is a smart memory hierarchy that uses knowledge about size/speed tradeoffs to make smart decisions about where to store things, but that won't look much like "swap".

Even with SSDs, it has become very unusual for "the system is currently swapping" to mean anything other than "the system's performance characteristics just became completely unacceptable". I stopped using a swap partition somewhere around 512MB to 1GB, and now my laptop that I didn't even try to max out the memory on shipped with 4GB. I can't hardly even fill up the _cached buffers_ in 4GB in any reasonable period of time.

(Please note the difference between "very unusual" and "never". In fact, I can cite a server in my care that benefits from using a bit of swap. But that server is the _exception_ , not the rule.)

<comment><USER13> We'll still have swap as long as CPUs support virtual memory. Nothing beats using a dumb programming language and process model, and without explicitly deciding anything, being able to scale past physical memory until your working set just barely fits.

Further, there may be some performance benefit to swapping idle process memory to make room for more buffer cache.

It's true that for interactive use (especially a laptop with spun-down drives), you may be happy without swap. But I don't see the facility being mothballed.</comment></comment></comment>


Ask HN: Can we not build a contact tracing app that respects privacy? - <AUTHOR> Every time the medias mention location-tracing apps, they say it's a threat to privacy and personal autonomy.
I am not a security engineer, but my understanding is that it's entirely possible to build an application that's anonymous (see: https://ncase.me/contact-tracing/) Yet, medias don't relay that. Am I missing something? <comment><USER2> Yes it's entirely possible to build an app that is anonymous.

What's missing is the legal framework to prioritise this anonymity (during emergencies like this pandemic), the political will (long shot) and the understanding the general population have about privacy in general.

The last point in also somewhat debatable, since people are scared currently. So, without having a certain level of knowledge about the technology and the implications of lack of privacy, they're more likely to 'barter' their privacy for security against getting infected.

</comment><comment><USER3> To me it seems that the most important reason is that most medias are not interested in informing and/or educating the public, but in attracting readers so they are going to see the advertisements that generate their revenues. You will not see a media proclaim: the tracking our website does, is a greater danger to your privacy than the tracking apps.</comment>


The Giant iPod Touch Theory: I Don't Know if I Buy It, But I Like It - <AUTHOR> http://technologizer.com/2009/03/20/the-giant-ipod-touch-theory/

<comment><USER2> I think what's appealing in this is that we all know that the tablet form factor is a valid one. There are hundreds of thousands of purposes for a lightweight, page sized tablet device that people could do basic computing from.

But at this point no one has managed to "do it right".

Apple is a company that tends to do things right when they actually move into a new market place so the "giant ipod touch" seems like the light at the end of the tunnel. That's why I think people who hear the rumor (myself included) tend to like the idea.</comment>


Google adds machine-learning capabilities to its BigQuery cloud data warehouse - <AUTHOR> https://www.geekwire.com/2018/google-adds-machine-learning-capabilities-bigquery-cloud-data-warehouse/ <comment><AUTHOR> Documentation, for anyone wanting to get started: [https://cloud.google.com/bigquery/docs/bigqueryml- intro](https://cloud.google.com/bigquery/docs/bigqueryml-intro).

(Disclosure: I'm on the BigQuery team)</comment>


Why Almost Everything You Hear About Medicine Is Wrong - <AUTHOR> http://www.newsweek.com/2011/01/23/why-almost-everything-you-hear-about-medicine-is-wrong.html

<comment><USER2> _Negative results sit in a file drawer, or the trial keeps going in hopes the results turn positive._

Therein lies the problem. We know the solution: tweak the publishing process.

Currently: 1) Do study 2) Submit paper, including conclusions. 3) Journal accepts, based on correctness of methodology and interestingness of conclusion.

To prevent this result, just change the order of things.

1) Submit paper explaining methodology, and "alternate endings" (positive and negative). 2) Journal accepts paper based on _methodology only_. Methodology section of paper is published. 3) Authors do study and submit results. Journal publishes appropriate conclusion section. 4) If necessary, authors submit additional observational results.

<comment><USER3> _Negative results sit in a file drawer, or the trial keeps going in hopes the results turn positive._

There is a more unfortunate scenario that often plays out. That is that they get a different statistician in who has a goal of getting positive looking results out of a negative result set though various means and then often carefully manipulating the wording of the success criteria to match their best attempt at making the data work. This is quite common, my partner (in the girlfriend sense) work within medial journal publishing and sees it quite a lot.

<comment><USER2> _...manipulating the wording of the success criteria to match their best attempt at making the data work._

While not impossible, it is very tricky to do this before you actually have the data. I'm advocating that the wording of the test criteria, statistical tests to perform, etc, are all decided _before the trial_.

Bringing in a different statistician post-hoc is pointless, all he could do is run the exact predetermined statistical test. In fact, you can do even better: submit code to the journal before the study, and submit data after the study. The journal runs the code on the data and decides which conclusion to publish.

</comment></comment></comment><comment><USER4> Always relevant, the article from Ioannides - the man who dedicated this entire career on examining the way in which research is done rather than the actual research.

[http://www.plosmedicine.org/article/info:doi/10.1371/journal...](http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124)

#1 in downloads on PLoS for many years.

<comment><USER5> Thomas Kuhn is another one worth reading although more philosophical (none the less - relevant)

"The Structure of Scientific Revolutions"

[http://en.wikipedia.org/wiki/The_Structure_of_Scientific_Rev...](http://en.wikipedia.org/wiki/The_Structure_of_Scientific_Revolutions)

<comment><USER6> And none the less widely misused.

</comment></comment></comment><comment><USER7> For a bit more detail:

The New Yorker

[http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_...](http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer?currentPage=all)

The actual referenced article (PDF)

<http://www.edwardtufte.com/files/Study1.pdf>

<comment><USER8> also this one

[http://www.theatlantic.com/magazine/archive/2010/11/lies- dam...](http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies- and-medical-science/8269/)

</comment></comment><comment><USER9> I've been writing about health and medical topics for a long time and this piece does a good job of discussing how the science can get 'icky.'

First, let me say that in general, modern medicine has done wonders, increasing duration and quality of life in astonishing ways.

That said, the system is set up to promote positive studies/findings, and there are _a lot_ of ways you can play with the data. Very competent statisticians/researchers are capable of helping studies, sometimes, be more positive than they might otherwise be.

That, plus the many millions of dollars you can make from a drug, has often resulted in questionable - but predictable - results.

<comment><USER10> right, less money to be made if people just ate much less and stopped smoking.

</comment></comment><comment><USER11> And there's the deeper trouble with method:

[http://en.wikipedia.org/wiki/Paul_Feyerabend#Nature_of_scien...](http://en.wikipedia.org/wiki/Paul_Feyerabend#Nature_of_scientific_method)

<comment><USER5> Kuhn, Feyrabend and Lakatos all worth reading. Even tough they might be a tad to postmodern for some people.

None the less they are all valuable thinkers.

<comment><USER12> What do you mean by 'postmodern'? Is this a real criticism of what they are saying?

<comment><USER13> Oversimplifying things a bit, these writers are saying that scientific objectivity is impossible. This line of argument tends to reduce the credibility of science in the eyes of the public, with negative consequences for research funding. This is a bad thing because, even if not "objective", science has singlehandedly transformed life for the better in the past century by reducing disease, poverty, manual labor, etc, and should continue to be funded.

<comment><USER5> I don't think any would claim otherwise.

But you don't need objective for that all you need is useful.

<comment><USER14> Except if some people think science is not objective, they conclude that religion is just as valid, and that, therefore, Creationism (er, Creation 'Science') needs to be taught in schools alongside the 'fact' that gays and liberals are evil, diseased beings. After all, if it's all subjective, everything is equally valid!

<comment><USER5> Science isn't objective as such. It's testable.

That is the defining factor between religion and science.

</comment></comment></comment></comment></comment></comment></comment><comment><USER15> The vitamin D claim is misleading. Sure, most people have enough vitamin D for bone health. But vitamin D does a lot more than make your bones healthy, so supplementation is still probably useful. Luckily, unlike vitamin C, too much vitamin D is not harmful so all you're wasting is money if the other claims for vitamin D also get disproved.

<comment><USER16> First of all: it's probably not a good idea to be dispensing (or receiving) medical advice on HN.

That being said-- Vitamin D is fat soluble, and too much vitamin D is _definitely_ harmful. Take a look here:<http://en.wikipedia.org/wiki/Hypervitaminosis_D>

Vitamin C, on the other hand, is water soluble, and much harder to overdose on.

<comment><USER17> The biggest Vitamin D supplements I've seen have 1000 IU per tablet, and the bottle suggests one tablet per day. According to the article you linked, 10,000 IU/day is safe (we can get that much from sunshine in a day), and long- term overdose has been observed at 77,000 IU/day. You'd have to go pretty hog- wild with the Vitamin D supplements to overdose.

<comment><USER18> 2000 and 5000 IU are quite common and can be bought at any US pharmacy and many grocery stores. It looks like 10000 to 50000 IU pills are easily available online: [http://www.amazon.com/s/?url=index%3Dblended&keywords=vi...](http://www.amazon.com/s/?url=index%3Dblended&keywords=vitamin+d3+10000)

For what it's worth, I supplement using the 5000 IU pills taken at about 7/week (irregularly, making up doses as necessary). I recently had my blood tested for D3 levels, and found it was close to but not exceeding the standard recommended level.

</comment></comment></comment></comment><comment><USER19> While this article is correct and talks about Ioannides's work—which is good, respectable stuff by the way—it does paint a picture like this should be a shock to you.

Folks, it isn't just medicine; _Science_ in general must be comfortable with the proposition of being wrong. The more complex the field, the more often replication will fail (and the tighter standards need to be). There are very few pursuits that our race engages in that are more difficult than understanding life itself.

One of the biggest problems is that physicians and the press tend to get very hasty with pushing new research. It's not their fault, mind you, they're trying to inform us about health. If a doctor could save a life with an experimental treatment that has yet to reach thorough replication, shouldn't s/he? And it's the media's _job_ to get us interesting new stories (albeit they could be far better at putting their information into context).

The nature of science is such that refinements occur over time. Unfortunately, the nature of most people receiving medical treatment is to believe it works or doesn't work based of extremely subjective data. If we want to attack that problem head on, we should begin with education. All the improvements in methodology imaginable cannot stop science from being what it is, and as such cannot eliminate the sawtoothed slope of medical progress.

</comment><comment><USER11> Also, Jaynes had a very interesting take on how to address these issues.

Probability Theory: The Logic of Science <http://bayes.wustl.edu/etj/prob/book.pdf> (draft)

[http://www.amazon.com/Probability-Theory-Logic-Science- Vol/d...](http://www.amazon.com/Probability-Theory-Logic-Science- Vol/dp/0521592712)

</comment><comment><USER20> Which just goes to show that you should learn about settled science before worrying about the latest findings.

<http://lesswrong.com/lw/ow/the_beauty_of_settled_science/>

</comment><comment><USER21> A place to hear things about medicine that are more often than not correct:

<http://www.sciencebasedmedicine.org/?p=8198>

<http://www.sciencebasedmedicine.org/?cat=11>

</comment><comment><USER22> One of the problems with drugs studies is that the very point is to develop a drug and that intent is generally commercial in nature. At no point does anyone seem to stop and wonder what might be _the best solution_. In addressing my health issues, I have had to infer a lot of things based on drug studies because there is a dearth of good studies on other topics (at least specific to my medical condition). Yet the general consensus is that diet and lifestyle play a major role in every major deadly medical condition. Encouraging people to live right just doesn't have the splash and dash and isn't generally as a lucrative.

It also gets a lot of resistance from the intended audience: An awful lot of folks don't want to do the hard work of changing their lives. In many cases, they just want a new drug, one that won't make them sick...etc (I'm sure someone can quote song lyrics better than I can).</comment>


Decrypt/Extract NitroKey HSM RSA Private Keys - <AUTHOR> https://raymii.org/s/articles/Decrypt_NitroKey_HSM_or_SmartCard-HSM_private_keys.html <comment><USER2> Author here, do note that device access and access to all previously setup passwords and PINs is required for this to work. It still beats the purpose of the HSM though. The article explains this a bit more in the #Rationale section. Happy to answer questions or comments.

<comment><USER3> From #Rationale (concerning the backup option that most commercial HSMs allow):

>> I do am of the opinion that HSM's should not offer this option (getting access to the private key).

It boils down to your opinion then. Within the industry "HSM" is pretty much synonimous of compliance to FIPS 140, which allows key extraction even at its highest level (Level 4), provided that the risk is mitigated appropriately (E.g. by means of a split secret policy and more in general by means of all the security measures that go _beyond_ the technologic ones provided by the HSM itself).

You are free to keep your opinion but the statement that such possibility "still beats the purpose of the HSM though" is not shared by many people.

Also (again concerning the backups):

>> Since it requires so many steps and so much access, I don't think this is a huge risk, but a rather nice convinience.

No, it is not a convenience. It is a manner to mitigate the risk of denial of service one may cause by purposedly destroying or stealing your HSM.

>> I've seen multiple big-name HSM's where their support was able to decrypt the key and transfer it to another HSM model, but since I've signed an NDA I cannot tell which ones that were.

I challenge this statement: I deal with most big-name HSMs (and there are not that many) and none of them can do this. I do know because that is a risk we explicitly check for when evaluating one.

In certain cases you may manipulate their API to extract some keys if they were generated internally in a certain way, but that is a known, public problem in the industry.

<comment><USER2> > You are free to keep your opinion but the statement that such possibility > "still beats the purpose of the HSM though" is not shared by many people.

I'm aware of the FIPS standard, therefore I worded the piece as my opinion. I think it's better to be able to rotate the keys then to restore to the same/other device. Since extraction might lead to decryption. I do also know that software-wise that is an inconvinience and in some cases not possible. Bur still, a device that protects private keys should not give them up. As you say, my opinion.

> No, it is not a convenience. It is a manner to mitigate the risk of denial > of service one may cause by purposedly destroying or stealing your HSM.

Stealing should be protected by the tamper-protection, thus causing a wipe of the device. In that case you might have a bigger problem, someone has access to your datacenter and racks. In that case a backup is very welcome. (I do find it a hard problem where the two sides, no backup, a backup, both are equally important.

> I challenge this statement: I deal with most big-name HSMs (and there are > not that many) and none of them can do this. I do know because that is a > risk we explicitly check for when evaluating one.

We've had a situation were two different revisions of the same device were not able to load a wrapped key. Support flew in and told us they could fix in in a new firmware in three months, no guarantees. We installed screen recorders and a keylogger, since they were on our production env (audit logging). They did, with undocumented commands, export the key from the device in an unencrypted format and loaded it into the other model so that we could continue our operation. Reviewing the recording and the logs showed that the commands did also work on other HSM's we had. I'm still under NDA so I cannot talk about which brand or model specific.

<comment><USER4> > They did, with undocumented commands, export the key from the device in an > unencrypted format and loaded it into the other model so that we could > continue our operation.

NDA or not, that company should be put out of business. Essentially they've backdoored that HSM.

Assuming it is true this is _far_ bigger news than the article you wrote.

</comment></comment></comment></comment><comment><USER5> Note that this is not a vulnerability, per se - it requires the attacker to already have the keys to the castle (DKEK).

<comment><USER6> The author addresses your comment in the abstract. Being able to ever recover the keys defeats the purpose of a Hardware Security Module.</comment></comment>


A Differentiable Programming System to Bridge ML and Scientific Computing - <AUTHOR> https://arxiv.org/abs/1907.07587 <comment><USER2> Happy that this paper finally made it to arxiv. The biggest reason for writing it was to try and showcase some of the breadth of applications we see for really high quality first class AD support at the language level. There are several communities that need this technology, so it makes sense to try and build one system that can address all of them and share tricks. I'm also hoping this gives people a sense of why our list of feature requirements for this system is so extensive (pervasive custom gradients, higher order AD, mixed mode, ultra fast scalar AD, etc). These days AD is often discussed only in the context of deep learning, but as the frontiers of deep learning are pushed and hybrid models become more and more popular, I'm expecting our focus on generality to pay off here.

<comment><USER3> Keno: first of all, let me give a big public _thank you_ to you and your colleagues. (For those here who don't know, Keno is listed as one of the authors of the paper, and works closely with Mike Innes, lead author and also lead developer of Zygote. Mike is also an active member of HN.)

Second, let me bring up what I think is a significant issue. My perception is that most deep learning researchers and practitioners -- that includes me -- tend to iterate very rapidly over new ideas. We want to test ideas in code as quickly as possible, and in practice we will not invest the time and effort necessary to figure out how to write cache-optimized kernels (e.g., for GPUs) every time we might need one. In fact, I'd say the default attitude is that it doesn't even make sense for us to use (what we perceive as slow) automated kernel-writing tools to search for and compile optimized kernels. In practice, it always seems faster and easier (from a developer-time standpoint) to write code that leverages existing, inflexible, prebuilt, handtuned kernels that have already proven to work well, are fast, and are already nicely integrated with frameworks like PyTorch and TensorFlow.

There was a good discussion of this topic in the forums a few weeks ago[a] in response to a recent paper published by some folks at Google in which they make a compelling case that this issue is holding back AI research.[b] As an example, they use capsule networks[c], the implementations of which have proven difficult to optimize (e.g., they copy a lot more data around than is strictly necessary, in order to slice and reshape data in a manner that is compatible with preexisting hardware-accelerator kernels).

What are your thoughts on this? Will Zygote provide any improvements or advantages on this front?

\--

[a] [https://news.ycombinator.com/item?id=20301619](https://news.ycombinator.com/item?id=20301619)

[b] [https://dl.acm.org/citation.cfm?id=3321441](https://dl.acm.org/citation.cfm?id=3321441)

[c] [https://arxiv.org/abs/1710.09829](https://arxiv.org/abs/1710.09829)

<comment><USER2> Zygote is an orthogonal piece of technology on this front and relies on a good optimizing compiler behind it to target actual hardware. Its focus is primarily on expressability. We've been talking about automatic kernel generation for a while (and when I saw kernel generation what I mean is basically search for access patterns), but note that it's not quite as bad a problem in julia, because you can use higher order functions to get a lot of composability (e.g. if there's a hand-optimized parameterized matmul, fusing in inputs and outputs is just passing in extra functions). There's some academic work at the JuliaLab that's promising. We're also in discussions with commercial partners who care about this kind of thing to see if somebody is willing to fund it, but so far existing solutions have been good enough while we work on higher priority items. I do agree that being limited to a few hand- tuned kernels is a significant drag on research, so I hope we can do something here.

<comment><USER4> In terms of trying to break free of dependence on hand optimized kernels: a few people, myself included, have been working on some theoretical approaches to generating cache-efficient rearrangements for neutral net like problems. We've worked it out for convolution like problems [1] and have some upcoming results generalizing these techniques to other problems. Please feel free to email if you'd like to talk.

[1] [https://arxiv.org/abs/1802.06905](https://arxiv.org/abs/1802.06905)

</comment></comment></comment></comment><comment><USER5> This is huge and I have high hopes for zygote,jl.

Observation/request: For higher-order derivatives (Hessian, Laplacian, etc), AD libraries typically provide API shortcuts. I have found it difficult to control or predict the memory footprint and the time complexity of these API shortcuts.

Laplacian is case in point: it is sometimes computed by first computing the Hessian by forward-over-reverse and then taking the trace. In most libraries you would have to dig deep in the documentation or the code itself to understand how intermediary values are accumulated in memory for instance.

I would love to have a summary table of the complexity of all these API shortcuts, similar to [https://wiki.python.org/moin/TimeComplexity](https://wiki.python.org/moin/TimeComplexity) for data-structures, but for these AD shortcuts (laplacian, divergence, hessian, etc). The table would show memory and time complexity in terms of the number of the input/intermediary/output dimensions and the complexity of evaluating the original function.

<comment><USER2> Unfortunately, this isn't super easy since the time complexity heavily depends on what optimizations the compiler will apply and how the higher order AD is exactly implemented (there's many ways to do so that all give you the same answer and you probably want the system to pick the best for you). What we could do fairly easily however is to have a compiler introspection tool that runs through the computation, but doesn't do any of the work, just record how much work it would have done. That would easily give you and idea and let you plot a parameter sweep over any dimension you care about with actual results corresponding to the capabilities of the system.

<comment><USER5> Thanks for the reply. Such introspection tool would be useful indeed--it seems already possible to do some inspection by hand with [https://fluxml.ai/Zygote.jl/latest/adjoints/#Gradient- Reflec...](https://fluxml.ai/Zygote.jl/latest/adjoints/#Gradient-Reflection-1)

Anyway, great work and I look forward to what happens next!

</comment></comment></comment><comment><USER6> I've just given this a cursory read but wanted to say that this looks really exciting. Julia in general seems like such an exciting language.

I've recently switched some Fortran simulations at work for a pure Julia implementation and it is great fun to write. DifferentialEquations is a phenomenal piece of work!

<comment><USER7> Is the performance equivalent in your scenario?

<comment><USER6> We have taken a slight performance hit, but the code is cleaner and much easier to extend.

<comment><AUTHOR> If there's any benchmarks on the DiffEq side, feel free to reach out to us with them and we can take a crack at optimizing it. If it's something that's allowed to be shared, we'd love to add it to our DiffEqBenchmarks.jl repository to track the performance over time and understand the performance characteristics of your domain.

<comment><USER7> Is there a general way to do this with many of the libraries in Julia?

Any chance ya'll could eventually use a test model and script from me in a benchmark test suite.

Ex: I have a large synthetic test model from a university and plan on writing some code that uses a bunch of the sparse matrix and linear solve functionality of the language. This would be on a sparse ~70k row/column square matrix.

<comment><AUTHOR> At least with DiffEq we keep track of a lot of different benchmarks here: [https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl](https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl) and it's always growing. Some of the best benchmarks have come from users who want us to track performance on their problem. Some other libraries have repos around, but not as formalized. We are getting things automated so that we can start helping others.

Your model sounds nice because it can use our automatic sparsity detection and matrix coloring. I'd love to give it a try.

[https://github.com/JuliaDiffEq/SparseDiffTools.jl#automated-...](https://github.com/JuliaDiffEq/SparseDiffTools.jl#automated- sparsity-detection)

<comment><USER7> Awesome! I wish I could give you a real industry one (even bigger), but a synthetic model should adequately reflect the problem domain and they have been the primary models used by researchers in my field for the past twenty years or so.

Edit: which Julia group should I get into contact with over this?

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> An alternative approach [https://www.microsoft.com/en-us/research/video/the- simple-es...](https://www.microsoft.com/en-us/research/video/the-simple- essence-of-automatic-differentiation/) where you define the right functional structures that allow a very compact and efficient way of expressing automatic differentiation and then let the existing compiler do the heavy lifting.

<comment><USER2> I've read the corresponding paper, and while I think it's a fun read, I'm not sure it does particularly much to clear up the underlying confusion here. Reverse mode AD on straight line primitives is describabable in about three lines of exposition. The tricky part is how do you handle control flow (equivalently recursion). The description thereof then depends on what your underlying IR data structure is, but describing this generally isn't horrible either. The real problem in the literature is that there is lots of confusion over what is the AD algorithm and what are the workarounds applied to make it work in the particular system described.

For example, all the ML frameworks generally combine a high level tracer with a straight line AD transform, which then sometimes gets presented as a fundamental aspect of AD (which it isn't, you're just using the tracer to get an IR you can handle). As a result, this field has a lot of confused terminology, but all the results have been known since the 70s.

The name of the game for all the latest generation tools is to cleanly separate out the semantic AD transform from the rest of the system and then just use an unmodified compiler thereafter. We do it by transforming Julia IR, the Swift folks do it on SIL, and the Scala folks have an implementation using shift/reset and LMS. If you do this, a bunch of traditional AD techniques become just special cases of general purpose compiler transforms (DCE, CSE, etc). See this talk I gave recently for some more details: [https://juliacomputing.com/blog/2019/02/19/growing-a- compile...](https://juliacomputing.com/blog/2019/02/19/growing-a- compiler.html)

As an aside, a pet peeve of mine is people calling these algorithms "tape free". High storage requirements, due to the need to remember (or alternatively, recompute) intermediate values are a fundamental property of reverse mode AD and you can't get rid of it. The best you can do is hide it (e.g. in compiler managed stack or closure environments), with all the same fundamental challenges. This is probably a terminology clash, with people wanting to use "tape" as a term for a much more narrow kind of data structure generated from a tracing AD system, but the requirement to have some data structure like it, no matter how hidden is fundamental to the algorithm.

<comment><USER8> Currently, I'm currently using TF 1.x for most of my AD needs, but encountering the limits so I'm always looking for neater solutions.

Conceptually, I enjoy the functional approach where the standard compiler does the work. But I'm still using some graph based approach, where the order of operations are recorded on a "tape", then replayed it in reverse order.

The main reason is to be able to manage the memory and computation placement. To distribute huge computations among machines, while limiting memory transfers between machines/devices to a minimum.

Also some kind of automatic gradient accumulation to reduce the memory need would be nice. Currently using the graph based approach, partitioning a big tensor into smaller ones and doing a map-reduce to accumulate the gradient works surprising well (except for the initial very long graph creation time).

Every time I do one of those manual optimizations I start dreaming : "that should be done by the compiler". You tell the compiler your cluster definition and devices, you write a few for loops as if the code was done on a single cpu. And the compiler do the loop splitting, reordering, parallelization across device in an optimal way given your available memory constraints.

<comment><USER9> Sounds like you'd love julia!

</comment></comment></comment></comment><comment><USER10> Is this somehow similar to the new Swift Automatic Differentiation feature? [https://github.com/tensorflow/swift/blob/master/docs/Automat...](https://github.com/tensorflow/swift/blob/master/docs/AutomaticDifferentiation.md)

<comment><USER11> Yes, it's the same idea. But Julia's differentiable programming capabilities are far more advanced and mature than Swift's. As far as I'm aware, Swift still doesn't support differentiating code with control flow (branches or loops), which, needless to say, eliminates pretty much all non-trivial programs.

Compare that to the situation in Julia: ∂P works _today_ on arbitrary programs—like the ray tracer and other examples in this paper—programs which are highly non-trivial and use iteration, recursion, mutation and global state. All of which Julia's ∂P can take derivatives through.

When you additionally consider Swift's essentially non-existent computational and data science ecosystem, it's a bit hard (for me at least) to rationalize the Swift ∂P effort. (Are we going to differentiate iPhone apps?) They're attempting to bootstrap their computational/data ecosystem by allowing calling Python code, but as soon as you call into Python, you lose all ability to take derivatives which only works for pure Swift code. So any program which relies on Python to do some of the computation you want to take a derivative of won't be differentiable, which kind of defeats the point of having ∂P in the first place. We'll see how it pans out but the Swift effort has considerable technical and social challenges to overcome.

<comment><USER12> How do if conditions handled? They are not differentiatable right? So are loops?

<comment><USER11> There's a whole paper explaining how it all works :)

</comment></comment></comment></comment><comment><USER13> Can't comment on the technical details/merit of the paper.

However, this is huge. From a conceptual perspective, everything in the physical world can be modeled as relative rates of change, which are basically differential equations. Having the power to more easily build, run and understand these models will help tremendously in advancing the computer/physical interface as well as all its potential applications.

Thank you to the authors for such a huge contribution. Looking forward to all the cool things that will be made possible through this.

</comment><comment><USER14> I keep orbiting Julia, because of developments like this one. I am an old Lisper/Scheme guy. How does this compare to say R6RS-AD[1] or cl-autodiff[2] or Deriva for Clojure/Java[3]?

I realize Julia has femtolisp inside, yet another Lisp bait for me!

[1] [https://github.com/qobi/R6RS-AD](https://github.com/qobi/R6RS-AD)

[2] [https://github.com/masonium/cl-autodiff](https://github.com/masonium/cl- autodiff)

[3] [https://github.com/lambder/Deriva](https://github.com/lambder/Deriva)

<comment><USER15> Those are based on overloading functions to add the new behaviors (computing the forward diff or building the graph within a special variable for the reverse mode) which in Julia is implemented for example in [1], [2], [3] and [4], while the one in OP is based on source analysis and transformation.

And the similarities with Lisp are more than just the parser being written in it, Julia's programming paradigm is based on the CLOS, everything is an expression, hygienic macros and reader macros and code is data. It mostly just lacks sexpr.

[1] [https://github.com/JuliaDiff/ForwardDiff.jl](https://github.com/JuliaDiff/ForwardDiff.jl)

[2] [https://github.com/dfdx/Yota.jl](https://github.com/dfdx/Yota.jl)

[3] [https://github.com/denizyuret/AutoGrad.jl](https://github.com/denizyuret/AutoGrad.jl) (from the Knet framework)

[4] [https://fluxml.ai/Flux.jl/stable/internals/tracker/](https://fluxml.ai/Flux.jl/stable/internals/tracker/) (from the Flux framework)

<comment><USER14> Wow, thank you for such an informative reply!

I am going to try this out. One of my pet peeves with Julia is that my main machine is a Windows 10 box, and things like cudanative.jl say they only install on Mac and Linux. I have an old 2013 Linux box (Lenovo T430u laptop).

BTW, do you recommend Julia Pro install or vanilla Julia and build up for more general technical programming, not just ML and DL.

<comment><USER15> I prefer vanilla Julia since it has all the packages at the latest released versions and it's the most used so it's easier to get help. CUDANative.jl documentation doesn't mention being mac/linux only (the only mention was in the 2 year old preview release), and if it is maybe you could try another library like ArrayFire.jl.

And more general technical programming is Julia specialty (it was built for that as a high performance interactive language), the DL hype started after the language was first released.

</comment></comment></comment></comment><comment><USER16> This looks great! The language features of Julia seems to be exploited to good effect. However, I'm curious: are there any aspects of Julia's design that, in retrospect, make differential programming harder or more inconvenient?

<comment><USER2> There's two competing currents here. One the on hand, Julia is extremely powerful and dynamic, so it has very high expressability for any possible differentiable programming you could think of. It also has a fairly simple core, so as long as you know how to properly transform the core, you can get a mathematically correct differential. However, the reason julia works so well is that the compiler is able to understand and eliminate all the dynamism and complexity for you at run time and generate very tight code as a result. When you add AD into the mix, the generated code becomes, much, much more complicated, so the compiler needs to work a lot harder. That can sometimes be a cause for frustration for us, because we get something really cool working with absolutely zero effort, but then need to go back and make the compiler better to reclaim some performance (static languages have the opposite problem, where you need to improve the type system in order to allow the thing to happen, but once you have described it in a specific enough type system, you usually also have good performance - if your language is any good that is).

Overall I think Julia is a pretty great language to implement AD (as evidenced I'd say by the 15 or so different AD packages that people have written for individual use cases before the latest push for a unified one), but it still is a very powerful language, so if you want your AD to handle the whole language (as we do), then you're gonna have to do a bit of work.

<comment><USER16> Thanks! So it sounds like what you're saying is that, as with other languages that have complex optimizers, there can be performance cliffs when you get too far off the beaten path?

For someone not working on the Julia compiler, how tricky it is to figure out what to do to improve performance?

</comment></comment></comment><comment><USER17> This looks neat . In python is this also a project that does the same thing ? [https://github.com/HIPS/autograd](https://github.com/HIPS/autograd)

<comment><USER18> Would also be curious to understand how this compares. They seem to account for recursion and if-branches as well.

<comment><USER9> They're not really all that close. For instance that Autograd library requires that you use its own version of numpy instead of the regular version because it can't differentiate C. Because most non-trivial python code is actually written in C, there's almost no performant programs you can just differentiate out of the box unless autograd itself has a fork of that package.

My understanding is that the python Autograd library is also quite slow (though it's been a while since I looked at it).

Zygote will give the same runtime performance as handwritten derivatives in many cases and works across almost the entire julia ecosystem since nearly all julia packages are truly written in julia.

<comment><USER19> JAX (github.com/google/jax), which is being developed by many of the authors of Autograd, is a probably a better comparison. At the cost of requiring you to rewrite control flow in a functional way, it eliminates Python's overhead by compiling NumPy code into XLA.

</comment></comment></comment></comment><comment><USER20> Does this support user defined data structures? This is a thing holding me back in Stan and other frameworks with AD (they only want scalars or arrays)

<comment><USER15> Yes

[https://github.com/FluxML/Zygote.jl](https://github.com/FluxML/Zygote.jl)

"Without compromising on performance, Zygote supports the full flexibility and dynamism of the Julia language, including control flow, recursion, closures, structs, dictionaries, and more."

<comment><USER20> Ah OK the link page here didn't mention structs.

</comment></comment></comment><comment><USER21> Cool and amazing. This thing should be used to enrich base Julia, not simply be an extension.

<comment><USER9> Julia is designed in such a way that third party 'extensions' (packages) are first class citizens. Very little actually needs to be in Base or the standard library. When one types `using Zygote` you're not using a programming language with a siloed AD exension. You've radically extenended julia into a differentiable programming language.

The beauty of Zygote.jl being a package is that we don't force one AD approach on the entire language ecosystem. Julia has several very promising AD systems in development, each of which is preferable for certain situations but are close enough in API that they're basically hot swappable.

When one of them needs a change made to base julia to better support the sort of compiler transformations they want to do, they make a PR and the new functionality gets implemented very quickly and all the other AD / compiler transformation packages benefit from the change.

<comment><USER2> Yes, this is exactly correct. We try very hard to extend Julia with "mechanisms" not "features", such that if you don't like the exact way we're doing it, you can just do your own (with blackjack), while still re-using as much of the rest of the system as possible. The same holds true for AD. We think (and we hope) that Zygote (and related packages) will be an extremely good, general purpose AD system that's a good default, but if you're in a specialized domain, you may be able to do a lot better with a specialized AD transform (e.g. your domain might want to do part of the computation in the frequency domain automatically, or have a domain-specific optimizations that are not generally valid, etc.). Our user base is to varied to be able to prescribe a one-size-fits-all approach. The only thing we can do is provide the necessary mechanisms and high quality default implementations (as well as good explanations as to how things work) and let it loose on the world. Everything being in the same language helps enormously here, since even users coming from the scientific side are well versed enough in the language to at least be able to read and understand the semantics of code in the compiler (even if they think it employs dark magic).

</comment></comment></comment><comment><USER22> Interesting project. I question the implications of using ML data shaping to define models. A sufficiently optimized model may be missing fundamental inputs & constraints, yet still perform better than a less optimized model with the correct inputs & constraints. This would effectively create a local maxima of understanding. It seems, while the predictive capabilities of complex systems may increase in some cases, the practice of fundamental understanding & articulation of the system will decrease (i.e. I don't know how it works, the magic black box has all the answers).

Perhaps there are ways to mitigate these implications in the scientific process.

<comment><USER15> In this case it might be the exact opposite. Regardless of the structure (as long as it has at least one hidden layer and enough parameters), neural networks are universal approximators, which is why they are effectively black boxes that can learn any model. So using more custom structure does not increase the theoretical capability of the network to find more complex models, but instead you're effectively adding more a priori information (by embedding information of the problem within it's structure), which can heavily restrict the search space and therefore making it somewhat less of a magic black box.

An example might be attention models, which while being more complex than the previous models it gives you more information about what the model does (by allowing you to visualize what inputs the following layer is using).

</comment></comment><comment><USER23> How is this different from JAX?

<comment><USER2> JAX is a sophisticated and well implemented high level tracer, combined with a backend compiler that makes use of XLA. High level tracing has some fundamental draw backs in key features we want (efficient scalar AD for example), so the approach being advocated here is doing AD as a compiler transform on the original source language. Mike has some details in an earlier paper: [https://arxiv.org/abs/1810.07951](https://arxiv.org/abs/1810.07951). As a shameless plug, you can plug (heh) Julia into XLA as well, e.g. for targeting TPUs ([https://arxiv.org/abs/1810.09868](https://arxiv.org/abs/1810.09868)). In fact, Julia and JAX are probably better at that than TensorFlow, because the TF execution model is a bit of a mismatch for what XLA expects.

<comment> one-more-minute The really big problem with tracing, `@jit` annotations, and similar approaches, is that you're no longer running Python, but something almost-but- not-quite Python – modifying the language's semantics is necessary to get performance.

In deep learning, tweaking some syntax and adding some annotations isn't a big deal, but for the use cases we're interested in we really don't want to rewrite every library to be AD compatible. Zygote is pretty unique (outside of the scientific computing world) in being able to take libraries that were written years before AD existed in Julia, and differentiate them correctly and efficiently.

There are other, more subtle and technical, issues with those approaches, but that's really the Big Deal.

</comment></comment></comment><comment><USER24> How can one get involved in this project? Is there a discord or something where I can see what's happening?

<comment><USER25> The Julia discourse ([https://discourse.julialang.org](https://discourse.julialang.org)) site is a good place to engage. PRs, issues, etc. should go to the relevant package repos on Github - Zygote.jl, Cassette.jl, ForwardDiff.jl, model zoo in FluxML org and so on.

[https://github.com/FluxML/Zygote.jl](https://github.com/FluxML/Zygote.jl)

[https://github.com/jrevels/Cassette.jl](https://github.com/jrevels/Cassette.jl)

[https://github.com/JuliaDiff/ForwardDiff.jl](https://github.com/JuliaDiff/ForwardDiff.jl)

[https://github.com/FluxML](https://github.com/FluxML)

</comment></comment><comment><USER26> Me reading the title: "doesn't Julia already have that?"

Me reading the paper: "oh"</comment>


Ask HN: md5 + size = file UID? - <AUTHOR>

Is md5 checksum plus the file's size is absolutely unique identifier for a file across the universe? <comment><USER2> No, for the same reason you can't fit n+1 pigeons in n holes.

<http://en.wikipedia.org/wiki/Pigeonhole_principle>

</comment><comment><USER3> No, it isn't.

MD5 gives 16 bytes of output, so consider all possible 17 byte files and their MD5 checksums. On average, each checksum will be shared by 256 17 byte files.

If you're worried about MD5 collisions between files, adding the file size isn't going to do much to help. Better to use SHA1 or some other algorithm in addition to MD5. E.g. 16 bytes of MD5 + 20 bytes of SHA1 = 36 bytes total output.

</comment><comment><USER4> No, because all hashing algorithms suffer from collisions. A perfect hash function for files would require space of at least a number of bits proportional to the size of your problem space.

In practice, you can go with SHA-2, for which no collisions have been found yet.

</comment><comment><USER5> No matter which hash function you use, you will _not_ get 100% unique identifiers, because you _can't_! Just do the freaking math, and you'll see for yourself - it's quite obvious, actually!

<comment><AUTHOR> Yeah, I was confused. I'm a product guy, so math isn't my strongest point. It looks like combining Md5 + size + Sha1 should be enough to get a almost unique file id. Thank you all for the replies.

</comment></comment><comment><USER6> _Is md5 checksum plus the file's size is absolutely unique identifier for a file across the universe?_

No. Use SHA256.</comment>


Ask HN: Why OSS projects do not provide a vagrantfile with their sources? - <AUTHOR>

This would make contributing a LOT easier. It doesn't really require too much effort in providing a ready-to-use vagrantfile yet almost no project-owner is doing it. Why? Is there any downside? <comment><USER2> Open source is about helping. If you see a project without one you want to use, contribute a vagrant file.

</comment><comment><USER3> It would be useful if there were a few platforms available, or maybe a quick ansible playbook to set up your own dev box.</comment>


Ask HN: Is anyone building a decentralized resilient messaging app? - <AUTHOR>

<comment><USER2> Depends on what you mean by "decentralized".

Email is technically decentralized. Many of us use hosted mail services, but we could just as easily use our own PC as a mail server.

There are similarly decentralized chat services, such as XMPP.

<comment><USER3> I'm keeping a close eye on [http://matrix.org/](http://matrix.org/).

To quote the homepage: "Matrix is an open standard for decentralised communication, providing simple HTTP APIs and open source reference implementations for securely distributing and persisting JSON over an open federation of servers."

It's not finished by far, since for example there is no end-to-end encryption yet [1], but the specs look really promising.

[1] [http://matrix.org/docs/spec/#end-to-end- encryption](http://matrix.org/docs/spec/#end-to-end-encryption)</comment></comment>


OpenAI Retro Contest - <AUTHOR> https://contest.openai.com/ <comment><USER2> Two months to advance the state of the art on a complex physics-based game with branching paths. Current approaches such as DQNs or god forbid DRL[1] barely reach the performance of my three year old cousin in atari game score maximization and are mostly non-transferable to new levels... Good luck.

[1] [https://www.alexirpan.com/2018/02/14/rl- hard.html](https://www.alexirpan.com/2018/02/14/rl-hard.html)

<comment><USER3> Of course it's a difficult problem, but why try and dissuade people from attempting it? Nothing novel ever came from somebody who didn't fail first : )

<comment><USER4> I don't consider it dissuasion, but a fair warning & proposition preparation. Seriously: Good luck!

</comment></comment></comment><comment><USER5> I'm a big retro Sega fan, and I've always wanted to look into doing something like this, but this seems... really difficult. Would the best approach be to jump right in and hope for the best, or are there any sources I should look into?

<comment><USER6> If you want to do ok and understand what's up, watch the awesome free lectures by Alphabet's DeepMind here (the people at the origin of AlphaZero): [https://www.youtube.com/playlist?list=PLweqsIcZJac7PfiyYMvYi...](https://www.youtube.com/playlist?list=PLweqsIcZJac7PfiyYMvYiHfOFPg9Um82B)

</comment></comment><comment><USER7> The demo GIFs show Sonic 1 and 3, but all 3 Genesis games have slightly different physics and mechanics which could trip up an AI if not trained on all 3 games. Is the challenge just using the Sonic 1 engine?

<comment><USER8> The challenge is using all 3 Genesis games.

<comment><USER7> Yes, just found it in the details: [https://contest.openai.com/details](https://contest.openai.com/details)

From the downloaded CSV of game states, _all stages_ from the 3 games are valid training stages, and the custom stages used for testing are derived from all 3 games. Yikes.

I really hope those custom stages use the same art assets as the original games.

<comment><USER9> Every test run could theoretically start with a jump or some other behavior to exercise the different physics or whatever among the 3 games, and use a discriminator network to feed into one of three networks individually trained on each game?

<comment><USER10> The whole point of the OpenAI exercise is to improve transfer learning- a winning project should be able to work off of a single model.

For an example of a system that does learn new rules using a single model check out this post from Vicarious[0].

[0] [https://www.vicarious.com/2017/08/07/general-game-playing- wi...](https://www.vicarious.com/2017/08/07/general-game-playing-with-schema- networks/)

</comment></comment></comment></comment></comment><comment><USER11> I'm wondering if there's a MOOC that takes you from zero to being able to build a system that learns how to play such a game, maybe focusing a bit less on the math (especially the proofs). I took Andrew Ng's course on Coursera but I feel like the gap between what I know and what's needed for this contest is huge. Am I wrong?

<comment><USER12> No, you're not wrong. That said, I don't think this is a competition aimed at people with a beginner level knowledge of machine/deep learning. I don't believe there is currently any MOOC that is going to hold your hand from 0 to expert. Given the current state of the field, I believe the only surefire path towards this level of knowledge is either university education or a /lot/ of reading.

</comment></comment><comment><USER13> Seems like a stretch to call this "transfer learning". Maybe training on Sonic and testing on Mario.

Would be cool to see some kind of adversarial competition. You train to, say, beat a game level but you test to beat someone else's submission. (Short on the specifics, I know.)

<comment><USER14> Transfer learning is what makes this contest currently unique. Usually in these machine learning contests, you hand off your model to the contest runner, and they just run it and evaluate its performance. It has to be already good at the task.

In this contest, you're not just submitting a trained model, you're submitted a docker environment capable of performing training, which they will run with their secret levels, with specified constraints. So you want to make a highly trained model whose capabilities can be transferred to secret levels.

</comment></comment><comment><USER15> Cynical prediction one: Nothing learned here will be readily transferrable to another domain. :(

</comment><comment><USER16> I expect training & testing to make use of emulators & ROMs - wouldn't Sega potentially have a problem with this or is it considered fair use?

<comment><USER7> The loophole suggested in the instructions is to use the ROMs provided by legally-purchased copies of the game from Steam.

</comment></comment><comment><USER17> Can we move away from video games ? I know games provide a closed loop for these kind of experiments/contests, but I want to see more practical application of RL. How about code generation that create programs based on the test cases or RL agent that can use a 3d design software.

<comment><USER18> The problem is that reinforcement learning is far from solved and doesn't work all that well yet, so these toy problems are probably what researchers will stick with for some time to come.

</comment></comment><comment><USER19> Couldn't a new level contain some sort of new scenario that would be completely impossible to navigate based on previous experience without some _very_ general AI?

</comment><comment><USER20> What's the input to the agent? Just all the pixels on the screen?

<comment><USER21> > Each timestep advances the game by 4 frames, and each observation is the > pixels on the screen for the current frame, a shape [224, 320, 3] array of > uint8 values. Each action is which buttons to hold down until the next frame > (a shape [12] array of bool values, one for each button on the Genesis > controller where invalid button combinations (Up+Down or accessing the start > menu) are ignored ... During training you can access a few variables from > the memory of the game through the info dictionary. During testing, these > variables are not available.

So yep, the RGB image.

</comment></comment><comment><USER22> Is this a fair representation of a human playing a game? Usually as a human player I don't go into a level expecting to clear it never having seen it before.

<comment><USER23> It's fair because you get better at gaming in general when you play games. For instance, your performance on Mario will be better after playing a lot of Sonic.

This is in contrast to current models which generally need to be trained up from scratch on each new game, even if the games are mechanically similar.

<comment><USER22> I think maybe a reasonable choice would be to give the ai the ability to store some limited amount of information, let's say 4kb, and it gets to populate this on a first run and draw from it on a second run. Second run scores are compared.

</comment></comment></comment><comment><USER24> I couldn't find - high score or fastest time? Totally different skills.

<comment><USER25> Best score

Check out [https://contest.openai.com/details](https://contest.openai.com/details)

I'm not sure how different those skills are from an AI development point of view though, why do you think they are?

<comment><USER26> If I'm reading that right they're defining score in terms of time so it's actually the same?

<comment><USER25> Ugh, you're mostly right, I skimmed too quickly.

> The reward your agent receives is proportional to its progress to the > predefined horizontal offset within each level, positive for getting closer, > negative for getting further away. If you reach the offset, the sum of your > rewards will be 9000. In addition there is a time bonus that starts at 1000 > and decreases learning to 0 at the end of the time limit, so beating the > level as quickly as possible is rewarded.

So mostly time - but past a certain threshold just completion. And if you don't finish "portion completed".

<comment><USER26> Well if you get to the time limit then it ends wherever you are, so this is literally just how long it took to complete, or how far you got if you did not complete. Completing gives you 9000 points and the max time bonus is 1000 so it heavily incentivises winning consistently versus higher variance strategies that may be quicker.

</comment></comment></comment></comment></comment><comment><USER26> Do you win anything?

<comment><USER27> Second to last paragraph:

      The contest will run from April 5 to June 5 (2 months)
      and *winners will receive some pretty cool trophies.*

</comment></comment><comment><USER28> Was this just released today?

<comment><AUTHOR> Yep!</comment></comment>