Npm operational incident, 6 Jan 2018 - <AUTHOR> http://blog.npmjs.org/post/169432444640/npm-operational-incident-6-jan-2018 <comment><USER2> > Unfortunately, the process was complicated by well-meaning members of the > npm community who believed that a malicious actor or security breach was to > blame and independently attempted to publish their own replacements for > these packages. Ensuring the integrity of the affected packages required > additional steps and time.

That is such a bad response to this.

The problem isn't that "well-meaning members of the community" decided to upload packages. The problem is that when their system decides that a package shouldn't be up it completely removes the package, as if it never existed, and allows the namespace to be reused immediately. Those "well-meaning members" should not even be able to hijack packages this way, as it means the people who aren't "well-meaning" can also do it.

What should happen is that they block downloads of the package while they investigate. That way people who attempt to download the packages get a meaningful error and people are unable to hijack the package name.

<comment><USER3> It's been literally years since node-forward got its talk about signing packages [1] with a lot of pushback from the npm team. Every time a new typosquatting article shows up, there's some more waffling by npm. left-pad happened to much consternation. Now this.

I used to really care about trying to harden the Node ecosystem, and last year it was one of my main goals. I tried to send multiple vulnerability reports, do mass static analysis of npm packages, and wanted to contribute more to the ecosystem, but the consistent ambivalent reactions of much of the community that I talked to turned me off of the project entirely. If npm wants to continue to be a security dumpster fire, let it burn. Node is a waste of security researchers' time and an honest goldmine for black hats looking to compromise relatively powerful novice webdev hardware.

I don't see it changing anytime soon. npm is a business that isn't focused on security. These things keep coming up, and yet npm install metrics I'm sure aren't decreasing. Until they face meaningful competition and/or the rest of the Node community begins to give even half a care to security outside of this forum, there will be no incentive for anyone to do anything about it. It's easier to play PR, give a little lip service to it and dodge the problem than it is to add any friction to their potential growth.

[1] [https://github.com/node- forward/discussions/issues/29](https://github.com/node- forward/discussions/issues/29)

<comment><USER4> This is your occasional reminder that package signing is not a panacea, and as typically proposed for community package repositories like npm, PyPI, etc. would likely do absolutely nothing.

For example, people often insist in the Python world that PyPI should support package signing. But _it already does_ \-- you can generate a signature for a package and upload the signature with the package. Django does this, and has been doing it for years. You can also get package download/install tools that will check the signature. But then what?

What people really _mean_ when they say there should be "signed packages" is that there should be a whole bunch of invisible infrastructure (set up by... who, exactly? Maintained by... who, exactly?) to decide which PGP keys are authorized to sign releases of which packages. And that's close to an intractable problem for an anyone-can-contribute community repository like npm or PyPI.

<comment><USER5> This is a very important point. I work for a company that publishes client libs for many different package indexes (although not npm). This is a fairly well automated process, but it takes minutes (if that) to push a new version to pypi, rubygems etc, but at least a few hours of fiddling about to get something on maven, which of course has this security infrastructure.

An analogy might be drawn with the app stores. We all know it is massively easier to get stuff in the play store than the iOS store. We all know there is a shit ton of spam, malware etc on the play store and not really in the other. But it's also much easier to contribute to. It's a trade off. Security is important, but sometimes I feel that people are unwilling to treat it as an input in a basic cost benefit analysis, instead turning it into a kind of absolute value. I accept that it is not treated seriously enough by many in the community, but overcorrection is not the answer.

Of course, other relatively 'open' package indexes exist that do not have npm's typo squatting issues, so there are other design issues at work in this particular case.

</comment></comment></comment></comment><comment><USER6> Very displeased about this response.

> In yesterday's case, we got it wrong, which prevented a publisher's > legitimate code from being distributed to developers whose projects depend > on it. We identified the error within five minutes and followed defined > processes to reverse this block. Unfortunately, the process was complicated > by well-meaning members of the npm community who believed that a malicious > actor or security breach was to blame and independently attempted to publish > their own replacements for these packages.

No. Assuming everything in that excerpt is true (and I happen to know it's not, but that's not even relevant here), that _wasn 't the problem_.

The problem is that NPM allowed packages to be re-uploaded by new authors after the initial versions had been spam filtered. Especially since allowing packages to be re-uploaded by new authors was the core issue of the left-pad debacle, and the _one thing_ NPM said they'd fixed in response.

Let's summarise here:

1\. NPM has a big issue

2\. They claimed they had fixed it

3\. They had not

4\. In their post mortem they're pretending the issue doesn't exist

This guts any remaining trust I had in npm. Even if I wanted to trust them, they're not even admitting the problem exists; how am I meant to believe they're finally going to fix it? They've stopped even _promising_ to fix this, and moved on to lies and denial.

Unacceptable. Literally. This is pushing me away from the node ecosystem because I am not prepared to accept this sort of weaponized incompetence from the primary package repo for node.

<comment><USER7> With all that money they've raised, it's puzzling why they don't take PR seriously. I guess they have a mentality of "where the fuck else are you gonna go, the node community is tightly coupled to our service"

<comment><USER8> It's almost as if the tech community has to learn once again centralized systems with a trusted authority at the top can't be trusted when that trusted authority makes mistakes.

<comment><USER9> Honest question: what's the alternative?

<comment><USER6> I've mentioned this elsewhere in this discussion, but: [https://github.com/ramasilveyra/gitpkg](https://github.com/ramasilveyra/gitpkg)

I've been using it a bit, and it's really nice. Yarn (and the npm command line tool) already provide nice methods to install packages _from_ arbitrary locations, including git repos; this provides a nice mechanism for publishing _to_ git repos. It's a super simple solution, but then, it's not really a hard problem.

</comment></comment></comment></comment></comment><comment><USER10> Very disheartening to see that NPM has not grown from the kik or left-pad incidents. Users should not be able to republish on top of old package names without some kind of intervention.

Of course, NPM's response to the kik/left-pad problem was also pretty awful. Make it so users can't delete packages. Cool. For those of you using NPM's private offerings, this also applies to you, so hope you don't care about cleaning up your private NPM registry content.

Getting pretty tired of this. Their open source operation seems to suffer from poor handling of community and technical issues, at least from a high level. Their private registry operation is very lean on features, and also suffers from very confusing limitations. I'm surprised at how long it took there to be read-only API keys. Until last year, you literally had to give your CI keys that could publish to your organization if you wanted access to private packages. And you had to pay for an extra user for the privilege of doing it wrong. It's fixed now but it still blows my mind that it took so long. Aside from faster installation, it's actually a lot better to just use private Git URLs instead of NPM's private offerings.

NPM will never properly take the fall. They will take just enough blame to seem responsible and then shift the majority of the poor decision making onto other actors in the community in their explanations. Third time in recent memory.

</comment><comment><USER11> > no malicious actors were involved in yesterday's incident, and the security > of npm users' accounts and the integrity of these 106 packages were never > jeopardized

Maybe not in the incident itself, but the sheer fact that many of the packages were replaced by other people constitutes a jeopardization of _applications_ that depend on NPM. The only reason why some big package didn't get replaced with code that exfiltrated data from production or subtly backdoored it is sheer luck.

<comment><USER12> it's beyond disingenuous and flat out dishonest for npm to say the integrity of the packages was never jeopardized. within minutes there were reports that packages had been replaced with questionable and malicious content.

[https://news.ycombinator.com/item?id=16087079](https://news.ycombinator.com/item?id=16087079)

this is probably a good indication of how a much more serious security event would be treated by the organization.. just sayin

<comment><USER13> At the very least, Npm should explain why we can be sure the packages now offered are exactly the same as they were before the incident, if, in fact, it is in a position to make that claim.

</comment></comment></comment><comment><USER14> How can they claim no malicious actors were involved when packages such as duplexer3 were apparently replaced with undesirable code as reported in [https://news.ycombinator.com/item?id=16087126](https://news.ycombinator.com/item?id=16087126) ?

<comment><USER15> This is not a claim or a fact, that's probably an uneducated guess, npm is broken on so many levels they can't say anything for sure. [1]

npm is full of fundamental bugs in their software, workflows, architecture, and community. We learnt this in March 2016 when one developers got a copyright claim and had to remove their package. [2] Why is it even possible to remove a package? Why is it possible to create new package with the same name? Almost no software published there have unit tests, asking developers how they tested it in a GitHub issue leaves the issue open with no response. Everything is centralized on GitHub, which was proven to fail many times when GitHub was down in a region or globally due to a DDoS. Why is Node allowed to exist? We're not in 1998 anymore.

[1] [https://news.ycombinator.com/item?id=16092584](https://news.ycombinator.com/item?id=16092584) (comment above)

[2] [https://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/](https://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/)

<comment><USER16> At least get the company name right if you have to make guesses about what they know internally and what they don't. (lol, and edited it silently so my comment looks dumb now)

<comment><USER15> >We don't discuss all of our security processes and technologies in specific detail for what should be obvious reasons, but here is a high-level overview

> they know internally and what they don't.

They know internally that they do a lot of things badly and this will result in more disasters for npm and JS communities, that's for sure. Check out how GitLab fixed their backup failure. They made new software for testing backups, they live-streamed their work on youtube, they made at least 2 write ups on this case. They were totally transparent about their mistakes and fixes, everyone applauded them for this. What did we get from npm?

</comment></comment></comment></comment><comment><USER17> The last paragraph is good! The other paragraphs are bad. The two main ways they are bad are:

1) A system that detects "spam" and then allows for the complete removal of packages as if they never existed, allowing anyone to replace them, should never be described in the neutral terms used in this post. This system appears to be an existential threat to the company and project, and one of its largest mistakes. It won't take a long investigation to figure that out; it should be obvious today.

2) The claim that the security and integrity of these hijacked npm package names was not jeopardized appears to be 100% pants-on-fire false. If it is not false, I think npm's users are owed an explanation today of why it's false, rather than a bare assertion.

I know it's hard to be in the hot seat. No animosity to any of the humans involved.

</comment><comment><USER18> It is really another embarrasment for NPM, considering that this is not the first time something similar happens (see: left-pad issue, 23 march 2016)

So they learnt zero from it.

Then, _WHEN ARE THEY GOING TO SIGN THOSE PACKAGES?_ Zero, nothing. They didn't learn from mistakes, they don't listen. NPM is still open to all sorts of malicious use.

Besides NPM problems, yesterday many packages won't work because package "pinkie-promise" wasn't available. This is the full effective source code of pinkie-promise:

        module.exports = typeof Promise === 'function' ? Promise : require('pinkie');

Not just a NPM fault, but a fault of the js community as a whole for accepting systems made from hundreds of one-line packages, a sort of spaghetti code for the modern era.

<comment><USER19> Someone should grab the 100 most used packages which contain less than 30 lines of code, merge it into one package and call it 'npm standard library'.

<comment><USER20> Guess I'll be that guy: [https://xkcd.com/927/](https://xkcd.com/927/)

<comment><USER21> I've gotten to the point where I recognize certain XKCD urls and no longer have to click on them to know which one it is. The ol' 927, I see.

</comment></comment></comment></comment><comment><USER22> > who believed that a malicious actor or security breach was to blame

This _was_ a security breach. Their anti-spam system should block repos by freezing the module name and returning blank files, not by deleting the entire module and subsequently allowing anyone to upload new modules. This is leftpad all over again.

Update: Woah, so I was checking out my old NPM namespaces and apparently someone took control over [https://www.npmjs.com/package/filesaver.js](https://www.npmjs.com/package/filesaver.js)

<comment><USER23> It's not a "breach" technically. No malicious person caused the packages to be removed by circumventing authentication/authorization systems. It is a security issue though. A bug and serious security issue, yes, but not technically a breach.

<comment><USER24> A breach didn't cause the issue, but there absolutely was a breach in response to NPM's incompetence. [https://news.ycombinator.com/item?id=16087126](https://news.ycombinator.com/item?id=16087126)

</comment></comment></comment><comment><USER25> I think it's time to replace npm in default Node installs. They've shown a history of negligence for package handling and procedure. Why should Node continue putting the trust of such a substantial part of not only their ecosystem but the JS ecosystem as well into such unreliable hands?

<comment><USER26> the official node.js Docker image comes with yarn as well as npm. but this is an issue with the npm registry, not the package manager shipped with node. both yarn and npm use the npm registry.

EDIT: it may be worth debating why npm (a binary produced by npm, Inc) is shipped with node.js at all, but that's upto the node foundation

<comment><USER27> I'm pretty sure npm is shipped as Javascript source by node.js

<comment><USER26> ah yes, you are correct. it isn't a binary

</comment></comment></comment></comment><comment><USER28> From what I've read (I've not yet worked on anything that used npm), it appears that npm uses a single shared flat namespace for all packages. If I publish a package named "order_pizza" nobody else can name a package "order_pizza", unless mine gets deleted first, and then they can...and anyone who was using my "order_pizza" will now silently get the other developer's "order_pizza" the next time they build.

Anyone know why they did it that way instead of giving each developer a subspace of the namespace that only that developer can publish under?

I'm thinking of the way it works on Java, where my package would be net.tzs.order_pizza, or GitHub (and numerous others) where it would be tzs/order_pizza.

<comment><USER29> I think it's annoying but also toxic to an ecosystem. You have to name your packages the same way you look for a domain name, people can squat on names, and it causes drama like how the 'kik' package was stolen from a developer and given to a company who never used it.

Nobody would've cared it the package was "tzs/kik". For example, the Elm package system does this right.

Another example of this failure is the Reddit subreddit system where a subreddit like /r/bitcoin makes it seem like the authority when it's a carefully moderated and biased subreddit. And nobody can really compete with it without getting a worse subreddit name like truebitcoin and bitcoin2 or something. Would be much more clear if it was /r/@theymos/bitcoin and /r/@spez/politics.

<comment><USER30> The failure is yours alone if you actually consider Reddit an authority on anything.

<comment><USER30> Downvoters: I of course meant an authority on anything ELSE than memes and karmawhoring. Sorry for not being clear about that.

<comment><USER29> Okay.

And you might say the failure is yours alone when you overeat, yet 30%+ of the USA is overweight. This little platitude only makes you feel better about yourself but isn't a good basis for designing something.

<comment><USER30> That's interesting. Unless I'm being forcefed, then who else is to blame? I have free will after all.

</comment></comment></comment></comment></comment></comment><comment><USER31> "We don't discuss all of our security processes and technologies in specific detail for what should be obvious reasons..." \- Security by obfuscation? That's not security. Your protocols and processes should stand up even if made public.

<comment><USER32> What is your home address and phone number?

<comment><USER33> If you think that your address and phone number are secret information, consider how easy it would be to find yours from the information in your HN profile. I'm pretty sure I found your entry on whitepages within a few minutes and I'm barely motivated to find it.

<comment><USER32> I think you are confused as to my intention

<comment><USER33> I thought you were asking a rhetorical question to show that obfuscation is a valid security measure that we all use. What was your intention?

</comment></comment></comment></comment></comment><comment><USER34> I left an Ethereum related project precisely because of the house of cards feel of it being built off JS and npm modules. They're a great example of how a project can be exploited by a malicious module which could proceed to extract all the tokens.

If my hat is black, I'm writing a daemon which monitors relevant npm modules and uploads subtlely modified versions if and when the possibility to do so occurs again. Particularly, but not only, targeting crypto currencies.

<comment><USER35> > If my hat is black, I'm writing a daemon which monitors relevant npm modules > and uploads subtlely modified versions ...

I started sketching out one yesterday, and then thought "What am I going to do with this? Probably just get in trouble" so I stopped.

</comment></comment><comment><USER36> If I was managing npm-using systems, this press release would not put my fears to rest. They need to publish a full review of the contents of packages replacements that were uploaded (even if just for 5 minutes), and publish those replacements in a safe form so they can be reviewed personally by any concerned npm user (any user - not just those who downloaded the replacements through npm itself).

</comment><comment><USER37> Lesson : in a name space, never delete records. Mark them as unused, invalid, inactive, "deleted" but never actually delete.

</comment><comment><USER38> Independent of npm's issues now and in the past, how vulnerable are other package managers to similar problems? There were typosquatting issues with pip, replacement issues with Rubygems, probably others that I don't recall. What's the current state of things for the more commonly used languages?

<comment><USER39> Maven/Gradle don't allow un-namespaced packages; nobody knows for sure how deep up their arse npm's designers have had their ass when they made non- namespaced packages possible (even the only way in the beginning; I don't know if it's still the default).

Additionally, there's the problem of npm's fine-grained dependencies (that have become a laughing stock in itself), in my Java large projects, while I don't know every single dependency, I know all the "vendors", i.e. trusted upstream namespaces I use, and their transitive dependencies. There's the occasional non-Eclipse, non-Google, non-Apache dependency (say, XStream), but I am aware of it and fix it to a specific version. npm with its thousands of dependencies by hundreds of pseudonymous developers for simple projects is an abomination.

<comment><USER40> `Crates.io` (the Rust package ecosystem) doesn't have namespaces and operates after the first come first serve principle. They won't resolve namesquatting issues so something like the left-pad debacle won't happen. I think their policy[0] is quite sane. At least I'm not aware of any issues of that kind regarding crates.io.

For typosquatting: I don't think that problem can be easily solved by package registries. They could warn you if there is a package with a similar name (and more votes, downloads, what ever) but aside from that, the developer should double check the dependency names and that's it.

[0]: [https://crates.io/policies](https://crates.io/policies)

</comment></comment></comment><comment><USER41> Npm isn't ready for production use. If you're going to use it you either need to ship the modules you need or run it against a private repo of modules.

<comment><USER42> I agree in principal about not relying on npm during deployment. For a novice, what is the best way to ship modules?

<comment><USER43> My first thought was "What about including node_modules in your version control?" Downside is that the repo would be much larger, but if it's going to expand to that size on the client after running "npm install", maybe that's not a big deal? But it feels kinda wrong to do that...

<comment><USER44> Committing your `node_modules` is a bad idea. The best answer is to either use an offline cache of package tarballs that have been committed to your repo, or an NPM caching proxy.

Yarn has an "offline mirror" feature built in, and Shrinkpack is a tool that can create an offline mirror based on an npm-shrinkwrap.json file.

I wrote about using Yarn's offline mirror here: [http://blog.isquaredsoftware.com/2017/07/practical-redux- par...](http://blog.isquaredsoftware.com/2017/07/practical-redux- part-9-managing-dependencies/#managing-dependency-packages-for-offline- installation)

<comment><USER23> I know you can use JFrog/Artifactory to cache pulls of Jar files from maven repos, as well as Docker images. Does JFrog support node?

<comment><USER45> Yes, JFrog Artifactory supports npm and so does Sonatype Nexus. More info:

[https://www.jfrog.com/confluence/display/RTF/Npm+Registry](https://www.jfrog.com/confluence/display/RTF/Npm+Registry) [https://help.sonatype.com/display/NXRM3/Node+Packaged+Module...](https://help.sonatype.com/display/NXRM3/Node+Packaged+Modules+and+npm+Registries)

</comment></comment></comment></comment></comment></comment><comment><USER46> As others have said countless times on the original thread: new packages from different users should never have been allowed to replace the missing packages.

Even if a package is removed as malware, a user should never unexpectedly download the work of a totally new author that they haven't vetted.

Some of the protections should be done on the user side, but that's no reason for NPM to have dangerous policies.

<comment><USER47> > but that's no reason for NPM to have dangerous policies.

Oversight != 'dangerous policies'. Seems like they thought they had this fixed.

</comment></comment><comment><USER48> They need to pull their socks up and start signing these packages

<comment><USER49> What would signed packages have achieved in this incident?

Who needs to sign the packages? The developers of the packages, right? Otherwise developer credential compromise subverts the entire point.

Which keys are authorized to sign for which packages? How to prevent credential compromise from affecting those authorizations?

What's the difference between a signed legitimate package and a signed malicious package?

If they introduced package signing, would packages would adopt it fast enough for users of packages to only use signed packages?

What happens when signing keys are lost or compromised? Do we need to use countersignatures from timestamping services as with other forms of code signing, so that CI systems do not break if a key is pulled?

I think this is very much not straightforward. npm+pgp may be well intentioned but seems grossly inadequate.

<comment><USER14> Android and the Google Play store works pretty well. The systems "locks on to" a public key signature so further updates for a given app must be signed with the same key. That would fix the problem with reproducible builds here. Only adding new top level packages to a project for the first time is at risk then.

</comment></comment></comment><comment><USER50> A bunch of folks here are complaining that the explanation isn't good enough, doesn't go into enough detail, etc.

Literally the second sentence is:

> Early this coming week, we will share a full analysis and technical > explanation of the incident.

I'm looking forward to reading the final post-mortem, but in the mean time I really appreciate their update.

</comment><comment><USER51> The only thing missing from this post is 'we take security very seriously'

</comment><comment><USER52> Am I missing something in thinking that a reasonable solution is to temporarily block downloads of potentially malicious packages? Why remove them?

EDIT: or better yet, don't allow people to download the update, just keep users on the old, ostensibly safe version until everything gets sorted?

</comment><comment><USER53> I'm curious of the manual review process. Is this synchronous, i.e. immediately when publishing a package? Or is it after the fact, where suspicious code code have already been distributed? There are plenty UX trade- offs in either direction, of course.

</comment><comment><USER54> I've used NPM only a little but this scares me. What are NPM alternatives?

<comment><USER47> NPM shouldn't scare you. Simple fact is you should not be relying on ANY package registry at the time of deployment.

<comment><USER55> > _NPM shouldn 't scare you._

It absolutely should, just like any dependency on any other third party code or servers. Especially when they regularly have incidents like this.

> _Simple fact is you should not be relying on ANY package registry at the > time of deployment._

This is true, but doesn't prove the previous claim.

<comment><USER47> > NPM shouldn't scare you.

>> It absolutely should, just like any dependency on any other third party code or servers

There's no need to be scared of any of those things if you understand the trade-offs and risks.

>> Especially when they regularly have incidents like this.

They don't.

>> Simple fact is you should not be relying on ANY package registry at the time of deployment.

> This is true, but doesn't prove the previous claim.

The 'previous claim' follows from it. If you don't treat a package registry like an essential part of your own infrastructure, there's no need to be scared of it.

</comment></comment></comment></comment><comment><USER56> Lots of people are piling onto npm here. This reads to me like a fairly simple unintended consequence of what seemed like a good approach.

Automated spam filters help to avoid dodgy packages. Spam filtering operates on heuristics so it's sensible to not publicise how they work.

The automated spam filter kills dodgy uploads, as these mostly happen on previously unused names a decision is made to not have the spam packages' name remain taken. Among other things this stops the spammers from leaving all the good names blocked.

The spam code gets a false positive and the above logic kicks in, leaving previously used names now available. This is noticed and corrected within a few hours.

The npm team will likely improve their spam filtering heuristics and also ensure that formerly good packages that get spam flagged do not release their names - they have indicated roughly this on Twitter.

This only just happened, and it's the weekend, so I'd expect a full write up will be released during the week when they've had time to do a post-mortem and work through the salient points.

<comment><USER6> > This reads to me like a fairly simple unintended consequence of what seemed > like a good approach.

Yes, but you need to keep in mind:

1\. This stuff is hideously important because getting it wrong is a incredibly serious security hole. Letting people publish a package with a name that other people are already requiring is a defcon 1 emergency.

2\. They already had a major issue with this and said they had fixed it. You get, at most, _one_ time when you can say "whups, we didn't realise how big an issue this was!".

3\. It's their job to get it right. This is, literally, npm inc. It's not some hobbyists, or a service being provided by people in their spare time.

So when you say:

> The automated spam filter kills dodgy uploads, as these mostly happen on > previously unused names a decision is made to not have the spam packages' > name remain taken.

I have to disagree. They needed to check how many times the flagged package had been downloaded, and after the left-pad debacle, they knew (or should have known) that.

They're not taking this seriously.

</comment></comment><comment><USER57> maybe everyone should wait for the full report before jumping to conclusions?

from what it appears, the reason someone else was allowed to register the same package name was because their database didn't pull in the entries for that package. an error? yes. an issue? yes. but it is not a repeat of the same incidence of left-pad.

i'm not going to pass judgement until i read the full report. i think at the very least, whatever mechanism they were using to copy/backup the database seems to have an issue with deleting entries and needs to be addressed

</comment><comment><USER58> Others may be interested to note this blog post from about a week ago by the CTO relating to npm's outage handling [0]. I'm not sure if I would classify this as an outage, but this seems to be the official narrative [1]. Given that status page was updated relatively quickly, I'm guessing that at least something _like_ this procedure kicked in.

I am a little shocked at the brevity of the procedure. I am guessing by the blog post pointing out the brevity that I am meant to be shocked, but if I turned out that 310 word procedure for downtime I'd be pretty confident it was not good enough. I'm not confident that I would be able to read that document and understand what to do if I was the one on pager duty. Maybe it would be different if I worked there and perhaps the procedures are more granular and each of the stages well explained in other procedures. Maybe.

I'm finding it hard to understand what npm does that is so different to other languages' package managers that it can have so many problems and such bad public relations. Linux distros have repositories that handle packages with far greater sizes and quite a few are purely voluntary efforts. They also manage to include package signing as well. I've known of a few hiccups, but none that been handled quite so bad as to completely undermine my trust in the tool altogether.

Is there any compelling technical reason why we couldn't have a tool that works more like go get for node? For those that haven't used it, you just point it at a git repository.

[0] [http://ceejbot.tumblr.com/post/169045932039/how-to-handle- an...](http://ceejbot.tumblr.com/post/169045932039/how-to-handle-an-outage)

[1] [https://status.npmjs.org/incidents/41zfb8qpvrdj](https://status.npmjs.org/incidents/41zfb8qpvrdj)

</comment><comment><USER59> I'm trying to figure out whether node2nix [0] could help avoid package name/version hijacking vulnerabilities. Node2nix can convert a package- lock.json file to something called a Nix expression [1], like this [2], where each dependency has a checksum. The purpose is to make the Nix expression deterministic so that each Nix package is reproducible.

[0] [https://github.com/svanderburg/node2nix](https://github.com/svanderburg/node2nix)

[1] [https://nixos.org/nix/about.html](https://nixos.org/nix/about.html)

[2] [https://github.com/svanderburg/node2nix/blob/22d0c19575ecd19...](https://github.com/svanderburg/node2nix/blob/22d0c19575ecd19208cbdfbb54e8c2228c553bf7/node- packages.nix#L13)

Edit: fix link formatting

</comment><comment><USER60> Why does a spam system delete packages that have already been in use for quite some time? I could understand if it blocked some newly updated ones, but it seems like it has deleted already existing packages that also were used by other packages?

</comment><comment><USER8> With this and leftpad etc., I think it's time for the tech community to remember Java and Oracle, Sourceforge, etc., or look it up if you are too young to remember. It is beneficial to learn from history instead of making the same mistakes all over again.

Centralized trusted authorities cannot be trusted if they are private companies who don't answer to you. If you aren't paying them and you don't have a management position in said company, things like this will happen.

</comment><comment><USER61> Everybody is mad/disappointed at/by the NPM for their handling...

Does this have any real impact on the NPM package adoption though? There is no alternative node module package directory, is there?

It seems like any action they could have taken would have had exactly the same effect (well, maybe some prospective products don't adopt JS backends, but you have all webpack projects locked in pretty tight, right?)

<comment><USER6> I've been using gitpkg ([https://github.com/ramasilveyra/gitpkg](https://github.com/ramasilveyra/gitpkg)) lately; it works pretty well. Basically lets you use a git repo as a package repo, instead of npm.

Very little about how I use npm requires it to be centralized; I don't _ever_ search npm looking for a package to install; I find a package I want and then look to install it.

<comment><USER62> Couldn't you just use github IDs in your package.json directly? npm and yarn both have support for it.

<comment><USER6> Short answer: Sure, but it works much better with gitpkg.

Long answer: Sometimes you can do that, yes. But you'll run into issues with "modern" frontned toolchains and complicated packages where you want to write ES2017 code, transpile it with babel, bundle it with webpack, make a universal lib, etc., and push the result to your package repo, but not commit it to your git repo.

Gitpkg handles all the wierd corner cases, and lets you keep build artifacts out of your git repo, and raw source out of your package repos. And if you use github you can even use the github IDs in your package.json, but the result is going to be a lot better. If you're just writing a plain one file package for use in node that doesn't need any processing, eh, no need for gitpkg.

</comment></comment></comment></comment><comment> vl-y Such explanation looks misleading to me. Even having the module detected as potentially malicious, why was it put into a state that such module doesn't exist, letting others publish package with the same name. For me it looks like there is something wrong at the npm side internally with the processes.

</comment><comment><USER63> How difficult is it to run your own private npm repo? Looks like that's what security-conscious folks should be doing, given this response. Any pointers/gotchas/battle stories much appreciated.

<comment><USER64> It's doable, and one very good option is Sonatype Nexus [https://www.sonatype.com/download-oss- sonatype](https://www.sonatype.com/download-oss-sonatype)

The one issue I've seen over the years is it seems npm Inc does not officially sanction on site npm repos (presumably to help monetize their private hosted repos), and finding info on how to set up a private repo is lacking.

</comment></comment><comment><USER65> Just when I started thinking NPM had turned the ship around after the previous incidents. Given the amount of people who depend on their service, this is extremely amateur.

</comment><comment><USER66> The pleasures of working with Node.js

</comment><comment><USER67> Is it related to a package that write logs in colors and send visa card numbers to hackers ?

<comment><USER68> No, that article was a warning that a package like that could exist in the wild and nobody would notice. It doesn't actually exists, as the author states in the blog post.

</comment></comment><comment><USER69> Since everything uses HTTP these days I think they could use one of the 20x codes to signal when a package has been affected by spam management or suspension.

That way the official clients would support it but you could easily ignore it if you wanted to.

</comment><comment><USER70> Has the community discussed adding cryptographic signatures to npm packages? It's generally a good idea in any case, but this outage further highlights the security risks of a public registry where packages are not signed.

</comment><comment><USER71> Looks like they don't allow comments on their blog posts??... I guess with good reason.

<comment><USER72> They use Tumblr as their blog platform, and Tumblr AFAIK doesn't use comments.

</comment></comment><comment><USER73> > We don't discuss all of our security processes and technologies in specific > detail for what should be obvious reasons, but here is a high-level > overview.

Isn't that the opposite of good practice? You shouldn't rely on obscurity. It's better to have the security processes out in the open so that it can be audited and flaws pointed out.

<comment><USER74> Yeah, I found this an important aspect of the article. I don't know anything about the culture at NPM, but this feels related to the mismanagement of communication with consumers/the public about the incident itself. i.e., being secretive for no good reason.

</comment></comment><comment><USER75> I think a major security improvement for npm and other package managers using almost exclusively open-source/ non-compiled code is to require the source code be linked from a popular open-source platform like Github and then to take the package directly from there, to ensure the code can be audited and nothing else can be snuck in.

</comment><comment><USER71> I've been meaning to set up a blog with hexojs for a few weeks now, and was seriously all set to install node and npm (don't do too much with node...), and start tinkering yesterday, and then I checked hacker news, and was like 'well sheeyit'...

<comment><USER71> I guess this would be safe enough probably:

[https://github.com/billryan/docker-hexo](https://github.com/billryan/docker- hexo)

</comment></comment><comment><USER76> Pay attention to the tags they themselves added to their report. Namely, "#SPAM IS WHY WE CAN'T HAVE NICE THINGS". Oh npm...

</comment><comment><USER77> noob question : would using yarn have mitigated any of the risks of missing/broken packages ?

<comment><USER78> No, both Yarn and npm use the npm registry.

</comment></comment><comment><USER79> idea: Peer reviewing packages with the ability to suggest a better name.

</comment><comment><USER80> A manual review misjudged 106 packages at the same time? That seems... dubious. Unless they have a bulk removal based on meta data, instead of careful analysis. Which would also be pretty dubious.</comment>


The Discord Store Beta - <AUTHOR> https://blog.discordapp.com/the-discord-store-beta-9a35596fdd4 <comment><USER2> Discord's user experience is incredible compared to Steam. The app is super responsive. If they add video streaming as well they can compete with Twitch & Steam at the same time. Both have abysmal ux.

<comment><USER3> I'm sorry, what? Steam is much more responsive than Discord.

<comment><USER4> Strongly disagree. Steam feels like browsing a slow web page.

<comment><USER5> Most of the time that's exactly what you're doing, if you're navigating through the 'store'. And it's horrible. Steam absolutely is not an application I would point to if someone asked me for an example of a well optimized and performing desktop application.

(And I still haven't forgiven Valve for killing WON, even if I know they had what they felt was a good reason for it)

<comment><USER6> Oh WON... Those were the days! HLDS (Half-Life Dedicated Server) was a very big part of how I became a sysadmin - I'm sure I owe a big chunk of my career to whoever wrote it and it's documentation!

</comment></comment></comment></comment></comment><comment><USER7> It's an interesting choice, and while I saw it coming after they added a page to view your friends' gaming habits, it's not the long-term move I thought they would take.

Discord basically started out as "Slack for gamers", and very quickly (IMO) surpassed both Slack and Skype in terms of functionality. I don't have to have a different account for every server I'm in unlike Slack, and their video and audio calling has always been much more stable than anything I've gotten on Skype.

With that in mind, I would've thought that their next plan would be to improve on what they do best, and try to market to the professional markets and take Skype and Slack head on. Instead they seem to be veering off and doubling down on the gamers. Judging by the number of people I see who have it, I imagine the profit from Nitro may be enough to keep the company going, and the Slack crowd won't be as lucrative as the opportunities from taking some of Steam's marketshare.

<comment><USER8> Anything marketed to corporate inevitably gets bogged down with nonsense. With their current approach they can keep working on what the absolute end-user cares about, and this is how they avoid repeating Skype's mistakes. I think it's interesting how many ways they are planning to mimic Steam's features with moves like these, I still wonder how much traction it can actually gain. But everyone I ask seems positive about it.

</comment></comment><comment><USER9> If you told me two years ago that anyone (even a major player like Amazon) was thinking about competing with Steam, I'd write them off immediately, but this makes a lot of sense. Steam has gotten way too complacent in its social features, and I am getting increasingly frustrated by bugs in the app. My ears picked up when Discord mentioned a "book shop" vibe. With all of the controversy about adult games and Steam's "grand bazaar" approach, I would love a store that was curated and made my gaming experience more fun.

I smell a game vendor war a-brewing. The results could be messy...

<comment><USER10> Well, almost exactly two years ago, Amazon bought Curse. I believe this is forming the basis of their Twitch desktop application. I haven't looked at it, but I imagine the intent is the same.. use Twitch network effect to build an install base, turn it into a distribution/store to take on Steam. That's what Discord had already been doing at the time, largely accomplished via the Twitch platform too.

They must have looked at Discord when they were evaluating Curse, given that they suggested the purpose of Curse was to help build community (though curse was originally a distribution mechanism for mods and add-ons that added community as afterthought, in my subjective opinion). I imagine someone at Amazon is kicking his or herself for the choice today..

<comment><USER11> I am still surprised Amazon didn't buy Discord, and my internal canon suggests they must not have liked they price. Between Lumberyard, buying Twitch and Curse and merging them, Discord is a sure thing fit for their gaming arm. Twitch in particular is a great/widespread streaming platform with utterly terrible chat, bringing Discord into that would patch up a big weakness there.

Microsoft has often been suggested as a suitor, but between having Skype and Teams and the Windows/Xbox Store already, I just don't see Discord meshing well with what they already have.

</comment></comment></comment><comment><USER12> Recently, Steam rolled out Discord-like chat/community features: [https://www.theverge.com/2018/7/24/17609872/steam-chat- new-f...](https://www.theverge.com/2018/7/24/17609872/steam-chat-new-features- roll-put-discord-like-competition)

Turnabout is fair play.

<comment> old-gregg Shouldn't discord be considered cheating by Valve? I haven't seen/used it, but I assume it's a voice chat outside a game, so people can continue to give out enemy coordinates even after they die. If that's the case, steam should just refuse to run or ban your account if discord presence is detected, no?

<comment><USER13> It's no different than using Skype or Ventrilo or even a phone/conference call. There's no real way to manage who talks to who while you're playing a game.

<comment><USER14> Hell, it's not different than being in the same room as other players. I've yet to hear of any games that ban teams sharing the same IP address.

<comment><USER15> Most poker games do.

</comment></comment></comment></comment></comment><comment><USER11> I've long held that Discord lacks an actual business model. This is the first time I could honestly say they have one, as the benefits of Nitro were a joke.

Of course, it's interesting they chose a business model that nobody has ever succeeded at: Competing with Steam. You need exclusives to draw people to your game platform, and with that on PC comes "why can't I just get this on Steam?" Without something like a "Games Anywhere" offering sync, you'd also have to have people buy games anew on your platform.

<comment><USER16> People can keep their old Steam games and still use Discord. That's what people have been doing since Discord came out (along with Epic Games, Blizzard, EA, etc.).

I'm curious to see how the social stickiness of Discord will affect their foray into a games/app store. I believe that, for multiplayer games at least, people first fire up the way they want to talk with their friends and then set about playing a game. From there, you grow the product into managing and buying your games.

If Discord provides some differentiated social benefits to the games bought through their store, then I could see this doing quite well.

<comment><USER11> With Steam comes the achievements, the trading cards, etc. Sure, you can copy all of that, but the challenge is that Discord isn't fighting Steam on day one, they're fighting an established library. How do you convince someone with 99 games in Steam that their 100th should be bought through Discord instead?

Sure, people can use Discord for chat while playing Steam games, but Discord really doesn't benefit from that. They need to convert chat users into game buyers, and I think you'd need something like Games Anywhere (or more simply, like GOG Connect), to bridge in what people already bought to start the process along. Bonus if you can find a way to bring over people's achievements and other progress.

<comment><USER17> > How do you convince someone with 99 games in Steam that their 100th should > be bought through Discord instead?

You don't have to, there are always people just starting their library. The Discord client is already significantly better than Steam's (which is why its so popular for chat).

I think people are pretty meh on Steam achievements since hacks to unlock them are pretty widespread, but Discord would need to provide other platform norms like cloud saves and DRM.

I also think Microsoft should swoop in and buy Discord before it's too late. The Xbox brand and UWP need a shot in the arm, and a Discord purchase could finally get PC gamers using Xbox services.

<comment><USER18> Because that audience didn't collectively jump ship from the last service Microsoft bought _to_ discord. They also don't have a long lasting mistrust of Microsoft due to GFWL, the Xbone attempt to stop game sharing, umpteen "We really care about PC gaming this time, honest" promises, etc.

Frankly, while programmers have benefited enough from new Microsoft that people are starting to give them a chance, gamers have not at this point a MS owned Discord would do more harm to Discord than good to MS. Users already see the Microsoft store as a flytrap to avoid, and any future Discord services would be tarred with the same brush.

</comment></comment></comment></comment></comment><comment><USER19> I've been a Discord Nitro subscriber since Nov. 2017, so I welcome any additional benefits. That said, I'm not entirely a fan of the game store model. I know Discord's primary audience are gamers, but that's not why I use Discord.

<comment><USER20> Agreed, they could have have generalized and eaten all other chat services including slack. I guess they want to take on steam but I don't know if it will work.

<comment><USER21> honestly i was so skeptical of discord at the beginning because as someone who was a regular vent user it really struck me as a niche thing. But they've done a great job to branch into non-gaming communities to get enormous growth. I'm not sure if they're gonna "win" against steam or even make a dent but it will be interesting to see them try.

</comment></comment></comment><comment><USER22> Discord is claiming that these new features won't add to bloat:

[https://blog.discordapp.com/how-discord-maintains- performanc...](https://blog.discordapp.com/how-discord-maintains-performance- while-adding-features-28ddaf044333)

Meanwhile I've noticed that Battlenet and Steam have rolled out even more Discord style features very recently.

</comment><comment><USER23> I remember telling s friend a few years ago that this is the natural next step for their business. Attacking he steam hegemony won't be easy though given people already have libraries there. But if anyone can do it it's them.

<comment><USER24> I've already moved beyond the everything-in-Steam model. I'm content to have games in Steam, in Origin, in UPlay, in my Microsoft account, in the Blizzard launcher, and standalone.

This is just one more account to keep in mind, though it looks like it's more of a rental model than a purchase model.

<comment><USER25> Looks like Discord can launch games from all of the above from within the app.

<comment><USER24> So can the Xbox app.

</comment></comment></comment></comment><comment><USER26> Steam works pretty great on Linux (with a surprisingly large selection of native games). I wonder if Discord is planning on competing there as well, or if they're just limiting games to Windows.

<comment><USER27> They have no hope until they fix the terrible audio quality on Linux that appeared several months ago.

<comment><USER28> For me and many others audio input doesn't even work. There is a ticket in their issue tracker, it has been broken for a long time already

</comment></comment></comment><comment><USER29> I love Discord and have been a nitro subscriber for a while now, but I'm disappointed at their responsiveness to bugfixing.

One of their latest updates added a really frustrating regression where emoji autocompletes are spellchecked on ios, correcting it nonsensical words. It took some digging to find their bug tracker and I noticed it's set to their lowest priority. Why was this allowed to be released in the first place?

</comment><comment><USER30> >So, we've curated some of these golden games and will be adding them to Discord Nitro.

They need to focus this curation in a big way. There's been a lot of flack directed at steam for allowing it to be essentially polluted by all sorts of low quality "games" recently, and discord could really hurt them by actually _caring_ about their community. Valve's hands off nature has not been easy on gamers

</comment><comment><USER31> This'll be really interesting. They might have the best shot so far (IMO) of actually competing with Steam in a significant way.

They have a very sticky app for gamers with best in class UX. People love using it and many have it open 24/7\. A lot of companies have tried going "store first" and it hasn't put much of a dent on Steam, but this approach might eventually. Assuming their library gets big enough. Still incredibly difficult though given how much players have invested in their Steam libraries.

</comment><comment><USER32> I just hope its not another store with a 30% commission. If they're charging developers 15% or less, I'll definitely get on board.

Valve need to up their game though. Their approach of 'anything goes' when it comes to allowing games on Steam is backfiring. They need to increase the fee to list from $100 to $2000. For legitimate devs that extra $1900 charge will more than pay for itself with the attention you will get launching a game into a de-cluttered store.

<comment><USER33> Microsoft now charges just 5%, if you do your own marketing.

<comment><USER32> Thanks, didn't realise that. Its a shame that the Microsoft store is so terrible.

[https://blogs.windows.com/buildingapps/2018/05/07/a-new- micr...](https://blogs.windows.com/buildingapps/2018/05/07/a-new-microsoft- store-revenue-share-is-coming/)

</comment></comment></comment><comment><USER34> Just wish they would add encrypted messaging. To me that's Discord's most important missing feature.

<comment><USER10> Well, their privacy policy allows them to sell all of those messages. Encrypting them in a non-reversible fashion (or at least non-reversible to Discord themselves) would impact this or any kind of research they want to do on the years of gamers' conversations.

I like Discord a lot, and to be fair, their people have publicly claimed here and elsewhere that they have no intention to sell the information. Unfortunately, those statements have no teeth.

While they state the same in their privacy policy (in non-binding form about intent), the same privacy policy goes on to specifically allow sale and transfer of user data.

See "OUR DISCLOSURE OF YOUR INFORMATION" here: [https://discordapp.com/privacy](https://discordapp.com/privacy)

The difference is that the privacy policy is a legal acknowledgement end users must accept before they use the software. The forum/blog statements and non- binding language we read everywhere else are just 'feel good' things for users to hear and for their employees to tell themselves.

If they just wanted protection against an accidental slippage of data then that privacy page could be changed substantially. Instead, they pave the road for the explicit sale of data at a later time.

Ideally they would deny it in current policy if they have no intent of selling the information. They could leave the option open to change in the future with a "this privacy policy is subject to change" clause, but clauses like this only have teeth if there is also a clause that ensures future policy changes won't apply retroactively to historically collected data without opt-in after the policy change.

But this weakens their position in an eventual acquisition by some company who finds monetary value in that data.

As it stands right now, in an eventual acquisition or even just some internal shifts of philosophy in the organization, all historical data is up for grabs for any potential use.

</comment></comment><comment><USER35> This is an interesting move. Mark my words, Valve will acquire Discord soon

<comment><USER36> Tencent will acquire Discord soon

</comment></comment><comment><USER37> Great, distracting games where I least want them.

<comment><USER38> In a gaming chat app?

<comment><USER37> People don't just use discord for gaming chat.

<comment><USER39> And people use bats for things other than baseball but it's doesn't stop manufacturers from catering to the athletes.</comment></comment></comment></comment>


Firefox Private Network - <AUTHOR> https://fpn.firefox.com/vpn <comment><USER2> I have this for my mobile devices that are running Android, but can't wait for the ability to use it on Linux. I do wish that Mozilla would publish some regular updates, though. I realize it's more or less in beta, but I know I'd appreciate at least having a timeline of when they're expecting support, or at least to make sure they're making progress in that direction.

I'm also excited by the prospect of Mozilla offering consumer-facing privacy- focused products besides Firefox and Thunderbird. I can only imagine there's a market for hosted alternatives to GSuite, etc. in that niche that they could capitalize on.</comment>


Microsoft's video accidentally reveals that they have only ~200k servers - <AUTHOR> http://www.istartedsomething.com/20080812/microsoft-enviroment-video-reveals-server-numbers-power/

<comment><USER2> Here's an interesting statistic: Based on 148357 servers consuming 72500 kW of power, Microsoft's average server is using 488 W of power at any point -- so clearly Microsoft is not following Google's approach of using smaller servers which each draw 200W or less.

Is the Windows kernel better at using large boxes than the linux kernel? Is Microsoft worse at managing clusters than Google? I'm sure there's _some_ reason why Microsoft uses fewer more powerful servers while Google uses more less powerful servers.

<comment><USER3> It's not clear from the video, but the utility power number probably includes environmental stuff like cooling and lights. Just dividing one number by the other isn't going to give you the size of the power supply for each machine.

<comment><USER4> Environmental control is one of the largest power sucks of a data center. It's part of why data center space is so expensive per square foot.

</comment></comment></comment><comment><AUTHOR> Does it mean that google still has less than 1M?</comment>


Ask HN: What Does the California Consumer Privacy Act Do for Me? - <AUTHOR> I understand the act goes into effect Jan 1, 2020. What can I now ask companies for or to do? <comment><USER2> From [https://en.wikipedia.org/wiki/California_Consumer_Privacy_Ac...](https://en.wikipedia.org/wiki/California_Consumer_Privacy_Act)

"The intentions of the Act are to provide California residents with the right to:

      * Know what personal data is being collected about them.

      * Know whether their personal data is sold or disclosed and to whom.

      * Say no to the sale of personal data.

      * Access their personal data.

      * Request a business to delete any personal information about a consumer collected from that consumer.

      * Not be discriminated against for exercising their privacy rights."

If it's anything like the GDPR you can send a letter asking for a print-out of all personal data. And request it gets deleted. Note that financial transaction must be retained based on other laws (tax law) so companies can't delete everything.

[https://www.techrepublic.com/article/how-to-request-your- per...](https://www.techrepublic.com/article/how-to-request-your-personal- data-under-gdpr/)</comment>


A new foundation for mathematics - <AUTHOR> https://www.rdmag.com/news/2014/09/new-foundation-mathematics

<comment><USER2> > The disadvantage of set theory is that its principles are very difficult to > translate into the programming language of a proof assistant.

I don't think that's true. There's an Isabelle theory for ZF, and the ZF axioms are actually simpler than the axiomatic system that Coq uses, the Calculus of Inductive Constructions.

Backing up for a second, I'm just not sure how HoTT will make using proof assistants so much simpler. I've done a couple of largish (> 1000 lines) proof developments in Coq. The hardest part never seemed to me to be that mismatch between CIC and ZF. The hard part was that proofs are actually hard to do in a careful and complete way. The proofs I did in math class were easy because we left out a lot of the details.

Let me give an example. This is from the HoTT book, Book version: first- edition-611-ga1a258c, page 368:

"The field of rationals Q is constructed along the same lines as well, namely as the quotient

Q : ≡ ( Z × N ) / ≈

where

( u, a ) ≈ ( v, b ) : ≡ ( u ( b + 1 ) = v ( a + 1 )) .

In other words, a pair ( u, a ) represents the rational number u/ ( 1 + a ) . There can be no division by zero because we cunningly added one to the denominator a. Here too we have a canonical choice of representatives, namely fractions in lowest terms. Thus we may apply Lemma 6.10.8 to obtain a set Q, which again has a decidable equality.

We do not bother to write down the arithmetical operations on Q as we trust our readers know how to compute with fractions even in the case when one is added to the denominator. Let us just record the conclusion that there is an entirely unproblematic construction of the or- dered field of rational numbers Q, with a decidable equality and decidable order. It can also be characterized as the initial ordered field."

If you have experience with using proof assistants, you know how handwavy this is compared to the work required to actually get this done. Here's that development in Coq right now:

[https://github.com/coq/coq/tree/trunk/theories/QArith](https://github.com/coq/coq/tree/trunk/theories/QArith)

Does HoTT really make these things simpler to write and prove? If so, what is the evidence?

<comment><USER3> I'm no expert, but I believe the hope is that quotients are easier to work with in HoTT than they would be elsewhere since you can treat the resulting "cosets" as true objects instead of structure with proof.

Also, it should be much easier to transport proofs from type to type according to isomorphisms that are discovered.

I'm hardly versed enough to say whether those things are (a) true (b) actually occurring (c) powerful enough to be worth the potential headaches HoTT incurs or (d) deserving of acclaim, but it's at least my understanding.

<comment><USER2> As far as quotients go, I have used setoids before for that purpose. Maybe HoTT makes quotients even more convenient.

[http://coq.inria.fr/V8.2pl1/refman/Reference- Manual030.html](http://coq.inria.fr/V8.2pl1/refman/Reference-Manual030.html)

<comment><USER3> Yes, my understanding is that quotients should be much more convenient to use than setoids. Univalance means you can more or less ignore equality in the quotient after you define it---proofs should just be transportable.

</comment></comment></comment></comment><comment><USER4> I'm curious - can anyone point me to an overview/exposition of this that is more hand-wavy than the book? I'm interested understanding the ideas in this work, because proofs as objects would be a major step forward, but I don't even know much about the existing systems of foundations (ZFC etc), nevermind type theory...

<comment><USER5> Did you read the Introduction from pp1-12 of HoTT? I thought it was a pretty good overview.

<comment><USER4> I tried. I was hoping for something more like: "a field is something you can add, subtract, multiply, & divide in" and less like: "a field is a set that is a commutative group with respect to two different operations (excepting 0), linked by the distributive law".

I need something to hold on to when reading the more formal treatment.

<comment><USER3> I highly suggest Robert Harper's lectures from the Oregon Programming Languages Summer School. I watched the OPLSS'12 lectures, but I'm sure they're great any year.

</comment></comment></comment></comment><comment><USER6> Vladimir Voevodsky is making an interesting step when he says that rewrite steps in math are clearly homeomorphisms. I would like to see some more formal examples of that. It sounds like a promising path.

Concerning the replacement of ZFC by Russell types -- contrary to lambda calculus' functions -- I have not run often enough in illustrations that demonstrate the usefulness of Russell types.

I am personally much more vouching for a replacement of Zermelo and Fränkel's sets by Alonzo Church's functions than Bertrand Russell's types at this point. It is probably just an issue of lack of familiarity, though.

</comment><comment><USER7> The Mizar project has been verifying a good share of math results for decades, not to demerit HoTT but to clarify the article.

<comment><USER8> Exactly. The HoTT folks seem to be unaware of developments like Mizar and the the Journal of Formalized Mathematics (where you submit a computer proof along with your paper).

<comment><USER9> They are almost certainly aware of Mizar. Mizar's foundation is ZFC+Tarski's axiom. If you are developing a foundation based on homotopy theory/type theory you probably don't care about results formalized in Mizar. Also the type theory people in general prefer constructivist mathematics, and would reject Axiom of Choice and friends being in the foundation.

<comment><USER8> But doesn't Mizar void the argument that we need a new foundation as set theory has not been conducive to mechanized proofs?

<comment><USER3> I don't think the argument is that it's impossible to mechanize proofs outside of HoTT---just that it's prettier and perhaps better there.

<comment><USER8> "Prettier" is subjective.

<comment><USER3> Absolutely. Never claimed otherwise.

</comment></comment></comment></comment></comment></comment></comment><comment><USER10> Is there anyone approaching the issue from the other side? Meaning, if I have a good understanding of programming and type theory, using that to explain homotopy theory?

<comment><USER3> I think many people are approaching it that way. Ultimately you can end up using higher inductive types as "synthetic homotopy", i.e. homotopy theory which is by construction valid to the definition instead of carefully isolated from other constructions.

So you can certainly just take the type theory and start to get at pretty large stuff in actual homotopy theory.</comment></comment>


Bounded Integer: Header-only C++ library replaces integers, adds explicit bounds - <AUTHOR> https://bitbucket.org/davidstone/bounded_integer/src/master/ <comment><USER2> At this point, when I see libraries like this all I can think is "oh good, my compile times aren't long enough, lets make EVERY INTEGER a template".

Also, while I can reasonably beliveve that most of the overhead goes away at -O2 or -O3, this has to just trash performance for debug builds, which is not unimportant.

<comment><USER3> I don't think you even looked at the library, but from I can tell, debug perf would not be impacted at all.

<comment><USER2> Of course it will be impacted. Libraries like these are "zero-cost" only because of very aggressive inlining (not generally available in debug builds), otherwise all its operations happen in subroutines. Operator overloading is the most obvious example: for regular integers, addition is basically a single instruction even in debug, but with a library like this, it has to make a subroutine call. A call which has who knows how many layers of abstraction that compiler optimizations hasn't shaken off.

In fact, the README says as much: it has only zero time/space overhead "assuming basic compiler optimizations like inlining".

And I did actually look at the code, but at a glance I couldn't get much sense of it. The main header imports like two dozen other headers, and the "detail" folder is filled to the brim with more headers. I tried cloning and using it, but I couldn't get it to compile (which is probably mostly my fault, but I really didn't want to spend too much time on it). I did run just the preprocessor though, and "#include <bounded/integer.hpp>" expanded to 48000 lines. 48000 extra lines to compile in order to use integers. For _every file_ you use it in.

I don't want to be too harsh here, I'm sure it's an excellent library and it does what it says brilliantly, and if you need bounded integers I'm sure it's awesome. But language like "[integers in C++] are mostly unusable" rankles a bit, and the the implication in this file is that this is something you should regularly use instead of integer types.

I personally feel that libraries like this are taking C++ in the wrong direction ("ranges" is another obvious example) and making the language less and less usable in my profession.

<comment><USER3> > but with a library like this, it has to make a subroutine call.

I mean, before I wrote my comment I checked and it's `constexpr` all the way down to the add instruction so if it's going to make a call, I'm not seeing it. There is definitely a lot of template machinery, but I can't be the judge of that immediately.

> I personally feel that libraries like this are taking C++ in the wrong > direction ("ranges" is another obvious example)

I think you're making an emotional argument based on some preconceived twitter-verse sentiment. I'm in games too (graphics specifically) and people are up and arms about the wrong thing. Maybe there are corner cases that are a bit overly complex, but what's wrong with being able to write `std::sort(container)` instead of `std::sort(container.begin, container.end)`. There may be things we don't like, but I don't think we should resort to hyperbole either.

Incidentally, I wouldn't use this library if only because for what I do, having explicitly sized data types is important.

<comment><USER2> constexpr functions are not inlined in debug builds. They can't be: the whole point of debug builds is that you can attach a debugger and step through the code.

Illustration, compare the assembly for "foo1" and "foo2": [https://godbolt.org/z/t9Zkx-](https://godbolt.org/z/t9Zkx-)

<comment><USER3> Ah ok, my mistake, I read "debug" as "-g" not "-O0." I generally run with optimizations on but with symbols.

</comment></comment></comment></comment></comment></comment><comment><USER4> Oh, that's nice. I wanted that decades ago, when I was working on program verification. Templates have made a lot of progress if this can be done entirely in C++ templates. I once wrote, but never published, "Type Integer Considered Harmful", back when there were still 16-bit integers in most C programs. I wanted ranges on everything, like Ada. As a practical matter, integer overflow became less of an issue with 32-bit.

Sizes of intermediates are a big issue. When you write

        int_32 a,b,c,n;
        ...
        n = (a*b)*c;

how big is each part? My thinking on this was that it's the compiler's job to prevent overflow in intermediate values where the final result will not overflow. So, above, you'd have to compute (m * n) as a 64-bit product, do a 64-bit divide, and only then check that the result fit in n.

is legal to compute in 32-bit, but requires overflow checking on the intermediates. If an overflow occurs, there will be an overflow in the result. (Although, the case where some values are zero is an issue. Suppose a * b overflows but c is zero so it doesn't matter. That's probably an error.)

Sometimes you have to use larger sized intermediates. For

        int_32 m,n,p;
        ...
        n = (m * n) / p;

how big is each part? Above, you'd have to compute (m * n) as a 64-bit product, do a 64-bit divide, and only then check that the result fit in n.

To do this right, you need something in the compiler that can do basic reasoning about machine arithmetic. Something that knows, for example, that

        uint_16 n;
        ...
        n = (n + 1) % 65536;

cannot really overflow and can be optimized down to a plain unsigned 16-bit add.

If you try to to this through linguistic type analysis only, it's not going to be satisfactory. You need to be able to prove out inequalities.

<comment><USER4> Botched the editing near the examples.

Sizes of intermediates are a big issue. My thinking on this was that it's the compiler's job to prevent overflow in intermediate values where the final result will not overflow.

When you write

        int_32 a,b,c,n;
        ...
        n = (a*b)*c;

that's legal to compute in 32-bit, but requires overflow checking on the intermediates. If an overflow occurs, there will be an overflow in the result. (Although, the case where some values are zero is an issue. Suppose a * b overflows but c is zero so it doesn't matter. That's probably an error.)

Sometimes you have to use larger sized intermediates. For

        int_32 m,n,p;
        ...
        n = (m * n) / p;

how big is each part? Above, you'd have to compute (m * n) as a 64-bit product, do a 64-bit divide, and only then check that the result fit in n.

</comment></comment><comment><USER5> Maybe Ada's approach of every number type requiring explicit bounds is a good one.

<comment><USER6> Or Lisp's approach of just using "big" ints by default (which is practically the same but with an implicit default of "infinity"). And trust the compiler to use fixints internally when the programmer has declared it safe to do so.

<comment><USER7> I just finished writing a custom FIFO allocator for an _extremely_ high bandwidth and low latency data processing (and UI) system written in C/C++ (even std::deque was doing WAY too many heap allocations, not to mention the allocations within each object passing through the system, despite use of move semantics to minimize redundancy).

Performance improved by _100x - 1000x_. And it was already blazingly fast before, if measured against performance standards we've become accustomed to from JavaScript and other GC languages.

In high performance systems (where the benefits of C/C++/etc. outweigh the downsides), dynamic allocations always come back to bite you.

If you rely on them too heavily from the start (and don't plan for custom allocation schemes in the future), you can even get into bad situations where it's infeasible to refactor to custom memory management (without a total rewrite), when you later need the performance gain.

Incorporating even the _possibility_ of heap allocations into a language's most fundamental data types will doom that language to being relegated to performance-insensitive and latency-insensitive tasks, if only because it requires that a heap exist (whereas C, Rust, etc can run on embedded real-time systems with no heap).

And that's okay! It's good that we have languages for that. But C/C++/Rust/etc. are definitely not where you can tolerate such a thing in the core language.

<comment><USER8> If you are doing high-throughput, and you ever allocate anything after startup, you are Doing It Wrong. Any brand of FIFO loses.

What you need is a big-ass ring buffer, mmapped on a hugetlbfs, fed by a process on a NOHZ isolcpu core. Readers are separate processes.

<comment><USER9> No. Thread local/core-specific FIFO is more cache efficient than a ring buffer because the address about to be allocated is significantly more likely to be in a high level cache.

With a ring buffer, you're constantly cycling out to L3 or worse and hoping the prefetcher figures out what you intend. It's basically a LIFO allocator.

Even if you want to use a separate processing core, you get better latency using a FIFO allocator between 2 threads on the same core complex and the code is simpler, reducing instruction fetch overhead.

Frankly, I see architectures like yours all the time from firms like Hudson River Trading and I think they suck. They incur tons of overhead, the process separation adds a useless layer of abstraction that's annoying to transcend, and you end up with this useless message protocol between cores that invokes tons of copies and breaks compiler inlining features.

<comment><USER8> Ring buffers have a strictly sequential access pattern, which prefetchers are specifically optimized for.

Anybody using a "message protocol" with ring buffers, or doing copies out of them, is Doing It Wrong. I routinely get 10x performance by doing away with FIFOs and buffer allocation and freeing.

Process separation means you can start and stop readers independently of any other activity.

</comment></comment></comment></comment></comment></comment><comment> ur-whale It takes quite some amount of stamina to keep on reading after "The built-in integer types in C++ (int, unsigned, long long, etc.) are mostly unusable"

<comment><USER10> I can see why you'd say that that way but the author probably feels that way because of the domain they work in. I've encountered research types who just use bigint everywhere because it's safer and they don't worry about the perf.

When I was coming up as a game programmer no one would use boost shared_ptr and weak_ptr because of perceived issues with it. Everyone rolled their own handle system.

In fact there were a lot of people that considered languages beside C/C++ untouchable due to perf. They couldn't fathom there were domains where that perf just wasn't that important.

I now work a lot with Unity and find people feel the same about C# foreach because it used to trigger an allocation.

You'll find people often make the mistake of thinking their domain specific problem is a problem for everyone.

</comment></comment><comment><USER11> > bounded::integer uses built-in integers as the template parameter to > determine its bounds. This means that it cannot store an integer larger than > an unsigned 64-bit or unsigned 128-bit integer (depending on the platform) > or smaller than a signed 64-bit or signed 128-bit integer. This restriction > should be removed at some point in the future.

So when an overflow happens here I assume it's a compile time error (UB due to integer overflow in constexpr context must be diagnosed by the compiler). A handful of multiplications can get you there easily.

<comment><USER12> I am not aware of any compiler or platform where int is 128 bits. The most common these days, even on 64 bit platforms, is 32 bits, which they don't even mention.

I am not sure why they chose to speculate about sizeof(int) in this way, vs. expressing the limit in terms of something more concrete.

<comment><USER13> How are you envisioning this library being used?

<comment><USER12> What?

I am saying the author did not express the limitation well, and instead made it sound like they don't understand sizeof(int). The explanation that leni536 offered makes a lot more sense, so maybe they should have put it that way instead of using an easily misunderstood term as "built-in integer" to mean "one of several integer types that we may choose based on some ifdef"

<comment><USER13> Why did you arrive at this confusion?

Why would you assume your observations are relevant to the use case?

<comment><USER12> Did you read this part:

> bounded::integer uses built-in integers as the template parameter to > determine its bounds. This means that it cannot store an integer larger than > ...

They say the limitation arises from using "built-in integers" as template parameters. It is reasonable to assume "built-in integer" means "int". It is more a stretch to assume "built-in integer" means any number of types depending on the evaluation of several ifdefs [which is what I found in the source].

There is nothing about a "use case" relevant to any of that quoted statement, so I find your reply very confusing too.

<comment><USER13> > It is reasonable to assume "built-in integer" means "int".

This strikes me as a little nuts for C but it definitely explains this thread. I just assumed people had already switched over to e.g. sint32/size_t to avoid this problem a long time ago. I couldn't tell if this was a legitimate complaint or someone's language lawyering tendencies gone too far.

<comment><USER12> First of all, we're not talking C. This is a C++ template parameter. The feature doesn't exist in C.

Second, it is not "nuts" to use int, I would argue it's quite a bit more crazy to use rarely-supported 128-bit quantities when you don't need them. Your suggestion of "sint32" (this is not a standardized typedef, you mean int32_t perhaps) would _be the same as int_ on most compilers today, and size_t is likewise _very often_ 32 bits. Your suggestion is basically a no-op in a lot of places. Domain-specific areas like file formats or network protocols are a different story, as those have to be explicit about specifying sizes.

Third:

> someone's language lawyering tendencies

I suggest you avoid passive-aggressive communication style and just say "you" instead of "someone". If you're going to criticize or give feedback, be direct. Don't vaguely tell me that "somebody" "might" have a problem.

Best regards to you, friend.

<comment><USER13> Hey I was just trying to figure out if you were bringing up anything of value :)

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER14> > The built-in integer types in C++ (int, unsigned, long long, etc.) are > mostly unusable

Weird. Ive been programming in C/C++ professionally for over a decade and have been using built-in integer types mostly without issue. Might want to qualify your hyperbole- is it worth reading on?

</comment><comment><USER15> > The built-in integer types in C++ (int, unsigned, long long, etc.) are > mostly unusable because of the lax requirements on bounds.

This must be some new definition of the word 'unusable.'

<comment><USER16> I agree with that statement. How can you use type when you don't know range of values. I want to store year. From 1850 to 2050. What type should I choose? I want to store colour value: from 0 to 2^24-1. What type should I choose?

<comment><USER17> int, long. The standard specifies their minimum sizes.

<comment><USER18> X86 has also had saturation arithmetic since MMX was introduced.

<comment><USER19> .... However, unless you use non portable intrinsics, you can't make the compiler use MMX. So in practice you get wrap on overflow.

Worse, the compiler is entitled to use saturation arithmetic if you've assumed wrapping.

<comment><USER20> > Worse, the compiler is entitled to use saturation arithmetic if you've > assumed wrapping.

C89 (draft[1], because the actual spec is not public) section 126.96.36.199 states:

> ... a result that cannot be represented by the resulting unsigned integer > type is reduced modulo the number that is one greater than the largest value > that can be represented by the resulting unsigned integer type.

So, unless I'm misunderstanding, the compiler is not entitled to use saturation arithmetic on unsigned integers.

[1] [http://port70.net/~nsz/c/c89/c89-draft.html](http://port70.net/~nsz/c/c89/c89-draft.html)

<comment><USER8> signed and unsigned have different semantics.

The compiler is allowed to do literally anything on signed overflow, including launching the missiles, and not running the program. Or both.

</comment></comment></comment></comment></comment></comment></comment><comment><USER10> Because of the field I work in I feel this much more keenly with floats. I have been bitten by their excentricities so many times I'd just much rather have a flexible fixed point type that let me choose how much precision I want on either side of the decimal for a given set of calculations.

<comment><USER21> Doesn't COBOL do this?

</comment></comment><comment><USER22> At first glance it seems to be similar to boost::safe_numerics. Are there significant differences?

[https://github.com/boostorg/safe_numerics](https://github.com/boostorg/safe_numerics)

<comment><USER23> Both libraries do the same thing. Boost lets you set your exception policy and I'm unsure if this Bounded Integer library gives the same level of control.

<comment><USER24> The bounded::integer types accept three template parameters: `integer<min, max, overflow_policy>`. That final parameter can be bounded::throw_policy, which is itself templated on the exception type thrown. The default exception policy is "overflow is undefined behavior". The other two policies supported out of the box are wrapping / modulo and saturation / clamping.

</comment></comment></comment><comment><USER25> > you don't pay for what you don't use

> throwing an exception on overflow or clamping the value to the minimum or > maximum are also possible by use of template policies (and those particular > use cases are already built in to the library)

> Never perform a run-time check when a static check would work instead

... but _sometimes_ do run-time checks?

> Have no space or time overhead

This is a very confusing set of claims. You can't have all of these at the same time. Dynamic checks will be needed more often than you'd think, even in cases that "we" can tell won't overflow, so you'll definitely pay for stuff that you use even though with a stronger system you wouldn't need to use it.

For example, I'm wondering if this system can eliminate dynamic checks in cases like these:

        int sum = 0;
        for (int i = 0; i < 10; i++) {
            sum += i;
        }

After this loop sum's type should be integer<55, 55>, but I'd bet (without having tried) that with this actual system it's inferred to integer<0, infinity> and you'd get dynamic checks. Unless you use the "null policy", in which case you don't need this library at all.

There is definitely a case to be made for a system that tries to trap overflows but also tries to use static analysis to be smart about where to put the dynamic checks. But I don't think this system is that.

</comment><comment><USER26> Reminds me of Pascal having dependent types like

        var
          age: 18 ... 130;

</comment><comment><USER27> what about support for older gcc (say 6), older clang (say 5) and old MSVC (say 2015/17)?

<comment><USER24> Only very old versions of the library are supported by those older compilers. My library is targeting the C++20 standard, so currently the only compile version that can compile it is clang with the concepts branch.

</comment></comment><comment><USER28> Ouch</comment>


The day he was arrested, Anonymous's "Sabu" helped the FBI - <AUTHOR> http://arstechnica.com/tech-policy/news/2012/05/literally-the-day-of-his-arrest-hacker-sabu-helped-the-fbi.ars <comment><USER3> I read elsewhere that he is the sole guardian of the two daughters of his aunt who's in jail. They're aged 6 & 10, IIRC. The kids have no one else and they would be shipped off to state custody if he ever goes away. Under those circumstances, I understand him wanting to go home before they come back from school.

<comment><USER4> Except that he was willing to break the law in the first place, including using home computers where those girls live.

There are other explanations that would cover both his initial Anonymous activities and his rapid cooperation with the police, like enjoying the attention and power that comes with getting one over on other people.

<comment><USER5> People don't really think of the consequences until shit gets real. His reaction is totally understandable. I know a lot of us would try to be tough and claim we wouldn't cooperate with the Feds and sellout but when the FBI busts in and you're in cuffs lets see how tough we all are. I know I'd take a deal in a second.

In the same way that trolling forums posting racist, awful things is easy from behind a computer screen the same is true for "cybercrime". I once committed a pretty serious felony and it was easy to do because it wasn't a violent crime and it didn't feel like theft. It felt like a victimless crime. Crimes that seem victimless are really easy to commit. Anyone who meets me cannot believe I'm anything but an upstanding respectable citizen but I also didn't think of consequences before I did what I did. So I can understand how he could not think of his kids before he started hacking. When I got caught I took a deal too. It's something you just can't understand until you've been there.

<comment><USER3> Some crimes I would commit in a heart beat, out of principle, because laws are sometime white and it's moral obligation to disobey them.

<comment><USER3> Android aut-corrected "shite" to "white". I swear ;-)

<comment><USER5> Haha I believe you. But you would really disobey some laws on principle? Which type? I can see people smoking pot and saying its on principle. But in a lot of cases it seems more like people don't break such laws on principle but simply because they want to do something that's illegal. For example take pot smoking and pirating media. I've done both k owing they were illegal. It was easy for me to break those laws because they seemed victimless and the likelihood of getting caught and the way I perceived the seriousness of the crimes to be minimal let me ignore the potential consequences. If I were to get caught I wouldn't protest to the police and judge and try to make a stand. I'd pretty much say "fuck. I shouldn't have done that" and try to get out of trouble. Now, I'm not saying this describes you but it seems like the whole "on principle" argument is an excuse for many. Most people don't smoke weed because they want to protest the law. They just want to get baked. Most people aren't pirating media because of their stance on copyright. They just want the new Justin Beiber single. There are laws that need to be broken in protest then there are first world laws that are conflated and compared to fights for freedom but are really just petty. Again, this is not directed at you specifically but I've heard this from others for whom it would apply - <USER2> individual breaking a law in protest does not help matters. Even if there massive amounts of people doing the same thing it still makes no impact. You have to unify, organize, and get active to truly change something that's wrong in the world. Getting a bunch of people together to smoke pot or pirate the new Dark Knight film in protest is nothing more than a self-important circle jerk where everyone pats each other on the back for protesting. Getting politically active, lobbying representatives (with or without money), getting out the vote, marching in the streets, educating the people who aren't aware of the issues are all far superior to creating a Twitter hash tag or getting together in an online forum. You need to get the attention of the uninformed, the opposition, and those with the power to execute any plans for change. Otherwise it just looks like a circle jerk. I would point to the protests against SOPA, PIPA, and CISPA as examples of the right way to protest. No one was breaking any laws. They just got the word out, made their case, and lobbied like hell to get those shut down. In addition the protests weren't exclusively online. Personally, I would've liked to see more rallies and physical protests but I wasn't involved so I can't say much. On the flip side, the "right way" can be equally ineffective. I'd point to the early days of the Occupy movement. There was a point in the Occupy protests where they just looked like a bunch of disorganized hippies out protesting that life isn't fair. They had a complaint but offered no clear solutions that they could rally behind and push for. They were out there basically screaming "the economy sucks! Someone please fix it!". These days it seems they have a more concrete plan and not just a list of complaints.

Sorry to rant. This wasn't directed towards you specifically. It's just that what you said reminded me of some people I know who like to "protest" but are all talk and full of BS.

<comment><USER3> I would happily shelter an "illegal" immigrant.

</comment></comment></comment></comment></comment></comment></comment><comment><USER6> Truly, there is no honor among thieves.

This surprises me, actually: he was willing to invest a ton of time and effort in this, but rolls over at the first sign of trouble? It's not like it's surprising that he eventually got caught...

<comment><USER7> And it's not surprising that once he was caught, the FBI could easily leverage his two daughters to terror him into cooperating?

<comment><USER8> It is surprising that as sole guardian of the girls, he would run the risk of them going into the state foster system just so he could get (illegal) lolz online. That's very irresponsible and I have no sympathy for him.

<comment><USER9> I agree, but it seems like the feds would have wanted to keep his status as a CI under wraps. I question why this is being published.

<comment><USER10> The publication weakens the whole movement. "See, they don't really care about those things they say they do...and neither should you. Look at how weak they are for cooperating with us!"

It supports law enforcement's narrative that Anonymous is a group of criminals with no honor or allegiance to one another.</comment></comment></comment></comment></comment>


Roll Your Own Bitcoin Exchange in Haskell - <AUTHOR> http://www.michaelburge.us/2017/08/31/roll-your-own-bitcoin-exchange.html <comment><USER2> A bit OT:

There's a project trying to build a protocol to trade tokens. Essentially allowing anyone to build their own exchange: [https://0xproject.com/](https://0xproject.com/)

There's also a JS library to interact with the protocol: [https://github.com/0xProject/0x.js](https://github.com/0xProject/0x.js)

<comment><USER3> Yeah I think this is being built with it:

[https://medium.com/@the0cean/announcing- the-0cean-85ae78e9bc...](https://medium.com/@the0cean/announcing- the-0cean-85ae78e9bc58)

</comment></comment><comment><USER4> It appears to me that the trouble with building an exchange isn't the software, but the securities regulations involved in running one.

Woe to they who attempt to do so without counsel.

<comment><USER5> Absolutely, speaking as someone who built an exchange and spent a lot of time trying to figure out a legal route to operate it (I coded, my co-founder was a lawyer).

A friend of mine built a very similar product and opened his doors while we were still hesitant, and this happened: [http://www.cftc.gov/PressRoom/PressReleases/pr7231-15](http://www.cftc.gov/PressRoom/PressReleases/pr7231-15)

<comment><USER6> Can't you incorporate in an off-shore jurisdiction?

<comment><USER7> That strategy didn't work for Liberty Reserve, which was in Costa Rica. If Uncle Sam doesn't like what you're doing, he'll find you wherever you are and get you somehow.

[https://en.wikipedia.org/wiki/Liberty_Reserve](https://en.wikipedia.org/wiki/Liberty_Reserve)

</comment></comment></comment></comment><comment><USER8>
        We won't work with actual bitcoins or wallets, since they add a lot of complexity and risk without making the article any more interesting.

Actually that is the most interesting part -- how to manage re-orgs & forks is an open question.</comment>


Ask HN: When should the company ever consider changing programming languages? - <AUTHOR> It seems like 99% of the time you shouldn't do it.  Seems like articles are only written about success stories though.  Do you have any anecdotes where switching a programming language was a bad idea? <comment><USER2> From Python to Go and Back Again [https://news.ycombinator.com/item?id=10402307](https://news.ycombinator.com/item?id=10402307)

<comment><AUTHOR> Thanks a lot!</comment></comment>