Is Filecoin a $257 million Ponzi scheme? [pdf] - <AUTHOR> https://github.com/backender/filecoin-survey/releases/download/0.5/filecoin_survey.2.pdf <comment><USER2> I think the top comment for this should be about Filecoin, instead of just another uninformed comments about Bitcoin that we see here every day.

That said, one thing most people don't talk about is: I think most people think IPFS is disruptive, but I think IPFS itself is susceptible to disruption.

The main reason why IPFS is useful is because there's not an easy way to do NAT traversal therefore it's super hard for people to run their own server on their laptop or mobile devices.

IPFS introduces all kinds of technologies to get around this, but in my opinion the ability to freely set up your own server on your device is the core problem.

You don't really need a globally addressable immutable file storage, because unless you're dealing with static images, a lot of files DO change all the time, and people want to store and share files privately.

Which means, if there's a new type of technology that lets people do just that --set up their own server anywhere and make it accessible via HTTP--then I don't see why we really need IPFS. This could be done by startups or other protocols, but even by some innovative ISPs, which decide to change their business model to compete against edge nodes that capture most of the value (such as Google, Facebook, etc.)

In my opinion, the reason IPFS is so hot nowadays is because it's riding the DApps wave where people want to build Ethereum apps with IPFS as a file storage. But after playing around with it a bit I think it's much better at this point to just use Github to host your file and use Ethereum as database because the whole point of immutable apps is in the data and transactions, not in the static files.

But I would love to be corrected. If anyone actually think IPFS is essential to DApps, feel free to correct me and educate me.

<comment><USER3> > Which means, if there's a new type of technology that lets people do just > that--set up their own server anywhere and make it accessible via HTTP--then > I don't see why we really need IPFS

I have a ton to say here, but I'll keep it to a few point. Tor onion services solve the setup-your-own-server issues, it's just not super easy yet. IPFS is needed regardless, because there is a need to distribute things when your machine isn't up and to solve discovery. What IPFS doesn't have natively is anonymity (tho there are onion transports conforming to libp2p transport iface) and distributed compute, but neither is that important.

<comment><USER4> > Tor onion services solve the setup-your-own-server issues, it's just not > super easy yet.

Lately I've been trying to make this easier. I developed ormesh ([https://github.com/cmars/ormesh](https://github.com/cmars/ormesh)) in an attempt to simplify setting up hidden services for private use.

I think both Tor and IPFS are both awesome and useful decentralization projects, they just have different use cases & characteristics. AIUI IPFS lacks fine-grained access controls. Last time I looked, it seemed like I either had to run all my own infrastructure of IPFS nodes with pre-shared private keys, or open everything to the public and encrypt on top of that... I prefer the Tor hidden service model, where I can decide on a per-service basis how I want to share it, revoke client access if necessary, and have my traffic securely routed on existing infrastructure.

Edit: fixed link

<comment><USER3> Saw this on /r/golang (I was the one asking why the full Tor browser). I have considered similar [0]. A simple service (or portable exe) that has Tor statically linked and lets you reach/manage a machine from anywhere (think webmin-like web console on onion service) would have real value. Of course it can be authenticated and have the option to open direct connection (webrtc?) if you need fast media playing or direct download.

0 - [https://github.com/cretz/software- ideas/issues/54](https://github.com/cretz/software-ideas/issues/54)

</comment></comment></comment></comment><comment><USER5> [Author here] short answer taken from our .PDF file:

"Considering that Dropbox [53] currently holds around 500 petabytes of user data [54], one could argue that Filecoin is overvalued."

Study we conducted with a master student at Delft University of Technology. Open lab notes when writing this paper: [https://github.com/Tribler/tribler/issues/3097](https://github.com/Tribler/tribler/issues/3097) Note TUDelft has currently 8 professors in their [http://blockchain- lab.org](http://blockchain-lab.org) Various scientists and 38 master students working on improving our own Filecoin-like system, based on our Trustchain fabric: [https://github.com/Tribler/tribler/issues?q=is%3Aissue+is%3A...](https://github.com/Tribler/tribler/issues?q=is%3Aissue+is%3Aopen+label%3A%22MSc+course+work%22) It's home to one of the largest blockchain labs of Europe.

<comment><USER6> Dropbox feels like the wrong thing to compare it to. IPFS isn't really for storing files privately. Public S3 buckets would be closer. I don't have any numbers to say if it's overvalued based on how much public cloud storage there is (I wouldn't be surprised if it were though).

<comment><USER7> Storage costs are hyper-deflationary. We have a NAS on a desk with 27 terabytes of storage and it cost under $1000 (with drives) and consumes about 40 watts of power. Ten years ago this would have cost closer to $5000-$10000 and consumed hundreds of watts of power. We bought it about a year and a half ago and already we could replace it for less than $500.

The idea of distributed storage at the edge is interesting but it's not something that would cost $250 million to build. We could probably build a friendly system for doing this for under $500k in developer time.

It's also a mystery how efforts like MaidSafe or whatever it's called this week have never managed to ship anything after getting multiple multi-million- dollar golden showers. Distributed storage of immutable content-addressable blobs is not _that_ hard of a problem. BitTorrent kind of solved it long ago, albeit with a different UI/UX metaphor and use case. Just take those ideas and pivot them a little and wrap them around the S3 UI/UX metaphor and add payment channels or some other accounting mechanism and you're done. You don't necessarily need the sexiest tech. In fact sexy tech probably makes it more fragile and harder to use. DHT + hashing + RAID-type redundancy techniques + wallets will do it.

Everyone over-thinks and over-engineers things these days and it drives me crazy. Stop it. Intelligence is for the things you _can 't_ do, not for doing the things you can already do in more baroque ways. Intelligence is also for simplification. Complexification is stupidity.

<comment><USER8> > we could replace it for less than $500

27 Terabytes of spinning rust with NAS hardware for $500? Not yet.

<comment><USER9> The sweet spot for drives is the 4Tb at about $100[1].

Seven of those will give you 28 raw Tb with $300 left over for case and power supply.

So not impossible.

[1] [https://www.backblaze.com/blog/hard-drive-cost-per- gigabyte/](https://www.backblaze.com/blog/hard-drive-cost-per-gigabyte/)

</comment></comment></comment></comment></comment><comment><USER10> Heh.. Yes, probably is now.

I dont know if the founders started out to make a Ponzi, but once the dollars started rolling in, I can imagine, they probably got "different ideas".

Who am I to judge, but if I was a founder of this, I would be driving my Ferrari in Thailand instead of slaving away trying to build some stupid file system. Who the hell cares about file systems :P

<comment><USER11> Driving ferraris gets old, I'd much prefer to be inventing decentralized file storage.

<comment><USER10> I agree, but those cars and the attention is just so tempting. And then in Thailand. Imagine kite surfing, drinks, anything you want for small cash. Hell, you could even contract someone to write that damn file system people are whining about.

</comment></comment></comment><comment><USER12> Filecoin is a huge disappointment to me.

IPFS is fantastic technology, but the main developers getting sidetracked with yet another scammy ICO is the last thing we need.

<comment><USER13> Hey, ipfs dev here. The amount of work being done on ipfs has been increasing continually. Work on filecoin is not detracting from ipfs development, and we are bringing on more people for ipfs, libp2p _and_ filecoin.

<comment><USER14> That doesn't discount the fact you are involved in yet another ICO. Cryptocurency integration and identity within IPFS could have been done without a new coin. I know you know this, yet you choose to do it anyway. That means we should question your intentions, across all your projects.

And, just for the record, a cryptocurrency fund raise is simply selling access to given private/public key pairs which contain values, like a database. Even hardened by mining, these values and keys are intrinsically worthless until they are used widely for something valuable. It's when convincing someone it's "valuable because you think it is" that it gets "tricky".

<comment><USER15> I don't get all the hatred for ICOs. Nobody is forcing anybody to invest in them, and for something like filecoin there is at least a chance of something extremely useful coming out of it. Many other ICOs seem pretty strange to me, but as long as I am not forced to invest in them, what's the problem?

<comment><USER16> Can't say so if it is a Ponzi scheme.

</comment></comment></comment></comment></comment><comment><USER17> _edit: Weird, I thought I was commenting on another thread, about Nobel laureates criticizing Bitcoin. I saw it on my phone, then opened HN at my notebook to comment. I misread the title and commented here, but now I can 't find the correct thread (was it flagged?)._

I am growing this idea that Bitcoin (and any cryptocurrency that is affected by aggressive price growth) is an anti-fragile indirect multilevel marketing scheme, the "diamond" of the digital era.

It is anti-fragile because it does not have a single point of failure, it is decentralized, its creators are unknown, it is adaptable to several use cases. So each attack on Bitcoin (be it exchange hacking, scams or celebrities' critics) that doesn't kill it, make it stronger.

It is a multilevel marketing scheme because it is mostly a zero-sum game, all earnings of bitcoin owners come from new money entering the game. It is indirect because the ones that are most fervourous advertising and defending bitcoin are holding it, not selling.

I believe it is the diamond of the new era because of some key similarities:

Both BTC and Diamond:

\- Its original bump in value come from a marketing campaign (even if with Diamond it was centralized and offline, and with BTC is decentralized and viral)

\- Its scarcity is artificially controlled

\- Its main use case is black market commerce (for one side of the argument) or money laundry (for the other side).

For BTC owners the good news is that the "diamond bubble" it is still strong decades (centuries?) after its modern boom. The bad news is that Diamond was able to correlate to status, a strong (if subjective) value that is part of humanity since always. BTC only appeal is its value as money, so bubbly reinforcements (up and down) apply.

<comment><USER18> _The bad news is that Diamond was able to correlate to status, a strong (if subjective) value that is part of humanity since always. BTC only appeal is its value as money ..._

Startup idea: Bitcoin wedding rings.

A beautiful gold ring with a private key engraved on the inside. Give your loved one the gift of BTC! Your commitment to each other, eternalized on the global blockchain.

Gentlemen may prefer blondes, but the modern woman prefers Bitcoin to carats of squeezed carbon. Status, store of value, it's all there.

(This is satire, but now I'm worried it will actually happen if the coinmania doesn't subside soon.)

<comment><USER19> I think that jewelry should have a property of showing off, so hidden private key wouldn't work that well. You need to have some mechanism how to show status. And that nicety factor was driven mostly by women decorum.

<comment><USER18> That's just a question of recognizable design and enough marketing spend, i.e. branding.

It's well known that people pay a major premium for things solely because they have words printed on them like "Calvin Klein", "Guess?" or "YSL". Jewelry branding is actually easier to make visually recognizable than clothes.

<comment><USER20> People don't pay a premium for the label, they pay a premium for the middlemen who host the label, because that is a presumably exclusive group and culture that they have access to.

</comment></comment></comment></comment></comment><comment><USER21> tl;dr conclusion from the article: "Filecoin's economic feasibility is hard to predict and, given a simple summary of risk related points, the biggest hurdles are probably going to be acceptance from the users. In addition, Filecoin shows certain characteristics of a Ponzi scheme but the trust built by the team members in the past leads us to believe that it is not one. Filecoin's Whitepaper introduces novel concepts and predecessor projects by the team members which prove their technical capabilities. Hence, it appears that Filecoin is poised to be an outstanding project although it remains to be seen if it will be adopted by the average cloud storage user."

</comment><comment><USER22> It's pretty frustrating to see something that has an easily accessible value (M MBs stored for N seconds consuming X bandwidth) is denominated in the same way that all the other cryptocurrencies are denominated in, as a deflationary good that encourages speculation. If you want to create some token that scale with the value of the system, that's fine, but don't force your users to price their contacts in something that is untethered from the thing they are trying to buy.

What I'd do instead is allow people to store random data to mint new MBseconds at a fractional rate and then the cost of a new contract is trivial. If you want someone to store 1GB for a day the cost of that is well known. The price of storage constantly drops, but the price of a deflationary good that is actually desired will continue to rise. This encurages contact churn.

If you want to have a token that scales with the value of the network/product then just create tokens that generate MBseconds proportional to the fees paid.

</comment><comment><USER23> No, it's a buy the dip scheme.

[https://www.youtube.com/watch?v=0akBdQa55b4](https://www.youtube.com/watch?v=0akBdQa55b4)

</comment><comment><USER24> I'm so lost, there are so many conversations here talking about IPFS compared to things like Dropbox and S3. Why?

To me, IPFS aims to solve a _completely_ different problem. IPFS does not exist because we don't have means to host and distribute files - that is a problem solved by a hundred different entities. IPFS, as advertised in many mediums (video/etc), is a platform to distribute data from a local-first P2P medium. Like Bittorrent, but without ever leaving the local network/etc. Eg, transferring a file/video/site to someone next to you would not download the data from across the world _twice_.

Has the marketing changed? Because all of the videos I watched on it from months ago were all about reducing global bandwidth usage, utilizing local and distributed sources of the data as much possible.

Why is everyone acting like IPFS is trying to solve file hosting?

<comment><USER25> Filecoin is trying to solve (paying for) file hosting through IPFS. Many of the existing projects around IPFS are "host X, but on IPFS".

<comment><USER26> No, it's more generally about hosting _any_ content-addressed data. That's much much more than files/directories, as S3/Dropbox/Sia are doing.

Files and directories are just one particular data structure you can store on IPFS. Think of it as a global unified content-addressed graph database really, and Filecoin can store any arbitrary (sub)graphs within that.

<comment><USER24> > No, it's more generally about hosting any content-addressed data.

Back to my original points, I don't think it _is_ about _hosting_ the data. It's the _location_ of the data. Eg, a room full of people downloading a large video don't need to leave the room for the majority of the data, they can share the data between each other - seamlessly. Leading to less clogging of the network of data when there's absolutely no need for us to be downloading from youtube/etc.

For me personally, another thing IPFS does is allow access to immutable data. Seeing as websites love to change data on the fly, it's hard to know what someone else saw, compared to what you saw. Was the article modified? Did X political candidate change what s/he's saying on their site? IPFS solves this, and this is my favorite part, tbh.

<comment><USER26> > Back to my original points, I don't think it is about hosting the data. It's > the location of the data. Eg, a room full of people downloading a large > video don't need to leave the room for the majority of the data, they can > share the data between each other - seamlessly. Leading to less clogging of > the network of data when there's absolutely no need for us to be downloading > from youtube/etc.

You're absolutely right here of course

<comment><USER24> Didn't mean to imply you meant otherwise of course - nor was my intent to argue. I just thought it was an important point for anyone _(you or otherwise)_ to read. I thought it came off a little ... wrong, so my apologies for that, if anything was perceived.

<comment><USER26> Not at all :) There's multiple important points about IPFS and I always forget at least one, so thanks :)

</comment></comment></comment></comment></comment></comment></comment><comment><USER27> How is Filecoin fundamentally any different from Bitcoin?

      Filecoin gave a deal to their buddies
      Bitcoin and Ethereum were premined

Both are variations on a Ponzi scheme where early 'investors' benefit from latter day victims.

</comment><comment><USER28> Anyone interested in reading the Filecoin offering info you can find it on the CoinList.co site

[https://coinlist.co/filecoin](https://coinlist.co/filecoin)

</comment><comment><USER29> From the paper:

> a total of $257 million – so far the biggest initial coin offering (ICO) as > of today (September 2017).

This appears incorrect, as the Sales Summary on [https://eosscan.io/](https://eosscan.io/) shows EOS having raised over $400 million by September (first 68 days).

</comment><comment><USER30> I think it would be helpful if the authors disclosed whether they invested in Filecoin. I didn't really see it anywhere. Did I miss it?

<comment><AUTHOR> Author here: we did not invest; nor are we invested in any of the competitor projects mentioned in the article.

<comment><USER5> other author here, no ICO investments. We're actually working for many years on a similar system, non-profit non-ICO based. We know this scientific area a bit (see my other comment).

<comment><USER31> >>We're actually working for many years on a similar system... Hmm, this is what you neglected to mention in the paper

</comment></comment></comment></comment><comment><USER32> Good discussion on the technical merits. However, let's take a look at the 'overvalued' claim and a critical error the authors make.

The analysis rests on this ratio calculated for Airbnb: "Airbnb is valued today at approximately $31 billion while holding around 3 million listings in total. The average apartment in the United States was 934 square feet in 2016. In a hypothetical scenario, Airbnb is therefore valued at $11.06 per square feet. If one compares this number to the median price per square foot in the United States, which is $123, the Airbnb ecosystem diminishes the median price by a factor of 11.12."

I get this is back of the envelope so let's ignore the international dimension. The critical error in these calculations is the apples-to-apples comparison of the square feet of an Airbnb listing to a purchased square foot.

Basically - when someone pays $123 per square foot (median price in the U.S.) they are paying for the use of that space for an entire year. Said another way, they are paying $0.34 per day per square foot. Airbnb listings are not in general year-long, which means the same square foot of space is rented out over and over again throughout the year, adding revenues along the way. The 11.12 scaling factor is therefore off and too high. This leads the authors to apply an erroneous scaling factor to the Dropbox sizing to come up with an invalid valuation assessment for Filecoin.

It looks like a good average stay for an Airbnb property is about 5 days ([https://blog.atairbnb.com/economic-impact- airbnb/](https://blog.atairbnb.com/economic-impact-airbnb/)). Let's also assume 50% occupancy rate ([https://www.mashvisor.com/blog/what-airbnb- occupancy-rate-ca...](https://www.mashvisor.com/blog/what-airbnb-occupancy- rate-can-you-expect/)).

That means an Airbnb listing is rented out about 37 times a year (365 / 5 days * 50%). The $11.06 per square foot times 37 revenue-generating occasions is $409.22 per square foot ($1.12/day/sq ft v. $0.34/day/sq ft for a purchase). So the correct apples-to-apples comparison here is to say that Airbnb _increases_ the median price of the asset by a factor of 3.32. Aka Filecoin would need to deliver 2,580 petabytes - not 95,185. That's only 5x Dropbox, which is a minnow compared to S3 etc.

Why would that be? Most directly, because Airbnb allows the owners of the asset (sq ft in this case) to ask for a higher per day charge since the length of stay is shorter and thus the owner of the asset has more uncertainty over utilization - it's part of why 30 nights in a hotel costs much more than a month of rent. This is a well-understood economic dynamic and one that would apply to Filecoin storage miners given how miners using Filecoin could theoretically 'rent out' the same gigabyte over and over again to different buyers throughout the year - but also cover the risk of less than 100% annual utilization rates of their fixed-cost asset.

Honestly, reading this article made me more confident in the future of Filecoin. The authors highlight multiple times the better technical features than competition, proven dev team, and unwittingly call out why economically speaking owners of gigabytes under Filecoin could expect a higher return than if you were going to buy the gigabytes yourself for your own use.

Thoughts?

</comment><comment><USER33> Where did the funding for this research paper come from, specifically?

<comment><USER34> It seems like the first author is still a student. So this could be a BSc./MSc. project, for example.

<comment><USER35> [https://github.com/backender/filecoin- survey/releases](https://github.com/backender/filecoin-survey/releases)

Seeing as the release is named "Release to turn in", I'd say so too.

<comment><USER34> I think it would be fair if the title is changed from

"... by Delft University"

to

"... by student of Delft University".

Or just drop it altogether and shorten the title.

<comment><USER5> Please don't alter our title. I'm the second author and tenured staff member of TUDelft, associate professor.

</comment></comment></comment></comment></comment><comment><USER36> y'all are forgetting that juan is a genius and still alive...

</comment><comment><USER37> Hey HN, this is jbenet -- an author of Filecoin.

We think it's great that people ask hard questions, and get involved. It's great to see others studying our work and we really appreciate the open discourse. There are a few things from this article I'd like to address. (Despite the length of this post...) these are quick comments, and not a proper in-depth response.

\- (a) The article gets some things right and some things wrong -- there is good summarizing of several of our projects, and discussion of many difficult aspects in these projects. The article discusses many technological aspects in good depth, and highlights difficulties in building these systems, aligning incentives, and the trials of past projects. The article also has significant inaccuracies. For example, the sale figure -- which appears in the title and impacts the analysis -- is incorrect. We raised $205M -- officially here: [https://protocol.ai/blog/filecoin-sale- completed/](https://protocol.ai/blog/filecoin-sale-completed/)

\- (b) The authors chose a provocative title. As some commenters have already pointed out, the conclusion is "[we] believe that it is not one." Despite Betteridge's law ( [https://en.wikipedia.org/wiki/Betteridge's_law_of_headlines](https://en.wikipedia.org/wiki/Betteridge's_law_of_headlines) ), many people who only read the headline will come to the opposite conclusion, and now we (not they) will have the burden of correcting those misunderstandings. Provocative titles, though they may drive imagination and clicks, can do a huge disservice to everyone in the space, and contribute to misinformation. Most people will only read the title, maybe the abstract, and use that to form and drive opinions. We choose titles of our research with diligence and care, and hope others do the same.

\- (c) The article has a great technical overview of the Filecoin stack, and how it fits with IPFS and libp2p. This is a large structure with many pieces, and it is rare to see articles grasping how all the pieces fit together so well, and then explaining it cogently. In particular, it's great to see this article diving deep and discussing advantages and disadvantages of low level technical structures (multihash, ipld, libp2p, and more). We modularized everything in the hope to generally improve peer-to-peer systems, and improve reusability. We hope these components will be useful to the author's Tribler project (a network similar in goals to Filecoin), and we hope that we can also learn from and leverage solutions they have made.

\- (d) many of the objectionable things described in this article are common in ICOs in general. Put another way, consider those claims also in terms of other significant token sales, such as Ethereum, Tezos, Polkadot, Blockstack, Cosmos, Golem. People were saying similar things about Ethereum when they did their sale in 2014. Perhaps worth doing a survey / analysis over all of them, comparing and contrasting the different things groups have done, how the ecosystem has improved, and suggest new directions.

\- (e) It's worth mentioning that the analysis gives a definition of a ponzi scheme, but their discussion does not map to that definition. Instead, the discussion centers on claims about future investor sentiment or speculation as the driver of value in the token, which is not the only way to establish value in token networks, and ignores the value of the services provided. That kind of analysis does not work for projects like Ethereum and other live and functioning crypto tokens. If the network is __useful __, and there is a way to __generate __or __introduce __value, by providing new or better services, and if the network can capture that value in the token itself (important step), then the tokens can hold value, based on the utility of the network as a service and not just or primarily speculation. Networks like Ethereum, Bitcoin, Zcash, and Filecoin aim to provide useful services, and much of the value stored in their tokens will be thanks to the utility of the networks.

Perhaps it's worth pointing out that most crypto token projects are compared to ponzi schemes at some point :(

      - http://www.google.com/search?q=is+bitcoin+a+ponzi+scheme
      - http://www.google.com/search?q=is+ethereum+a+ponzi+scheme
      - http://www.google.com/search?q=is+zcash+a+ponzi+scheme
      - http://www.google.com/search?q=is+tezos+a+ponzi+scheme

\- (f) The article discusses the SAFT and assurances to investors, but does not discuss them in contexts of other token sales and ICOs. Most ICOs are structured as __donations __(not investments) to a project, with little to no legal recourse -- even though many people refer to these "donations" as being "investments". In our case, we raised investment through an instrument (the SAFT) that is a direct liability to us, and gives investors greater guarantees on the completion of the project, or consequences otherwise. If we fail to deliver the network, we must return the proceeds of the token sale. Few token sales ever have such a clause. Our structure gives investors greater accountability, not less. The article discusses this in sec IV, but does not take into account that startups are similarly risky (i.e. that startup dissolution events return only remaining capital from the efforts), and does not mention how our structure improves on the ICO landscape in general.

\- (g) We do share the legitimate concern that ICOs need stronger accountability, and some are structured in a way that leads to abuse. The community as a whole needs to raise the bar on accountability and ethical behavior. We have taken significant steps in this direction, not just in our sale but to improve the ecosystem -- the SAFT project, which was a gargantuan undertaking that many other networks are now using, is one example. Many other networks are introducing and improving structures. We believe token networks present a very important new way to form capital, with promising advantages to users, investors, and creators, but the space is still in its infancy, and significant changes are still ahead. Token sales have improved dramatically in the last three years, and we hope they continue to improve to find the right balance and protection of the interests of all parties involved with the network.

Thanks, Juan

</comment><comment><USER38> Filecoin is one of the ones I'd stay far away from. VCs got in before anyone else with special preferences, which taints the whole project. There are alternatives which don't come with the VC problem attached to them.

<comment><USER39> What is your recommended alternative? Is there any unusual involvement beyond participation as a user that you should probably disclose?

<comment><USER38> Sia is more legitimate. I don't own any Siacoin or Filecoin, but I would hold Siacoin if it was supported by the Ledger hardware wallet.

<comment><USER40> [https://sia.tech/funding2016/](https://sia.tech/funding2016/) ? How is this any different? I realize that this doesn't concern an ICO, but still there is plenty of VC involvement to "taint" the project.

<comment><USER38> Sia didn't do an ICO. Every coin has been mined, with no premine (although I assume the developers mined the first few blocks at least). The investors could have mined too, if they wanted, along with anyone else. This seems more egalitarian to me.

</comment></comment></comment></comment></comment><comment><USER41> Are they not all (bitcoin, eth etc.) ponzi schemes?

<comment><USER42> You're aware that a Ponzi has a _very_ specific definition, more specific than "scam," correct?

Neither Bitcoin nor Ethereum have a central operator generating returns for older investors by disproportionately funneling revenue paid by new investors to them. When the price rises, new and old investors profit equally. You could use BTC as a currency _in_ a Ponzi, Bitcoin _being_ a Ponzi would be impossible by definition.

</comment></comment><comment><USER43> All XXXcoin currencies are Ponzi schemes, in effect. Their "growth" is fueled almost entirely by new investors dumping cash into the system from which no one takes (can take[1]) anything out.

That's a pyramid. Eventually when the investment money runs out[2], the support will fall over. The only distinction between this and a Ponzi scheme is that there are (probably) no single fraudsters at the top ready to run off with the money. It's an unintentional Ponzi scheme.

Coin boosters simply don't get this: there's not enough money in the world to keep bitcoin (and all the copycat coins) growing forever. _What is your plan for when the investment money runs out?_. There is no plan, because you can't get your money out.

[1] You simply can't dump a large bitcoin holding. There's not nearly enough liquidity to do that in reasonable time.

[2] And it will.

<comment><USER44> What a load of BS. Of course you can sell off a large Bitcoin holding. People do it all the time. That you think otherwise is ignorance and bias.

<comment><USER45> You can sell as long as people are buying.

<comment><USER46> That also applies to shares, real estate, apples, Teslas, ...

<comment><USER45> Yes. I was just stating the obvious, which appeared to be not so obvious to the commenter.

<comment><USER46> Not sure we should bash that comment. If large means 1 million dollars it isn't hard to find liquidity to sell 100BTC.</comment></comment></comment></comment></comment></comment>


Facebook expanding its Canadian operations to RIM's home town - Waterloo - <AUTHOR> http://www.techvibes.com/blog/facebook-expanding-its-canadian-operations-to-waterloo-2011-11-26

<comment><USER2> This isn't confirmed yet: [http://www.therecord.com/news/local/article/630801 --facebook...](http://www.therecord.com/news/local/article/630801--facebook- reportedly-planning-major-presence-in-kitchener)

> "... but nobody at the city, Facebook, the company redeveloping the > buildings or the high-tech community could confirm it."

Besides, RIM has very little to do with it. Though keeping an eye on the growing cluster of startups and competing with Google for UWaterloo's co-op / recent-grads talent makes Kitchener/Waterloo an excellent position for Facebook.</comment>


Let's turn censorship into sensorship - <AUTHOR> https://medium.com/i-m-h-o/f2fb0aea0bee

<comment><USER2> Somehow, the author's self description containing "I'm a sucker for persuasion. You can talk me into anything." severely undermines the point he is trying to make in this post.</comment>


How do I find tech/startup events in Boston? - <AUTHOR>

I'm looking for something to do in Boston in November. Any cool events there? <comment><USER2> <http://www.marksguide.com> is a great resource. Also:

<http://dorkbotboston.com/upcoming-events/>

<http://bostontweetup.com/calendar/>

<http://microsoftcambridge.com/Events/tabid/57/Default.aspx>

[http://upcoming.yahoo.com/search/?type=events&rt=1&r...](http://upcoming.yahoo.com/search/?type=events&rt=1&rollup=&q=&loc=Cambridge)

</comment><comment><USER3> Boston groups:

[http://www.meetup.com/cities/us/ma/boston/groups/business/?r...](http://www.meetup.com/cities/us/ma/boston/groups/business/?radius=25.0)

[http://www.meetup.com/cities/us/ma/boston/events/?radius=25....](http://www.meetup.com/cities/us/ma/boston/events/?radius=25.0)

</comment><comment><USER3> <http://www.meetup.com/></comment>


A Brief Introduction to Provisioning - <AUTHOR> https://stephenmann.io/post/a-brief-introduction-to-provisioning/ <comment><AUTHOR> This is the fourth post in a series about building production web applications. The first post in the series started with the high level motivation for doing this, and each subsequent post works its way down from highly abstract down towards more concrete implementation details. By the end of the series, I hope to have an overview of every aspect of building a modern, production web application.

Let me know what you think!</comment>


SOPA Protests Sway Congress: 31 Opponents Yesterday, 122 Now - <AUTHOR> http://techcrunch.com/2012/01/19/sopa-opponents-supporters/

<comment><USER2> Yesterday had some impact, but these bills are still very much on track to become law. All the post mortems today aren't communicating that very well.

The closure vote in the Senate on the 24th needs at least 41 no votes to prevent the bill from moving into markup. While more senators have announced opposition or doubt, current headcounts put "no" votes on closure only in the single digits.

Unlike subcommittee hearings, the markup hearings are closed door and allow the Senate to draft a final version of the bill without public scrutiny or testimony from tech leaders. If this legislation follows suite with other controversial legislation, the bill that emerges will have enough back room deals attached to let both the latecomer opposition and the White House to declare the bill a healthy compromise irregardless of the facts.

People need to be on the phone to their senators. Urge them to vote no on closure so that the inevitable compromise bill can be debated in public with input of experts.

<comment><USER3> Additionally, the lobbying groups aren't about to stop lobbying suddenly. It just means that they need to try harder. I wouldn't be shocked in the least to find some well funded senator trying to slide some of the language from this bill into a "must pass" 10,000 page budget bill.

What's scary enough is that some of this will get through in some way or another eventually.

<comment><USER4> So perhaps the best way to appease the lobbying companies would be to pass SOPA, but remove the most destructive pieces first. Perhaps narrowing the definition of an infringing or "rogue" site would be a good start.</comment></comment></comment>


People are spending much less time on social media apps - <AUTHOR> http://www.cnbc.com/2016/06/06/people-are-spending-much-less-time-on-social-media-apps-said-report.html <comment><USER2> So number of installs went up and average usage went down?

The data is incomplete and the graphs are poorly labeled, but this story might actually be telling us that total time spent on social media has gone _up_ quite drastically...</comment>


Show HN: isoc – yet another "standard" container format - <AUTHOR> https://gist.github.com/proppy/20ca5dec630aafc1d35c

<comment><USER2> Two questions:

\- Have you tried to influence both the Docker and APPC formats first?

\- Is it a personal project or something officially from Google (because of the copyright notice)?</comment>


The Accusations Were Lies. But Could We Prove It? - <AUTHOR> https://www.nytimes.com/2020/03/18/magazine/title-ix-sexual-harassment-accusations.html <comment><USER2> There's a more information-dense article at [https://reason.com/2020/03/18/title-ix-arizona-state- univers...](https://reason.com/2020/03/18/title-ix-arizona-state-university- viren-tecedor/)

</comment><comment><USER3> I feel very bad that this couple was falsely accused of sexual harassment under Title IX.

I'm sure almost all claims of sexual harassment under Title IX are valid, but as this article shows, not all are. I think the number of false accusations are extremely small, but not zero.

The article also shows the limited options of the people who are accused. In this case, the accuser made a few critical mistakes which allowed the innocent to be exonerated. If it weren't for those few pieces of evidence, the professional careers of both would have been over, with no recourse for appeal.

This article shows that Title IX, although written with good intentions, is tilted too far in favor of the accuser. Unfortunately, it does not allow the accused to defend themselves adequately.</comment>


Price comparison of the top four VPS providers at all levels between $5 and $80 - <AUTHOR> https://docs.google.com/spreadsheets/d/1LQ9voKCAklqYYD-H4XHV57QEpWisqpo93BHbqV2aupg/edit?usp=sharing <comment><AUTHOR> I made this to more easily compare VPS prices. You can create a filter view to sort or filter it to your own requirements.</comment>


Web developers: The Internet is trying to kill you - <AUTHOR> http://www.anchor.com.au/blog/2011/11/the-internet-is-trying-to-kill-you/

<comment><USER2> Indeed, networks are a hostile environment. I think most people who would be on HN understand the basics of programming for the public.

<comment><USER3> you'd think, right? :)

But you would be amazed by the amount of crazy stuff that we see here at Anchor from Web Developers who we think know better.

Obviously, it's not the same for every web developer, but reminders of just how volatile the Internet is in generally can never hurt.

<comment><USER2> I am guilty of considering HN members as being more educated than average, I admit.

</comment></comment></comment><comment><USER4> come on, if you're making a web app and you don't know about stopping injection, you should be an intern, not a developer.</comment>


'Murmurs of Earth: The Voyager Interstellar Record' (1978) - <AUTHOR> https://wearethemutants.com/2017/07/17/murmurs-of-earth-the-voyager-interstellar-record-1978/ <comment><USER2> For those interested in the multimedia content from the Voyager golden records, I created this interactive a few years ago.

[http://voyager.damninteresting.com/](http://voyager.damninteresting.com/)

Self-link, yes, but there are no ads or anything on it. I just decided to build it since I was unable to find a complete collection online (also, I had thought of a nifty way to make a spinning record effect, and I wanted to try it out).

</comment><comment><USER3> This is a nice bibliophile piece.</comment>


Show HN: Ruby queueing library built on Amazon SQS for background processing - <AUTHOR> https://github.com/Denwen/fifo

<comment><AUTHOR> List of features and uses from github:

Features

Built on Amazon's reliable and scalable queue service, Connections to SQS are opened lazily negating any initial load time, Multiple queues can be used simultaneously, Doesn't poll the database, Rails ActiveRecord objects maintain state through the queue, Built-in retry mechanism.

Uses

FIFO is extracted from the Mine (<http://getmine.com>) codebase. Here are some of the things we use it for:

Sending emails, Processing images, Indexing, Sharing to social networks, Cache management, Launching cron jobs, Communication between disjoint parts of a complex application (cron, web, processing servers)</comment>


Internet collaboration still in infancy: Wikipedia founder - <AUTHOR> http://www.breitbart.com/article.php?id=081101154919.8q6zayy9&show_article=1 <comment><USER2> I think Wikipedia is a successful collaborative site because there's much less room for people to argue over simple, declarative text than any other medium (though of course they do argue, all the time). The arguments that could be raised over the editing of a collaborative documentary are much less cut and dry.

Also, the density of something like video make it much harder to review changes. If someone moves around a few scenes and in the process splices in still frames of porn, how is anyone going to know without carefully viewing every frame of changed video? Short of someone creating a very sophisticated diff viewer for video, that would have to happen for every single change that's committed. It would become much harder to differentiate between real changes and blatant vandalism, which would inevitably drag down the quality of the videos produced.

</comment><comment><USER3> Surely rather the "other fields" of collaboration are things that orbit the blackhole known as the free market. (wait, can you actually orbit a blackhole...?)</comment>


Washington was on brink of war with North Korea 5 years ago (1999) - <AUTHOR> http://edition.cnn.com/US/9910/04/korea.brink/ <comment><USER2> Could be considered a "break down this wall" moment for great former President Jimmy Carter.</comment>


A Robot With a Little Humanity - <AUTHOR> http://bits.blogs.nytimes.com/2014/07/16/a-robot-with-a-little-humanity/ <comment><AUTHOR> Jibo's "expressive orb" color screen reminds me of the colors used by drones to express mood in Iain M. Bank's _Culture_ novels:

[http://zrajm.org/nerd/drone-colours/](http://zrajm.org/nerd/drone-colours/)</comment>


Firefox about:addons is using Google Analytics - <AUTHOR> https://twitter.com/NicolasPetton/status/884694176515936256 <comment><USER2> WTF! (also verified) You can block it with the uMatrix plugin ( [https://github.com/gorhill/uMatrix](https://github.com/gorhill/uMatrix) ) but you have to manually enable it for about:addons :/

</comment><comment><USER3> wtf mozilla. (verified here, open addons and click "get add-ons")

<comment><USER4> Same with Firefox 54.0.1 and uBlock Origin.</comment></comment>


MSR funding a 2-year project to mature Parallel Haskell for commercial users - <AUTHOR> http://blog.well-typed.com/2010/04/parallel-haskell-2-year-project-to-push-real-world-use/

<comment><USER2> Almost exactly 20 years ago, I started my grad-school research on this very topic: auto-parallelizing Haskell. Shortly, though, my advisor left and I had to choose a new advisor and a new research topic. It's been surreal to see Haskell suddenly (it seems) get so popular lately, and now to see a commercial push for the very thing I was going to work on is very cool. It's incredible that it's taken so long, but better late than never.

</comment><comment><USER3> I have a few friends who program largely in C++ or in interpreted languages like Python, Ruby, and Perl. They've scoffed at my love of Haskell, claiming it's too academic and overly complex and that they'd rather just get things done.

Just last week I asked them what piece of software was missing from their lives in an attempt to get ideas for a new project to start. One of the first things on all their lists was software that would automatically parallelize an algorithm for them. Inwardly I wanted to put palm to face because the easy parallelization in Haskell is one of the killer features I'd been trying to convince them of.

I think the major barrier to adoption is lack of real-world results to show people that's viable. All the benchmarks in the world won't convince these people to change their ways, but if they see other people using it to good effect, they will seriously consider it.

<comment><USER4> I wasn't aware Haskell had good multi-threaded support.

How is Haskell better at auto-parallelizing an algorithm compared to say, java's fork/join or .NET's equivalent?

<comment><USER5> Haskell's tools mostly have the same sort of power as forkjoin (scheduling work to be done on separate threads; creating the right number of threads based on the hardware; providing an easy-to-use set of functions that call for work to be done in parallel).

The major difference is that the tools compose with the language better. Instead of having to keep "I'm working in a multicore environment" in your head all the time when you're writing your client code (as you would in Java, worrying about concurrent mutation and so on), Haskell enforces immutability throughout the language. Often you get the extremely convenient situation where you observe that something could be parallelized, change _map_ to _pmap_ and you're now using all your cores.

<comment><USER6> Yes. But immutability is not the whole job. You have to make sure to use the right data structures. Linear data structures like lists don't parallelize well. You need to use something that breaks into more equal parts instead of car and cdr.

Compare Guy Steele's talk "Get rid of Cons" at the International Conference on Functional Programming 2009. For the slides go to <http://news.ycombinator.com/item?id=814632>, the video should also be available online somewhere.

</comment></comment></comment></comment><comment><USER7> This is really cool. Kudos, Microsoft.

<comment><USER8> This is Microsoft Research; I believe it's not really the Microsoft behind Windows and co.

<comment><USER9> > This is Microsoft Research; I believe it's not really the Microsoft behind > Windows and co.

I'm curious as to what you mean with this statement, given that Microsoft Research is a part of Microsoft.

<comment><USER8> Well, I just meant that 'is part' is different than 'is'. And that I think the freedom that enjoy MSR is not exactly related to the creation of future consumer software.

It is generally accepted to make a difference between, for instance, the marketing communication and the work quality accomplished by the engineers. I think it is as important to make the distinction between Microsoft and its Reserach part.

<comment><USER10> >And that I think the freedom that enjoy MSR is not exactly related to the creation of future consumer software.

Where do you think the funding for MSR comes from? Who do you think decides to allocate that funding?

</comment></comment></comment></comment></comment><comment><USER11> Great to see major investments in moving programming languages forward.</comment>


What (actually) is Web-Scale IT? - <AUTHOR>

I was just doing some research into this, but I can't really seem to nail down a solid explanation for what it actually is. Is this just another re-purposed buzzword thrown out by Gartner to spark people's interest, or is there actually some solid new challenges and technology coming out of it? Any information would be helpful. This is more for my own curiosity than anything else. <comment><USER2> When I read the term, I associate it with "designed to scale horizontally to arbitrarily large capacity". I always imagine datacenter-sized setups too.</comment>


The Flying Crowbar (1990) - <AUTHOR> http://www.merkle.com/pluto/pluto.html <comment><USER2> How's the new Russian version coming along?

<comment><USER3> they confirmed the uninhabitable if it crashes part.

</comment></comment><comment><USER4> Today I learned that Ralph Merkle's dad ran Project Pluto. I never knew about that connection.

</comment><comment><USER5> i remember reading this when it was published, as a child.

i had nightmares for three days.</comment>


Lies, Damned Lies, and Averages: Perc50, Perc95 Explained for Programmers - <AUTHOR> https://www.schneems.com/2020/03/17/lies-damned-lies-and-averages-perc50-perc95-explained-for-programmers/ <comment><AUTHOR> > Well, when it comes to performance - you can't use the average if you don't > know the distribution.

...and if you have the distribution, you no longer need the average!

Latency as experienced by the end user is dominated by the fat fail, for both technical reasons and psychological ones.

Technical ones are probably the most convincing: it is very rare to have users submit single requests and then be done. Especially in the cloudified, service-oriented stacks of today, even single requests lead to a cascade of requests inside the system. Whenever you have tens or hundreds of requests for a single user, it starts becoming very likely that they hit that fat tail at some point in their journey.

Given that latency is dominated by "outliers", looking at anything but p99 _and beyond_ is meaningless.

What's worse is this: since most people at best look at p95 or p99, they tend to optimise for "the common case" at the cost of tail latencies! They introduce insane variance in latencies making benchmarks better, but things actually get worse for real users.

Sorry, this is a pet peeve of mine.

</comment><comment><USER2> > Well, when it comes to performance - you can't use the average if you don't > know the distribution.

This is frankly wrong. Performance comes in multiple flavors. Latency is one of those, and there we know that percentiles really matter (see Andrew Certain's section of this talk: [https://www.youtube.com/watch?v=sKRdemSirDM&feature=youtu.be...](https://www.youtube.com/watch?v=sKRdemSirDM&feature=youtu.be&t=175) for Amazon's experience).

But for others, like throughput and scale, you don't need to know the distribution. In fact for throughput, the only thing that really matters is the long-term mean latency. For concurrency, it's that and long-term mean arrival rate. I wrote a blog post about it a while back ([http://brooker.co.za/blog/2017/12/28/mean.html](http://brooker.co.za/blog/2017/12/28/mean.html)).

The core point here is that _all_ summary statistics are misleading. You need to be clear on what you care about, and making absolute statements about the mean isn't a good way to do that.

Edit: This came across a bit more confrontational than I had intended. The OP makes some good points, but I think his point about the mean is overly broad.

<comment><USER3> > The core point here is that all summary statistics are misleading. You need > to be clear on what you care about

I couldn't agree more. A few months ago I gave a talk that tried in part to emphasize this point ([https://www.youtube.com/watch?v=EG7Zhd6gLiw](https://www.youtube.com/watch?v=EG7Zhd6gLiw)). mjb, I hadn't seen your post until just now but I wish I'd known about it earlier.

Another hard-earned lesson on many teams I've worked with is that humans just aren't very good at judging the variance that's intrinsic to many [summary] statistics. Even when your system is operating in what a human would consider a steady-state, summary statistics are naturally going to bounce around a bit over time. The variance is often higher for tail percentiles just because the density of the PDF is lower in that region. When faced with a question like "did the behavior of my system get worse?" in response to an external change (such as a config change, a code deploy, a traffic increase, etc.), it can be difficult to come up with a reliable answer just by eyeballing a squiggly time series line.

</comment></comment><comment><USER4> Shameless plug. Just wrote a paper about this:

[https://arxiv.org/abs/2001.06561](https://arxiv.org/abs/2001.06561)

Containing a survey of the most popular Latency Aggregation methods used in the industry (Prometheus Histograms, t-digest, HDR-Histogram, DD- sketch/histogram).

<comment><USER5> The Circllhist algorithm is interesting, but I get an uneasy feeling from the use of a _relative error_ measure to evaluate the performance of quantile estimation. Note that other authors don't use this measure.

Dunning uses Mean Absolute Error in his latest T-digest paper: [https://arxiv.org/pdf/1902.04023.pdf](https://arxiv.org/pdf/1902.04023.pdf)

Cohen uses Normalized Root-Mean-Squared Error to evaluate sampling schemes, which are equally capable of estimating latency quantiles: [https://dl.acm.org/doi/abs/10.1145/3234338](https://dl.acm.org/doi/abs/10.1145/3234338)

The problem with Relative Error as a measure of accuracy is that it depends on the location of the distribution. The same size absolute error becomes a large relative error near zero and becomes a small relative error farther up the number line.

Another thing about this study is that only one value for the T-digest quality is tested. Of course, the T-digest quality parameter equates directly with compressed size, so it's unsurprising that T-digest's size is fixed throughout the experiment. I also suspect that the choice of data set matters quite a lot in this study. If your latency values were clustered around a small range, then the algorithms like DDSketch and Circllhist will indeed have relative error less than 5% (as they prove) but T-digest will be significantly more accurate.

<comment><USER4> Thanks for your comment!

Relative error is a practical choice, since it allows to cover an extremely large value range (essentially all floating point numbers) with small size ( O(log(range)) ) and zero-configuration. You can't have that with bounding the absolute error.

Also the relative error is what you are interested in most of the time as a practitioner. (200ms+/-10ms; 1year+/-15days)

DDSketch uses relative error for estimation as well.

> If your latency values were clustered around a small range, [...] T-digest > will be significantly more accurate.

That is correct!

One point of this example was to demonstrate that merged t-digest can have unbounded errors. In the t-digest paper it was speculated that merged digest have bounded error, but the proof was just more difficult. As it turns out, if you have heavy merges and an extremely large value range, you can get unbounded errors.

</comment></comment></comment><comment><USER6> Using the median in lieu of the average isn't always a good idea either. A service could completely fail to respond almost 50% of the time and you'd still get a low median. Same holds for perc95, but to a lesser extent.

The main problem is that people try to summarize their data too early. What you want is a measure for how good or bad a _single_ datum is and only _then_ can you summarize the end result. And usually averages aren't a bad choice at that stage.

Averages have some particularly nice properties when dealing with dependent variables, sums of variables, and when you want to minimize the distance between your estimate and the actual value. However to take advantage of that your measure actually needs to make sense. For companies the holy grail is if you can directly express how much money you make / lose because of that single datum, but failing that you'll need to find something that's at least somewhat proportional to it.

<comment><USER7> My normal approach is to measure and set up separate alerts for "error rate" and possibly "timeout rate". You definitely want to know about those, but "mean latency" mixes those metrics with the latency metrics for successful requests, which makes it less sensitive to changes in either one.

In general I agree that averages aren't always bad. One additional advantage is that it's often possible to generate robust confidence intervals for averages, but it's often not valid to generate CIs for medians/percentiles without introducing other probably flawed assumptions.

<comment><USER6> Error rate and timeout rate are good examples of an average that measures the thing you're actually interested in.

The whole point is that trying to average the latency and only then try to figure out what it means is backwards.

</comment></comment></comment><comment><USER8> Every time I hear about percentiles I am thinking: Why not just show the whole distribution instead of picking a few values? I was immediately thinking of just showing the latency distribution in a histogram and was pleased by the article doing exactly that. Of course graphing percentiles over time is much easier because they just represent a single value. Percentiles are very useful for finding latency spikes but not that good for analyzing them.

<comment><USER9> > _Of course graphing percentiles over time is much easier because they just > represent a single value._

Ridgeline plots (joyplots) are severely underutilized.

<comment><USER10> R [https://cran.r-project.org/web/packages/ggridges/vignettes/i...](https://cran.r-project.org/web/packages/ggridges/vignettes/introduction.html) ; d3 [https://observablehq.com/@d3/ridgeline- plot](https://observablehq.com/@d3/ridgeline-plot) ; py [https://github.com/sbebo/joypy](https://github.com/sbebo/joypy) .

</comment></comment></comment><comment><USER11> I honestly think most teams would be better off measuring max on the latency curve. It at least isn't subject to the many transform errors introduced by most metric pipelines & it is easy to explain to people without getting into why 95 vs 99 vs 99.9.

<comment><USER12> Max is always infinity/timeout/highly variable isn't it?

And it doesn't tell you when you just made all your requests 1s slower.

<comment><USER11> It's not infinite, it better be timeout but frequently isn't and yes there is variance to it but usually (handwave) not any more variant than the tail percentile that teams choose.

I've had the experience multiple times where simply shifting a graph to max shows that timeouts/load shedding is t working, then teams get to the point where they are hitting timeouts way more than they thought. Only after working through those issues do you get to actually improving latency.

The upside of simplicity in the number is only overtaken by the downside when you start chasing real time constraints in systems that don't need it.

</comment></comment></comment><comment><USER13> I talk about this a lot when giving talks or working with folks on their reliability. This article does a great (if a bit long winded) job on explaining why it's important to know your p50,p95, and p99.

But what rarely gets mentioned is that there is no one right answer as to which to use.

It's a business decision on a per product basis.

In some cases, it's totally fine if 5% of the customers get an awful response. In some cases, p99 must be sub 5ms or your customers will leave.

This is one of the key areas where engineering and management need to work together -- deciding which percentile is key for which metrics.

</comment><comment><USER14> I may have missed it, but in many scenarios, the reason p95 etc matters isn't that it is 5% of cases ('of course 3g users are slow') but each user may issue many requests. Ex: If serving 20 assets over a session, most users will be hit w a great p50 and bad p95.

Trouble shootong at the session-of-indiv-requests level is tough, so being able to zoom in/out is the power of correlation IDs and observability stacks (vs this kind of monitoring view afaict).

<comment><USER15> And similarly, within a microservice architecture the tail latency is amplified through all its downstream requests or if there is fan-out:

 _Consider a system where each server typically responds in 10ms but with a 99th-percentile latency of one second. If a user request is handled on just one such server, one user request in 100 will be slow (one second). The figure here outlines how service-level latency in this hypothetical scenario is affected by very modest fractions of latency outliers. If a user request must collect responses from 100 such servers in parallel, then 63% of user requests will take more than one second (marked "x" in the figure)._

"The tail at scale", Jeffrey Dean and Luiz André Barroso: [https://dl.acm.org/doi/abs/10.1145/2408776.2408794?download=...](https://dl.acm.org/doi/abs/10.1145/2408776.2408794?download=true)

</comment></comment><comment><USER16> The "how not to measure latency" talk by Gil Tene is another really good explanation of this topic: [https://youtu.be/lJ8ydIuPFeU](https://youtu.be/lJ8ydIuPFeU)</comment>


Daily Crunch: IBM is getting out of facial recognition - <AUTHOR> https://techcrunch.com/2020/06/09/daily-crunch-ibm-is-getting-out-of-facial-recognition/ <comment><USER2> Previous HN thread: [https://news.ycombinator.com/item?id=23462568](https://news.ycombinator.com/item?id=23462568)</comment>


Basic Authentication over HTTP – The Evil - <AUTHOR> http://adnanahmed.info/blog/2018/03/28/basic-authentication-the-evil/ <comment><USER2> What an idiotic article.

> The server simply uses base64 to decode the Authorization header to retrieve > the given username and password to authenticate against.

The encoding had fuck all to do with security.

> The HTTP protocol unlike its sibling HTTPS is not encrypted. So it means > that everyone on internet can see what you send.

No, that's not how it works at all. I have to be somewhere in between you and the server to see what you're transmitting in plain text. I can't just read random fucking data from Joe Bloggs.

> It means that all of our HTTP calls are open to any malicious hacker sitting > in between us and the server

Oh ok, you know that. So why sensationalize your previous point?

> I did have to stick my credentials in the Authorization header when I made > the request.

You're missing the point, if it's unencrypted it doesn't matter where the data is.

<comment><USER3> I agree that the article is very poorly written, and inaccurate in places.

But at this point, having seen thousands of customers use Basic auth over HTTP, I don't actually care how sensationalized the article is. If it gets people to stop doing it, I'm all for it. We can explain the nuances later.

Frankly, the nuances of /why/ it's bad are somewhat less important than just knowing it's bad. This isn't true for all cases, but I would argue is true for what is probably one of the most common issues we see.

</comment></comment><comment><USER4> What a pointless post and sensationalized title. The problem is not Basic Authentication, it's HTTP.

<comment><USER5> Yeah, it's just three paragraphs explaining that HTTP is not encrypted.

<comment><USER6> Plain HTTP is fine if we send it over an encrypted connection. Hmm... I wonder if we could use TLS for that?

;)

</comment></comment></comment><comment><USER7> "Despite its simplicity, you should never ever use basic authentication over HTTP! Basic authentication should only be considered over HTTPS."

Small correction: You should NEVER EVER use ANY authentication mechanism over HTTP. Basic authentication has its uses, and for those use cases it works fine using HTTPS

</comment><comment><USER8> Another way to look at this article is that Basic authentication may be a perfectly acceptable implementation over HTTPS, as opposed to something like Oauth2 when you aren't even using it for authorization. I've seen APIs that aren't user-facing at all and use Oauth2 merely for authentication. Pointless. Your best line of defense is HTTPS.

</comment><comment><USER9> Suppose I develop a website that uses basic authentication. Because this is 2018, I store my user's passwords using bcrypt/scrypt/PBKDF2/et al.

Every single request supplies the claimed password in the clear (wrapped in HTTPS) so I would need to run the long hashing process for every single request.

I could maybe cache those clear-passwords that I know are good in memory, but that's getting dangerously close to storing user's passwords in the clear.

Is there a good practice for using basic authentication without over burdening the server's CPU?

<comment><USER10> yes, get more CPU's if your auth server is hitting it's limits.

Scale it by moving auth to it's own URL, so you can then load-balance it differently if needed (and can add more security like disallowing Javascript at all - via headers, etc)

But first, verify you are actually hitting CPU max because of bcrypt/scrypt/etc and not just because of crappy code or something.

</comment></comment><comment><USER11> Revelation of the century...

</comment><comment><USER12> Upvote bot?</comment>


BigML is now open for business - <AUTHOR> http://blog.bigml.com/2012/10/23/bigml-open-for-business/

<comment><USER2> Great news!!!</comment>


Support ticket for de-duplication of files within (Google+) Takeout archives - <AUTHOR> https://fixato.org/temp/archived/gplus-feedback-takeout-duplicates-usage-analysis.html <comment><AUTHOR> As the Google+ shutdown looms, users and communities are wrestling with whether, and how, to archive their data.

One problem we're encountering: Google's Data Take Out pads its archives with _gigabytes_ of redundant data, mostly image files. Archives of up to half a terabyte have been reported.

People are trying to access this data over mobile links, metered links, slow residential broadband, dial-up, or unstable and slow links in Africa, Asia, and Indonesia, with little success. De-duplicating data would help tremendously.

My own was 15 GB, of which over 95% were images, most of little value, though I've no way of excluding them from the collection. The textual content I'm interested in is under 500 MB (and probably a small fraction of that).

Google have been almost completely noncommunicative since the G+ shutdown was announced, with two exceptions of which I'm aware.

The first shortened the time-to-live of the platform by another 4 months, a 40% reduction over the initial notice.

The second ... slightly ... improved instructions for Data Take Out.

(I'm moderator of the Google+ Mass Migration community and am helping coordinate other activities in moving people off Google+, preserving and porting data, and keeping communities together.)

In the even the server is hugged to death: [https://web.archive.org/web/20181217103934/https://fixato.or...](https://web.archive.org/web/20181217103934/https://fixato.org/temp/archived/gplus- feedback-takeout-duplicates-usage-analysis.html)</comment>


X - <AUTHOR> https://xkcd.com/2309/ <comment><USER2> Even without technically changing font, there are more than you might expect: XxˣₓẊẋẌẍⅩⅹ⠭ꭖꭗꭘꭙ🄧⒳ⓍⓧXx𝐗𝐱𝑋𝑥𝑿𝒙𝓍𝓧𝔁𝔛𝔵𝕏𝕩𝖃𝖝𝖷𝗑𝗫𝘅𝘟𝘹𝙓𝙭𝚇𝚡🅇🅧🆇

[https://unicode-table.com/en/blocks/mathematical- alphanumeri...](https://unicode-table.com/en/blocks/mathematical-alphanumeric- symbols/)

<comment><USER3> I never understood why Unicode has bold versions of characters, that's the function of the font, not the character set.

<comment><USER4> Symbolically same but semantically different characters are most of the time represented differently in the byte level so that searching in large text can still be implemented as simple byte comparison, if Unicode can be called simple.

<comment><USER5> "simply byte comparison" is not a term associated with Unicode. Unicode is usually in strings, and reverse engineering a string is -- very hard, and surprisingly I haven't found a paper proving whether it is NP complete.

</comment></comment></comment></comment><comment><USER6> If you think that it is just a sick joke, take a look at this and think again: [http://www.zifyoip.com/wysiscript/sigbovik.html](http://www.zifyoip.com/wysiscript/sigbovik.html)

<comment><USER7> Curious... I can't figure out if this is completely serious. There are what are either tongue in cheek remarks, or they are taking themselves very seriously...

> "removing the awkward necessity of writing text"

> "as powerful as other languages and significantly more colorful"

> "we believe it will appeal to many types of students who may not have an > interest in traditional text-based programming, including painters, graphic > designers, and the illiterate."

I think it might be more effective to teach the illiterate to read first rather than code... Maybe it's worth it if learning to code inspires them to become literate?

<comment><USER8> > I can't figure out if this is completely serious.

"Most programming languages in use today use only a single color for all numeric literals, which is unnecessarily confusing and makes it impossible to distinguish different numbers using only a spectrometer"

:)

</comment></comment></comment><comment><USER9> Back in the late nineties I worked with a software called _Origin_. It had a BASIC like programming language where variable names consisted of single letters. If you now think 26 variable names should be enough: Some of the letters, like X and Y, had special predefined meaning and you couldn't change that. If you used seem you encountered strange behaviour, like values changing behind your back. To add insult to injury I could never find documentation about which variables were special, so I stuck with the few I new were safe, which were not many...

<comment><USER10> The old TRS-80 Model 1, Level 1 had 4KB of memory and the only permitted variables were integers (A-Z), and two strings A$ and B$. No decimals, floats etc.

Level 2 allowed any number of letters in a variable name... but only the first two were recognised as unique. CUSTORDER was seen as the same variable as CUSTLINE.

<comment><USER11> > allowed any number of letters in a variable name... but only the first two > were recognised as unique.

That was a well-known Microsoft BASIC feature that got passed down licensed descendants like TRS-80 and Commodore BASIC, and lasted through earlier 6502 BASIC implementations.

</comment></comment></comment><comment><USER12> Reminds me of this comment from Reddit [https://old.reddit.com/r/rust/comments/5penft/parallelizing_...](https://old.reddit.com/r/rust/comments/5penft/parallelizing_enjarify_in_go_and_rust/dcsgk7n/)

Go allows characters from the Canadian Aboriginal Syllabics for identifiers, so you can define structs or methods like this:

      type ImmutableTreeListᐸElementTᐳ struct {

You can pretend that go has generics :)

<comment> 0-_-0 Isn't that a syntax terror?

<comment><USER13> Names just need to be valid UTF-8, which it is

<comment> 0-_-0 My misspelling was intentional... but now it's not funny

</comment></comment></comment></comment><comment><USER14> A good set of variable names could be found in Unicode's confusables list, starting at line 3795 (or search for CANADIAN SYLLABICS FULL STOP → LATIN SMALL LETTER X)

[http://www.unicode.org/Public/security/latest/confusables.tx...](http://www.unicode.org/Public/security/latest/confusables.txt)

<comment><USER15> Greek question mark can be especially put to good use, in particular if a colleague walks away forgetting to lock the machine: [http://www.fileformat.info/info/unicode/char/037e/index.htm](http://www.fileformat.info/info/unicode/char/037e/index.htm)

</comment></comment><comment><USER16> I envisage Randall's thought process for this was:

\- Thinks of "variable variable" title text joke

\- "Now I just need to come up with a scenario that will allow me to make this joke"

<comment><USER17> I was reminded of php's variable variables [1], which often makes for a good joke.

[1]: [https://www.php.net/manual/en/language.variables.variable.ph...](https://www.php.net/manual/en/language.variables.variable.php)

<comment><USER18> But it makes reflection so easy. I've never known a "modern" language to have such simple reflection without a bunch of boilerplate (Java, C#, Python gets close, Scala)... People harp on PHP a ton, but it's really darn powerful if used correctly. The trick is finding people who know how to use it correctly.

<comment><USER19> Oof, I'm not a PHP coder and I didn't know it had this feature before reading these comments but I definitely do not envy it in the languages I use. The potential for abuse is immense, this is one step removed from generating code at runtime by concatenating strings and calling eval() on them.

I'm sure there are situations where this feature can be really valuable (and I do sometimes use getattr/setattr in python, which is somewhat similar) but I think the developers should really think hard and make an educated decision about when to use it. As such having it require a bit of boilerplate is a feature more than a bug IMO. I definitely wouldn't want to have to maintain code written by someone who abuses these types of indirections everywhere.

I can definitely imagine a novice coder using these "variable variables" in lieu of a proper hash table or dictionary object for instance.

I mean it suffices to read some of the comments to the article posted above to find a bunch of people posting code snippets which, IMO, are a very poor way of implementing what they want. For instance:

        $price_for_monday = 10;
        $price_for_tuesday = 20;
        $price_for_wednesday = 30;

        $today = 'tuesday';

        $price_for_today = ${ 'price_for_' . $today};
        echo $price_for_today; // will return 20

or even:

        class foo {
          function bar() {
            $bar1 = "var1";
            $bar2 = "var2";
            $this->{$bar1}= "this ";
            $this->{$bar2} = "works";
          }
        }

        $test = new foo;
        $test->bar();
        echo $test->var1 . $test->var2;

I'd probably quit if I had to maintain code written like that. Having boilerplate would at least make it obvious that something unusual is going on with these variables and make the intent clear while reading the code.

<comment><USER20> Thats because your use case isn't real. In js, its usually used to generate select option (example in react):

        for(let optValue of Object.keys(options)){
          optDom.push(
            <option key={optValue} value={optValue}>{options[optValue]}</option>
          )
        }

Of course array also works, but with json object, it guarantee that the value will be distinct and you don't need specific array object type.

</comment></comment></comment></comment></comment><comment><USER21> I think there is a design issue in that language. X in a fixed width font should be a constant. X in a variable width font is a variable.

</comment><comment><USER22> My Commodore Vic 20 running basic only allowed 2 character variable names.

<comment><USER9> That is not entirely true, from my recollection variable names could be longer but only two characters were significant.

<comment><USER23> That was also the case with AppleSoft BASIC

</comment></comment></comment><comment><USER3> Obviously inferior to brainfuck, which uses just two characters.

[https://en.wikipedia.org/wiki/Brainfuck](https://en.wikipedia.org/wiki/Brainfuck)

<comment><USER24> Which is itself inferior to Whitespace [1], which uses only spaces, tabs, and linefeeds.

[1] [https://en.wikipedia.org/wiki/Whitespace_(programming_langua...](https://en.wikipedia.org/wiki/Whitespace_\(programming_language\))

<comment><USER25> > (...) the Whitespace interpreter ignores any non-whitespace characters (...) > A consequence of this property is that a Whitespace program can easily be > contained within the whitespace characters of a program written in another > language (...) making the text a polyglot.

This is actually pretty brilliant as a sort of steganography in code type thing.

<comment><USER26> So one could write an interpreter for Whitespace along with Whitespace code to run in the interpreter, so that the interpreter interprets itself. Hard to do better at maximizing the usage of negative space!

<comment><USER25> Whoa, you just blew my mind. We need this in our lives :)

</comment></comment></comment></comment></comment><comment><USER27> Unrelated question: what are the main reasons (aside from the obvious file format problems) for which we don't have rich text syntax?

Imagine having a e.g. link in the code. Wouldn't that be useful?

<comment><USER28> > Wouldn't that be useful?

Useful - for what? Do you want that link to be written & maintained by you, or to be automatically generated by the tools? Because we already have the latter... (e.g. in many IDEs, ctrl-/cmd-click on symbols performs useful actions, eg. takes you to the definition or finds all references if you're at the definition site).

<comment><USER27> It does, but in dynamically typed languages it's obviously less capable.

To deal with this one could of course have a static type system instead, but that necessitates learning the DSL behind it and its idioms - the lack of which is the main appeal of dynamic typing to begin with.

<comment><USER28> I actually did code hinting for PHP in a fairly popular commercial product, more than 10 years ago - and I disagree with you that it has to be less capable. You can use heuristics - people are forgiving because they understand you can't get everything right. You can use configuration files & user hints to help the semantic analysis. You can absolutely get the hints mostly right - and it's automatic, so probably closer to being correct than anything that'd have to be maintained by hand.

</comment></comment></comment></comment><comment><USER29> I think this hits close to home. I wish creators of new programming languages would reuse syntax of existing programming languages (ideally the ones in the mainstream). Or at least think about it.

Also think if you even need a new language. It seems that too many new languages have features that can be handled by functions or macros in existing languages.

Some languages like Forth, Lisp or Haskell are very powerful, have strong metaprogramming facilities, and most likely can already do anything that would need a new language and new syntax.

Think about it - if you avoid creating another syntax variation, and work with an existing syntax (or better existing language), your chances of somebody actually using your efforts will vastly increase.

<comment><USER30> After following Make A Lisp [1] and seeing how easy it is to parse and get all that power, other syntax just seems sadistic for both the language developer and the users.

[1] [https://github.com/kanaka/mal](https://github.com/kanaka/mal)

</comment></comment><comment><USER31> I heard a story that someone giving a talk at IAS only used M's on the chalkboard (lowercase, uppercase, doublestroke, script, etx) before he was told to cut it out. I can't remember who it was if anyone else heard the story.

</comment><comment><USER32> I had a class in grad school where the textbook ran out of english, greek, and hebrew letters and subscripts and resorted to serif vs sans fonts. It was interesting.

</comment><comment><USER33> So, basically coding like a mathematician or physicist?

</comment><comment><USER34> Came here thinking it was another jab at the venerable X11 display server. It seems we need a Unicode raster font to make this post more humourous.

</comment><comment><USER35> This API is reminiscent of the VA Fileman API for which X is used over and over again. Fileman also uses Y though.

[http://www.hardhats.org/fileman/pm/cl_dic.htm](http://www.hardhats.org/fileman/pm/cl_dic.htm)

</comment><comment><USER36> I am _delighted_ to see an xkcd comic make it to Hacker News's front page, but also a little disappointed it happened to be _this_ one. It's a perfectly fine comic, but there are so many others that could have engendered even more interesting discussions, which I'd love to have been able to read.

Limiting myself purely to more recent ones:

• [https://xkcd.com/2307/](https://xkcd.com/2307/) \- Alive or Not

• [https://xkcd.com/2224/](https://xkcd.com/2224/) \- Software Updates

• [https://xkcd.com/2221/](https://xkcd.com/2221/) \- Emulation

I'm also really missing the xkcd forums. The quality of the discussion was below what I'd expect to read on HN†, but it was still nice to have something for each of the comics.

† No disrespect intended to anyone who was on those forums—you're all great people, but there's less of you, and, well, there's nothing quite like HN.

</comment><comment><USER37> [https://www.explainxkcd.com/wiki/index.php/2309:_X](https://www.explainxkcd.com/wiki/index.php/2309:_X)

<comment><USER38> > This is also a jab at mathematicians who are in addition to using variable > names which are short and nondescript (e.g. "x"), are also "typeface > sensitive" (in addition to case sensitive). In other words, one can > typically find a statement involving three different "X" variables, > referring to three different objects, and they are distinguished by their > font and case.

The pain, across math, engineering, and physics, is all of the formulae with overloaded notation, sometimes from the Greek, sometimes the Roman alphabet.

Formulas are great as a summary and reminder for the seasoned practitioner, but can be a huge barrier to entry for the student.

Perhaps GitHub repos with well-written code and test cases for the neophyte can, as they say these days, "flatten the curve".

<comment><USER39> I remember always being annoyed at spherical coordinates when I was taking both math and physics classes in university. They use the same symbols but two of them are swapped.

<comment><USER38> As an undergrad I fantasized about a steel cage match between all of these practitioners to force convergence on a single set of notation.

<comment><USER40> [http://www.wastedtalent.ca/comic/non-dimensional- fame](http://www.wastedtalent.ca/comic/non-dimensional-fame)

</comment></comment></comment></comment></comment><comment><USER41> nobody has mentioned Color FORTH yet?

[https://en.wikipedia.org/wiki/ColorForth](https://en.wikipedia.org/wiki/ColorForth)

</comment><comment><USER42> Sometimes it seems like people writing Coq are trying to do this. Would it kill you to use variables besides n and x with subscripts?

</comment><comment><USER43> Ok just one variable - X[], use an enum for indexes into that. Enums aren't 'variables'.

</comment><comment><USER44> probably you can have a ML program that generates variable names, given the situation in the code.

<comment><USER45> There is something like this already and it is quite good. It is used for deobfuscating javascript [0][1].

[0] [https://www.sri.inf.ethz.ch/research/plml](https://www.sri.inf.ethz.ch/research/plml) [1] [http://jsnice.org/](http://jsnice.org/)

</comment></comment><comment><USER46> just make a hash map where key = actual variable name and value = x

</comment><comment><USER47> Who's the first to announce X in Rust?

</comment><comment><USER48> But what is the title??

</comment><comment><USER49> Good laugh, Thanks!

</comment><comment><USER50> Apart from the 'x' part, this is basically CSS.

</comment><comment><USER51> Why not just use x1, x2, ... Problem solved.

<comment><USER52> Back in high or middle school, I wrote a Tetris game. Being terrible at programming, I stuffed all of the row clearing logic into one function, which ended up having variables named, x, xx, xxx, xxxx... I think I stopped at four x's. You can imagine the inner loops. No, it never worked right.

<comment><USER53> I propose modifying a minifier to make all variables different combinations of the characters VvWw, functions FEf, objects O0o, etc

Bonus points: instead of minimizing space, all identifies are, say, exactly 6 characters long.

</comment></comment></comment><comment><USER54> Thanks for reminding me to catch up on the last few months of XKCD! Ended up sending a bunch of coronavirus-related comics to my biologist wife.

</comment><comment><USER55> For those of you on mobile who want to read the alt text, check out m.xkcd.com, the oft forgotten mobile UI.

<comment> frosted-flakes In Android Chromium, you can read the alt-text of images by long-pressing the image.

</comment></comment><comment><USER56> This is great

</comment><comment> DC-3 I like xkcd as much as anyone (I've even got the books!) but this isn't really HN calibre imo.

<comment><USER57> "HN caliber" is determined by the community voting on posts which determines what posts make it to the front page.

</comment></comment><comment><USER58> What makes XKCD so great? Is it that he sticks to noncontroversial subjects - everytime people pull up a "relevant xkcd" it's something most of the people agree on.

Even with controversial subject - the covid lockdown, he chose to highlight facts where everyone (well ~90%) agree on.</comment>


The Rise Of Company Builders - <AUTHOR> http://techcrunch.com/2013/02/16/the-rise-of-company-builders/ <comment><USER2> I am building a company on this model, and quite successfully so far. I'm in college right now but will be hiring in the future. I'm glad the model is finally getting some validity, as I'm sure it's a good one.</comment>


Unit testing in Coders at Work (2009) - <AUTHOR> http://gigamonkeys.wordpress.com/2009/10/05/coders-unit-testing/

<comment><AUTHOR> Earlier discussion here: <http://news.ycombinator.com/item?id=863219></comment>


Subtweet: Conversations for Twitter - <AUTHOR> http://subtweet.co

<comment><USER2> What I would like even more is to take a discussion I've already had on Twitter - all of it - and extract and export it in a sensible format for indexing and archiving. Taking the conversation private is not the problem for me.

However, that's the problem you've addressed, so here are some questions:

* Does everyone in the conversation have to sign up for your service?

* Is there going to be an option for signing up not via Twitter?

* Can I export the conversation?

* Can I extract tweets already on Twitter into the new conversation on Subtweet?

* Will there be a more complete description available soon?</comment>


We've been asking them to learn to code. They've been listening. - <AUTHOR> http://mariarociopaniagua.com/2012/02/21/all-of-the-code-new-developments-pt-2/ We've been asking them to learn to code. They've been listening. <comment><USER2> It's great to finally see people, taking initiatives to learn and evolve in the digital age. I'm of the opinion that everyone should learn how to program, so seeing people all over the global, actually do it from scratch is great.

Also, don't use w3schools... <http://w3fools.com/>

</comment><comment><USER3> Interesting. As a Mexican entrepreneur now I see a proper growing ecosystem. Efforts like these will produce results for the years -and entrepreneurs- to come. The start-up scene is organizing better, but we have to keep the push and support.

</comment><comment><USER4> This is really motivating all the efforts you are making to be prepared for the bus. Im really proud that Mexico is on the map of the start up world and this kind of initiative are arriving.

</comment><comment><USER5> :D thank you all

</comment><comment><USER6> Congrats Rocío.</comment>