Ask: How to use memcached to handle a large multi-to-multi relationship? - angus-t

Ask: How to use memcached to handle a large multi-to-multi relationship ?
We are using MySQL and memcached.
We are going to implement a feature "User subscription", which allow our users to subscribe our diff. category of news.
There are two main data query. 1) Get list of User subscribed / unsubscribed this Category. <<It mainly used in Cron Job 2) Get all subscribed Categories by a User
The problem is action 1), the data size will be very large for memcached,
say 10 millions of user x 500 of categories.
If we simply use CategoryId as cache key and the value possibly has millions items,  it will caused performance issue to get large cache from memcached servers.
We come up a method, to
- group every X user into pages e.g. categoryA_page1=[user1, user2, ...] ; categoryA_page2= - maintain a index in memcached e.g. user1_categoryA=page1 ; user1000_categoryA=page2
Please suggest if there is better solutions <comment><AUTHOR> If I were you I would have inspired myself from iNode structure found in ExtFS. Here is what I would have done:

Category -> Array of tuples (imagine nested array) where each tuple contains (Key to a group, Number of elements in group).

You can use something like JSON (I won't recommend that) or MsgPack (this will work best for you my guess) for serialising your array. Now the nifty part comes in holding the number of elements in a group you can have fixed sizes or you can find out average number members in a group (do the maths) combine it with your standard deviation and make an intelligent decision. Adding requires you looking at the last element in category if you filled the cap create a new group and append it into category container. If you face yourself with value size do yourself a favour and compress the values against keys ( I've two complete blog posts to get you convinced [http://blog.creapptives.com/post/27331500697/msgpack-vs- json...](http://blog.creapptives.com/post/27331500697/msgpack-vs-json-in-the- jaws-of-compression) and [http://blog.creapptives.com/post/25026698783/speeding-up- you...](http://blog.creapptives.com/post/25026698783/speeding-up-your- datastore-with-compression)) use something fast like libsnappy or lz4 to compress your values. Hope it helps!

</comment><comment><USER2> In your testing, was MySQL too slow after proper indexing? I don't think you need to cache that sort of query. Personalized queries are usually not the best candidates for caching.</comment>


The returns to entrepreneurship  - <AUTHOR> http://startupboy.com/2009/11/09/the-returns-to-entrepreneurship/ <comment><USER2> Excellent article.

I think code is just one tiny piece of the puzzle, but here's an example: I'm one guy who programs in his kitchen. I would up at 6:12 AM last Saturday and, unable to get back to sleep, decided to write a new feature into my website. I found it easier than I expected and eventually ended up writing two new features. Then I set up A/B tests for both of them. I then got done at 10:37 AM and spent the rest of my day playing videogames.

I was able to "write" two features in 4.5 hours because it was a matter of snapping two bits of OSS code together with a bit of UI glue drizzled on top. (The A/B testing framework is also OSS, though I wrote it.) My customers don't care about that, though: they care that they couldn't do X or Y last Friday and now they can. (Well, if they're in the A/B test group.)

You know what isn't anywhere in the above description? A four hour planning meeting between six people discussing whether X or Y should go into the next version of the product. I've been at that meeting before. I think we all have. It is a tremendous waste of time when I can just ship the feature in an A/B test and tell you a week later how 5,000 users reacted.

(Answer: feature X increased task success. Feature Y decreased it. Neither effect was statistically significant. As I had been delaying making these primarily out of fear that featureitis would hurt task completion, in the face of repeated customer requests for both features, I'll be shipping them to everybody.)

And I'm one guy doing this from his kitchen table in his spare time in a not- too-rich niche. There is nothing about these techniques that make them less effective for bigger, better funded, more focused teams in richer niches.

</comment><comment><USER3> _Examining a list of the 75 richest people in the history of the world, Gladwell notes that "an astonishing 14 are Americans born within nine years of one another in the mid-19th century."_ <http://features.csmonitor.com/books/2008/11/17/the-outliers/>

England's colonial shipping trade and railroads were also technologies that changed the economics of business, making fortunes in the process.

</comment><comment><USER4> > It seems that the entrepreneurs who "hit" these days are doing it more > quickly, making more money, and doing it at a younger age.

Citation needed! I think 'younger age' is likely to be true. Thanks to the internet, it's easier to build up a network and get informed earlier. I wish _I_ had had HN ten years ago! The rest of it... well, I'd like to see some facts.

</comment><comment><USER5> A very interesting reminder of just how low the barriers are to doing a start up today.

<comment><USER6> The barriers are low as long as the brainpower is high. It isn't just any person off the street who can flesh out and grow a startup company from scratch. It takes a certain type of person with a certain amount of horsepower under the hood to achieve that objective.

<comment><USER4> Ok, but the point is that everything else being equal, that person has a much better shot at things today, because of the very small amount of capital necessary for a web-based business.

</comment></comment></comment><comment><USER7> > _Most web products have no marginal cost of replication, so adding a new > customer is pure profit._

This is my single gripe about this article. This isn't true: although there's a higher profit margin because the cost of one customer on a general web application is close to nil, the cost of acquiring a customer is sometimes pretty high. Especially if you factor in that 1% of free users will convert to paid users (in a freemium model.)

Unless if you hit it out of the ballpark with some crazy lucky viral marketing, I'm pretty sure you're going to have to do some solid marketing to gain traction.

<comment><USER2> On the plus side, if your trial -> purchase conversion rate is in fact 1%, then a single A/B test which increases your conversion at any step of the funnel between trial and purchase by 5% results in an almost immediate 5% increase in top-line revenue. And 95% of that (allowing a little bit for cost of payments) will flow straight to the bottom line, because you typically pay for COCA whether the person converts to premium or not. (Well, unless you're doing CPA marketing. Even then, there are compelling reasons to do conversion optimization, as it will make your effective payout for affiliates higher, attracting more of them to your banner.)

Actually, come to think of it, in a freemium subscription model a 5% increase in conversion doesn't necessarily do 5% to revenues instantly because a portion of your revenues are produced by existing subscribers. Sorry -- I do one-time purchases, sometimes I forget about the economics for other folks.

Still: what other businesses can assign one junior employee to work on button designs for a week and walk away with 5% added to the value of the business? And close to risk-free, because you can experiment on a fraction of your customers and terminate it almost instantly if it doesn't pan out?

That is leverage, broadly defined, that would make hedge fund managers green with envy. ("Hey, can you figure out a way to make us 5% extra on all our capital? Oh yeah, and do it risk free. Oh yeah, and I'd like whatever you do to be basically repeatable with a minor variation next week, so that we can institutionalize this and do it as a process rather than a one-off. Alright, call me when you've made me a billion, I'll be on the golf course.")

<comment><USER8> I agree with your point, but think someone should mention a point I believe I saw on HN the other day.

Basically, trial -> purchase conversion for freemium seems to max out at around 1%, so your efforts are usually better focused on improving retention rather than initial conversion.

If anyone remembers what presentation this was from please post the link, I can't seem so find it.

<comment><USER2> You may be thinking of the discussion here. <http://news.ycombinator.com/item?id=926506>

The presentation it references for support of the "1% is the limit" myth is here: <http://particletree.com/features/web-app-autopsy/> "Myth" is strong language for me, but I believe it is justified in this case. If you play "Hey I got a BA degree for a reason, let's play secondary researcher", it is based off of a misstatement of a repetition of a claim that doesn't even appear in the primary research. All the primary research says is "Three particular firms we have convenient access to say they have about 1% visitor -> purchase conversions. That's probably typical, then." (Even the primary source skips over the fact that one of the three actually reported 1.14%, which is a 14% difference that falls straight to the bottom line, and the other two reported to exactly one significant digit.)

You'll note that visitor -> purchase is very different than trial -> purchase, but don't beat yourself up, different folks routinely quote both numbers at 1%. It should be self evidently obvious that both groups cannot possibly be equally right, but as it turns out they're probably both just equally wrong.

In addition to being incorrectly repeated from a misreported summary of anecdotal data, the claim is empirically false. I can't tell you other people's numbers, but I can tell you mine: 2.45% trial to purchase conversion. Feel free to do much better than that -- it is certainly possible. (Important possible apples-to-oranges alert: my freemium app is not sold on a subscription basis.)

This post is not meant as a personal attack: I have heard this statistic, and its older brother ("1% of shareware downloads convert") for years. Vaporizing it in the harsh light of actual evidence is one of my numerous personal crusades, because it leads to poor decisions that actually cost people money, like "Well, conversion is a fixed number so I totally can't add 5% to my bottom line by working on buttons or something absolutely stupid with a shopping cart redesign."

<comment><USER9> Practically:

* As you "launch" there is like 1% of conversion

* As you keep optimizing, conversion rate increases (in your case by 1.45%)

* Something like +1% per major optimization (and targeting) (compound rate ?)

Can you confirm it?

And don't forget that you can always hack the buyer psychologically.

</comment></comment></comment></comment></comment><comment><USER10> Is anyone else's Bubble Alarm going off? These are the kind of articles that are often a sign.

<comment><USER11> My bubble alarm has been going on for years, but I predicted that neighter Facebook nor Twitter would ever have a viable business model and I have already been proved wrong on Facebook at least - I assumed that it was the harder of the two.

Having said that, it still doesn't smell right.

</comment></comment><comment><USER12> Overnight success is a myth.

<comment><USER13> depends on your definition of success

<comment><USER14> or your definition of "overnight"

</comment></comment></comment><comment><USER15> If making technology products has become accessible, similarly barriers to entries has been reduced. When there are lots of players, Nasim Taleb describes the situation as winner takes it all.

<comment><USER2> I think the Internet makes two very different things common. I'm not sure what world any given business falls into:

1) The Internet is vast, and this tends to create everybody-wins. There is a virtually inexhaustible range of niches out there. Seriously -- one giant, flowing river of value creating. You are able to dip a bucket into it. You may not end up with your own private ocean (hello, Google) but you will also probably not die of thirst.

I honestly think that this is big, like Industrial Revolution big. Big like "bigger than all the hype about the dot com bubble" big. That was a bunch of PR-funded hype which collapsed in on itself. This is about creation of value -- huge, staggering amounts of aggregate value -- and cutting out a lot of the waste that previously prevented that from reaching people. Industries are being reinvented, and some are being created out of whole cloth.

2) The Internet is deep, and this tends to create winner-takes-most. At the top levels of any intellectual labor there is a step function difference in outcomes between #1 and #2, #2 and #3, etc. We talk about the Long Tail because it is new and exciting, but the fat head is still there. That fat head of SEO means, for example, that executing 1% better than the guy in the #2 spot more than doubles your revenues. By itself. The difference between being Google-successful and being Facebook-successful is several orders of magnitude, even though both of those examples made their founders rich beyond the dreams of avarice. The productivity difference between the best programmers and the worst programmers is widely rumored to be an order of magnitude: I think in the future, we'll see the pay catch up, outside the fairly narrow range of quants on Wall Street. (And the difference between the average programmer and the best day laborer is probably going to keep climbing, too. Returns on one of them keep increasing, you do the math.)

It is a great time to be alive for those of us who can leverage these trends. If I were committed to equality of outcomes, though, I would be terrified.

<comment><USER15> Regarding #1, I think you are right that there are a lot of niches which can be tapped and which will make one a moderate amount of money. But problem arises when tons of creative people choose a single niche (e.g. todo lists). Choosing a niche and executing well is important but what stops from other fishes having their lunch at your part of pond?

Regarding #2, do you think luck is involved there in getting to jump the step function?

<comment><USER2> _Choosing a niche and executing well is important but what stops from other fishes having their lunch at your part of pond?_

The pond is so vast and the river flowing into it is so wide that all the other fishes are welcome to anything they find in the general vicinity of me. I'm hardly starving. I also find time to publish a blog titled Tasty Worms Here and release open source swimming techniques. (Plus, although many people think my section of the pond is the size of a mud puddle if that, I think I'm probably, hmm, fourth biggest fish in these here parts? Maybe third. I don't spend too much time thinking about it -- too many tasty worms to eat, not enough time to worry about other fish.)

I don't believe in luck, and I think it lacks explanatory power. For example, the top 1% of iPhone apps rake it in next to the bottom 99%. If you were to ask me why, I'd talk very little about luck and very much about how the App Store is virtually designed to encourage churn and blockbusters, which exacerbates the Winner Take Most effect to the point where it gets almost absurd.

<comment><USER16> All great points, so I'll throw one more out there. Not all fish have the same tastes, needs, or swimming abilities. If the "niche" that you are filling is occupied by a huge number of fish, is it still a niche? ;-)

</comment></comment></comment></comment></comment><comment><USER17> Great positive outlook on the rewards of entrepreneurship when the media is constantly droning on about the negative effects of the recession.

I too am looking forward to seeing where we'll be in 10 years time.</comment>


SICP Distilled - <AUTHOR> http://www.sicpdistilled.com/ <comment><USER2> I started reading SICP for the first time over the weekend. I've been developing software for the last 5 years or so with a lot of ad hoc knowledge (Im self taught and mostly do PHP or Ruby based web projects), but I feel like lisp is interesting.

The concepts are familiar, but it feels more abstract in a nice and useful way.

I started with the videos, but they were wayyyyy too slow. Then I was reading the html version of the text. But this version is quite good. I am not yet sure what the differences between common lisp, closure, and scheme look like, (I'm still early in the book) but I think that this distilled version is marginally easier to follow.

</comment><comment><USER3> This looks great, thanks for sharing.

Has anyone tried doing SICP in several different languages, and have thoughts on how Clojure compares to others?

</comment><comment><USER4> I'd funded a kick starter campaign a couple of years back, wonder I'd this is that</comment>


The Mandelbrot Monk (1999) - <AUTHOR> http://users.math.yale.edu/public_html/People/frame/Fractals/MandelSet/MandelMonk/MandelMonk.html <comment><USER2> According to Wikipedia:

      Udo of Aachen (c.1200–1270) is a fictional monk, a
      creation of British technical writer
      Ray Girvan, who introduced him in an April Fool's hoax
      article in 1999. According to the article, Udo was an
      illustrator and theologian who discovered the Mandelbrot
      set some 700 years before Benoît Mandelbrot.

      Additional details of the hoax include the rediscovery of
      Udo's works by the also-fictional Bob Schipke, a Harvard
      mathematician, who supposedly saw a picture of the
      Mandelbrot set in an illumination for a 13th-century carol.
      Girvan also attributed Udo as a mystic and poet whose
      poetry was set to music by Carl Orff with the haunting O
      Fortuna in Carmina Burana.

[https://en.wikipedia.org/wiki/Udo_of_Aachen](https://en.wikipedia.org/wiki/Udo_of_Aachen)

<comment><USER3> I have to admit I am a little sad this was an April fools joke. It would have been so cool if some obscure medieval monk had discovered the Mandelbrot set centuries ago.

Then again, IIRC, complex numbers were not invented (discovered?) until the 1500s, so good old Udo would have had a hard time figuring out the math.

<comment><USER4> He "did" invent complex numbers! From TFA:

"Initially, Udo's aim was to devise a method for determining who would reach heaven. He assumed each person's soul was composed of independent parts he called "profanus" (profane) and "animi" (spiritual), and represented these parts by a pair of numbers. Then he devised rules for drawing and manipulating these number pairs. In effect, he devised the rules for complex arithmetic, the spiritual and profane parts corresponding to the real and imaginary numbers of modern mathematics."

<comment><USER3> Serves me right for not reading the "article" thoroughly. ;-)

</comment></comment></comment></comment><comment><USER5> Fun story! But look at the date at the bottom...

<comment><USER6> I was chatting about this with a friend recently. I spent most the last 15 years thinking it was real.

<comment><USER7> Had you written your own Mandelbrot set renderer or not when you first came across the story?

<comment><USER6> I can't remember. It was within a few years of me writing such a renderer on my graphics calculator, but I'm not sure which came first.

</comment></comment></comment></comment><comment><USER8> While this is a fun hoax, there is an 13th century illuminated manuscript that does get the imagination going:

[https://pavlopoulos.wordpress.com/articles/frontispiece- of-a...](https://pavlopoulos.wordpress.com/articles/frontispiece-of-a-bible- moralisee-ca-1250/)

</comment><comment><USER9> This is clearly not a true story. I think someone should flag it or put a warning in the title.

<comment><USER10> I think this comments section is enough. It's more fun to go into the story without knowing that it's fake.

<comment><USER11> NO it's not fun. I just sent it to all my colleagues...

<comment><USER12> You are not alone: so did I.

</comment></comment></comment></comment><comment><USER13> Reading the headline I first thought it is about "Buddhabrot".

[https://en.wikipedia.org/wiki/Buddhabrot](https://en.wikipedia.org/wiki/Buddhabrot)

</comment><comment><USER14> Notably, 1999 is also the year which saw the beginning of intensified studies on the Archimedes Palimpsest[1]. Some inspiration may have been drawn from this.

[1] [https://en.wikipedia.org/wiki/Archimedes_Palimpsest](https://en.wikipedia.org/wiki/Archimedes_Palimpsest)

</comment><comment><USER15> got me.

those plurality-of-worlds Lorites sure do get around.</comment>


Show HN: Franklin – Out-of-Network Health Insurance Claims Filed Online - <AUTHOR> https://www.getfranklin.co <comment><AUTHOR> Hi everyone, here's more info on our project. Would love any feedback!

\--

In 2019, my cofounder Hiroki and I started Neb, a startup to find people therapists. We would call and email therapists, and screen them, to recommend the best therapists to our users. After finding therapists for 3 months, we came to the conclusion that it was simply impossible to consistently find in- network or affordable therapists. We often had to reach out to 50+ therapists per user to find an available, affordable match, and for in-network therapists in particular, emails or any other form of online contact weren't available, which meant we had to call 50+ therapists. And sometimes there wouldn't be a good match - no available therapists in the area who were in-network or within the user's budget.

We could only consistently find therapists if users could pay $150 or more per session. We were quite demotivated by this discovery. We didn't want to make a luxury product. We wanted to make healthcare more accessible and this went against our values. Our experience did show us there was an opportunity to help Americans afford care, so we pivoted to filing out-of-network claims.

(Better was a startup doing the same thing that unfortunately had to shut down last year because their founder passed away. We know about them and have high regard for them.)

If you have a PPO or high-deductible health plan, which close to 80% of Americans with private insurance have, after you hit your out-of-network deductible, even if you see an out-of-network provider, you can usually get reimbursed for a portion of your expenses. Even with rising deductibles, this means some people can save a significant amount on healthcare. Related but increasingly significant, insurance covers a growing number of over-the- counter medical purchases, from supplements to nicotine gum to eyeglasses, but users have to file claims themselves.

Mental health is the sector for which this is most relevant. First, mental healthcare involves recurring visits - a full course of psychotherapy often involves 15 or more sessions. Second, for various reasons a lot of mental healthcare providers don't take insurance. About 45% of psychiatrists in the US won't see new patients with insurance.

We don't have hard data on this yet, but from our experience we suspect a significant and growing number of other healthcare providers - therapists, chiropractors, physical therapists - are out-of-network. And 11% of doctors across all specialties don't take any insurance. According to some sources, $60 billion in out-of-network claims are processed annually, and the amount that could be reimbursed but isn't, because people don't file claims, is possibly much higher.

Unfortunately, out-of-network insurance is both not well-understood and frustrating to use. Many people don't know what out-of-network insurance is at all, and simply pay for services in cash. The filing process is utterly archaic. Your healthcare provider has to give you what's known as a superbill, an itemized bill with diagnosis and treatment codes needed by insurance to reimburse. You submit this superbill along with a completed form to your insurance, most commonly by mail. Claims can take over a month to be processed, and funds aren't deposited into your bank account - you get a check.

A significant number of claims are denied because the superbill was incomplete or inaccurate, or because of some unknown confusion on the insurance side. To resolve claims, you have to call your insurance. Claims customer support staff are trained to deal with only very specific common issues unrelated to your claims, and are often offshore. Two different representatives may tell you two different things. As a result of all these factors, people are not reimbursed enough for their out-of-network and other out-of-pocket expenses and are spending a lot more on healthcare than they should be.

At Franklin, we're building a service that files out-of-network claims for people. Simply send us a photo of your superbill, and we file your claim for you. If a claim is denied, we talk to your insurance to understand the problem, and then reach out to your healthcare providers to get the information needed to resolve the issue. You simply receive the check mail in the mail. (We want to have direct deposit at some point in the future.)

It is insane that so much money is being left on the table and people are paying these massive healthcare bills entirely out-of-pocket. With Franklin, we intend to help make mental healthcare and other healthcare more affordable.</comment>


Build a Business, Not an Exit Strategy - <AUTHOR> http://melanie.io/2012/10/18/build-a-business-not-an-exit-strategy/ <comment><USER2> The critical mistake here is a misunderstanding of how probability works. If we know that x% of startups succeed, that doesn't mean that each group of founders starting a startup have x% chance of succeeding.

Some people are orders of magnitude more likely to succeed than others. For those it's a good idea to start a startup. For the rest (a much larger group) it's a bad idea.

<comment><USER3> There is no misunderstanding of how probability works here.

100% of people who start businesses believe they can succeed, otherwise they wouldn't start one.

Clearly given that such a high percentage of businesses fail, you are not qualified to judge your own chance at success. Therefore the only time you can know that your percentage chance is higher than the average is when you have a 3rd party that is skilled at evaluating such things that tells you so.

In the absence of such, your chances just fall back to the raw figures that the article gives and so the analysis stands up.

<comment><USER2> Your mistake in turn is to assume that it's impossible for people to judge their own abilities.

Lots of people think they could write a decent novel. Most are wrong. Suppose only .01% actually could. If your argument were correct, JK Rowling should assume her chances of writing a decent novel are .01%. She feels fairly confident that she could, but she has to discount that, because people are often mistaken about such things, and "fall back to the raw figures."

It is possible to know that one is good at something.

<comment><USER3> That is not an example of JK Rowling judging her own abilities. Her abilities were judged by the market when she published her books.

It's possible to know one is good at something, it's just not possible to _judge your own abilities_.

<comment><USER2> I'm not talking about an existing book. I'm talking about the case where she's trying to predict the chance she'll be able to write a new book, before she's started it.

Her previous books should give her confidence that she'll be able to write another. That is one way in which she can _judge her own abilities._

</comment></comment></comment></comment></comment><comment><USER4> Hear hear. I've been saying a similar thing for a long time.

If the business is so great why sell? The act of selling implies you think it's worth less than what someone else is willing to pay which is dishonest. If you thought it was worth more you wouldn't sell (under most situations, there are always exceptions).

Why shouldn't a VC hold on to the great business they've built for the sake of future cash flow? A great business should be able to get funding.

It also drives the wrong behavior. Instead of building a real business that lasts better build the appearance of a business that will fetch a good price. Such and such multiple of sales (what about profit? future cash flow?). Such and such many users (who may never pay you a cent). This behavior doesn't stop at the first exit, it perpetuates throughout the lifetime of many public companies; focus on looking good rather than being good.

I always thought the original purpose of the stock market was for companies to raise money to go after bigger things- it seems the purpose today is to "exit".

EDIT with another thought: To me, build a business vs. build an exit strategy should be orthogonal to VC vs. bootstrap. You can bootstrap and work towards an IPO and you should be able to VC without selling the business. I think everyone would benefit from a frame of mind that is about building successful, sustainable, long lasting businesses.

<comment><USER5> > The act of selling implies you think it's worth less than what someone else > is willing to pay which is dishonest.

Any trade involves both parties giving up what they have for what the other person has. Both parties believe they gain from the exchange. There doesn't have to be any dishonesty involved : Both parties can come out ahead of where they started (because they value what they had before vs having after in different ways).

A startup entrepreneur can reasonably value freedom+cash more highly than a cash-cow business. The business buyer may value proven yield above their other alternatives for their cash. Both people win by doing the trade.

<comment><USER4> There doesn't have to be but the way IPOs are set up there's incentive for dishonesty. At the very least, the buyers of the shares are at a information disadvantage compared to the issuer. If you were to personally buy an entire private business you would never accept the level of disclosure that people buying shares in an IPO have to accept. You'd have your experts review every corner of the business.

</comment></comment></comment><comment><USER6> Although I am not going to fault the conclusion, there are a number of questionable assumptions in this analysis.

One is that the expected value of an effort is not the probability of trying times the expected value of succeeding. Multiplying by 1% since only 1% of companies raise VC funds is not relevant. The correct proportion should be the percentage of companies that try to raise VC funds which are successful. Say this is 10%.

Two is that you are not going to spend the 10 years used for comparison trying and failing to raise VC funding. After 6 months you should give up, and spend the next 9.5 years trying to build a small business (expected value = (90% chance of failing to raise VC funding) * (9.5years/10years)* $356.4k ~ $305k).

Three is that the expected value of an exit is not the probability of a minimum exit cutoff times that exit value. There is a power law distribution to success, so the expected value of an exit is much higher than $100M * 2%. I'll be lazy and guess that the expected value is 3x higher due to the power law distribution of success (I imagine 3x is grossly underestimating here).

This makes the expected value of the VC success $3.3k * 10x * 3x ($100k) + the expected value that you give up on VC funding and start a small business ($305k), or approximately $405k. Not nearly so obvious a choice as painted in this blog post.

</comment><comment><USER7> I love this. It sets a realistic expectation for the vast majority of entrepreneurs. Odds are, you're not getting VC funding. If you want to be an entrepreneur, which should mean you are inherently risk adverse, get to revenue/profitability/CF positive as quickly as possible. Again, with the odds of landing VC funding so low, if landing VC funding is the foundation of your business plan, odds are you are fucked and so is your business.

I own a B2B SaaS company that only took $20K from an accelerator, and we have just recently hit $1M ARR. Outside of the $20K, we did it through blood, sweat and tears. No angel money, no VC money. Not that I'm opposed to outside money, but I liked the challenge of CF financing a company (wasn't always easy), but our initial product fills a niche and doesn't ramp up to the $100M in 5 years that gets the VCs' investment weenies going. Instead of spending the potential enormous amount of time that it could take to raise money, we decided to just build a business. And we are doing that shit...

<comment><USER8> Could you email me? I'd like to talk more about this offline: ryan at dailypath dc

</comment></comment><comment><USER9> It seems like each week or so we have an existential crisis on HN by an author who has realized the "go big or go home" mindset/lifestyle might not work for them. There is a HUGE fallacy in all this expected value rationalization for building a lifestyle business. You are going to DIE someday. You don't have unlimited time, and I'd rather take crazy bets toward building something risky and radical than be comfortable and safe with "a profitable, small web-based business in just a few years, take a great salary and work 30 hours week".

Yes I know I could do that, I was able to do that at 19. I'm doing a startup because that isn't enough for me. That would be like retiring at 19.

Blog posts like this feel like "why I settled at 20-something". Come on, really?! Ugh

<comment><USER10> Posts like this always essentially boil down to one statement:

"The startup (in the pg sense of the term) lifestyle is not for me."

And that's ok. For most people, making a dent in the universe is not an existential need.

<comment><USER11> I always forget the name of that startup where Torvalds wrote Linux ... he couldn't have made a dent without a startup, could he?

<comment><USER12> But the same reasoning applies. What was Torvald's "expected value" at the time he wrote Linux? Probably -XXXXX$ if we factor in opportunity cost.

</comment></comment></comment></comment><comment><USER13> This post is based on so many false assumptions that it's meaningless. Most small business owners/bootstrappers are NOT millionaires, most are just barely scraping by. Less than 3% of small businesses make $250K or more in profit.

([http://voices.washingtonpost.com/plum- line/2010/09/boehner_c...](http://voices.washingtonpost.com/plum- line/2010/09/boehner_concedes_only_three_pe.html))

So if you consider "failure to make over $250K" as failure, 97% of small businesses are failing.

<comment><AUTHOR> That is a fair point. However, even if you adjust the assumptions for expected value to be 3% (vs 25%, which is what I use), it is still an expected value of $43,000, over 14x greater than that of your average "I want to be the next Instagram" app. I wanted to use 2 hypothetical examples with some reasonable assumptions just to show the staggering difference in probabilities of each scenario.

<comment><USER13> Yes, but then the expected value is less than getting a job.

</comment></comment></comment><comment><USER14> I don't think there's a real formula that will allow you to calculate the expected value as there's so many other variables that come into play.

A lot of it comes down to the founder(s) personality, their immediate network, and the true need in the market for their business/idea.

That said, this is exactly the way I've felt about most startups. You're making a time machine for people's Twitter? That's not a billion or even million dollar idea. If you can figure out how to generate revenue it's a small business at best.

Thanks Melanie. This article was a breath of fresh air.

</comment><comment><USER15> This is one of the few business advice pieces I've seen lately on HN that is both USEFUL and USEFUL OUTSIDE OF THE USA (or outside any other "growth accelerating" places that popped up in a few other corners of the world...), despite being specific and inspired by US based business experience only.

Nice to see the other side of the pond is not really a "different planet" :)

</comment><comment><USER16> As with all things, it's not so black and white. I know it's popular to hate on venture funding here on HN, but the industry exists for a reason. There are certain business models that only work at scale, and in the meantime must be supported by VC cash. For bootstrapped revenue-generating companies, VC funding can be the difference between lifestyle business and IPO. Congratulations to the OP for building a profitable business, but let's keep in mind there are many paths to success.

<comment><USER14> I don't think it's popular to hate on VC on HN. Sure there are a small number of people who are bitter who speak out against it but I think most people on here are totally for VC funding.

</comment></comment><comment><USER17> The expected value is wrong here, because it imagines a distribution where you either go big or you go home with bubkis.

The truth is that there are a lot of things in between. Owning 33% of a company that is making millions, and is funded, but not sold for $100M, gives you a nice income and you work on something you like. And all this time you were hiring great people and receiving a good income. If you compare that with the lifestyle business, where you have to grind it out, you have a lot more risk in the lifestyle business actually.

So no, not only was the math in the calculation wrong, but really, VC is about scaling a startup (in the Paul Graham sense) into being worth tens and hundreds of millions of dollars and beyond. It's often worth it for the people you meet and the potential exit.

<comment><USER18> > Owning 33% of a company that is making millions, and is funded, but not sold > for $100M, gives you a nice income and you work on something you like.

Odds are that your VC isn't going to see eye-to-eye with this approach.

<comment><USER17> Not sure where you get such a conclusion about all the myriad VC firms out there. Founders who are getting a nice income and are able to work on changing the world can do well for the company and its investors:

<http://www.youtube.com/watch?v=u6XAPnuFjJc>

</comment></comment></comment><comment><USER19> Why don't we strive to have "dent in the universe" mean "make the world a little better" rather than whatever the heck we want it to mean?

Sure, be excited about tech, but be more excited about the possibilities it opens for the people using it.

</comment><comment><USER20> Great article. The analysis is correct and to the point.

This reminds me of an article about entrepreneurship in a small town in Germany, where they are known for high technology materials engineering. One interviewed businessman was shocked at the offers he received from large, multinational companies. His family business had begun generations ago, and his goal was not to get rich quick, but simply make a living doing what he enjoyed.

That's my definition of success. To each their own.

</comment><comment><USER21> Awesome post...useful advice for solo founders like myself.

</comment><comment><USER19> Sometimes it seems that we miss the forest for the trees.</comment>


Orchid: a new surveillance-free layer on top of the existing Internet - <AUTHOR> http://orchidprotocol.com <comment><USER2> Hello! My name is Jay Freeman (saurik), and I was both deeply involved in the design of Orchid's protocol as well as in charge of the initial implementation of the networking and routing logic, and am happy to try to answer any questions people might have about what we are working on!

(edit:) g_simonsson, who now has an account on Hacker News and has joined this thread, is Gustav Simonsson, another developer working on this project in charge of our smart contracts and payment logic. He was one of the core developers who worked on Ethereum, specifically focussing on blockchain security.

<comment><USER3> 1\. Based on the paper, it seems ORCHID by itself (without bandwidth burning. not sure if this is trivial to get right. is there tooling?) is as vulnerable to traffic correlation attacks as one might expect. Are there any other known weaknesses? I would expect a "limitations" section in red letters on the front page, instead of the nebulous and likely wrong "the NSA can't hack it either" claim.

2\. Given that todays secure overlay networks generally offer extremely bad performance in the Mbit region compared to using the underlay where 10G uplinks are becoming widespread for servers, and the extra hops increase latency, do you think secure overlays will ever be near enough to underlay speed to become widely used? Do you have experience how well ORCHID scales up? And out?

[edit: Don't mean to sound too harsh. I am very glad you're doing work in this area.]

<comment><USER2> 1) First off, as a fast apology and as mentioned lower in this thread, that FAQ answer about the NSA was rewritten two nights ago and failed to end up on the website; the updated answers have now been pushed (to say "Yes", actually, as the first sentence, instead of "No" with a qualification that essentially changes the answer to "well, yes").

That said, you are absolutely correct, and my colleague and long-time friend David Salamon (the person who has been the lead author of the whitepaper) wants that to be very clear. We _are_ implementing bandwidth burning (and are even discussing some interesting extensions of this scheme involving both Turing-complete-w/-limited-execution-time programmable bandwidth burning as well as a form of global bandwidth burning), so whether you call that a "known weakness" for all of Orchid or not is slightly confusing ;P.

When we have actual software that someone could possibly use I am going to be pretty adamant that there are good disclosures in the UI with respect to the security tradeoffs, and for right now the whitepaper goes into extreme lengths to qualify what is or is not possible, in our current understanding, with and without bandwidth burning in place. (We are also looking into taking aspects of the algorithm and building it out with a proof assistant to have even greater assurances.)

2) One place where Orchid will hopefully shine (in the future tense, as this part is decidedly not yet implemented) is that we are working to allow multiple routes to/from the destination exit node using a scheme similar to MPTCP, which solves a lot of the bottleneck issues you normally find running through an overlay network.

<comment><USER4> please note that we at Delft University of Technology have been working on a similar bandwidth-as-a-currency system since August 2007.

BBC New article from 2007: [http://news.bbc.co.uk/2/hi/technology/6971904.stm](http://news.bbc.co.uk/2/hi/technology/6971904.stm) Running decentral market code and tech portfolio: [https://github.com/Tribler/tribler/wiki](https://github.com/Tribler/tribler/wiki)

Notes after reading your whitepaper, latency and complexity matter. We copied the Tor packet layout and deployed a simple NAT puncture method to avoid difficulties. Your proposed approach looks like many years of work.

</comment></comment></comment></comment><comment><USER5> They call it a protocol but this is very much a startup/ICO, having raised nearly $5m in seed. Not that there is anything wrong with this, it seems like a very cool project. But it does seem to be very intentionally presenting itself as something that is closer to a not-for-profit a la Tor.

<comment><USER2> It is maybe worth noting that we had actually looked into whether we could be registered as a non-profit and still raise money by these means to build out our implementation and market it to the world, and that was simply not possible :/. We are, however, all deeply committed to open source, and are working on how to best ensure that this is enforced in the company's charter (such that even if there are any future changes, there will never be any fear).

All of our work is to be released under the AGPL3; and, while we have filed for a patent, we will be licensing the patent to the world in a manner similar to how Mozilla manages their patent portfolio. Brian Fox, who is in charge of making sure we have a successful and inclusive open source project, was the original developer of the bash shell, was for a while the maintainer of GNU emacs, and was the first employee of the Free Software Foundation (as Richard Stallman was a volunteer), and so is keenly aware of how important it is for technology to be available to everyone.

<comment><USER6> This is.. very exciting & excellent :)

</comment></comment></comment><comment><USER7> Very interesting. Visited Noisebridge a while back and there was a post on the door telling visitors what to do when the FBI visits to ask about the TOR exit node. I wonder how this tool avoids the exit node problem.

Also, from the FAQ:

>Can't NSA just hack into this too?

>No. Because of its fully decentralized approach, distributed architecture, and the size of the global network, Orchid cannot be easily hacked by any single government or entity.

That's not really a satisfactory answer. First, it doesn't answer the question. The question was not "can NSA easily hack into this." And I don't think the NSA is necessarily deterred by something being not easy. The bar needs to be higher than not easy, even if "not easy" is a polite understatement. Also relying on the size of the network means there is a bootstrapping problem, right? Hopefully they will get there.

This doesn't mean the system is bad... I'm just saying the FAQ answer is bad.

On the positive side, given the cred of some of the people involved (saurik!) I am optimistic this may well have a shot at working.

<comment><USER2> Ugh. FWIW: we agree. That FAQ answer was rewritten, and it failed to go on the website.

Here is the updated text that was written a couple nights ago by one of the people who helped design the protocol with me after being confused by the answer on the website.

> Yes. Our initial release targets China as the adversary, which is a more > tractable problem. We may implment full Chaumian mixes in the future (which > are immune to metadata/traffic analysis), but they are unlikely to be > complete for our first public release.

<comment><USER8> Would you consider changing "China" to something like "Chinese government"? Or, if I'm understanding it correctly, maybe something like "Our initial release operates with a threat model of an authoritarian government's internet censorship, such as those seen in China, Turkey, or Russia."

<comment><USER2> I agree with this edit and have made a similar change, pushed it to our server, and are poking others for review. Thank you for your feedback: that is a much clearer answer (as otherwise it sounds like China is hacking into places as opposed to "we mean their firewall").

</comment></comment></comment></comment><comment><USER9> I understand the plan is to be open source, but can someone help me understand why it's not developed in the open? I mean, you can tell people you're not accepting external input at this stage, but why not let people watch it being developed? I can't think of any reason that doesn't make me think that the goal may be openness/transparency but not the practice. Or did I miss the link to the repo?

<comment><USER10> There is no code. This is a cash grab. The author politely states they needed review of white paper concepts, so no they are not sure it will even be worth writing. The idealology of decentralization also reforms the patent system, so this project will fall like other over hyped icos, will get normalized on listing. Probably a huge discount for "seed".

<comment><USER2> Yet... there has been no monetizaton mechanism described that doesn't involve us releasing code, as we have been very clear that we refuse to do a token sale until such time as we have a fully working system. So, I am not sure how your theory works out here, but it will be interesting to see if you comment back here once we release code for you to actually run.

As for why we want whitepaper feedback: that is because we believe in peer review of design.

</comment></comment></comment><comment><USER11> Why were venture capitalists interested in funding this?

What do they get out of it?

How and when do you intend to 'exit'?

(Apologies for the skeptical tone, I think this seems like a cool idea.)

<comment><USER12> Good question, especially since we made it clear to them that we have no business model and no plans to ever have one.

There is a fixed number of Orchid Tokens that are used for payments within the network of relayed traffic (source nodes pay relay and exit nodes).

The only way for anyone to gain financially is for these tokens to appreciate in value, which is tied to the utility of the network.

So there's no exit, rather it's a continuous token incentive for all stakeholders similar to the Ethereum network which was funded by selling the promise of future tokens.

<comment><USER13> _The only way for anyone to gain financially is for these tokens to appreciate in value..._

Sounds like a security.

</comment></comment></comment><comment><USER14> > This would be a disruptive inconvienance [sic] for Chinese business people, > deeply effecting [sic] commerce, as well as quality of life for the average > Chinese citizen.

This sentence has two typos. It should be changed to:

> This would be a disruptive inconvenience for Chinese business people, deeply > affecting commerce, as well as quality of life for the average Chinese > citizen.

But TBH, stating that the lack of WebRTC or VPS web hosting would, "deeply affect the quality of life for the average Chinese citizen," seems like hyperbole. Maybe:

> This would be a disruptive inconvenience for Chinese business people, deeply > affecting commerce and web access for millions of Chinese citizens.

<comment><USER2> You caught this answer right before it was updated with a much longer answer that I had written a couple days ago (and which fixes at least one of those typos). Would you mind reviewing it to see if you still think it is a stretch?

</comment></comment><comment><USER15> > we believe will force anyone trying to block our traffic to end up blocking > large areas of the internet--such as every server being hosted on Amazon Web > Services--which would cause a serious problem for Chinese business people, > deeply affecting commerce, as well as quality of life for the average > Chinese citizen.

Not really. China has several replacement solutions for AWS. Ali Cloud, Baidu Cloud, etc.

It would just mean blocking a few more websites hosted on AWS and used by Chinese people, GitHub maybe? Then again, it is easy to build a GitHub clone within China with the decentralized (irony much?) git protocol.

<comment><USER2> We believe, as do many people as far as I have seen, that if a country like China could block the entire world of Internet, they would. Regardless: I consider it my mission to make them have to seriously consider doing just that, and if I read a news report saying "in a shocking turn of events, China has decided to cut off all access to the Western Internet in an attempt to shut down Orchid", I will consider myself to have won that round, not lost ;P.

<comment><USER15> > I will consider myself to have won that round, not lost ;P.

If fame and victory are what your company is going after, instead of taking care of your potential users, then I am sorry that you won't have my support.

Btw, thank you for creating cydia if you were the author.

<comment><USER2> To "win" here means "win a battle on the road to an Internet free of surveillance". I (personally) firmly believe that if China were to do that, that the people would not stand for it; any form of "half solution" or "compromise position" where the Chinese govnerment can block just enough to remove access to material that bothers them while only minority inconveniencing users, is a world in which this form of freedom is actually less likely, as people are willing to tolerate a sad lot before try finally snap back :/.

It is also worth noting that if they went that far, at least some of the other Western countries that are currently willing to tolerate their seemingly- limited control over the Internet would no longer feel comfortable standing idly by and might have to start making resolutions against them. Again: forcing that chess move from them would, in my (personal) book, be a strong win in this fight.

FWIW, you seem to be taking a very short-term definition of "taking care of your potential users" that I have sometimes seen in attempts to make excuses for Apple over the past decade of my work on Cydia (such as when I have stated that one of my "win conditions" for Cydia Impactor is for Apple to remove entirely the Free Developer profile system), and I (personally: I do not speak for anyone else on my team when I say this) do not feel the loss of your support on this particular day (though maintain hope that in a few years you will thank us as you have now me for Cydia).

<comment><USER15> I think you might have misunderstood what I mean by "taking care of your potential users". It is not a short-term definition. On the contrary, I want a long-term solution, where users trust the platform that your company provides, trust that your company will be there for the users when Chinese government bans your software, instead of celebrating the ban as a success and call it a day (like what Google did).

To celebrate the banning of your platform would be what I call a short-term solution, because in the long run, it is merely a temporary attempt at solving the problem, that people will eventually forget when a long-term solution prevails. Chinese people, from all spectrum of political views alike, do not thank Google for exiting China, because it does not solve their problem. The act merely throws the problem into limbo state awaiting for other people to solve.

Also, I think you mixed up "support" and "gratitude". I thank you for creating Cydia as well as attempting to solve this problem with surveillance and censorship, as always. However, I disagree with your approach and therefore will not support this platform unless I can see that you have something bigger in mind beyond "celebrating being banned".

<comment><USER2> What you are asking for--somehow being immune to China deciding to ban the _entire Internet_ \--is an apology for China's behavior and fundamentally, plain and simple.

After all, what is the alternative? In the case of Google, the other option you seem to think is reasonable is for them to censor themselves... but that _is_ a loss... that is in fact the _primary loss condition_.

That said, was Google leaving China the same kind of win for the Internet as what I am even talking about? No, because Google is but one company. My win condition is "China is forced to block some incredibly large percentage of the entire Western Internet", not "they block us" or "they block a few websites".

Google gave up long before I would say they won: their next step should have been to start doing exactly what we are now doing, shipping it as part of Google Chrome, and figuring out ways to distribute copies of the software in China.

There is a big difference between "celebrating being banned" and "celebrating your opponent being forced to make the difficult decision between salting their own earth to keep you from invading or accepting defeat".

This is all particularly the case given that my argument is rather consistent here that that won't happen: that China can't do that, as that is just too drastic of a move and it will cost them the support of businesses, citizens, and governments.

As for your final paragraph, every single decision made about Cydia was made within this moral framework... one which you do not support. You either should not be thanking me for what I did while working on it if you fundamentally refuse to support my methods and goals.

Honestly: it sounds like you would have preferred a world where I built a centralized Cydia and allowed Apple's censors access to the catalog... that completely misses the point of Cydia, so I don't know what you possibly wanted. With this attitude I not only don't feel the need to have you support Orchid, I would like to hand you back your false gratitude for Cydia :(.

</comment></comment></comment></comment></comment></comment><comment><USER16> The FAQ says

"Bandwidth contributors simply install Orchid and activate their Internet connected device as a node - either as a relay or proxy - and then they set permissions like sites they want to blacklist or whitelist, and they earn tokens into their Orchid wallet for sharing their bandwidth."

So hopefully the blacklisting will eliminate the problem of nasty content that plagues anonymous networks like Tor.

<comment><USER17> For that purpose I would think optional whitelisting would make more sense. Instead of spyware lists you subscribe to in uBlock Origin, you would subscribe to a vetted, whitelist of known to be generally acceptable Orchid content.

Others might want to have empty filter list to be a complete transit/peer.

Therefore I don't think there's much room for users who want to blacklist but are not rather looking for a whitelist.

The blacklist will grow and grow, while the whitelist size will be pretty stable.

That said, I haven't used it, so the implemented blacklist approach might already support the above cases and be sufficient.

<comment><USER18> If you're only using a "stable whitelist" of websites on the internet then you're not the target audience

<comment><USER17> I know what you mean, but for this to be more widely used than Tor there is a need for me to know that nodes I operate only process stuff that won't be identified by 3rd party to sue me into a jail or financial ruin.

When I say whitelist, I do assume prefixes in the DHT (or whatever design they're using) can be used, as otherwise the whitelist may also grow too big to be practical. If however it will gain the concept of domains or such (which is also kinda a prefix), a whitelist will also be more practical.

Tor exit nodes are not operated by many in various places where they would/could because of concerns of the exit node addresses being enough to legally ruin operators' lives, even though technically it's a pure transit. Nobody sues the county because someone committed vehicular manslaughter on their public road. But because laws are skewed against the Internet right now, all someone needs is an exit node IP to make you regret.

If, and there's little to analyze/go by right now public, nodes and everyone is totally oblivious to what packets are transmitted; and if also the packets stored on nodes' disks are encrypted/sealed, then one could assume a filterless system to be practical.

This is all speculation, based on the little info there is. I really hope the team has come up with better designs that obliviate the concerns surrounding Tor, and ideally also not suffer from Freenet like slowness.

EDIT: Of course, once you have a filter, you will need to deal with the responsibility like Youtube does. If you do not know what's on disk or passing through, which is the ideal technically, then it would be best for the Internet and free communication. So, I'm not sure if a filter is a good idea, if that means you get subpoena'd and held liable for enabling one too many whitelist subscriptions.

</comment></comment></comment></comment><comment><USER19> Regarding the alternative approaches section (which I'm assuming when complete will include an actual comparison to these alternative approaches?), another alternative approach you might want to mention is that improved mixnet approach that people were talking about last year, that once set up is supposed to be faster than Tor, or maybe mixnets in general since they seem to have a long history.

Edit: Riffle, that's what it was called: [https://en.wikipedia.org/wiki/Riffle_(anonymity_network)](https://en.wikipedia.org/wiki/Riffle_\(anonymity_network\))

</comment><comment><USER20> Thank you very much for taking time to thoroughly answer questions. that combined with whitepaper and faq has left a very good impression to me, both socially and technologically. Best of luck to your team!

</comment><comment><USER21> The liquidity offered by the token model means that early VC/hedgefund backers often get heavily discounted tokens in the presale, and can then turn round a month or two later and dump a portion of their tokens on public investors for a sweet profit or to mitigate their own risk.

Polychain did this with 0x project: [https://www.reddit.com/r/0xProject/comments/6w3zv0/0x_being_...](https://www.reddit.com/r/0xProject/comments/6w3zv0/0x_being_dumped_by_polychain_capital/)

The effect was attenuated in this case by the success of the project, however in lower performing ICO's like Kik's Kin, where the market price is 50% of public sale price or less, big early investors dunking on public token sale participants feels pretty wrong.

In addition to this, VCs are also often playing a role in promoting public sales shortly after their own pre-sale investments, without disclosing their own discount or terms of obtaining tokens. Its kinda like telling people to buy a bunch of stock at $10 when you bought all yours last week for $5 and you're keen to dump a portion of them for $15.

[https://medium.com/@rzurrer/raiden-fundamental- infrastrucutr...](https://medium.com/@rzurrer/raiden-fundamental- infrastrucutre-led-by-a-great-team-96c725fba9de)

It may be legal, you may think it's their right, and to be fair there is risk involved in being an early backer in a potentially failed project, however it is something to think about.

</comment><comment><USER22> This is needed but there should be more insight into the current surveillance problem and how Orchid solves it. This seems to be focused on China.

Because surveillance is a potentially serious problem, it should not be used cynically to market some new product that just adds another layer but doesn't really solve the problem.

For the average person its not China or Russia but their own state that is a bigger problem as they have power over them and can interfere with activism, journalism, dissent etc. The intensive profiling by Google, Facebook and others is another problem.

Was the Internet designed to be anonymous? Governments control access and can stomp down at will. ISPs and telcos know who you are, server and vpn providers know who you are, Facebook and Google have a lot of data on you. Only the very committed and technically adept can perhaps attempt any sort of anonymity.

Something like Musk's satellites with some basic free access may perhaps be the first of many steps to some kind of anonymity.

</comment><comment><USER23> There's a lot of projects that sound exactly like this. What they all have in common is that they all use an Ethereum (ERC20/22) token and all have very large amounts of money flowing into them, from private investors, companies, ICOs, etc.

It would be nice to see more projects take routes that projects like tor take, where the founders do not accept millions of dollars of money before they even have a working and complete protocol.

Ethereum has some amazing potential and technology behind it but I lost interest in this the second I saw they had already raised $5M and haven't even done their ICO yet. Their FAQ section seems woefully incomplete given this amount of money.

<comment><USER24> Check out cjdns for a less stupid, less blockchain BS take on fixing the internet.

[https://github.com/cjdelisle/cjdns](https://github.com/cjdelisle/cjdns)

<comment><USER25> One question I still haven't found answer to: what's with the name? What does it mean / why is it the way it is?

Should it be read as CJ DNS, i.e. something to do with DNS? Or CJD (author's initials) NS (??? again, name server?)? C (?) J (JavaScript/Node) DNS?

Related, what the hell it is, actually? You say in other comment that it's a mesh network, they say it's a network, but I see a _program_ to install. How a program is a network? What does it do?

<comment><USER26> Tor is a network, and yet you have to install a program to connect to it. You can think of the cjdns program like a driver that allows you to connect to a cjdns network.

To answer your question on what it does : cjdns is a routing protocol that intends to replace the current IP protocol. Like the standard IP protocol, it needs a way to communicate with another node to function (the default gateway). From there, you can communicate with the rest of the network. Your packets will go to the default gateway, who will then redirect it to the "closest node" recursively, until it reaches its destination.

You can setup cjdns over the current internet (essentially making cjdns an "overlay network" like tor) using UDP, or through an ethernet link.[1]

CJDNS indeed works in a mesh : anyone can route other people's traffic, somewhat similarely to tor. The only difference is that you need to manually configure your "Entry Node" (to lift tor terms). See [0] for the reasoning.

About the name, the creator is Caleb James DeLisle, so I guess CJDNS stands for Caleb James DeLisle's NameServer ? Doesn't really make much sense but I don't see how bad a thing it is.

[0]: [https://github.com/cjdelisle/cjdns#2-find-a- friend](https://github.com/cjdelisle/cjdns#2-find-a-friend) [1]: [https://github.com/cjdelisle/cjdns/blob/master/doc/configure...](https://github.com/cjdelisle/cjdns/blob/master/doc/configure.md#connection- interfaces)

<comment><USER27> Caleb James Delisle's Networking Suite

</comment></comment></comment></comment></comment><comment><USER28> So the business plan is to become the oligarchs of the new network?

<comment><USER12> Hopefully not! Ideally token allocations are fair and do not skew ownership towards any individual or entity while still providing good incentives (we're still working on figuring out what good allocations look like)

<comment><USER28> Are there any public details on the Simple Agreement for Future Tokens?

Sorry for nagging, but I'm one of those skeptical of greedy people acting maximally greedy saving the world as a process waste.

Edit: typo

<comment><USER12> The main reason there's currently not more details around the SAFT and the token model/allocations is that it would distract from the overall project and tech, given the current hype of blockchains and tokens.

There's certainly economic incentives for all participants, and what I find interesting is that since we're open sourcing all the code, anyone can easily deploy another network with a different token (model).

This puts pressure on us to propose fair token model/allocations else we risk disruption by forks or simply new Orchid networks.

</comment></comment></comment></comment><comment><USER29> the project seems very cool. related: [https://medium.com/@stevewaterhouse/how-token-sales-can- be-a...](https://medium.com/@stevewaterhouse/how-token-sales-can-be-an- instrument-for-social-change-91b6d01792d0)

</comment><comment><USER30> This is a funded startup.

How do they intend to make money?

<comment><USER31> Sustainability isn't even the greatest concern: where there are customers (people who pay for access to the network) there is a database of customers (in order to allow authentication) that has to be kept out of the grasp of government agencies.

<comment><USER2> Actually, that's the great part about what we are doing: it is all built on Ethereum, so there is no centralized database, and the users are generally pseudonymous! OK, you might then say "isn't that just a decentralized database?", but in addition to a form of "probabilistic micropayments" that ends up shrouding most of the participants, we are also working on integrating other techniques to make the payments fully anonymous (and have brought on a team of advisors which includes a professor of cryptography who specializes in this area).

</comment></comment></comment><comment><USER32> What's the weakest point in the design so far?

<comment><USER2> The answer you aren't looking for is probably something like "traffic cloaking" or "connection bootstrap", but my personal answer to that question is "so you sit down at a computer and download our software. you have a wallet full of cash and credit cards... now what?".

</comment></comment><comment><USER33> Is there any thought given to the response to a node operator who is detected/proven to be operating the node for surveillance or compromise? What would the response be? Will tokens not be sold to such a node operator?

<comment><USER2> The goal of our network is to make it so that the network is protected as much as possible from users running compromised nodes. If you are a single node, you should not be able to do anything against the network. As the number of nodes you are in the system increases, the probability you will be able to pull of various forms of attacks against users of the network also increases. We are working on documenting these security tradeoffs in detail in our whitepaper (and any attempt I make to describe them here will be less accurate and thereby dangerous ;P).

However, it is worth noting at a super-super high level that we are hoping to get a large number of nodes running the network and we have parts of our system that involve proof of work in a way similar to a cryptocurrency blockchain like Bitcoin. If a node in the Bitcoin network is operating for the purpose of compromise, it need not be banned from mining blocks: the network is designed in a way to make it so that such a node would not be able to pull off the kinds of attacks that people care about unless it owns a significant percentage of the resources being applied to securing the network. Our network has similar properties.

<comment><USER33> "We are working on documenting these security tradeoffs in detail in our whitepaper (and any attempt I make to describe them here will be less accurate and thereby dangerous ;P)."

Maybe a whitelist of nodes themselves? (mostly in jest there)

I suppose an agency with funding who can endlessly sockpuppet could gain a substantial portion of network nodes, but if they're unable to censor more than per-random-packet, a surveillance goal seems more likely. While this may not meet conventional definition of 'censorship', I wonder of the chilling effect. I suppose any such move toward a censorship-free network should expect surveillance. c'est la vie.

Thank you for taking time to answer my question.

</comment></comment></comment><comment><USER16> This sounds like a great project. I have some thoughts on how to communicate about it when it is ready to go.

With the general public, you might start with some things they are unhappy about with the present internet, and for each of them explain how things would be better for them if they used Orchid.

For the more techy minded, you could do a list of similar things like tor or IPFS, and for each of them explain how Orchid is similar or different, and what are your reasons for deciding to make it that way.

<comment><USER16> A further thought. You should focus on comparing it with Tor, like maybe making it the first one to compare with, since it is both similar and well- known. And then maybe add on other ones.

Another thought: compare early on with Ethereum and IPFS. Yeah, they are really different things, but as soon as you say web decentralization, people who are not techies but read about such things will think of them, so you need to explain early on how this is different so they get pointed in the right direction.

</comment></comment><comment><USER34> Ok, I wasted a few hours reading the whitepaper. It was just as idiotic as you'd expect. Here are a few notes:

It seems like this takes Bitcoin and Ethereum electrical power waste to a whole new level

      To produce a medallion, a peer takes a public
      key K , and the most recent Ethereum block hash
      E , then (iteratively or in parallel) locates a
      salt S such that H ( K, E, S ) ≥ N ,  where N
      is some difficulty scaling factor.

And with all that "an _anonymous_ " network _does not provide payment anonymity_ (while amount paid correlated with amount of data sent). So this will disclose how much data each user put into the network. This is valuable if you can observe large amounts of data out of the exit nodes (like a government-scale actor might). This might even allow correlation between data and users.

      We will argue that The Orchid Payments
      (section 7.12) fulfill all but the
      anonymity requirement.

Yup... not anonymous at all...

      The pseudo-anonymity of Orchid payments
      is equivalent to what can be achieved in
      regular Ethereum transactions

But it gets better. While you can use anyone as a relay, you PREPAY them before they do any work for you. This is claimed to solve the problem of nonpayment, but nobody mentions the opposite problem: I take your money and do no work for you. The money cannot be taken back - I own it now. cool.

      If there is some setup cost to Alice and
      Bob's relationship ( S Alice , S Bob s.t.
      S Alice > xy, S Bob > xy ), the answer is
      yes.

And for fun, it also messes with SSL. Allegedly only "for good", but who'll notice when this changes?

      (see the entire section 10 in whitepaper)

And then there is the completely not scalable method to prove you will route as required by the network. For that you need to literally get EVERY node on the network to sign off on the fact that you can route to them. So the size of this proof, its computational cost, and the time to complete it will grow as the network does. Total work done just to provide for network joining grows as N^2. Oh, and god help you in case of temporary partitions (like has happened in the internet a number of times when various underwater cables were cut). For extra fun, nobody mandates you actually _do_ route anything. Only that you prove you can. I see nothing in there that prevents packets from being purposefully dropped after collecting payment for "forwarding" them.

      (see whitepaper section 11.5)

Oh, and if you thought this will provide _INTERNET_ access, you are mistaken. Apparently HTTP w/ DNS only. As per whitepaper, exit nodes can filter based on domain. I am not even going to mention that running an exit node without a whitelist is idiotic (one user downloads CP, you go to jail). So everyone must have a whitelist? And just how big do you figure a typical "whitelist" of safe domains is? 100GB, 1000GB? Good thing you need to send it as a reply to _Get Offers_ request.

      (see whitepaper section 11.9)

Blocking this is easy (as the whitepaper concedes) by DPI and blocking Ethereum. Whitepaper claims "this will be solved later" but offers not even a handwavy explanation of how this will happen while remaining compatible with mainline Ethereum clients...

      (see whitepaper sections 12.3 & 15.3)

Of course, besides handwaving on "it will totally be ok"(tm), there are no real world performance numbers.

      (see whitepaper section 13.3)

But funny most of all, these guys didn't address the simplest attack of all. Pretend to be an exit node, /dev/null the traffic, pocket the payment.

My guess: the scheme is simply to raise an ICO and laugh their way to the bank. (as always)

<comment><USER35> (Note I'm David Salamon, one of the white paper authors.)

I agree with your payment analysis. As you point out, we explicitly disclose that payments need modification before they will be at the level of anonymity required for Orchid. Anonymous payments for the Ethereum platform are being worked on right now by very talented teams. As we don't believe we have anything substantial to add to that conversation, we are deferring to them, and will simply audit and adopt the methods they design as soon as doing so is feasible.

> And for fun, it also messes with SSL. Allegedly only "for good", but who'll > notice when this changes?

This is a fully general argument. Even if we didn't check the validity of certificates (which many phone apps don't do but should, hence our adding the feature), couldn't you still claim we might act maliciously in the future without anyone noticing?

(As you don't point out, our software will be AGPL3 licensed / open source. If one person notices and says something, everyone will notice. This is the logic behind trusting Linux, OpenSSL, Tor, etc... Are you aware of a better solution? I'm skeptical one exists.)

> Total work done just to provide for network joining grows as N^2

No, connection proofs are O(routing_table_size * per_element_proof_length). Routing table size is O(log(n)) and per-element proof length is the routing tables of the point-to-point route taken O(log(n)^2), so the proof is O(log(n)^3) -- likely much less in practice, as the successor and predecessor nodes are very likely to share _every_ connection that the joining node needs to make.

This is good feedback, we should explicitly do that calculation in the whitepaper, added to the TODO list. Thanks!

> Blocking this is easy (as the whitepaper concedes) by DPI and blocking > Ethereum. Whitepaper claims "this will be solved later" but offers not even > a handwavy explanation of ohw this will happen while remaining compatible > with mainline Ethereum clients...

The data which needs to be received is already defined by Ethereum's gossip protocol, so in theory adding Ethereum proxy support would be trivial. (Just proxy Ethereum traffic the same as any other traffic.)

We have not specified that, however, because we are worried about the possibility of a malicious proxy hiding some transactions from a customer. Once we feel like we have a good handle on how to analyze these kinds of attacks, and have a solution which mitigates them, we will add a solution to the whitepaper. This is somewhat outside our core product, and so has suffered a bit of neglect. It might be as simple as just receiving gossip from multiple independently chosen Relays, but I'd like to be more sure before making that the official method.

> Of course, besides handwaving on "it will totally be ok"(tm), there are no > real world performance numbers.

We felt that publishing real-world performance numbers would be best left until after there was a real-world system.

> But funny most of all, these guys didn't address the simplest attack of all. > Pretend to be an exit node, /dev/null the traffic, pocket the payment.

We do in a way. If a proxy behaves this way, Chrome disconnects, resulting in the attacker not being paid beyond a couple of kilobytes. This is not profitable, but still annoying. So the attacker causes a widespread nuisance, spends most of their time waiting for new customers, and suffers the loss of their CPU to proof-of-work. It's not profitable, but I expect some users will still do it for the lulz.

> My guess: the scheme is simply to raise an ICO and laugh their way to the > bank. (as always)

(Note I'm not a cofounder)

I joined in spite of not being able to negotiate myself to anywhere near cofounder levels of token ownership -- I would probably have made more by accepting a different job doing mobile A/B testing software company.

I do think some of the cofounders are very much in this for the money, but I also think every single person on this team hates what's happening on the web with respect to privacy and censorship. If that changes, I'll probably end up leaving and working on this separate from them. (Well, or demanding more money?)

So basically: if I don't quit you can conclude they've either paid me to keep quiet, or I still like them. Best I've got for you.

> Ok, I wasted a few hours reading the whitepaper. It was just as idiotic as > you'd expect. Here are a few notes:

Thank you very much for your feedback, sorry you felt it wasn't worth the time. Maybe next draft? :p

<comment><USER34> > Maybe next draft? :p

Looking forward to it :)

Also, what about the other half of my points? ;)

<comment><USER35> > Also, what about the other half of my points? ;)

Oh, sure, sorry about that. Did you edit the post I replied to? It looks like I only missed two points:

> But it gets better. While you can use anyone as a relay, you PREPAY them > before they do any work for you. This is claimed to solve the problem of > nonpayment, but nobody mentions the opposite problem: I take your money and > do no work for you. The money cannot be taken back - I own it now. cool.

Relays are sitting there burning their CPU while they wait for a customer to show up. This is a continual cost, which makes Relay/Proxy operation unprofitable unless they receive a steady stream of transactions. If they "take the money and run" they only pocket the initial payment, maybe as much as a couple kilobytes of bandwidth in tokens. If they "take the money and stay" they get a stream of payments from the customer. Therefore it's economically irrational for them to run -- the PoW's cost makes them boundedly trustworthy if they are economically rational.

(I sort of covered this in the /dev/null attack you proposed, as it's the same case except /dev/null has the attacker also uploading a bit of useless traffic. I should have been more explicit.)

> Oh, and if you thought this will provide INTERNET access, you are mistaken. > Apparently HTTP w/ DNS only. As per whitepaper, exit nodes can filter based > on domain. I am not even going to mention that running an exit node without > a whitelist is idiotic (one user downloads CP, you go to jail). So everyone > must have a whitelist? And just how big do you figure a typical "whitelist" > of safe domains is? 100GB, 1000GB? Good thing you need to send it as a reply > to Get Offers request.

Lol, yeah. Whitelists are dumb, but (as we seem to agree) not having them is dumber. I agree with your analysis. If they do become 100GB, etc we'll add queries to mitigate the size issue ("hey, do you support xyz.com?"), or any of the other relatively well studied solutions here (have I finally found a use for bloom filters? Unlikely.)

More broadly on the whitelist issue, my hope is that paying Proxies will result in actual "Orchid Proxy ISPs" springing up, which will have empty whitelists, actual legal protection as ISPs, etc. That's the beauty of real money being involved. Whitelists are there as a stop-gap while the market is being built.

</comment></comment></comment></comment><comment><USER36> Why promote it now when you have nothing to release? Without any actual software to use and test, this just comes across as the usual overhyped ICO vaporware.

</comment><comment><USER37> How is this different from Freenet or Tor?

<comment><USER2> (edit: When I wrote this answer, the person I was responding to had not yet added "or Tor".)

Freenet builds its own domain of content where people post websites that are hosted in a distributed fashion by the platform. What we are working on with this initial implementation is a fully-decentralized tunneling service to access existing content posted on the internet (so if you were to compare it to an existing technology, you might look at Tor, or the "out-proxies" from I2P).

<comment><USER37> So what are the benefits over Tor?

<comment><USER31> It seems _intended_ to be more anonymous and decentralized than Tor, and safer thanks to the strength of numbers, but their whitepaper is diseharteningly incomplete and disingenuous, particularly about problems that are shared with Tor.

For example:

"The distribution of Entry Nodes is a difficult topic. If oppressive governments are able to access this list, they will block user's abilities to access the list."

Or simply, you know, go after whoever runs entry nodes. Or run their own entry nodes and, even if they can't compromise the network, trace the evil cypherpunks who want to use encryption.

Unfortunately, some practical and political problems cannot be solved with improved cryptography.

<comment><USER12> Certainly incomplete as it is still a draft, though I'd contest "dishearteningly". Also, why do you find it disingenuous? We're not shying away from what entry nodes and bootstrap of user clients is one of the hardest problems to solve.

</comment></comment></comment></comment></comment><comment><USER38> How is the route decided? If it uses a centralized server for that, then wouldn't it be easy to block that.

<comment><USER2> We definitely do not have any centralized servers. The currently-specified mechanism to determine routes is that they are decided by connecting to a distributed peer-to-peer network that is modeled in many ways after a distributed hash table, at which point you do lookups of random keys in the hash space, which results in a randomized scalable lookup. There are mechanisms in play (and which I assume are already described in our in- progress whitepaper) to protect against eclipse attacks, sybil attacks, and some other miscellaneous attacks on the routing infrastructure. That said, this is also some ongoing work, and we are actively engaging with and being reviewed by people both in academia and in the field with respect to our techniques.

</comment></comment><comment><USER16> This sounds like a good idea. One problem with things like tor and i2p is you want to have tons of traffic so as to hide things from the various spies, but it is hard to get enough people to support nodes. Add a token and perhaps that problem will be solved.

<comment><USER39> A token seems to solve the supply side (relay and exit nodes) but what about the demand side? Why would anyone who doesn't care about subverting censorship or surveillance pay for this?

<comment><USER12> Because people currently pay for VPNs, and Orchid effectively provides some of the features of VPNs, with the added benefit that there is no central VPN provider who logs your (meta) data and sells it for profit.

And as Orchid is a P2P market place with no middlemen or fees, it should find a market equilibrium that is more efficient than VPNs, with a lower price per data relayed.

<comment><USER16> >people currently pay for VPNs, and Orchid effectively provides some of the features of VPNs, with the added benefit that there is no central VPN provider who logs your (meta) data and sells it for profit.

When you roll Orchid for use, I think this should be stated prominently on the web page as a way of selling it. Something like "If you are like millions of people, you use a vpn. But vpns have some problems" and so on.

</comment></comment></comment></comment><comment><USER40> It seems like it's like Tor, except instead of just paying my ISP, I also now get to pay more money to use this new POC protocol with a subset of Tor's features. Am I missing something?

</comment><comment><USER41> In 5 years, how might this be changing our lives? What will I think of when I look back and think "I'm so happy that they created Orchid"?

</comment><comment><USER42> When are they doing ICO!?

</comment><comment><USER43> What would convince my mom to use this though?

<comment><USER2> So, I don't know your mother, and can't say what kind of extreme technical needs she might have as part of her lifestyle or job; but, I guess I will presume you asked this question from the depressingly stereotypical perspective of "assuming peoples' mothers are not very technically skilled as they are older women, how would a mother use it" and answer your question anyway :(.

So, she might not use the first version, and if she does it might be installed or otherwise set up by you (so this hinges a lot on how much she trusts you ;P) or another friend of hers. However, we are building a simple user interface to make the experience of using it friendly and pleasant, and I guess it is worth adding that we also chose a name that we hope does not turn non-technical users away ;P.

<comment><USER39> So the reasons she might use it are:

* you install it on her computer

* it has a simple user interface

* it has a non-technical sounding name

Is that correct?

What if you reinterpret "my mother" to be "average internet user, in a country with average internet freedoms, who doesn't read HN etc"?

<comment><USER2> You could make that same argument about VPNs in general, which happens to be a 25-30 billion dollar industry. While I may or may not have reasons why a random "average" person might use this service, I find the question confusing... is the product not capable of being a success if people exist who don't want to use the service?

This kind of question always comes up for just about every service ever developed, and as the discussion continues the goalposts tend to shift and the user gets less and less sophisticated and more and more stubborn until at some point you demonstrate a user exists for which I can't prove they want it, and then victory is somehow declared. It is a sport I don't see much interest in playing.

To flip the question: we are providing a replacement for a service that already exists which is used by people worldwide and makes up a sizable market; our replacement is more secure and likely to cost less. Given that beachhead, you can now ask "who else would want to use this service if only it were cheaper and more secure?", and have a really fun brainstorming session.

<comment><USER39> If the answer is "the same reasons someone might use a VPN or Tor" that's fair enough, and a much better answer than "you install it on her computer".

(Though there are a variety of use-cases for VPNs and I doubt Orchid addresses most of that $25-30B market)

</comment></comment></comment></comment></comment><comment><USER44> Can I set up a relay node now?

<comment><USER2> Sorry, but we have not yet finished writing the code for this (let's just say it: "my fault" ;P). Even _I_ am currently unable to start and integrate a new relay into a running network; this should be fixed in the near future and we will have all of our code available as open source under the AGPL3 license, and will be working hard to build up an inclusive community around the project.

</comment></comment><comment><USER45> Who is this being built for?

<comment><USER12> Anyone who wants to access the Internet without being surveilled or censored, and for anyone who wants to earn tokens by offering their bandwidth for relaying such traffic.

</comment></comment><comment><USER46> It's Tor but with a worse website and more decentralised?

I think the website could be better if it said right at the top what it actually does and what it offers over competing services.

<comment><USER47> Are you talking about [https://www.torproject.org](https://www.torproject.org)? We have different standards for websites it seems. That looks like it jumped out of the mid 90s to me.

<comment><USER46> Tor's website actually tells you what it does.

And the design is nothing to scoff at. It's not hip, but it's functional and readable.

</comment></comment></comment><comment><USER48> buzzwords I see. what does it do?

<comment><USER49> Like Tor, but with capitalism.

Financial incentive for running relays.

</comment></comment><comment><USER50> As soon as I see "Ethereum" I stop reading.

<comment><USER51> Call me zealous, but I'm the same way. I'm wary of the potential for monetary incentives to skew actual demand. I've seen how digital advertising has more or less destroyed much of the internet. Hopefully decentralization will prevent monopolies or oligopolies from forming, but I'm not so sure about that.

At this point, it's hard to say what actual distributions of wealth might look like if people start using cryptocurrencies in their daily lives, for example to route traffic over a network. But at the moment, the distributions of wealth are quite nuts. But if there is an overall "unfair" atmosphere in the crypto-economy, it's could make depending on this system of routing kinda suck in the long term, especially if peoples actual livelihoods are under the pressure of massive ad gaps. People "sell out" when they're perceiving some stress.

If we go 'all-in' on this kind of system, it might actually end up creating the same kinds of de-facto power imbalances we see today with conventional social media.

I'm not anti-cryptocurrency either. I actually own a little bit of cryptocurrency. I'm curious about its potential to dissolve existing dominance relationships non-violently. That said, I don't see it as an end. Moreso, I see it as a potential conduit of healthy, non-violent economic chaos.

I also recognize that this strange new alien could be "out of the frying pan and into the fire".

<comment><USER33> I read your comment and I picture a sort of information balkanization via endless whack-a-mole of privacy tech and organizational surveillance.

It may just be how it will ever be.

<comment><USER51> Information is the real resource.

</comment></comment></comment></comment><comment><USER52> I saw this, realized I had never heard of it, and knew right away it had to be a blockchain. It all falls into place. Moving on.</comment>


Mandatory DNA collection on arrest is unconstitutional, says CA court - <AUTHOR> http://www.latimes.com/local/lanow/la-me-ln-dna-court-ruling-20141203-story.html

<comment><AUTHOR> I edited the title slightly to highlight the fact that this is a California rather than a federal legal development.</comment>