What's Y Combinator's conversion rate? - <AUTHOR> http://statwonk.com/yc.nb.html <comment><USER3> This is a very poor metric. Companies running out of money and doing a small exit counts as "good" ("conversion") while a successful and growing company is interpreted as something negative (no exit).

<comment><USER4> It's a terrific metric for founders who would like an exit. Statistically, a smaller exit works out much better for a founder than an IPO. Doesn't work for the VC so much. Maybe YC is anti founder?

<comment><USER5> This seems to have changed over the last several years. YC used to be very pro-founder in this regard, from an outsider perspective.

There's a pg essay that I can't find, but in essence states: Our rule is that it's up to the founder. If they want to aim small, that's fine. We ourselves aimed small with Viaweb. We didn't want to work on it the rest of our lives. It would be pretty lame to push founders into doing something that we didn't want to do. (EDIT: Found it: [http://www.paulgraham.com/swan.html#f1n](http://www.paulgraham.com/swan.html#f1n))

Contrast that with the recent Paul B interview: [https://blog.ycombinator.com/paul-buchheit-on-lessons- learne...](https://blog.ycombinator.com/paul-buchheit-on-lessons-learned-from- investing-in-200-startups/)

> Jasper: Sam Altman has said that the only criterion Y Combinator uses to > evaluate applying companies is, "Can this be a $10 billion plus company?" > like Airbnb and Dropbox. While this model works great for a fund, there is > an Early Exits movement that suggests individual entrepreneurs have a much > higher likelihood of success when they raise less capital and target exits > in the $20 million range. What do you think this view?

> Paul: The math does not support this strategy but if other investors want to > try it that's fine. Also, it is not just returns we are looking for but > really impactful companies. When you sell too early you don't realize the > full potential.

> For example, Facebook had an offer from Yahoo for a billion dollars, which > everyone told Zuck to take. Fortunately, he said no. Had he said yes, it > would have been another failed Yahoo acquisition and Facebook would not have > nearly as much impact. The reason we have these big and influential > companies is the founders believed in a long- term vision.

I think all YC companies have always aimed big, but some, like Zenter, were fine with being acquired by Google for a smaller amount. And YC was fine with that outcome in the sense that there weren't any institutional forces set up to discourage founders from pursuing it.

YC's signals now seem to say, "A small exit is equal to failure." And while that's true for YC as an investor, I'm not sure it was true for YC's institutional forces until the last few years. If you're in an environment where all of your peers consider you a failure if you sell for a few million, are you going to want to resist that peer pressure? Especially when they're your friends and mentors?

I want to be very clear that all of this is an outsider's perspective. Hopefully someone in YC will issue a smackdown if it's mistaken. I was just giving some supporting data for why people are feeling this way.

<comment><USER6> I think you're drawing a false dichotomy here.

As a founder of a YC company, I can tell you there's no signal from YC that "a small exit is equal to failure".

From personal experience, I can tell you that YC holds the wellbeing of the founders it funds as a top priority, even if it might seem to be against YC's own short-term interests.

It's true that YC's economic model is to fund companies that _can_ be $10B+ successes, because having a few companies that make it into that league enables them to fund all the other companies that _don 't_ make it that big.

But they're still happy for founders who don't achieve "unicorn" status but make an exit that yields them a life-changing amount of money, or indeed any outcome that leaves them better off than they were before.

Still, they'll encourage all their founders to _try_ to build a breakout company, because by doing that, even a "small" exit will often be much bigger than the exit you'd make if you started out with deliberately modest ambitions.

<comment><USER5> I see. Thanks for the clarification. It's a topic I've wondered about for some time.

</comment></comment></comment></comment></comment><comment><USER7> I realize the conversation in the thread pertains to exits and funded companies. But I keep thinking why is there a general negative perception or often a sense of ignorance/neglect towards successful, ongoing, revenue- generating businesses (that haven't taken any or very little funding before reaching profitability) whereas even a minor exit (acqui-hire) will often put you in the metaphorical league of 'successful' founders?

<comment><USER8> Because most tech start ups in the valley are funded by private investors, and for them usually the best outcome is either an exit or an IPO. When the industry is dominated and controlled by this kind of funding, these goals end up idolized because that's just what rules the land. The VC wants to flip it to Google, the entrepreneur just hopes that he doesn't fuck up and that his investor gets his outcome.

<comment><USER9> The dynamic you're talking about is intrinsic to the mathematics of funding startups. It's not a value judgement.

</comment></comment></comment><comment><USER10> I found YCList to be out of date and inaccurate for the handful of companies I've looked for on there in the past. Not sure about its general overall accuracy though.

<comment><USER11> It's very inaccurate for my batch (S10) at least, for which I know of about twice as many exits as are listed.

And as others have mentioned, the bigger challenge is that "exits" aren't a good metric for either gauging YC's success or for making founders rich. The companies that have "failed" to convert (i.e., exit) from S10 include Stripe, PagerDuty, and Docker.

<comment><USER4> Sure, but that error rate is probably relatively consistent. Probably exits aren't listed for earlier years either. Unless everyone is getting more secretive about exits, which is interesting in itself.

Anyways, maybe someone from YC will comment. They know the truth and it's a worthwhile figure to publish.

<comment><USER12> It is probably not that exits are more secret and more that since YC has scaled up the number of entrants, so the number of exits that people care to publisize becomes a smaller and smaller percent of the total number of members in a batch.

</comment></comment></comment></comment><comment><USER13> I think it'd be interesting to see the rate of success for rejected companies.

</comment><comment><USER14> "Conversion" rate, as defined here, doesn't make sense for YC's (or any startup investors') financial model.

Valuation could be useful. Maybe track the change in the companys' valuation over time.

<comment><USER15> But I don't think the point of this is to look at it from the investors' point of view. The point is to look at it from the founders' point of view. That is, as a founder, what's your basic (binary) chance of your company being a success? Founders don't get the chance to diversify across many companies - all of their eggs are in one (their own company's) basket. Thus, for them, a "percentage chance of a positive outcome" is an important metric.

<comment><USER14> Then why doesn't the metric account for companies that have not shutdown and not exited, i.e. have grown beyond a certain value? Isn't that closer to the most favorable founder outcome?

<comment><USER15> That metric (grown beyond a certain value) is much harder to get for private companies, and private valuations are all pretty fuzzy anyway. Exits represent a real, verifiable, concrete event.

I don't think his metric is perfect, and there are other ones (how many companies just shut down and returned what was left to investors) you can look at, but overall I think it's a good proxy for "what percentage of ycom companies have a positive outcome".

</comment></comment></comment></comment><comment><USER16> Surprising. I would have expected the success of YC startups to have increased with time -- more selective program, higher prestige value, bigger network of fellow YC startups to work with.

<comment><USER17> If your metric is exits then obviously newer companies are going to be 'worse'. It takes time to build something valuable.

</comment></comment><comment><USER18> I'm not familiar with the term "conversion rate" in this context. Conversion rate in marketing is someone that sees your ad and buys the product. For the post and the comments here conversion rate means rate of startups that have exited. From where does that meaning comes from? I googled but only found references to the marketing business.

</comment><comment><USER19> Your conclusion is odd about 15% over 10 years. The conversion rate is declining. Doesn't that mean ycombinator is failing? I mean worse than being stagnant, the value of ycombinator to startups is declining over time. The conversion rate is going to zero, if you ask me.

<comment><USER20> The goal of a VC is not to have a high conversion rate... it's to find the big winners. Airbnb + other 'big wins' are 1 in 1500 or higher, but account for the vast majority of returns.

YC started out with a sub-dozen class size. Now it has more than a hundred per batch. The conversion rate is going down, but the hit rate, which is all that matters, likely is going up. More hits with more failure == more success for yc.

<comment><USER21> You're excluding brand dilution as a function of class size. Knowing you are going to be one of 100+ companies going through the program each semester makes you less enthusiastic to join if you already have a very promising or successful company, and it makes it harder for VCs to determine signal; "It's from YC" stops being good enough.

A big reason for the class size explosion is almost certainly for network effects, a fleet of ready customers who support YC and drive growth out of the gates.

It creates the space for another brilliant investor to launch a rival YC that caters to a small size of elite founders. Isn't that almost inevitable?

<comment><USER22> > It creates the space for another brilliant investor to launch a rival YC > that caters to a small size of elite founders. Isn't that almost inevitable?

What would the attributes be of an elite founder?

<comment><USER23> Sell their startup for an outsized sum during a bubble to a failing giant only for it to be defunct within a short time? Just to go by viaweb's example.

That would put at least Mark Cuban in the contending since they both sold to the same failing giant.

</comment></comment></comment></comment></comment><comment><USER4> Worth reading about the early exits movement: [https://capitalandgrowth.org/page/unicorns](https://capitalandgrowth.org/page/unicorns)

</comment><comment><USER24> Not sure that the conversion rate is the most important metric.

[http://fortune.com/2014/05/05/each-new-y-combinator-class- wo...](http://fortune.com/2014/05/05/each-new-y-combinator-class-worth-more- than-a-billion-dollars-president-says/)

      >>  Y Combinator... creates more than a billion dollars of value with each class it graduates...

</comment><comment><USER25> Well, you can't harsh these findings too hard. Considering yc is a pretty successful unicorn hunter, having many exiting startups in their portfolio is par for the course. I suppose the board at yc would rather see a bunch of exits and a few IPOs than one Uber and a bunch of "great ideas".

<comment><USER26> _Considering yc is a pretty successful unicorn hunter_

Is it? Based on what? 0.73% of their companies are unicorns, is that a lot?

<comment><USER25> [https://en.m.wikipedia.org/wiki/Unicorn_(finance)](https://en.m.wikipedia.org/wiki/Unicorn_\(finance\))

<comment><USER26> I know what an unicorn is.

</comment></comment></comment></comment><comment><USER27> [deleted]

<comment><AUTHOR> Not sure I understand. These are the default survival::survfit CIs. I'd be very interested to hear why they're a poor choice in this case (I don't think they are) [https://stat.ethz.ch/R-manual/R-devel/library/survival/html/...](https://stat.ethz.ch/R-manual/R-devel/library/survival/html/survfit.formula.html)

Also, I have to address the claim of small samples. There is no sample size inherently too small. We make estimates of uncertainty given information. We might wish to extract more certainty (statistical efficiency per n), but that's part of the art. Smaller sample sizes are more work for the statistician but not less valid in any way.

<comment><USER28> At least one issue is that since the survival curves are different for each batch (as established by your final 'Y Combinator conversion rates by vintage' section), your data is not i.i.d, the K-m curves are changing over time, and so the overall survival curve and its accompany CIs isn't really meaningful: it's sort of the weighted average of the nonparametric K-M curves of each batch further weighted by right-censoring of more recent batches... or something like that.

You could try instead showing stratified curves by year, or directly modeling how the hazard rates change over real time. 'survival' claims to support the latter by the 'crazy clock' method (but when I tried doing that in [https://www.gwern.net/Google%20shutdowns](https://www.gwern.net/Google%20shutdowns) I never could figure out how it worked).

One thing I wonder is whether YC is scraping the bottom of the barrel and this is why the conversion rates get so much worse, or if it has become more difficult in general to be acquired; if the former, perhaps including the size of the batch as a covariate would be a meaningful test? Data quality might also be an issue from the other comments, and you could try auditing it by picking a few supposed non-exits and researching them intensively to estimate how much measurement error you have.

It might also be interesting to see if the survival curves look more like type I, II, or III curves or what parametric distribution might fit best - <USER2> like it could just be exponential/memoryless which is interesting.

<comment><AUTHOR> Right on! Great feedback! I was also thinking about using competing risks to introduce company foldings.

Also parametric modeling as you mention. Try out the Gamma and see which of the family it divines.

<comment><USER28> Yeah, you could do competing risks. I see yclist.com does distinguish between 'dead' and 'exit', although your CSV doesn't. You can use 'comprsk', I remember it as being no harder to use than 'survival' when I tried it out briefly for [https://www.gwern.net/DNM%20survival](https://www.gwern.net/DNM%20survival) . Any competing-risks model going to be very imprecise, though, since if you can't estimate the overall survival curve very precisely, you definitely can't estimate it split into multiple outcomes.</comment></comment></comment></comment></comment>


Micropython code to implement a GSM mobile phone - <AUTHOR> https://github.com/jeffmer/micropython-upyphone <comment><USER2> I thought it would be an actual GSM stack written in Micropython (like OsmocomBB etc.), but alas, it's only the UI and the real work of GSM is handled in a prebuilt module.

The UI reminds me of Apple's iPhone prototype: [https://thenextweb.com/apple/2014/10/05/apple-skankphone- bui...](https://thenextweb.com/apple/2014/10/05/apple-skankphone-built- original-iphone-release/)

<comment><USER3> Wow, they made an iphone _and_ an android.

</comment></comment><comment><USER4> Very nice, I wish this were around when I was writing my A6 lib:

[https://github.com/skorokithakis/A6lib](https://github.com/skorokithakis/A6lib)

Hopefully the MicroPython ecosystem will expand with efforts like this one, how did you find performance? I'm always worried about memory usage with μPython.

<comment><USER5> Based on the hearsay, µp actually has much better memory usage than CPython (but obviously worse than C).

Considering it's striking a pretty nice balance between high-level features and low overhead, I'd say it might be worth it, especially if you're aiming at a very hackable phone.

This is definitely going on my "list of things to do when I have too much time".

<comment><USER6> It isn't really heresay. You can run µp on systems such as an esp8266, which you couldn't run cpython on. It does run, but lacks much of stdlib (by default).

<comment><USER7> Is Expressif firmware still required to run µp on ESP8266/32K?

<comment><USER6> Hmmmm I'm not sure. They've got an open source esp firmware, but I don't know if it works with µp or not. I generally use the Expressif firmware

</comment></comment></comment></comment></comment><comment><USER8> so the core GSM is done by a SIM800L module. Will that work on any normal cell network? I know a certain class of cheap GSM chip only works with a 2G network, and (as I understand it) T-Mobile is the only operator of such a thing in the US, and they're trying to phase it out. Is this correct? Could someone enlighten me as to how all this works?

<comment><USER9> The SIM800L seems to be a Quad-band GSM module with GPRS.

So yes it's only 2G, but you've got the potential for SMS and maybe even 512kbps data. No idea about an audio channel (voice calls).

Whilst 2G is old, it's supprisingly battle tested and goes through walls quite well. A year ago we were told to expect the phasing out of 3G networks and to expect carriers to expand 4/5G with 2G as fallback.

Also, you gotta bear in mind, this module can be bought for £5. Getting a PCI-E 2/3/4G micro-pci card for my laptop is priced at closer to £200, so this is a great starting place. Expanding to a module that supports multi-networks either later or as an upgrade wouldn't be overly difficult - this is just a super cheap way of being able to test with real SMS, real networks and real recipients.

<comment><USER8> I understand that it's much cheaper than a 3G/4G/LTE connection, but if it's going to become non-functional in the foreseeable future, I'd like to at least be aware of that before building something on it.

<comment><USER10> At least some mobile operators are getting rid of their 3G networks and shifting focus to 4G/LTE, but keeping 2G because of how common it is.</comment></comment></comment></comment>


S-expressions: The most powerful data structure available - <AUTHOR> http://www.ymeme.com/s-expressions-most-powerful-data-structure-available-68.html

<comment><USER2> Arrays are the most powerful data structure available. You can't express arrays using S-expressions, but you can express S-expressions using arrays.</comment>


During the Pandemic, Grubhub Should Be Thriving. It's Not - <AUTHOR> https://themarkup.org/coronavirus/2020/05/27/during-the-pandemic-grubhub-should-be-thriving-its-not <comment><USER2> One in three drivers admits eating customer food. Why should I every consider using this service?</comment>


The best discussion of fading anonymity I've ever read. - <AUTHOR> http://skepticlawyer.com.au/2012/10/19/manners-cost-nothing/

<comment><USER2> Great Article. Thanks.

You might find the following research interesting.

<http://www.cam.ac.uk/research/features/privacy-by-design/></comment>


What Silicon Valley should learn from Windows Phone 7 app developers  - <AUTHOR> http://scobleizer.com/2010/11/08/what-silicon-valley-should-learn-from-windows-phone-7-app-developers-first-looks/

<comment><AUTHOR> Big pro for me: "Developing for it is easy and consistent across the devices that it runs on, unlike Android, which requires testing and specialized development for each different handset."

Big negative: .NET</comment>


Understanding Uber: It's Not About the App - <AUTHOR> https://www.londonreconnections.com/2017/understanding-uber-not-app/ <comment><USER2> Of all the things that have been revealed about Uber, this is among the worst:

"The facts are that on the 30 January 2016 a female was sexually assaulted by an Uber driver. From what we can ascertain Uber have spoken to the driver who denied the offence. Uber have continued to employ the driver and have done nothing more. While Uber did not say they would contact the police the victim believed that they would inform the police on her behalf.

On the 10 May 2016 the same driver has committed a second more serious sexual assault against a different passenger Again Uber haven't said to this victim they would contact the police, but she was, to use her words, 'strongly under the impression' that they would."

Horrible things will happen at scale. But here Uber was directly involved in enabling sexual assault and misleading the victim.

EDIT - some comments believe that this is overly harsh in judging Uber:

1) Uber has had ongoing issues with sexual assault in the US preceding this event. If preventing sexual assault was a priority they would have designed the right policy. Lack of strong policy on sexual assault is an explicit choice:

"firstly it seems they are deciding what to report (less serious matters / less damaging to reputation over serious offences) and secondly by not reporting to police promptly they are allowing situations to develop that clearly affect the safety and security of the public." *

2) If a serious crime (like this one) is suspected, companies have an obligation to report this and not just conduct their own investigations which will have an inherent bias.

3) Both passengers came away with the impression that Uber was going to take action. If Uber had no intent to take action, this made the situation worse by reducing the likelihood of direct reporting and enabling further sexual assault.

* [https://cdn.londonreconnections.com/2013/12042017-NB-to-Hele...](https://cdn.londonreconnections.com/2013/12042017-NB-to-Helen-Chapman_Redacted-1.pdf)

<comment><USER3> How often do you think this kind of thing happens with regular taxis? public transport?

My gut says it's extremely common, you just are not hearing about it.

<comment><USER2> I am not following your defense of Uber. Let's assume sexual assault is equally common among Uber drivers and regular taxi drivers.

It is still not material to the complaint. The complaint is that regulators are getting adequate and timely reporting from regular taxi companies so they can investigate. But not from Uber. This is leading to unnecessary safety issues for Uber riders.

<comment><USER4> I don't read it as a defense, its a necessary prerequisite question to ask to put this situation in its proper context.

Every sexual assault is horrific - but what if we look closer and discover that the rate of sexual assaults per ride from Uber is less than that from established cab services?

That completely inverts the situation and we would need to ask "What is Uber doing right, and how can the other services emulate their success?"

What if we discover they are about the same? Well, then Uber isn't anything exceptional and we have an industry-wide problem that needs addressing at all levels.

Either way, we don't even know where the real problem is if we don't start with those kinds of questions.

<comment><USER2> First, no matter what the statistics look like, it doesn't excuse Uber's failure to report sexual assault and protect the people who have placed their trust in Uber.

Second, the statistics do not support Uber being any safer:

 _" A search query for "sexual assault" within Uber's customer support ticketing system yielded 6,160 results, or 6,160 cases, for the period between December 2012 and August 2015. Plugging in the search term "rape" resulted in 5,827 tickets, while "assault" revealed 3,524 and "sexually assaulted" yielded 382. In the same report, BuzzFeed obtained data from the company indicating that it had received "fewer than" 170 sexual assault claims and five claims of rape over the same time span.

In a response to the BuzzFeed article, Uber's heads of communications, safety and global support attributed the thousands of results to misspellings of the word "rate," words and names that contain the word "rape" and discussions of the issue by passengers who were not personally facing assault.

A January 2015 report by the libertarian Cato Institute found ride-sharing services provided by Uber and its main competitor, Lyft Inc., to be about as safe as taking a cab, which, for drivers, was not exactly reassuring."_

[http://www.ibtimes.com/uber-sexual-assault-statistics- ride-s...](http://www.ibtimes.com/uber-sexual-assault-statistics-ride-share- companys-safety-drivers-passengers-2494823)

<comment><USER5> I think it's fair to say that India is a special(ly bad) case. I'd like to see the figures for London, which is what would be relevant here. From reading that excerpt in the context of this article, it would seem that ULL is doing a really bad job. Are the statistics in that article global?

*edited

</comment></comment></comment></comment></comment></comment><comment><USER6> "Indeed if TfL have any kind of ulterior motive for their actions, it is simply that they dislike the impact Uber are having on congestion within the capital"

As compared with official London taxis (aka 'black cabs'), cars operated by Uber:

1) After dropping off a passenger, drive to their next pickup point (vs. black cabs which drive around for an average of 25 mins waiting to be hailed).

2) Pay the London congestion charge (vs. black cabs which are exempt despite the fact that they're generally on the road for about 10+ hours per day than the average car, and spend a far greater proportion of that total time within the congestion charging zone).

<comment> vertex-four The goal of the congestion charge is not to make money off cars in London - it's an attempt to reduce the number of cars in London. The number of cabs is limited largely by the complexity of The Knowledge, while the number of minicabs is rather less limited, so in conspiracy theory land, TfL have motive to ban minicab operators wherever possible.

In any case, it's very unlikely that this is an ulterior motive - there's no reason why the problems aren't exactly what was listed, i.e. mostly administrative failures on Uber's part where other minicab companies are more cooperative with the license requirements.

<comment><USER7> > it's an attempt to reduce the number of cars in London

Not sure if that's true. I think it's mainly to reduce the polution in central London. Otherwise, electric/hybrid cars wouldn't be exempt but charged the same amount!

<comment><USER8> >> I think it's mainly to reduce the polution in central London.

I lived in London when it was launched, it was also to do with making the city centre flow again, so that those who actually needed to be there could travel in a reasonable amount of time.

<comment><USER9> You mean, the ones who have money to pay the congestion charge. :)

<comment><USER10> Correct. Until we have a better economic system, this is how authorities often make incentives.

</comment></comment></comment></comment></comment></comment><comment><USER11> Was just about to post this myself, it's an excellent summary of what the dispute is really about. Also includes some history; I hadn't realised that Hailo was British in origin, or that TfL had the same legal structure as a local authority.

<comment><USER6> "it's an excellent summary of what the dispute is really about"

Except that, like other articles I've read about the dispute, it talks only about Uber/ULL and its transgressions in isolation. Stats about number of offences are provided, but without context. Where are the comparable numbers for other minicab firms (of which there are many)? Are ULL's offences per ride/mile higher or lower than the worst/best/median minicab firm?

<comment><USER11> The issue is not the quantity, but the inaction, as explained: once made aware of the incidents, Uber should have reported them to the police.

(Also the operation of "greyballing"!)

<comment><USER6> "once made aware of the incidents, Uber should have reported them to the police"

Is that what other minicab firms do?

<comment><USER12> In the UK there is no legal requirement that someone knowing a crime has occurred to report it to the police.

Uber seem to be reporting crimes where they are the victim (the article says they report a lot of small fraud) but not crimes where others are the victim.

Which leaves an interesting thinking experiment: would Uber report a murder? The victim obviously can't report it. It would be illegal for Uber to do anything to assist the murderer in escaping the law, e.g. by cleaning the car etc; but would they actually report it?

<comment><USER13> Shouldn't the victims be reporting these crimes? If the victims aren't reporting them, maybe they don't want them reported? Maybe they have a good reason for not wanting them reported? I have heard victims advocates state that it should be up to the victims what legal action/investigation/etc should take place. I am not an expert in this area, but maybe that is what Uber is doing?

<comment><USER14> According to the article, the victims were under the impression that Uber would report it to the police as part of their investigation in to the driver.

<comment><USER13> Thanks for pointing that out- now I wonder why they got that impression? Uber should definitely be making changes to their interactions to make it clear. I guess I am not familiar with the customs around such things in London.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER15> "it seems that every journey only covers 41% of the costs involved in making it."

This statistic is dumb.

There is a distinction between marginal and average cost. Uber's marginal costs are negligible, except for the new markets and products where they pay the driver's fare rather than collect a portion. The "costs involved in making" a trip are born only by the driver. Uber's marginal costs are only the probability they have to refund the fare for whatever reason, or get sued or something.

This might seem academic, but recognizing it belies the idea Uber will one day have to raise fares by 41%. The 41% statistic is derived from Uber's total losses. But this involves the significant fixed costs of running Uber...some of which are investments in IP and infrastructure...and which scale much less- than-linearly with demand. If Uber's demand grew enough they would be profitable even with today's fares.

I do not know if Uber will one day be profitable or raise fares, but tying total losses to the price of an individual trip is sloppy.

<comment><USER16> > Uber will one day have to raise fares by 41%

FWIW, if the fare covers only 41% of the costs, you need to raise it not by 41%, but by 143% to cover the costs.

</comment></comment><comment><USER17> It's extremely painful to see people defending or justifying Uber's actions on HN. You can absolutely find Uber useful, and if you do and you are a Londoner, you can contact Uber to ask them to revise their policy when it comes to collaborating with the police in case of sexual assault instead of defending the indefensible.

There is something very wrong with a community where people think Uber's actions are normal. It's extremely unsettling.

Uber has always been an "HN favorite", but there are limits... Some comments here are just insane and completely out of touch with reality, or just don't take sexual crimes seriously, which is even worse...

</comment><comment><USER18> London Reconnections is consistently good. For anyone interested in public transport, it's worth disappearing down the rabbit-hole of their previous articles. (It also has a better-than-average comments section.)

<comment><USER19> If anyone is in project management the story about Kings Cross at Christmas 2015 is must read article on how what seems like the best contingency plans can go wrong [https://www.londonreconnections.com/2015/know-run-story- behi...](https://www.londonreconnections.com/2015/know-run-story-behind-xmas- kings-cross-problems/)

</comment></comment><comment><USER20> Great article. That Uber drivers of London alone have been accused of 32 sexual assaults in the past year [1] is hard to swallow.

Also, it sounds like Uber would be better off targeting public outrage towards the mayor. Any thoughts on why they went after TfL and not where the power actually resides?

[1] [https://www.google.ca/amp/www.independent.co.uk/news/uk/uber...](https://www.google.ca/amp/www.independent.co.uk/news/uk/uber- drivers-accused-of-32-rapes-and-sex-attacks-on-london- passengers-a7037926.html%3famp)

<comment><USER21> They started a petition [1], that is directed at the mayor.

[1] [https://www.change.org/p/save-your-uber-in-london- saveyourub...](https://www.change.org/p/save-your-uber-in-london-saveyouruber)

<comment><USER22> >the livelihoods of 40,000 drivers

Uber are fully supporting the livelihood of 40k drivers? Doesn't seem likely. They're depressing the market price whilst simultaneously creaming off more of that price for themselves, that surely leaves drivers over all worse off.

What sort of pension are Uber's employees getting, the one's whose livelihood Uber are taking care of? What sort of holiday pay do they get? Do they get perks above the lowest legal level?

</comment></comment></comment><comment><USER23> I'd be interested to know what rights, in English law, Uber think are being breached here:

> Uber hold a position not to report crime on the basis that it may breach the > rights of the passenger. When asked what the position would be in the > hypothetical case of a driver who commits a serious sexual assault against a > passenger they confirmed that they would dismiss the driver and report to > TfL, but not inform the police.

<comment><USER24> (i) British law, not English law.

(ii) Perhaps they think it's a privacy violation to share the drivers personal data with the cops, or maybe defamation?

<comment><USER25> > (i) British law, not English law.

English law is technically correct here. British law would be something that applies to England, Wales, and Scotland collectively.

In this case, London taxis are regulated by the London Cab Order 1934, and minicabs are regulated under the Private Hire Vehicles (London) Act 1998, so this is law that only applies to England (Wales and Scotland have other acts that apply.)

</comment></comment></comment><comment><USER26> I'm still surprised Uber isn't being rejected simple for the reason that they are making a huge profit to drive out (no pun intended) competition.

Something that made me laugh:

"... Uber are a bully. Unfortunately for them, TfL are an even bigger one."

and of course it's all down to this:

"It is also worth noting that all TfL really wants Uber to do is comply with the rules. Despite the image that has been pushed in some sections of the media, TfL has not suddenly become the champion of the embattled London cabbie (many of whom would laugh heartily at the idea)."

<comment><USER27> I assume you mean a huge loss?

<comment><USER28> Well, some people at Uber are making a lot of money.

<comment><USER29> And a lot of people are making some money.

</comment></comment></comment></comment><comment><USER30> This is an informative article, it's a shame about the straw-man that repeats throughout it. Nobody is saying this is "about the app". Obviously, the usefulness of the app is tied into the licence. It's about the licence. Everybody knows that.

</comment><comment><USER31> It's very interesting to compare some extracts taken from a similar thread that apparead some time ago about Uber being temporarly banned in Italy to the tone in the comments here:

"Innovation almost always sidestep regulations, by definition", "Italy is not a place for innovation", "few corrupt lobbies that are against innovation and competition" , "US consistently outperforms the EU in GDP growth and innovation", "Forcing innovators to "respect the law" squarely is basically killing innovation".

I don't see anybody blaming the UK for being anti-innovation here. Why?

<comment><USER32> Because the facts don't support it. There are a number of app-based minicab services that are able to comply with TfL regulations. Uber here is failing an extraordinarly low bar - report serious crimes when your drivers commit them, do not implement a system to decieve regulators. The regulators are not coming down Uber-like apps in general, or on some theoretical - they are coming down on Uber in particular for its history of action, with evidence-backed specifics.

</comment></comment><comment><USER33> Regular taxis are in every facet worse than Ubers. So safety etc is just an excuse.

In the end, this is about economics. Why send all this money to a US company, when you can keep it in the UK?

<comment><USER16> That's just wrong in so many ways...

London black cabs can take more pax than most cars (5 or 6 plus luggage), they have a very small turn radius, are wheelchair accessible, the drivers have "the knowledge" and actually know where they're going, and if you want, you can also call one online.

You can argue about the overall experience of London cabs vs Uber, but worse "in every facet" is just silly.

Lastly, Uber fares cover only a fraction of their cost, so they're not "sending money to a US company"; at most they're contributing to Uber's $2 bn annual loss.

</comment></comment><comment><USER34> Relevant: [http://www.independent.ie/business/technology/ubers-new- ceo-...](http://www.independent.ie/business/technology/ubers-new-ceo-writes- open-letter-apologising-to-london-for-taxi-apps-mistakes-36165372.html)

</comment><comment><USER35> I don't know the situation in London, but Uber in general has improved 2 situations immensely:

1\. Normal cabs are horrible. The drivers are often rude, always try to price gouge or even outright trick you. There is no rating system, meaning no public 'oversight'. Hell, cabbies always tried to get rid of regulations, but now that there's Uber they use it like a weapon: that alone should tell you enough about how scummy and opportunistic they are.

2\. Booking a ride in a foreign country. Many drivers don't speak English, so good luck explaining where you want to go. You also usually have no idea what a typical fare should be, so (especially around airports) prepare to be hoodwinked.

In contrast, Uber has a set price, easy way to hail and a rating system that works beautifully. Before Uber, I hated cab rides. Now it's a pleasure.

<comment><USER36> Did you read the article? Nobody is denying either of those points.

Though generally in London the standard for taxis has been much higher for a long time, primarily due to The Knowledge and the high price of black cab vehicles.

<comment><USER16> Well, let me just go ahead and deny the first point. Normal cabs are horrible _in some jurisdictions and places_ (try getting a cab at Manila airport, Denpasar (Bali) airport, Singapore, Addis Abeba).

In other places, where they're well regulated, they're perfectly fine by and large (London, Germany, Hong Kong).

</comment></comment></comment><comment><USER37> > Most of all though remember: it is not about the app.

Yeah, we get it; it's about bureaucracy and regulation, and you fucking peasants will do as we say.

I hope Uber does what they usually do in these cases: ignore this bullshit and continue providing value to common people.

<comment><USER8> And I hope that continuing that behaviour gets them banned from more countries.

They aren't providing value for "common people", they are trying to capture the market with massive subsidies. That itself is cause for a huge amount of concern.

<comment><USER38> Which "massive subsidies" are they receiving, exactly?

<comment><USER14> Billions of VC investment money. Actually, I don't agree with the GP. In my view Uber represents one of the biggest transfers of wealth from VCs to the common man in the last three decades. When the funding eventually dries up and Uber are forced to raise their prices, the common man will just move on and leave Uber to go bankrupt. It's beautiful really (at least the LP money coming from pension funds etc. is finding it's way back to the originators in the form of discounted travel rather than going up in smoke).

<comment><USER39> The common man being the users or the drivers?

<comment><USER14> AFAIK Uber are subsidizing users and drivers, so both? Although I think the users are getting the better end of the deal.

<comment><USER16> I'm not sure the drivers really have a good understanding of their cost base. I doubt very much that Uber covers the average cost per mile (though hopefully the marginal cost). Would be interesting to see a proper analysis.</comment></comment></comment></comment></comment></comment></comment>


Signal Is Down Without Explanation - <AUTHOR> Is this AWS's response to their earlier cease & desist?  https://signal.org/blog/looking-back-on-the-front/
Live Twitter feed with @signal keyword for on live discussion:
https://twitter.com/search?f=tweets&vertical=default&q=%40signalapp <comment><USER2> [https://news.ycombinator.com/item?id=17200865](https://news.ycombinator.com/item?id=17200865)

<comment><AUTHOR> Thanks for the link. Mods can close/delete this thread.

</comment></comment><comment><USER3> the us-east-1 region is having power outage issues.</comment>


Facebook Vanity URLs Available this Saturday - <AUTHOR> http://blog.facebook.com/blog.php?post=90316352130

<comment><USER2> For all those who want to calculate when is 12am EDT in your local time, go to <http://www.facebook.com/username/> where you got a count down.. (6 am saturday here in Switzerland)

<comment><USER3> <http://www32.wolframalpha.com/input/?i=12am+EDT> automatically detects your time zone and tells you your local time for that event, in case you're too lazy to determine it from the countdown on the official page

Edit: nevermind, I just saw they added that to the official site already, my bad - I only went there and saw the big countdown, didn't read the small-ish stuff above. Sorry again.

</comment></comment><comment><USER4> One of the things I always liked about Facebook was the lack of a global namespace. Every name on Facebook was contextual, when you logged in, so that Steve Andersen to me would mean something different than Steve Andersen to you, just like in real life. Sorry to see that go, but I suppose it's a tradeoff and a lot of good things will result from the new namespace.

<comment><USER5> I remember about a year ago Zuckerburg was running on about how we're just mapping the existing social graph so we don't need usernames and deriding myspace for it... amazing how a few months and a little tweetie bird can change your mind.

</comment></comment><comment><USER6> Nice to see that they're going to place new accounts on 'probation' of sorts to (try and) prevent name-squatting.

</comment><comment><USER4> woah, I just noticed names can have "." chars. Who's gonna get "home.php"? ;)

</comment><comment><USER7> Quick, somebody write a Web app that hammers Facebook on your behalf.

</comment><comment><USER8> Having a name no one else in the world has does have advantages

<comment><USER9> it has its disadvantages too, you are very easy to find using Google.

</comment></comment><comment><USER10> They could tolerate squatting but explicitly support a secondary market powered by their credits system (and take 30%)

</comment><comment><USER11> I just hope I remember when friday night comes along.

<comment><USER12> <http://calendar.google.com> \+ SMS reminder

</comment></comment><comment><USER13> thats awesome, i missed out on the google profile name rush.</comment>


Ask HN: Non-technical day trading strategies? - <AUTHOR> I do fairly well with my stocks and cryptocurrency investments. My general strategy is to buy and hold. I have done well by sticking with what I know (tech).
I'm interested in dabbling in a bit of cryptocurrency day trading (maybe stocks later) but I don't really believe in technical analysis and crazy graphing.
I'm curious to know what other kinds of strategies exist for day trading with the goal of making small profits every day. Listening/watching for news?
Perhaps I should simply continue my buy and hold strategy and simply liquidate a small amount every month? <comment><USER2> Only certain kinds of investments generate income, and those are profitable businesses that pay dividends.

The key to investing in those is to buy them at a low price. Pretty hard for average people to do that these days since we're in a stock price bubble.

You can only purchase at a low price if you have better information than the market about what you're investing in.

<comment><AUTHOR> Many investments do not generate income, and the profit is made from the appreciation of the asset (ex. A Da Vinci painting, or gold).

You can make a profit off of the mispricing and misjudgment of others.

Better knowledge exists in many ways, not just insider news/current events. Example: I understand what hashing algorithms are, the difference between Proof of Work and Proof of Stake, therefore I may be better at judging the feasibility of a cryptocurrency and thus better at valuing it.

<comment><USER2> That is essentially what I am saying.

What I think OP was asking for is strategies to beat the market by buying and selling at the right time. I don't have any strategies like that, and would advise against following any such "strategies."

If OP wants to take regular income from investing, the smartest thing to do is just invest in businesses that pay regular dividends. Unfortunately, those stocks are all very expensive now, but as they say, "them's the breaks." I personally only invest in crypto and don't take any income from it.

<comment><AUTHOR> Hey, thanks for replying. I am the same person.

I'm not really asking how to time buys and sells. I'm asking for models or strategies that traders might use alternatively to graph reading. Ex. event driven strategy (trading on news and rumors).

<comment><USER2> Hah! Woops, didn't look at your username :D

I'm definitely no authority on it but I would highly recommend Warren Buffett's essays on investing.

<comment><AUTHOR> Thanks, although I believe Buffet's general strategy is to buy undervalued businesses for the longer term.

I was hoping for day trading strategies.</comment></comment></comment></comment></comment></comment>


A $650M Donation for Psychiatric Research - <AUTHOR> http://www.nytimes.com/2014/07/22/science/650-million-psychiatric-research.html?smid=tw-share&_r=0

<comment><USER2> Behavior genetics research on schizophrenia has been going on since I was born, in other words since long before most Hacker News participants were born. The crucial issue in the research, mentioned in the article kindly submitted here, is that any one gene only has a small effect on risk for schizophrenia. "Dr. Lander cautioned that each variant accounts for only a tiny portion of the risk of developing schizophrenia. 'It shouldn't be used for a risk predictor,' he said." Behavior geneticists, such as those I know locally,[1] and the pioneer of behavior genetic research on schizophrenia[2] hope that large-scale genomic studies of the kind that are now possible will help tease out how gene differences between one individual and another interact with environmental influences to increase or decrease risk for schizophrenia, which should lead eventually to better treatments and better prevention strategies, through deeper fundamental understanding of the disease. It is plain that environmental factors matter too, because sometimes identical ("monozygotic") twins are discordant for schizophrenia.[3] This research path will be slow and frustrating, precisely because it is well known already that no one gene has strong effect, and no particular gene acts without interaction with the environment, but this kind of fundamental research is indeed necessary to refine the diathesis-stress model[4] of the development of schizophrenia.

[1] [http://www.psych.umn.edu/research/areas/pib/](http://www.psych.umn.edu/research/areas/pib/)

[2] [http://www.psych.umn.edu/people/facultyprofile.php?UID=gotte...](http://www.psych.umn.edu/people/facultyprofile.php?UID=gotte003)

[3] [http://www.virginia.edu/uvanewsmakers/newsmakers/spiro.html](http://www.virginia.edu/uvanewsmakers/newsmakers/spiro.html)

[http://link.springer.com/article/10.1007%2FBF01564274#page-1](http://link.springer.com/article/10.1007%2FBF01564274#page-1)

[4] [http://college.cengage.com/psychology/bernstein/psychology/6...](http://college.cengage.com/psychology/bernstein/psychology/6e/students/web_tutorials/chapter15/schizo/schizop5.html)

[http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2632355/](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2632355/)

AFTER EDIT: I'm still learning the rules about informed, thoughtful discussion here after 2073 days of participation as a registered user on Hacker News, so I'd be glad to understand what someone considered objectionable about this comment. I was just surfing by this comment to add a link emailed to me by Irving Gottesman, the prominent schizophrenia researcher who was mentioned in the acknowledgements of the book _A Beautiful Mind._ That link discusses the study in the journal _Nature_ published today.

[http://www.schizophreniaforum.org/new/detail.asp?id=2068](http://www.schizophreniaforum.org/new/detail.asp?id=2068)

</comment><comment><USER3> This is a huge donation for the Broad and probably equals all their previous fundraising. I think a lot will ask if it's too much to one institute or too little to solve the problem, but overall I'm very excited. Biology is changing very fast right now, and it's probably the right time for a big new push. Psychiatric disease and brain research is the next big frontier and private funding like this allows places to take risks they otherwise couldn't on grants. The Broad Institute is the best genomics institute by far, albeit a little light on the clinical side. The money will not be wasted on anything hokey. All that said though, there is a large chance that it still won't be enough to solve a very tough set of problems.

<comment><USER4> I really appreciate that the Broad keeps a lot of attention on basic (or at least, pre-clinical) research. That kind of work is critical to making big discoveries but it's harder to defend for funding in a world where we want to put drugs on shelves. I still want to see drugs on shelves, but starting at the clinic isn't the best way to make discoveries that lead to real therapeutics.

<comment><USER5> I don't think for psychiatric research we should start with the presumption the end goal is necessarily "drugs on the shelf".

it might or might not be a good goal for some conditions.

<comment><USER4> That's fair. I was responding to the critique of the Broad is being not very clinical, and that was a figure of speech.

</comment></comment></comment></comment><comment><USER6> I'm happy that they're putting money into developing better diagnostic methods for mental disorders, because, frankly, the existing tools are terrible.

After 9 years of misdiagnoses by several different doctors, earlier this year one astute doctor was able to uncover that I had bipolar 2 and not unipolar depression or SAD. I had mixed feelings about it (ironic) but I was put on the correct medications and they have made a world of difference. In fact, I read that it can often take close to a decade, on average, to diagnose types of bipolar properly, because doctors and patients simply don't communicate well. Any tools that shorten that time gap would benefit individuals and society as a whole.

<comment><USER7> I think there is a lot of basic research to be done. I hope this research amounts to real advances!

<comment><USER8> Strangely enough it may be that our attempts to create artificial people (AGIs) will permit us to understand what mental disorders are all about.

<comment><USER9> Strangely enough it may be ____________ will permit us to understand what mental disorders are all about.

( ) Hypnotism ( ) Psychoactive drugs ( ) Functional programming ( ) Meditation ( ) Phrenology

<comment><USER8> None of your examples entail building a mind. Also there's a kind of irony in my example, since as a branch of computer science it is independently funded and seemingly unrelated to mental health.

<comment><USER9> But you're guilty of assuming your conclusion.

You have a model of the brain. You do not have a model of the mind. You assume that by simulating a brain with significant details, a simulated mind will emerge. I find that's a big pill to swallow.

<comment><USER8> Nah, I don't assume it. Hence 'may' and 'attempts'. But the explanation is simple enough. If you can discover what it takes to code a mind then you're also going to learn some of the systemic faults that minds, qua minds, can develop. Of course, I could be wrong about this. It _could_ be that mental diseases are all purely down to hardware issues (brain health). But it seems like a bad bet. For example: addiction, or at least specific types of addiction, depend on people's culture and choices.

</comment></comment></comment></comment></comment></comment></comment><comment><USER10> I was initially worried when the headline said "psychiatric" research, but was pleasantly surprised when I read the article and found out that what they really meant was neuro-biology and its genetic origins, not study of patient behavior. The "top-down" approach has failed psychiatry (or more accurately, psychology) for half a century. I'm glad to see the bottom-up approach (i.e. starting with genes and cells and then working your way up to thoughts/feelings/behaviors) taking the lead.

<comment><USER11> It's at least worth noting that it's only within the last decade that such an approach has become viable -- we're still entirely unable to establish any relationship between genetics and psychiatric illness via theory, so massive, inexpensive genetic analysis was a prerequisite to this sort of study (which still relies on correct _diagnosis_ in order to find genetic correlates).

</comment></comment><comment><USER12> It is a wonderful donation, but _I do wish_ that philanthropists would spend money on lobbying congress and The President to increase science budgets by the billions of dollars per year. Money could also be donated to those running in congress who are favorable to increasing the science budget of NIH, NSF.

<comment><USER9> Why do you prefer that model?

In fact, I would say your model has it backwards -- as a philanthropist, rather than devote my money to a cause, I should coerce/influence politicians with money, so as to compel through force of law other peoples money to flow to that cause.

<comment><USER13> One could argue that increased government funding for science is more likely to fund basic science and other "non-sexy" but still important and likely to pay dividends research.

<comment><USER14> In theory (and hopefully) but all of the non-sexy stuff ends up as political wedge issues. And when that happens we have people like Palin decrying research on fruit flies in Paris, France.

<comment><USER13> Honestly, at this point, "Science" is a wedge issue.

</comment></comment></comment></comment></comment><comment><USER9> This article is a enlightening counterpart to the 'emerging neuroscience' chorus, who would have us believe that a full model of consciousness and the brain is just around the corner. That those trying treat schizophrenia _haven 't a clue_ what causes the onset, and why certain drugs are effective tells me that we really are at the stage of the absolutely primitive models of brain activity.

<comment><USER15> Anyone who says a full model of the brain is right around the corner is either a fool or a charlatan. The BRAIN initiative was a first start to _develop the technologies_ needed to take the measurements that would underly potentially explanatory brain models.

In Conneconomics ([http://dx.doi.org/10.1101/001214](http://dx.doi.org/10.1101/001214)) we try to estimate the cost and time needed to map the connectome alone (only one component of a holistic brain model).

</comment></comment><comment><USER16> I'm glad to see more money spent on these issues, hopefully we will have some breakthroughs soon.

While the current medications surely do help a lot of people, I've watched really smart friends devolve into zombies while on some of them. People I've known for 15 years who are definately not the same people they were when they started. They themselves don't notice the change, of course, because it happens slowly.

We really need better options with less severe personality change side- effects.

</comment><comment><USER17> An interesting article on the U.S. Army's Comprehensive Soldier Fitness Program was published today. Wonder if this money is being well spent! [https://thewinnower.com/papers/a-critical-examination-of- the...](https://thewinnower.com/papers/a-critical-examination-of-the-u-s-army- s-comprehensive-soldier-fitness-program)

</comment><comment><USER18> Very good episode on psychiatric research: [http://www.londonreal.tv/episodes/rick-doblin-mdma- therapy](http://www.londonreal.tv/episodes/rick-doblin-mdma-therapy)

(and raising money, and all stuff in between)

</comment><comment><USER19> I applaud any efforts being made in the mental illness space. It's difficult because it can't be "seen" and anything we can do to make diagnosis as objective as possible would be great.

</comment><comment><USER20> ok, ok I'll take the bait...

That's crazy!

</comment><comment><USER21> This is a great donation, that seems to be going to exactly the right place. However, it's a little unfortunate that Mr. Stanley only started donating to this cause after a member of his own family was affected.

<comment><USER22> I'm fairly certain you'd find that the vast majority of donations to _any_ cause are from people who are personally affected in some way.

<comment><USER4> Agreed. And in this case, I think it's significant that the family member affected has controlled his illness long ago. The point of this donation is to help other people for whom such control is not yet possible.</comment></comment></comment>


Now we know that Amy Winehouse is dead we see that we need better noise filters. - <AUTHOR> https://plus.google.com/111091089527727420853/posts/HKcsYKSN27A

<comment><USER2> Happy to be downvoted for this one.

So a person dies, and robert scoble is concerned about noise filters in Google+? Literally 1-2 hours after their death ?

Please - show some respect for a person and choose a time to post a more adequate title. Amazing that tech pundits have reduced themselves "oh, she's dead BUT MORE IMPORTANTLY! Lets focus on noise filters!"

Sorry. Don't think it's cool.

</comment><comment><USER3> There's a pretty serious tinge of narcissism to using a woman's death to complain about too much noise in your social stream. He couldn't have waited until the next time Apple's rumored to be building a TV to post this?

<comment><USER4> Agreed. I don't care what Amy Winehouse did with her life, but to hear some professional Twitterer bellyaching that her death is wasting his time is sickening.

</comment></comment><comment><USER5> re-framing problem for clarity:

"Man who follows everyone he meets complains about noise in his social streams..."

</comment><comment><USER6> I can't think of a more perfect polar opposite to Winehouse than Scoble.</comment>


Pachyderm Raises $10M to Bring Data Provenance to the Enterprise - <AUTHOR> http://www.pachyderm.io/2018/11/15/Series-A.html <comment><USER2> I was part of an experimental neuroimaging group that tested Pachyderm OSS years ago and at the time we were really impressed with the versioning capabilities it provided. For us at the time it made it easy for each researcher to grab and change data as needed for their own development without requiring support from eng.

<comment><USER3> How well does that work when you datasets are a sizeable percentage of available storage capacity, though? Is there some sort of deduplication at work?

<comment><AUTHOR> Pachyderm does a ton of data deduplication, both for input data that's added to pachyderm repos as well as for output files.

Pachyderm's pipelines are also smart enough to know what data has changed and what hasn't and only process the incremental data "diffs" as needed. If your pipeline is just one giant reduce or training job that's can't be broken up at all, then this isn't valuable, but most workloads include lots of Map steps where only processing diffs can be incredibly powerful

<comment><USER3> This is super cool, thanks for pointing that out. Is the hard part done by Pachyderm or as some layer over container file systems?

<comment><USER4> Pachyderm does it. It's like half of what pachyderm does, manage the versioned data, and schedule workers to run your containered processes against them.

FYI, it's ridiculously easy to get going playing with Pachyderm if you just want to check it out. You can run it on Minikube.

<comment><USER3> > You can run it on Minikub

Thanks for the tip. I just started down the k8s path from bare metal cluster and will try this.

</comment></comment></comment></comment></comment></comment><comment><USER3> I have a "data science pipeline" coordinated with a Makefile and run on CI/CD (GitLab) with reports generated as build artifacts. Big stuff checked in with Git LFS.

Why would I use Pachyderm?

<comment><AUTHOR> Pachyderm pipelines can be run in a massively distributed fashion with data being sharded across many workers. Pachyderm also offers much better failure semantics than Make + CI. For example, if one shard of data of your pipeline fails such as a node or container dies, Pachyderm will automatically make sure that the data get rescheduled to another worker.

Each pipeline can have separate resource requirements (e.g. GPUs, lots of memory, etc) and gets scheduled by Kubernetes.

Finally, Pachyderm is versioning all of the intermediate steps in your data pipeline so if a downstream step fails, you don't have to restart from scratch, you can pick up right where it left off.

<comment><USER5> Congrats on your round! FWIW, (me being in a space adjacent to yours) I see these kinds of comments all the time about the "my 1 off pipeline seems good enough".

Most people don't need "industrial strength" till you hit a certain scale. They tend to optimize for ease of use/simplicity. It's one thing if you don't have to change their workflow, it's another if you have to not only have people change their workflow but also teach something new.

Convenience matters more. There's a pain threshold of "new tool" vs "this costs me x amount of time".

How are you guys overcoming this? Even though we're also in the infra space, I've never seen you guys out in the wild. Where would I bump in to you and what is the scale I would want to add the complexity of k8s + pachyderm + whatever other deps you guys have over just using an S3 bucket?

Another question: Why hasn't AWS just packaged this up and offered it as an extension of their k8s service? How are you guys going to overcome that?

Usually I see "hybrid cloud" or "on prem" as the response. If it is on prem, are you guys relying on the presence of k8s at customers? Do you guys use something like gravitational?

<comment><USER3> > my 1 off pipeline seems good enough

Not what I was describing. We've been using the setup for a while with multiple projects and for the data science end of a clinical trial.

I like building new setups incrementally out of tools I already know, so the question is obviously why is this new tool a radical improvement

</comment></comment></comment></comment><comment><USER6> He was holding these two pieces of pizza...

<comment><USER7> Hi, Pachyderm CEO here. We have been discussing this comment internally for the last 15 minutes and still don't understand it. But we really enjoy it. Would you mind explaining a bit more what you are saying?

<comment><USER8> Not the person you are responding to, but the comment you're replying to is referencing Seinfeld, due to the name of your company.

[https://en.wikipedia.org/wiki/The_Stand_In_(Seinfeld)](https://en.wikipedia.org/wiki/The_Stand_In_\(Seinfeld\))

<comment><AUTHOR> What a wonderfully obscure reference.

</comment></comment></comment></comment><comment><USER9> I've also wondered why I should use Pachyderm. Decided to give it a try, and wrote the following blog about it : [https://medium.com/bigdatarepublic/pachyderm-for-data- scient...](https://medium.com/bigdatarepublic/pachyderm-for-data- scientists-d1d1dff3a2fa) " Finally, version control for your data "

</comment><comment><USER10> Congrats guys! Data provenance is only becoming more important

</comment><comment><USER11> This is great news. An amazing team with a good solution to a huge problem. I look forward to following your progress.

</comment><comment><USER12> Love this project! Git for data is such a brilliant concept

</comment><comment><USER13> Congrats to the team at Pachyderm!

</comment><comment><USER14> much deserved, been following this project for sometime and have continued to be impressed.

</comment><comment><USER15> Congrats Pachyderm team.

</comment><comment><USER16> nice!</comment>


Amaranth – nutrition - <AUTHOR> https://en.wikipedia.org/wiki/Amaranth#Nutrition <comment><AUTHOR> The leaves of some amaranth species, cooked as a dry curry, are damn tasty. Can't remember where and when I had it, except that it was long ago, and in India. Probably was made with some onions and common Indian spices, but that alone did not account for the good taste - it's the plant itself. Might have been in North or South India - the Wikipedia article says it is used in both those areas, and also many other areas worldwide.</comment>


When 'int' is the new 'short' - <AUTHOR> http://googleprojectzero.blogspot.com/2015/07/when-int-is-new-short.html <comment><USER2> I've never really liked the short/int/long definitions. They're just out there to confuse you, especially if you've written your share of assembly.

For as long as I can remember I've just used size_t, uintptr_t, uint32_t/int32_t (or any of the 16/32/64 variants), exactly because I want to be explicit about the machine word sizes I'll be dealing with. Before that, I always used similar (u32/i32, LONG/ULONG, ...) platform-specific typedefs on proprietary systems too.

For all practical purposes, int/unsigned int has been at least 32 bits since the early 90's (well, on modern platforms) but why use those if you can explicitly declare how many bits you actually need.

(I've bumped into a few archaic platforms where stdint headers weren't present but it's easy to just add a few build-specific typedefs somewhere in that case.)

<comment><USER3> Well I know historically it was because there were potentially performance issues with using explicit sizes - int was basically the guarantee that you would be working in the CPU's native size, and hence at it's most efficient, no need to shift and mask bits to get the value that you wanted. Obviously this leads straight to a bunch of relatively subtle bugs, but I guess for some applications the tradeoff of speed vs safety was worth it.

<comment><USER4> On 8 bit machines it's the reverse, an int being 16 bits minimum requires more operations to handle than an 8 bit number. Pass an 'int' push two things on the stack. Etc. Causes your code size to balloon noticeably.

<comment><USER5> Many 8bit processors such as AVR have instructions that work with 16 bit numbers (stored in register pairs). So that's not the case.

<comment><USER6> That only applies to adds, subtracts, and register moves. 16-bit Booleans, shifts/rotates, multiplies, and load/store still need to be done with multiple instructions.

<comment><USER4> I did a little mucking on some AVR code of mine. Sometimes going from an uint8_t to a uint16_t saves a couple of bytes Sometimes adds a dozen.

One case changing an index in a for loop to an int, code went from 34024 bytes to 34018 (saved four bytes). But changing uint8_t i, j, k; to uint16_t i, j, k; code compiled to 34068 bytes, gain of 44 bytes.

</comment></comment></comment></comment></comment></comment><comment><USER7> Conversion to and from C's `int` is one place where I think Rust is failing too :(

Rust allows silent truncation of values in numeric conversions and considers it "safe", because other features for memory safety will catch buffer overflows — but it doesn't care about cases where the program will do a memory-safe logically-invalid thing (e.g. write to a wrong location within a buffer).

That's because Rust has no integer size promotion at all, which means `len as usize` and `len as c_int` are required _all over the place_ when interfacing with C (and the `as` operator has no overflow checking by design).

<comment><USER8> I think integer size promotion would only make it worse.

I do agree that "as" should have range checking. Some of it can be done statically (e.g. converting to a strictly larger type, and when the range of possible values are known) and the rest can be done with minimal overhead if they emit the code right.

<comment><USER9> It's not minimal overhead in many cases. Think about vectorization, for example.

</comment></comment></comment><comment><USER10> It's kind of bizarre that chrome goes to exceptional lengths to sandbox things into lots of mutually untrusting processes, then parses network input outside of that protection.

</comment><comment><USER11> Many many C/C++ programmers, even many of those employed by google, have a natural tendency to use a old C types like short/int/long etc. for integral types without thinking through X-platform issues or api interactions with other code.

Also, size_t is a frustrating beast. It's meaning is dependent on the platform. The Single Unix spec only calls for size_t to be an unsigned integer type. Now imagine you're writing code to compile over multiple mobile platforms as well as on x86_64 on the server side. Can you tell me what is the largest number you can address with that type -- without getting into a long google/stackoverflow session or hitting the compiler manuals for each of those platforms? If you absolutely want to make sure that your type can handle the values you expect it to handle, better give it well defined types provided by stdint.h (uint32_t is sooo much better than just int or unsigned int or even size_t for this purpose).

Now granted, you'd need to interact with external libraries (including libc/libc++) that'll want to use size_t etc. Not much you can do here but be very careful when passing data back and forth between your code and the library code. But that's been the lot of C coders since time began.

<comment><USER12> I disagree.

All you need to care about for cases like these, when you're talking about the size of something, is that both malloc() and new[] handle allocation size using size_t.

That, to me, says pretty clearly that "the proper type to express the size, in bytes, of something you're going to store in memory is size_t".

It can't be too small, since that would break the core allocation interfaces which really doesn't seem likely.

You don't need to know how many bits are in size_t all that often, and certainly not for the quoted code.

<comment><USER13> For cross platform interoperability, API with the exact size type helps remove any ambiguity. Using size_t might be fine for intra-process usage, but as soon as we are dealing with data across platforms, exact size type definition is a must.

<comment><USER14> > Using size_t might be fine for intra-process usage, but as soon as we are > dealing with data across platforms, exact size type definition is a must.

I don't know why you are downvoted, but this is very important.

Never send anything "on the wire" (or to a file) unless you know its exact size and endianness.

<comment><USER15> That is correct. For file formats and packets etc you must use exact sizes.

However for cross platform support using size_t in an API (as in what is exposed via .dll or .so) is a must. It's exactly the correct way to write cross platform code.

</comment></comment></comment></comment></comment><comment><USER16> I think the most interesting bit here is this:

> Now, the more astute reader will point out that I just sent over 4 gigabytes > of data over the internet; and that this can't really be all that > interesting - but that argument is readily countered with gzip encoding, > reducing the required data to a 4 megabyte payload.

This was pretty much my first thought on seeing the IOBuffer signature - "That exploit payload is going to be huge". But things are not always as they seem and using gzip to generate a large string on the client is something I had not previously considered.

<comment><USER17> You can blow up all sorts of things with gzip: [https://en.wikipedia.org/wiki/Zip_bomb](https://en.wikipedia.org/wiki/Zip_bomb)

</comment></comment><comment><USER18> I don't entirely understand why 'int' is the new 'short' here; int hasn't been a particularly good way to store sizes since C99.

Good spot though; I kind of doubt that this was a conscience design decision and probably just a slip up.

<comment><USER19> Agreed on both points. Plain int hasn't been very good for a long time. C99 <stdint.h> is really the way to go, but since this is C++ we're talking about, both the C++ committee and Microsoft Visual Studio deserve most of the blame for why people weren't using them since neither recognized/supported it for the longest time. (Visual Studio just _finally_ got stdint and stdbool, about 15 years late.)

And agreed, good catch.

<comment><USER20> Visual Studio has had stdint.h since its 2010 edition. Before that there were readily-available substitutes (like [https://code.google.com/p/msinttypes/](https://code.google.com/p/msinttypes/)), or you could do it yourself by typedef'ing [unsigned] __intNN as [u]intNN_t.

<comment><USER21> Somebody elsewhere pointed out to me that these will give types that are not aliases of the common ones. I.e. __int8 isn't the same as either unsigned char or char. Probably won't make a difference most places, but what does?

<comment><USER20> That is also the case in usual implementations of stdint.h, where int8_t is defined to be `signed char`. In C and C++, `char`, `signed char`, and `unsigned char` are different types, and `char` is not guaranteed to be signed or unsigned---that's up to the implementation.

EDIT: looking at the documentation, it appears that __int8 is supposed to always be an alias for `char`, even as far back as 2003: [https://msdn.microsoft.com/en- us/library/29dh1w7z(v=vs.71).a...](https://msdn.microsoft.com/en- us/library/29dh1w7z\(v=vs.71\).aspx). However, the workaround found in msinttypes suggests that Visual Studio 6 does have this problem. I weep for those still using it.

</comment></comment></comment></comment></comment><comment><USER19>
      Now; on x86_64, with gcc and clang an int is still a 32-bit integer type;

Minor nit. The size of int is typically defined by the platform ; the compiler follows along. All the major/popular ones happen to define int as 32-bit, so that's what you are seeing with gcc/clang. Maybe on Solaris you might see it as 64-bit.

</comment><comment><USER22> Can anyone shed some light on the design rationale for using integer types (most commonly _int_ it seems, as here) followed by a check if the number is not negative, whereas one could just use an unsigned type right away?

<comment><USER23> I disagree with it, but the Google C++ Style Guide offers the rationale you're looking for (under "On Unsigned Integers"): [https://google- styleguide.googlecode.com/svn/trunk/cppguide....](https://google- styleguide.googlecode.com/svn/trunk/cppguide.html#Integer_Types)

<comment><USER14> Yeah this doesn't convince me

The bug example there, sure, needs a signed type, so you can't blame the type for its wrong usage

I've had more bugs coming from using signed types that I'll not be bothered by writing 'unsigned' ever again

<comment><USER24> I disagree that a signed type is the correct solution to that, rather you need a while loop:

      std::size_t i = foo.size();
      while (i != 0) {
        --i;
        ...
      }

<comment><USER25> The point is that C int types are an unsafe mess, so it's better to have one simple rule than memorize all the corner cases and address them all the time.

<comment><USER24> I am addressing the specific example given by the Google style guide. The code for counting down using a signed integer as they do has more corner cases than the while loop I have shown. Using their way, you have to remember to subtract 1 from the size at the start and then use >= in the loop conditional. My way is just the inverse of what you do while counting up.

It's also worth pointing out the style of loop they give can't be used at all if you are counting down iterators or pointers instead of numbers.

</comment></comment></comment></comment></comment></comment><comment><USER26> I've recently tried to fix JOE (a UNIX portable program) so that it will compile without warnings and with minimum casting with -Wconversion. This is what I've found:

Chars: I hate it that they are signed because I like the convention of promoting them to int, and then using -1 as an error. It's easy to forget to convert to unsigned first, and the compiler will not complain. In the past I've used 'unsigned char' everywhere, but it's a mess because strings are chars and all library functions expect chars. My new strategy is to use 256 as the error code instead of -1. The only problem is that getchar() uses -1, so it's weird. IMHO, it's a C-standard mistake that char is signed.

I used to use int for indexes and long for file offsets. But these days, int is too short on 64-bit systems and long is not large enough on 32-bit systems.

ptrdiff_t is the new int. I've switched to ptrdiff_t in place of int and off_t in place of long. Ptrdiff_t is correct on every system except maybe 16-bit MS- DOS (where it's 32-bits, but I think it should be 16-bits). Off_t is a long long if you have '#define _FILE_OFFSET_BITS 64'. Ptrdiff_t is ugly and is defined in an odd include file: stddef.h. It's not used much by the C library.

The C library likes to use size_t and ssize_t. The definition of ssize_t is just crazy (it should just be the signed version of size_t, but it isn't).

I understand why size_t is unsigned, but I kind of wish it was signed. It's rare that you have items larger than 2^(word size - 1), so signed is OK. You are guaranteed to have -Wconversion warnings if you use size_t, because ptrdiff_t is signed (even if you don't use ptrdiff_t, you still get a signed result to pointer differences so you will have warnings). Anyway, to limit the damage I make versions of malloc, strlen and sizeof which return or take ptrdiff_t. They complain if the result is ever negative. Yes this is weird, but I think it's better than having many explicit casts to fix warnings. Casts are always dangerous.

<comment> david-given chars are only signed on some platforms (x86, for one). On others they're unsigned (ARM, for one).

One knockon effect of this is that strcmp() will return different values on the two different platforms for UTF-8 strings (because 0xff > 32, but -1 < 32)...

Incidentally, I don't know if you know about intptr_t; it's an int large enough to put a pointer in losslessly. It's dead handy. (My current project involves a system with 16-bit ints, 32-bit long longs, and 20-bit pointers...)

<comment><USER26> I did not know that chars were unsigned on arm- interesting.

I try to be conservative with the definitions I use, so I'm worried that intptr_t might be too new.

</comment></comment></comment><comment><USER27> In my experience we can't rely on manual handling of these integer overflow issues, especially with changing compiler behavior over time.

I've noted some compile time and run time checking options at:

[http://www.pixelbeat.org/programming/gcc/integer_overflow.ht...](http://www.pixelbeat.org/programming/gcc/integer_overflow.html)

</comment><comment><USER28> A signed type for a buffer size in Chrome. Sigh, C/C++ just wasn't made for humans.

<comment><USER29> Signed types are the sane ones and give you error checking possibilities (negative size doesn't make sense; huge positive size may be right or an error). Unsigned types break trivial math like x < y => x-1 < y-1.

<comment><USER30> Do signed types not also "break trivial math" like that, just at a different boundary? Genuine question. (The 0 boundary is obviously going to be more commonly hit than the 2^32 boundary, but nonetheless.)

<comment><USER28> Signed types have the same "wraparound" problem, i think the OP meant that they don't have this problem at the zero boundary.

<comment><USER31> Signed types have a worse problem. They _typically_ wrap around, but the behavior is undefined. That means the compiler can assume it never happens and optimize your code accordingly, which can lead to all sorts of entertaining misbehavior.

</comment></comment></comment></comment></comment><comment><USER32> How can they hide the bug in an open source project after fixing it. Isn't commit public?

</comment><comment><USER33> How exactly are you going to serve a 4GB certificate? Who in the world is going to wait for that to load?

<comment><USER34> The article mentions that it can be compressed to 4 MB...</comment></comment>


DoorDash is learning just how binding arbitration is - <AUTHOR> https://qz.com/work/1801652/doordash-is-learning-just-how-binding-arbitration-is/ <comment><USER2> Judge Alsup continues to impress with his well-reasoned judgments that demonstrate a deep understanding of the fast-moving technology world and a preference for the "larger good".</comment>


Twitter Strengthens Its Mobile Platform:Buys Clutch.io, Shutting Down Service - <AUTHOR> http://techcrunch.com/2012/08/13/twitter-strengthens-its-mobile-platform-buys-ab-app-tester-clutch-io-shutting-down-service-in-november/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+Techcrunch+%28TechCrunch%29&utm_content=Google+Reader <comment><AUTHOR> Just found initial discussion based off of the Clutch.io blog

<http://news.ycombinator.com/item?id=4376503></comment>


Guess What, You Don't Own That Software You Bought - <AUTHOR> http://gizmodo.com/5635269/guess-what-you-dont-own-that-software-you-boughthttp://gizmodo.com/5635269/guess-what-you-dont-own-that-software-you-bought

<comment><USER2> Other articles:

arstechnica: <http://news.ycombinator.com/item?id=1680185>

Wired: <http://news.ycombinator.com/item?id=1679906> and <http://news.ycombinator.com/item?id=1679974>

And groklaw: <http://www.groklaw.net/article.php?story=201009101658045></comment>


Spoil: Send an anonymous mystery surprise to your friends - <AUTHOR>

https://www.spoil.io<p>We'd love to hear your feedback of the product! We're only shipping Spoils to NYC, but that doesn't mean you have to be from NYC to order. If you use the word "hackernews" in the "where you heard about us" section we'll take off $5 from your order! <comment><USER2> Based on the colors chosen for the site (blue for the button - i.e., boys click the button - and pink for the title logo - i.e., the gift is for girls), the images (jewelry, women's shoes, a notebook that says "LOVE"), and the 'spoilee' suggestions (your lady crush, that girl at work), this site seems unfortunately and unnecessarily supportive of a heterosexual and outdated men- giveto-women gender binary.

I feel like if you made the imagery and wording more inclusive, you could have a really neat thing that appeals to a much broader audience of gift-givers.</comment>


3P Web - <AUTHOR> https://www.thirdpartyweb.today/ <comment><USER2> I think an interesting takeaway is that mapbox loads more than 2x of google maps. Is that because of poor/more implementation or something on mapbox's end?</comment>


New "Caching" Startup offers 5TB high-speed RAM data storage as a network appliance - <AUTHOR> http://www.infoworld.com/article/07/05/14/gear6-cachefx_1.html <comment><AUTHOR> "Pricing for the appliance begins at $400,000. CACHEfx comes in 0.25TB, 0.5TB, and 1TB iterations, which can be added in building-block fashion to increase caching capacity on the fly."</comment>


Is it Rails? - <AUTHOR> http://IsItRails.com remote fingerprinting service for potential rails applications. enter url + check. <comment><USER2> Result

We've made some checks on <http://news.ycombinator.com/> We think that <http://news.ycombinator.com/> is

maybe a Ruby on Rails driven site (85 points).

<comment><USER3> Versus 550 for <http://rubyonrails.org/> or 545 for the site itself ("damn sure").

Pity nothing like an explanation of the rating system.

</comment></comment><comment><AUTHOR> You can now subscribe to our twitter feed and get a new Rails driven site per hour ;-)

<http://twitter.com/IsItRails>

</comment><comment><USER4> At the very least, Cloud27 a Pinax site (Django) get 20 points out of 100.

<comment><AUTHOR> well, 100 is not the max ;-)</comment></comment>


He Hunted Bin Laden for the CIA. Now He Wants Americans Dead - <AUTHOR> https://www.thedailybeast.com/michael-scheuer-hunted-bin-laden-for-cia-now-he-wants-americans-dead?ref=author <comment><USER2> Are these people being monitored closely? There are a lot of right wing people who have come out of the woodwork in the past 2-3 months that claim they're fighting domestic terrorists but are actually building domestic terrorist cells themselves to kill people who express their 1st amendment rights (claiming BLM is a terrorist organization).</comment>


From Literature to Coding: How I Broke into the Software Industry - <AUTHOR> https://triplebyte.com/blog/from-literature-to-coding-how-i-broke-into-the-software-industry <comment><USER2> I wrote this article! One interesting angle we explored in the interview (but which didn't make the final edit) was Kevin's thoughts on academia and the time he spent there:

 _Do you feel any regret from switching from academia to tech?_

I certainly spent many years of my life studying things that only a handful of people care about. I think there's an answer to that question where, "Oh, of course, if I'm thinking about this 10 years ago, in terms of where would I want to be, financially, today, then no, I'm not going to do that." But do I regret those years I spent training for that? No, I don't.

I still love literature, I still love to read. I feel like, "Okay, I have this knowledge and appreciation of literature, and now I have this other technical job that I'm still able to learn really interesting things, and make a decent salary, and have this enriched, enlightened soul." I wouldn't necessarily advise people away from studying humanities. I would try to advise them to pick up another skill, just in case. Learn coding, or learn something, and just make sure you know what you're getting into. But I'm pretty sure I wouldn't have listened to anyone who said that when I was getting into it. So you have to consider it, yourself.</comment>