You can probably use deep learning even if you don't have a lot of data - <AUTHOR> http://beamandrew.github.io/deeplearning/2017/06/04/deep_learning_works.html <comment><USER3> This post doesn't even mention the easiest way to use deep learning without a lot of data: download a pretrained model and fine-tune the last few layers on your small dataset. In many domains (like image classification, the task in this blog post) fine-tuning works extremely well, because the pretrained model has learned generic features in the early layers that are useful for many datasets, not just the one trained on.

Even the best skin cancer classifier [1] was pretrained on ImageNet.

[1]: [http://www.nature.com/articles/nature21056](http://www.nature.com/articles/nature21056)

<comment><USER4> This is how the great fast.ai course begins - download VGG16, finetune the top layer with a single dense layer, get amazing results. The second or third class shows how to make the top layers a bit more complex to get even better accuracy.

<comment><USER5> Can't recommend the course highly enough!

<comment><USER6> I'm skimming through the content and it seems really great! I'm interested in the last lesson (7-Exotic CNN Arch), but I'm afraid of missing other cool stuff in past lessons.

What do you suggest for someone who has experience with Deep Learning?

EDIT: found this wiki with the course notes: [http://wiki.fast.ai/index.php/Main_Page](http://wiki.fast.ai/index.php/Main_Page)

One can use it as a guide to avoid missing anything.

<comment><USER7> I think one of the strengths of the course is that Jeremy shows parts of the process of working on a ML problem. If you have time, I recommend watching earlier lessons, even if you know the theoretical aspects of the content covered.

</comment></comment></comment></comment></comment><comment><USER8> You probably can... but is that really the issue?

I think the problem isnt that you cant solve problems with small amounts of data; its that you can't solve 'the problem' at a small scale and then just apply that solution at large scale... and that's not what people want or expect.

People expect that if you have an industrial welder than can assemble areoplanes (apparently), then you should easily be able to check it out by welding a few sheets of metal together, and if it welds well on a small scale, it should be representative of how well it welds entire vehicles.

...but thats not how DNN models work. Each solution is a specific selection of hyperparameters for the specific data and specific shape of that data. As we see here, specific even to the volume of data available.

It doesnt scale up _and_ it doesn't scale down.

To solve a problem you just have to sort of.... just mess around with different solutions until you get a good one. ...and even then, you've got no really strong proof your solution is good; just that its better than the other solutions you've tried.

Thats the problem; its really hard to know when DNN are the wrong choice, vs. you're just 'doing it wrong'

</comment><comment><USER9> Andrew Beam's post offered very persuasive evidence that Jeff Leek's intuition (deep learning yields poor performance with small sample sizes) is incorrect. The error bars and the consistent trend of higher accuracy with a properly implemented deep learning model, particularly with smaller sample sizes, is devastating to Leek's original post.

I think this is a fantastic example of the speed and self-correcting nature of science in the internet-age.

As an aside, @simplystats blocked me on Twitter, which I assume is in response to this tweet: [https://twitter.com/ErickRScott/status/871586233599893505](https://twitter.com/ErickRScott/status/871586233599893505) and it seems that I'm likely not the only one blocked: [https://twitter.com/jtleek/status/871693250947624961](https://twitter.com/jtleek/status/871693250947624961)

What's most concerning about @simplystats blocking activity is the chilling effect it has on discourse between differing perspectives. I've tried to come up with a rationale for why highlighting the most recent evidence in reply to someone who sympathized with Leek's original post (btw, @thomasp85 liked the tweet) is grounds for blocking , but I can't come up with a reasonable idea.

Further aside, is irq11 Rafael Irizarry?

Update: after emailing the members of @simplystats they have removed the block on my account and offered a reasonable explanation. SimplyStats is a force for good in the world ([https://simplystatistics.org/courses/](https://simplystatistics.org/courses/)) and I look forward to their future contributions.

</comment><comment><USER10> It's an interesting conversation but really weakened by failing to take on the generalization problem head on. This is something I see in a lot of discussions about deep nets on smaller data sets, whether transfer or not. The answer "it's built in" is particularly unsatisfying.

The plots shown certainly should raise the spectre of overtraining - <USER2> rather than handwaving about techniques to avoid it, it would be great to see a detailed discussion of how you convince yourself (i.e. with additional data) that you are reasonably generalizable. Deep learning techniques are no panacea here.

</comment><comment><USER11> Ppl keep saying a lot without even thinking is a relative term. For images a lot means enough to get to x percentage accuracy, for OCR for a single font, a lot means 26 letters + special chars and numbers. Stop saying a lot blindly like every one underatands.

</comment><comment><USER12> ...but why would you?

The fact that there are people "getting their jimmies up" on questions of training massively paramterized statistical models on tiny amounts of data should tell you exactly where we are on the deep-learning hype cycle. For a while there, SVMs were the thing, but now the True Faithful have moved on to neural networks.

The argument this writer is making is essentially: "yes, there are lots of free parameters to train, and that means that using it with small data is a bad idea in general, but neural networks have overfitting tools now and they're flexible so you should use them with small data anyway." This is literally the story told by the bulleted points.

Neural networks are a tool. Don't use the tool if it isn't appropriate to your work. Maybe you can find a way to hammer a nail with a blowtorch, but it's still a bad idea.

<comment><USER13> I think you're missing the point. The jimmies are getting rustled up because someone provides false information about the performance to make his own argument seem better. This is something anyone should be against.

<comment><USER12> but they didn't.

the writer makes an unconvincing claim that the original post was wrong. the data presented shows only that if you try really hard and get lucky enough, you can probably do as well as a simple regression in this case.

the author himself admits that deep learning is probably misapplied here, and that training with such small data is difficult, at best. which again brings us back to the important question (i.e. the point being made by the original post): why would you ever do this?

<comment><USER14> _if you try really hard and get lucky enough, you can probably do as well as a simple regression in this case._

Maybe you aren't familiar with deep learning: but this isn't "trying really hard." This is doing basic stuff that anyone using deep learning probably knows.

And deep learning doesn't just "do as well" as the simpler model. It does meaningfully better at all sample sizes.

<comment><USER15> Define "meaningfully better." Perhaps you mean statistically significantly better? It may have better accuracy, but it has significantly less interpretability. What does it capture that regression couldn't capture? At least with regression you can interpret the relationship between all of the variables and their relative importance by looking at the coefficients of the regression. With deep learning, the best approaches for explanation are to train another model at the same time that you use for explanation. Additionally, it was proven that a perceptron can learn any function, so in some senses the "deep" part of deep learning is because people are being lazy because at least you could get a better interpretation of the perceptron. I don't mean to imply that there's not a place for deep learning, but I think this isn't a great refutation of the argument that fitting a deep model is somewhat inappropriate for a small dataset.

<comment><USER14> The model we are comparing against makes 10X as many errors.

I hadn't imagined someone would argue that's not a meaningful difference.

Though the difference is statistically significant too.

<comment><USER15> Not sure what kind of argument that is. If something overfits it will have less error, does that make it better? It may mean it would generalize a lot less when run on more data. Whether or not something is meaningful depends on what you take the meaning to be.

<comment><USER16> Not the OP, but it wanted to point out it has 10X less error on _the holdout sample_ so it is not simply overfitting.

<comment><USER15> It doesn't matter that it's on the holdout, he's partitioning an already small dataset into 5 partitions and talking about the accuracy in using 80 points to predict 20 points. The whole argument is usually that in the law of large numbers you can now have a statistically significant difference in accuracy. When you're predicting 20 points each with 5 (potentially different) models you likely don't have enough to talk about statistical significance.

<comment><USER14> _We tried to mirror the original analysis as closely as possible - we did 5-fold cross validation but used the standard MNIST test set for evaluation (about 2,000 validation samples for 0s and 1s). We split the test set into 2 pieces. The first half was used to assess convergence of the training procedure while the second half was used to measure out of sample predictive accuracy._

Predictive accuracy is measured on 1000 samples, not 20.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> Of course you can use it but does it perform better than "shallow" methods such as Gaussian processes, SVMs, and multivariate linear regression ? either through theoretical or empirical evidence ?

<comment><USER18> The original post used a linear regression (and apparently misimplemented an intended logistic regression); this post sees better results at all sample sizes with a proper deep learning approach.

</comment></comment><comment><USER19> Maybe what you should do is deep learn some data and then do some deep learning with your deep learned [deep] data.

Deep.

</comment><comment><USER20> aka Wisdom of Crowds

</comment><comment><AUTHOR> Also, not to mention transfer learning is a big one.

Case in point, the silicon valley "not hotdog" classifier which they stopped at hotdog or not due to lack of training data when in reality they could've just used a pre trained net on imagenet. Lol, I was literally cringing through that episode so hard xD

<comment><USER18> The developer behind the real-world app mentioned why they did not use pretraining, and why you can't always use pretraining: [https://news.ycombinator.com/item?id=14347513](https://news.ycombinator.com/item?id=14347513)

> We ended up with a custom architecture trained from scratch due to runtime > constraints more so than accuracy reasons (the inference runs on phones, so > we have to be efficient with CPU + memory), but that model also ended up > being the most accurate model we could build in the time we had. (With more > time/resources I have no doubt I could have achieved better accuracy with a > heavier model!)

<comment><AUTHOR> You could still have pre trained the custom architecture.

</comment></comment></comment><comment><USER21> This debate is meaningless for several reasons:

1\. The original argument is a strawman. What do they mean by "data"? Is it survey results, microarrays, "Natural" images, "Natural" language text or readings from an audio sensor? No ML researcher would argue that applying complex models such as CNNs is useful for say survey data. But if the data is domain specific, such as Natural Language text, images taken in particular context, etc. using a model and parameters that exhibit good performance is a good starting point.

2\. Unlike how statisticians view data (as say a matrix of measurements or "Data Frame"), machine learning researchers view data at a higher level of representation. E.g. An image is not merely a matrix but rather an object that can be augmented by horizontally flipping, changing contrast etc. In case of text you can render characters using different fonts, colors etc.

3\. Finally the example used in the initial blog post, of predicting 1 vs 0 from images is itself incorrect. Sure a statistician would "train" a linear model to predict 1 vs 0, however I as an ML researcher would NOT train any model at all and would just use [1] which has state of the art performance in character recognition in widely varying conditions. When you have only 80 images, why risk assuming that they are sampled in an IID manner from population, instead why not simply use a model thats trained on far larger population.

Now the final argument might look suspicious but its crucial in understanding the difference between AI/ML/CV vs Statistics. In AI/ML/CV the assumption is that there are higher level problems (Character recognition, Object recognition, Scene understanding, Audio recognition) which when solved enable us to apply them in wide variety of situations where they appear. Thus when you encounter a problem like digit recognition the answer an ML researcher would give is to use a state of the art model.

[1] [https://github.com/bgshih/crnn](https://github.com/bgshih/crnn)

<comment><USER22> Are you using some accepted definition of Statistics that I'm unaware of? Because I always thought Machine Learning was a branch of Statistics.

<comment><USER21> >> Machine Learning was a branch of Statistics

Thats incorrect.

""" Machine learning is the subfield of computer science that, according to Arthur Samuel in 1959, gives "computers the ability to learn without being explicitly programmed." """

\-- Wikipedia

<comment><USER22> That's fair enough, although I'm still not really sure why statisticians have to think about data in such a different way to machine learning researchers. I feel that if a statistician _didn 't_ look at the bigger picture of what their data is actually about and whether there are existing techniques to tackle that problem, they'd make a pretty terrible statistician.

<comment><USER23> I dont know if the commentator's assertions are correct, but my anecdotal experience follows: A statistician asks me if adding a feature could make a model worse (R^2), and I am like of course! He gets testy, and snidely chimes back 'i don't know why you think that could happen'. And I think 'I don't know why you don't think you can over fit data...'

then it hit me, statisticians thinking revolves around running the model on the entire data set, where my thinking revolves around how it performs on test data

<comment><USER21> There is a good paper by Leo Breiman [1] which discusses the exact issue that you mentioned. E.g. Statisticians believe that there is a "True" model that generates the data, and the errors observed are merely noise. ML on the other hand does not assumes existence of a "True" model, the assumption is that only the Data is source of the truth and any model that predicts the data with lowest error is preferable subject to performance on test/cross-validation etc. This is a powerful approach and distinguishes ML as separate field from statistics.

My favorite example is predicting prices of real estate. A statistician will build a multi-level model that takes into account various effects (Zip code level, city level, school district level, year/month of acquisition etc.) and then build a regression model. The errors then would then be simply noise, an ML approach would be simply using weighted K-Nearest Neighbors, with geographic location as part of distance metric. Sure there are no effects to adjust but the K-NN regression model can account for difficult to capture quirks of geography by being able to represent local non-linear decision surface.

[http://projecteuclid.org/euclid.ss/1009213726](http://projecteuclid.org/euclid.ss/1009213726)</comment></comment></comment></comment></comment></comment>


Choosing a career that a robot won't steal - <AUTHOR> https://www.sokanu.com/blog/choosing-a-career-that-a-robot-wont-steal/ <comment><USER2> Plumbing... It's highly unlikely that plumbing will be outsourced to robots any time soon. The same goes for electrician work for the most part too. For the most part, pretty much any traditional builder trade craft will be around for a while.

Beyond this, I don't think good DJs are really going anywhere... I really prefer a good DJ's matching of music together over Pandora and the like. I don't think the algorithms will ever really match people in that respect...</comment>


Cologne's aftershocks - <AUTHOR> http://www.economist.com/news/europe/21688418-ultimate-victim-sexual-assaults-migrants-could-be-angela-merkels-liberal-refugee <comment><USER2> It's rather hard to catch a criminal before they commit the crime. Sad, but true.

If we were actually good at predicting crime, it wouldn't exist anymore; but we simply don't understand criminals (or ourselves) well enough to do it, so the best we can do is punish.</comment>


Using Samsung's Internet-Enabled Refrigerator for Man-In-the-Middle Attacks - <AUTHOR> https://www.schneier.com/blog/archives/2015/08/using_samsungs_.html

<comment><USER2> From the comments:

> what do we gain by connecting all the fridges out there to the Internet.

Samsung gains the same thing that Coca Cola gains by selling you a 20 oz container instead of a 12 oz.

</comment><comment><USER3> "The internet of infrequently updated things built by under qualified contractors"</comment>


A large majority of health news shared on Facebook is fake or misleading - <AUTHOR> https://www.fastcompany.com/90301427/a-shockingly-large-majority-of-health-news-shared-on-facebook-is-fake <comment><USER2> A large majority of content on social networks are fake or misleading.

<comment><USER3> img file=shocked.gif

</comment></comment><comment><USER4> A Shockley huge number of research papers are bunk. Couple guys from MIT made a fake paper generator and had to break the news because it went undiscovered for so long. Lipitor was promoted for years as a medicine that lowers the risk of stroke. My Mom and two of my close friends had stroke. Two of them are dead. Real confirmation research about Lipitor was done in Europe and Japan and found that it did little if anything to improve the actual outcome that matters.

So yes there are medical quacks everywhere online. Even and especially in places people don't expect.

</comment><comment><USER5> Also true: A shockingly large majority of health news is fake or misleading

<comment><USER6> Oddly enough, despite the clickbait headline, the article supports this thesis. The only place where social media is used here is in the selection of which articles to consider judging for misleadingness. Otherwise, these articles were all carried in regular mainstream publications.

</comment></comment><comment><USER7> Social media is not a platform to consume news. Nobody is vetting the content.

</comment><comment><USER8> Is anyone surprised by this?

</comment><comment><USER9> Ummm..I think this can be generalized: a majority of news shared is fake or misleading or incomplete. So, what do we do about it? Even if we had a perfect filter, would we want to use it?

</comment><comment><USER10> this issue transcends facebook, and social media. a large majority of health news shared by the media is fake or misleading

</comment><comment><USER11> And this is a huge part of why we have a measles outbreak happening just a few miles from where I live. Fortunately everyone in my household is vaccinated, but it's not even supposed to be a thing in the US anymore! And yet it is, not in small part because of widely shared anti-vaccination articles on social media.

<comment><USER12> My wife is pregnant and the doctors realized she isn't immune to measles. The vaccination she had was either not effective or she didn't develope immunity.

I read a comment on Reddit where a student said they had whooping cough but were vaccinated. Turns out most of the people who had the cough went to the same doctor to get vaccinated. The vaccine he gave was presumably a bad batch.

I'd caution against exposing your family to anything if you don't have to. Especially measles.

<comment><AUTHOR> For pregnant women might be different, but CDC says go ahead and do another dose if in doubt [https://www.cdc.gov/vaccines/vpd/mmr/public/index.html](https://www.cdc.gov/vaccines/vpd/mmr/public/index.html)

Frankly, we should research and then instead of buying a new gadget, just double up, just in case....hep vaccines, tetanus etc etc. Go to a travel clinic and pay cash</comment></comment></comment>


Smart locks opened with nothing more than a Mac address - <AUTHOR> https://www.zdnet.com/article/smart-locks-opened-with-nothing-more-than-a-mac-address/ <comment><USER2> Ouch!</comment>


Understanding Dictatorships: The question of legitimacy - <AUTHOR> http://www.reason.com/news/show/135775.html

<comment><USER2> Interesting piece. The pace dragged a bit, and I almost gave it up. But I'm happy I read the whole thing as there was 1-2 very interesting insights mixed in there. The different legitimization "strategies" was the most fascinating.</comment>


Ask HN: What did you think of Google I/O today? - <AUTHOR> Personally I thought it was meh. Just seemed to be a lot of the same: convenience-friendly data-hungry services masked with baby pictures and "family" sentiment.<p>I did enjoy hearing about how they have mini-ML models on each device, feeding data back to a master learner, which in turn sends updates back to the device. <comment><USER2> I wonder if things like I/O are heading to the Intel "tick/tock" model where expectation of change has to push out to 2 or more cycles, because it turns out endless innovation is just not sustainable.

So a new gizmo in even years, and a cheaper model plus some interesting extras in odd years?</comment>


Ask HN: Who needs a business model? - <AUTHOR>

If you think you have a great technology or app but your not sure how to make money from I can help provide you with a business model. In exchange I will write about it on my blog. If I don't think you can make a business out of it I will explain why. <comment><USER2> Oh, I need a business model but I don't have technolory or app (yet). I have figured out how to get myself well in the face of an incurable "dread disease" (a form of cystic fibrosis, which is genetic). I want to eventually make a game (simulation) to more effectively convey what I know. In the mean time, I work a full time job and am drowning in financial problems. I don't know how to get there from here. I need to leave so I have time to learn a programming language, write the game, etc. You probably can't help me, the gap is too big, I'm just very frustrated so replying anyway.

<comment><AUTHOR> Your situation is not as bad as you think. Although you have financial problems at least you have a job. Remove all unnecessary expenses from your life, only buy what you need to survive. Focus on a few things, focus one work and your side project. You must eliminate distractions and time wastes. It's easy to say, harder to do.

<comment><USER2> I have eliminated all unnecessary expenses. My health issues are still quite pricey and the past three months have been very ugly in that regard, which is a big part of why I am so stressed out about the whole thing. The debts I owe are a lot _for me_ , given my current income and situation, but not a lot in the grand scheme of things (like compared to what my medical condition typically costs to treat using conventional approaches). It's very, very frustrating because I am saving gobs of money for other people in terms of taxes not spent on keeping me alive: I am a former military wife and could have kept my military medical benefits and had nearly "free" medical care at the expense of taxpayers. Doing the right thing is all coming out of my hide financially and it just feels like the universe is pissing on me. I could be rewarded for keeping my problems alive and not solving them but can't seem to catch a break for actually rolling up my sleeves and fixing sh*t.

It would make financial sense in the big picture -- not just for me, but for the national economy - if I could get some support so I can leave my job and work on this. Even if others with my condition don't get well, even if all they do is get less sick and need to be hospitalized less and need half as much medication, by my estimate the US is spending around $3 billion/year on medical care for roughly 30,000 people with my condition and most of that is covered by insurance, state aid, federal aid, charities and write-offs (when they can't pay their medical bills). And that doesn't begin to factor in disability payments and other forms of support that chronically ill and dying people often qualify for. But I know of no way to tap into that $3 billion and I know of no way to fund-raise to put myself in a position to work on this.

I think I need about $50,000 or so to resolve my financial issues, walk away from my job, and have some money in the bank to live on while making the transition to some kind of entrepreneurial income based on my various websites. I know there is "demand" for some of what I know and a couple of my websites have been around for several years (one of them is about 8 years old, another is over 5 years old and I have a couple of newer things). I don't know how to monetize it and, given the health issues in the family (I and my oldest both have CF), trying to put in the extra effort to achieve launch while working a full time job is proving to be too much.

Thank you for replying. I really didn't expect much (if any) help. I have figured out how to "cure" cystic fibrosis which most of the world says cannot be done. If resolving my financial problems were a simple matter, I imagine I would have done it by now. I'm not bad with money. I've just been doing "the impossible" and there are consequences for that.

Peace.</comment></comment></comment>


The Next Sourceforge: Bought by Slashdot, Open-Sourced - <AUTHOR> https://sourceforge.net/create/ <comment><AUTHOR> Everytime when sth. on SF is linked someone comments "Oh, that crapware site". Thus I thought maybe we should check out what SourceForge is trying to become. I'm by no way affiliated to them (nor a user ATM).

<comment><USER2> I am a long time user of Sourceforge (I registered in 2004-05-01). While I agree that Sourceforge experienced troubling issues when it was (along with Slashdot) bought by that media group, I still prefer it to Github when it comes to searching for tools in the same category (there are no categories in Github) or tools that are suggested by their recommender. It was and is still great to learn different techniques to solve some problem.

Ah those good old times!</comment></comment>


Why REST (Representational State Transfer) ? - <AUTHOR> http://blog.dhananjaynene.com/2009/06/why-rest/

<comment><USER2> I actually liked the article because I used to be confused about the fact that anyone would choose this limited protocol over RPC but I'm starting to get it.

I personally will still stay away from it. I'm sure that if I implemented RESTful system, eventually I would need some special functionality that would force me to either emulate RPC over REST (what was the point implementing it in first place if I do that) or end up with 2 interfaces - one for REST and other for RPC, making things confusing.</comment>


SaveTheInventor urges people to lobby for stronger patent laws in the US - <AUTHOR> http://www.savetheinventor.com/? <comment><USER2> I couldn't find out who is behind this organization but I did find this:

[http://www.patentprogress.org/2015/02/02/real-inventor- adven...](http://www.patentprogress.org/2015/02/02/real-inventor-adventures- save-inventor-didnt-tell/)</comment>


GM to Introduce Hands-Free Driving in Cadillac Model - <AUTHOR> http://www.bloomberg.com/news/2014-09-07/gm-to-introduce-hands-free-driving-in-cadillac-model.html

<comment><USER3> I worked at Google for 3 years and GM for 5 years (including engineering in Cadillac) so when I see news about the companies, my ears tend to perk up.

A few things should be set straight. Google was not (and by a long shot) the first to think of and try to develop self driving technology. I'm disappointed to see technical people in silicon valley not think a bit more critically. All OEMs and many tier 1 suppliers have thought about and (especially larger players) tried to build self driving cars. When you're a company like Denso or Bosch, this is a massive opportunity and not to aim a few people of your hundreds of thousands of employees to the problem is hard to imagine even if you know nothing about automotive. To be clear when I worked at Bosch from 2004-06, this tech was in development.

Secondly, the HN crowd specifically is notably biased. Check out Google self driving announcements vs GM (or any news about automotive manufactures other than Tesla). Having worked at both companies in product and engineering - <USER2> they are of course different. But not nearly as much as you think. The single most notable difference is that Google is a very margin rich business. That allows the company to attract and retain top talent but it doesn't mean engineers that doesn't work at Google are bozos.

Thirdly, the technology that traditional OEMs have tried to develop is around 2 axis - cost and safety. Google is approaching the problem differently. Google's approach of a $x0,000 LiDAR system might prove to work if LiDar costs fall but thats a big risk. Also, Google admits that their cars don't work in things like rain. I'm not saying Google won't win this space - the contrary, I think they have a great shot. What i'm saying is this is significantly complex tech and its complexity is hard to understate. There can and will be different solutions that work.

Lastly, GM and Google are not the only ones in the game. Nissan and Mercedes have announced they will have cars with similar tech in 2017/18\. Ever single automotive player is investing huge amounts of resources.

<comment><USER4> While this may be true, it's also been the observation of a lot of people here that, historically, it's been difficult for non-software industries to build effective consumer facing software teams, no matter how well capitalized.

It's not inconceivable that car companies are simply not equipped to manage the kind of complexity intrinsic in building fully autonomous vehicles, independent of the amount of resources they deploy.

<comment><USER5> I agree that automakers haven't been the greatest with consumer facing software/UI; I've yet to use a stock touchscreen interface in a car that doesn't cause bouts of rage.

However, automakers are probably up there with the best of the best when it comes to embedded "replace the consumer" software. Almost every aspect of a modern car is software controlled or software monitored in some way. I think that the fact that many people don't see their car as a network of computers, or recognize that it's pretty much all software is proof of that. Automakers have slowly been replacing the driver with software, and many people don't notice or think of it like that, because it just works, requires no input or interaction, and doesn't fail.

Honestly I think that in the end, Google will probably end up with a better product, but today, there's probably no one better equipped than car companies to make autonomous vehicles that align with consumer expectations of what an autonomous vehicle should be, both in engineering and resources.

Edited to add:

Look at what's already been done as far as software controlled driving. Mercedes has a system that will stop your car if you're about to rear end someone/hit something. A bunch of manufacturers have systems that will warn you when you're drifting out of your lane. Maintaining speed to the car in front of you instead of just setting a fixed cruise control speed is getting pretty widespread too. All of these by themselves seem like simple iterations, but when you start putting them all in one package, its moving closer and closer to an autonomous car, and all of it works so well that many people don't even notice.

</comment></comment></comment><comment><USER6> Anyone remember reading an old article about the relationship between safety equipment and risky behavior? The idea was that the more safe you feel, the more likely you are to engage in unsafe behavior. The example given was that cyclists with helmets were much less cautious of traffic than bicyclists without helmets.

So I'm not sure if this has to do with cars feeling more safe or just the prevalence of smart phones, but driving down the interstate and paying close attention to what other cars are doing is terrifying. Literally every day I see a number of people driving 70+ MPH and slightly drifting over the lines or driving with their phones perched in front of them on the steering wheel. And it is not just teens or other classes of people typically associated with risky behavior, it is everyone.

This transition from smart-cars to self-driving cars is scary. As the cars become more attentive it seems we get less attentive.

<comment><USER7> That's an interesting thought, but we are already incredibly inattentive drivers - as evidenced by the 33,000 automotive deaths last year in the U.S. alone. Self driving cars can't come soon enough.

Fortunately, the same technological advances that have given us hand-held computing to distract ourselves are also going to give us self driving cars to save us.

</comment></comment><comment><USER8> Expectations have downscaled: [https://news.ycombinator.com/item?id=3909931](https://news.ycombinator.com/item?id=3909931).

</comment><comment><USER9> >>"The roads will be equipped with sensors and cameras that enable roads to communicate with cars to alert drivers to hazards and congestion. The technology, to be deployed along stretches of Detroit's busiest freeways, will monitor vehicle speed and position, though that information will be anonymous and police won't use it to ticket drivers, Lauckner said."

Call me skeptical, but I highly doubt that this type of tech would not be used by police

</comment><comment><USER10> I wonder if the "vehicle-to-vehicle" technology could be implemented faster if it was included in GPS mapping apps. Sure, your GPS can't slam on your brakes but it could warn you of hazards, and your data might be used to slam on the brakes of the car behind you (and prevent you getting rear-ended!)

</comment><comment><USER11> Actual news about this will tell us how the system deals with distracted or sleeping drivers.

Right now we assume a computer driven car can handle the highway. How it handles its human occupants is the real question.</comment>


Switzerland stops contact tracing, reduces quarantine to 5 days - <AUTHOR> https://www.lematin.ch/suisse/quarantaines-courtes-suisse/story/13175495 <comment><USER2> I've heard yesterday Switzerland is really not prepared very well. They are relying on hospitalization, but clearly the calculation is they'll only take in high risk cases.

first commenter says when her child developed flu like symptoms they were told they can't guarantee what she has, go home and see what happens the next 5 days, wearing a mask not to spread it to others...

</comment><comment><AUTHOR> Infections were doubling every 2 days until they stopped testing mild cases. Schools are open, border with Italy is unchecked, nobody is wearing masks. I'm here and honestly not sure what to do.</comment>


Diagramming Product/Market Flow at Lean Startup - <AUTHOR> http://torgronsund.wordpress.com/2010/02/28/diagramming-productmarket-flow-at-lean-startup/ To achieve a product/market flow state (fit), a balance must be struck between customer development and product development. If a startup's focus is too problem-oriented or to solution-oriented – drifting too far along one of the axis without iterating – flow cannot occur <comment><AUTHOR> To achieve a product/market flow state, that is product/market fit, a balance must be struck between customer development and product development. If a startup's focus is too problem-oriented or to solution-oriented – drifting too far along one of the axis without iterating – flow cannot occur</comment>


Amazon to give power to brands to remove fakes from website - <AUTHOR> https://www.theguardian.com/technology/2019/mar/04/amazon-to-give-power-to-brands-to-delete-fakes-from-website <comment><USER2> So, how will this help with things like fake SD cards? Those are usually sold unbranded, but are still faker than fake.

IMHO, it's not a good compromise to push the responsibility for fake-detection onto brands and manufacturers. Small makers might not have the resources to effectively police _Amazon 's massive inventory_, and some brands might abuse this power to squash smaller competitors. It's really Amazon's job to keep fakes out off of its platform, and I think they should do the work themselves.

However, cooperative things like their "serialization" feature seem like a pretty good idea.

<comment><USER3> > So, how will this help with things like fake SD cards? Those are usually > sold unbranded.

How can an unbranded SD card be fake? Fake with respect to what?

A brand brings this value: expectations.

If I buy a no-brand item, I'm not sure what expectations I can have, and I'm not sure I can even claim that it's a fake, since I have no reference point as "the true item".

I mean that's literally the biggest value that a brand brings to you, the consumer.

That's also why brands work hard to build up and maintain good reputations.

<comment><USER2> > How can an unbranded SD card be fake? Fake with respect to what?

In respect to not being the claimed capacity, but modified in a data- destroying way to make that difficult to detect:

[https://www.eetimes.com/document.asp?doc_id=1326059#](https://www.eetimes.com/document.asp?doc_id=1326059#) (from 2015, capacities have obviously changed since then):

> Tests by the Counterfeit Report found that the cards will work at first, but > generally speaking, buyers are purchasing what they think are cards with > capacities of 32GB and up. Instead they are getting are cards with 7GB > capacity. Counterfeiters simply overwrite the real memory capacity with a > false capacity to match any capacity and model they print on the counterfeit > packaging and card, Crosby explained. Users can't determine the actual > memory capacity of a counterfeit memory card by simply plugging it into > their computer, phone, or camera. When the user hits the limit, the phony > card starts overwriting files, which leads to lost data.

> The Counterfeit Report often comes across cards in capacities that don't > exist in any product line, and the cards it purchases and tests that are > 32GB and up are usually always fake. The counterfeiters make a great profit > on the fake cards, and there's no consequence.

<comment><USER3> Please see my reply in sibling comment. It addresses this.

</comment></comment></comment></comment><comment><USER4> About time. I've ordered quite a few skincare & haircare products that were obliviously refilled with random things. Shampoos that smells like pool cleaner, etc.

I used to always order everything from Amazon. Now I've reached a point where I avoid it.

What's annoying is that the fakes and the reals have the same listings and share the same reviews. So you glance at the reviews and see mostly 5 stars... only to notice that all recent ones are 1 star written by people who received fakes.

Ex: [https://www.amazon.com/Joico-111772-K-pak- Conditioner-33-8-o...](https://www.amazon.com/Joico-111772-K-pak- Conditioner-33-8-ounce/dp/B0018Z4DAE/ref=cm_cr_arp_d_product_top?ie=UTF8&th=1)

<comment><USER5> Switching products like this seems to be very common on Amazon. I always make sure and go to all reviews and check the most recent before ordering. Review Meta is also a great resource for weeding out less trustworthy reviews.

</comment></comment><comment><USER6> This will help with fraudulent copies of things like books, but what about utter fraud where they steal content and make fake books?

[https://twitter.com/billpollock/status/1091840257073471488](https://twitter.com/billpollock/status/1091840257073471488)

</comment><comment><USER7> But who will remove the other non-brand fakes? Just yesterday i noticed a 1Tb micro-sd for only $20 instead of the usual $60 for other fake 1Tb cards... I would have flagged the product. If only Amazon had that function.

All these fake items could destroy Amazon's credibility pretty fast...

<comment><USER8> How do you become trusted ?

do a good job(Amazon's brands ), while making the alternatives look bad.

And it's one huge step ahead against their biggest competitor, aliexpress.

</comment></comment><comment><USER9> Why has it taken so long? Why isn't Amazon doing it themselves? Why have they not been sued for enabling fake item sales?

<comment><USER10> Probably the same reason mom and dad don't try to figure out who started it.

</comment></comment><comment><USER11> Surprised it took this long. I'm curious if the implementation is right, in both directions. You want to cede enough power to kill counterfeits, but not enough to kill workalike products, secondary market sellers like those that buy genuine merchandise from, say, unsold WalMart stock, etc.

<comment><USER12> You're surprised it took this long, then outline some of the harder issues that would make it take a long time to implement correctly.

Not only that, but those are policy / design issues. The technical implementation of this at Amazon scale isn't trivial, then adding in fraud detection <for this tool> so folks don't abuse it too.

The assumption that Amazon doesn't care about this problem, the fake reviews problem and others killing their marketplace vs. the problems being difficult to solve that permeates the comments here gets tiring.

There is no logical reason why they wouldn't care when they try to cultivate customer satisfaction and loyalty (they're one of the most loved companies in America) in every other aspect of their business.

<comment><USER11> Amazon FBA, which made this problem much harder, was launched in 2006. I don't feel like "took this long" is unreasonable.

<comment><USER12> And it became an issue immediately or was it only an issue in the last 2-3 years as bad actors found and exploited holes in the system?

<comment><USER11> It was an issue from the outset. By 2016, Apple was claiming 90% of Apple chargers on Amazon were fake. If there's truth to that, it was way past being an "issue" then. I'd consider 5% fakes of a popular product to be a serious issue.

Edit: I'll concede that it's been a serious PR problem for Amazon only in the last 3 years. It's been a problem for brands, sellers, and buyers for much longer, and Amazon has known about it for more than a decade.

<comment><USER13> Apple claimed that 90% of the ones they bought were fake. They didn't claim theirs were statistically representative of the ones on the marketplace.

I've seen plenty of false Apple claims regarding Amazon and I've talked to people who sued Apple and won. There's also a famous case (search hard 2 find accessories) that sued Apple and lost because the judge basically said Apple had immunity under the 1st amendment to make claims even without vetting them. I think this is wrong but it's tough to go up against Apple in court.

</comment></comment></comment></comment></comment></comment><comment><USER14> I miss when Amazon was just a retailer, not a marketplace (or at least when it had the marketplace adequately fenced-off from its own retail). It had a lot more value to me as a consumer then.

</comment><comment><USER15> I suspect some serious first-sale violations will occur with this new system.

</comment><comment><USER16> Could there be negative unintended consequences here? If someone else sells your product cheaper than you do on Amazon, could this be used to remove them and keep prices higher?

<comment><USER13> Yes. I've personally received fake complaints of counterfeit from multiple 9 and 10 figure brands. There are active lawsuits (see careful shopper LLC vs tp-link and Johnson vs incopro which was just settled) for this. Shady law firms and "brand protection companies" are advising big brands to do this and it's hurting legitimate sellers. I know sellers with dozens of false complaints, it's getting out of hand.

</comment></comment><comment><USER17> No reason to link to blogspam. The actual release of information is here: [https://blog.aboutamazon.com/company-news/amazon-project- zer...](https://blog.aboutamazon.com/company-news/amazon-project-zero)

<comment><USER18> The Guardian isn't a blog, and this story isn't blogspam. The linked Guardian story provides a lot of useful context about Amazon's counterfeiting problem that isn't included in Amazon's marketing post.

</comment></comment><comment><USER19> This has existed for the better part of 4 years BTW. No idea why it's suddenly newsworthy - slow week?

<comment><USER15> No, it hasn't. Anti-counterfeit controls have existed for a while. This is new, and part of Project Zero, which was only officially announced a week ago.</comment></comment>


StackOverflow breaks 15M questions - <AUTHOR> https://stackoverflow.com/questions <comment><USER2> It seems like a vast majority are low quality questions asked by people with a questionable grasp of English.

I'd like to see some studies on quality through the times as SO grew.</comment>


Digital Needle - Playing phonograph records with a scanner - <AUTHOR> http://www.phys.huji.ac.il/~springer/DigitalNeedle/ <comment><USER2> And for the record, there are some pretty unique distortions here. I know a lot of artists who'd be pretty excited about the samples themselves...there's something special about them being a byproduct of a unique process as opposed to a clicked button or three in protools.

<comment><USER3> Any idea what those ~2Hz distortions are? In the a and c series recordings, they have a very strong pitch change, coming in groups of 4...

<comment><USER2> Was wondering the same thing, but I've got no idea. Curious if blur from the visual scan of denser grooves has anything to do with it. Really interesting sound.

</comment></comment></comment><comment><USER4> It seems to me that I read once of a device at the Library of Congress which is a purpose built laser scanner for extracting such waveforms from old records, even the drums.

<comment><USER2> Yeah they're IRENE scanners. Pretty amazing, actually. Able to lift sound off the most delicate old recordings. Probably a lot of possibility for repair of lost fragments visually:

<http://irene.lbl.gov/>

And there are laser turntables for commercial purposes as well. Not a visual product, in interesting still...

</comment></comment><comment><USER5> Related: <http://www.elpj.com/>

<http://en.wikipedia.org/wiki/Laser_turntable>

<comment><USER6> Some time ago I wondered if this could be done with a small CCD (Like the one in your mouse, see the mouse-scanner project: <http://spritesmods.com/?art=mouseeye>). I tried it once, but it's very hard to do this in real-time.

</comment></comment><comment><USER7> so, how long before we get an augmented reality app that lets you play your (parents) records just by pointing your iPhone at them?

<comment><USER8> Actually, that sounds like a pretty good app. Shopping for a CD and want to hear it (or at least previews) now? Scan the barcode or album cover and boom.. music

I'd pay $4.99 for that app.

<comment><USER9> People still shop for CD's in stores?

</comment></comment></comment><comment><AUTHOR> Reading up on Wikipedia, I think he got the encoding wrong. The samples he plays are the diff between left and right. Now I want to try this... but I don't have a scanner handy. Can anyone scan a record for me?</comment>


macOS is required to develop – no questions asked - <AUTHOR> https://www.fanfario.com/post/1110-macos-is-required-to <comment><USER2> The tribalism is stupid and it annoys me a bit. I understand it though in an operating system landscape space, let me explain why.

When it comes to developers time, it is zero sum. People who work on our non- preferred platform take time and energy away from our preferred platform. Nowhere is this more evident than video games which essentially require one (paid/opaque/proprietary) operating system.

It's also zero sum in our own minds. If we are expected to understand our tools (which is a fair ask) is it better to have 2 tools or 50 tools? Those 50 tools all attempting to be the same thing but in only slightly different ways and the main differentiator being what is developed on the platform.

FWIW I would consider myself a bit of a Linux zealot, but I use a Mac at work and I Parsec into a windows machine purely for visual studio. For me computers (And operating systems) are a tool, but I cannot deny the overhead when switching, the change in hot keys, in basic commands and available features always causes more friction than if I was working with _just_ Linux or _just_ macos and so on.

<comment><AUTHOR> That is totally true and I was just pointing out, that bashing the other system is not OK, just because you want to minimize your overhead. Windows might be the inferior platform or not for a specific purpose. But talking bad about the neighbor, because you dislike his house color is not something I want to have in our dev community (/personal opinion).

</comment></comment><comment><USER3> Oh what I would give for Xcode on Windows. I develop crossplatform apps using Flutter, and I am 99% of my time using Android Studio on Windows, and if I want to publish on iOS I have to use a Mac just to sign my app.. Nothing else pretty much, since it is all one codebase for 2 platforms.

It does become pretty irritating trying to adapt to these "closed" platforms, especially when I fail to see how benefical is a move like that for consumers, except for monetary purposes for Apple.

I want Apple to convince devs to use Macs for development purely because of the features and experience it provides, not because it closes it's platform and says "You cannot develop for iOS unless you cash out $2000 for our laptop".

<comment><AUTHOR> I know, the requirements for a Mac just to publish your app ALSO for iOS is a bit exhausting. Cordova or Flutter are pretty cool environments to write once and deploy every... wait, need to fire up the mac...

</comment></comment><comment><USER4> I've similiar setup except Mac, setup VNC on Linux and RDP on Windows, so I only need to open single laptop + external monitor.

I use Mac on virtualbox to build the iOS and some as gitlab runner

</comment><comment><USER5> What laptop stand is that?

<comment><AUTHOR> The one at the bottom for the Dell? That is an older SVALT stand ... the one up with the Mac is just one of these arms, you can get cheaply on amazon...</comment></comment>


Clipboard.js: Modern Copy to Clipboard - <AUTHOR> https://github.com/zenorocha/clipboard.js <comment> david-given Text field autoselection and programs being 'helpful' by copying things to the clipboards is one of the banes of my life. I wish there was a way to just disable all this, globally. If I want something selected, I'll select it myself, thanks.

My problem with text field autoselection is that it's wrecking my muscle memory of how to interact with text fields. My reflexes know that a single click places the cursor, a double click selects a word, and a triple click selects a line... except when the text field has autoselect turned on, which I can only find out _after_ I've committed to the action. And they all behave slightly differently.

Example interaction:

\- click on text field to place cursor.

\- text field autoselects. I swear.

\- click again to place cursor.

\- hasn't been long enough since the last click, which means it's interpreted as a double click.

\- text field selects word. I swear.

At this point I'll either force myself to wait for a second or so, to make the interaction time out, and then click _again_ , or else I'll be too impatient and the madness continues as my third click is interpreted as a triple-click.

Seriously, wish there was a way to make this all stop. If I want my text fields selected, I'll do it myself, thanks.

<comment><USER3> If there was a Firefox extension that simply disabled 100% of the ability to control or influence the selection, I'd install it in a second.

Manipulating the selection is probably a specific case of the larger problem of terrible, incomplete reimplementations of UI that already exists. This inevitably ends up demonstrating the exact gap between what user environment is, and what the webpage author believes it _ought_ to be.

<comment><USER4> From a quick glance it seems quite possible to make such extension: simple dark prototype crippling for inputs and textareas like

        Selection.prototype.addRange = HTMLTextAreaElement.prototype.select = HTMLInputElement.prototype.select = function(){ console.info(this,' arificial selection disabled, sorry.') }

If you use some userscript manager, try it with [https://greasyfork.org/en/scripts/12762-disable-arificial- se...](https://greasyfork.org/en/scripts/12762-disable-arificial-selections)

<comment><USER5> Hate to be that guy but just wanted to point out the typo in the console message/userscript title:

"arificial" should be "artificial" (missing 't').

<comment><USER4> Hanks for pointing out, good catch! Fixed in userscript, also updated it a bit. Not able to fix it here. Seems like quite frequent mistype BTW :]

</comment></comment></comment></comment></comment><comment><USER6> I was thinking -- why isn't this more common?

Then I read the bottom of the readme, and noticed safari was holding back webdev once again.

I've been casually following this for the last 5 years and it seems almost too convenient that safari is regularly behind on mission critical features.

<comment><USER7> Funny. I find Safari leaps ahead of Firefox, and even further ahead of Chrome, when it comes to browser performance (CPU use, memory footprint, and especially battery life), with snappier scrolling and animations and fewer weird graphical glitches or mouse/keyboard interaction bugs. For my own browsing habits, Firefox is _very_ sluggish, and Chrome is completely unusable: it just can't handle the load and brings my whole computer to a crawl. The Safari user interface also feels much more platform-native and predictable (though it has unfortunately copied several Chrome UI elements in the last few years, which I think make the basic UI worse than ~5 years ago), and I don't constantly feel like someone is trying to gather and analyze my keystrokes or sell me out to advertisers. (I don't have a Windows machine, so I couldn't tell you what IE is like these days.)

Maybe there's just a focus on different things between vendors: Google has tons of 20%-time engineers pushing new half-baked features in every direction (yes this is a bit of a caricature), while Apple is taking a more measured approach with a focus on performance and reliability.

<comment><USER8> > For my own browsing habits, Firefox is very sluggish, and Chrome is > completely unusable: it just can't handle the load and brings my whole > computer to a crawl.

Any example of those pages that brings the computer to a crawl?

<comment><USER7> It's not any particular page that makes Chrome die. It's opening more than 30 at a time. (I typically have anywhere from 50–200 browser tabs open at once.)

<comment><USER9> Do you use task manager (Hotdog menu > More tools) to kill tabs that you are not currently using? Also chrome 46 looks to be getting much better[0]; although I have also noticed a sizeable improvement with 45.

[0] [http://thenextweb.com/apps/2015/08/20/chrome-is-finally- gett...](http://thenextweb.com/apps/2015/08/20/chrome-is-finally-getting- faster-on-mac/)

</comment></comment></comment></comment></comment><comment><USER10> What does this do that OS native clipboards don't? Serious question - <USER2> there platforms I'm unaware of where the functionality doesn't exist but a web browser does? Seems unlikely to me, but I might be wrong.

<comment><USER11> It gives you a button you can click to select a predetermined piece of text and add it to your clipboard. That's it. This is handy from a usability perspective in some circumstances.

</comment></comment><comment><USER12> The documentation is a bit coy about this, but the usual purpose for this sort of library is so you can modify the copied text before it hits the user's clipboard, to insert an advertisement or copyright notice. (See the 'data- clipboard-text' attribute.)

So, for evil. Safari not supporting execCommand copy and paste is a _good_ thing for users.

(This claims to work in all major browsers except Safari, but I'm see the same fallback when I try it in Firefox, on OS X at least -- it shows the key combination in a popup instead of actually copying or cutting the text as it does in Chrome.)

</comment><comment><USER13> I needed a simple "click to copy some data" so I made this simple function: [https://gist.github.com/mdamien/a0f2bd91750a2d45c8ab](https://gist.github.com/mdamien/a0f2bd91750a2d45c8ab)

I feel like the need for a whole library here is a bit overkill if you have a "simple" usage.

</comment><comment><USER14> Does it copy/paste image data too? My brief look at the docs doesn't seem to mention it but it would be ace if it did...

</comment><comment><USER15> Glad to see new libraries for the new clipboard API! I was recently making a color picker website that would not use flash for copying color values ([http://0xrgb.com](http://0xrgb.com)) and I was really missing a library like this. Anyway thanks, I hope it will replace zeroclipboard very soon!

</comment><comment><USER16> I'd love to see the ability to paste without using the keyboard too. Is that something possible/on the roadmap?

<comment><USER17> I don't think so. Ability to access information in user's system clipboard seems like a huge security and privacy issue.

</comment></comment><comment><USER18> Just switched [https://wakatime.com/](https://wakatime.com/) to this yesterday... it's so much better than ZeroClipboard!

1\. ZeroClipboard has issues with Bootstrap tooltips on the copy button

2\. This doesn't depend on Flash!

3\. Gracefully downgrades to a "Press Ctrl+C to copy" message for Safari

<comment><USER19> The irony of telling safari users to "press control c" is ridiculous.

Safari on iOS doesn't have a control key and on OS X copy is "Command-C"

But honestly the idea that someone can't select text and copy it themselves (and thus need this helper) is a stretch of reality anyway so why am I surprised that it doesn't reflect reality in its "fallback" either..

<comment><USER20> It's not a stretch. If you have a product that "real" people use and phone number for customer support, you will quickly learn the value of a copy-to- clipboard feature.

People cannot even successfully select complete URLs at an alarming frequency.

<comment><USER19> > If you have a product that "real" people use and phone number for customer > support, you will quickly learn the value of a copy-to-clipboard feature

 _Real_ people have being using _regular_ select + copy operations on their device of choice, for literally decades.

I honestly don't understand why every web app developer assumes that their user base is full of people that just picked up a device with a browser and have never used it before.

<comment><USER21> We don't assume anything. We take calls from customers where they can't figure out how to copy and paste. We take calls where customers don't even know what a browser is, or a web page, or in some extreme cases how to select a textbox and type in it.

Just because your customers (assuming you do web dev) can figure things out doesn't mean everyones' can.

</comment></comment></comment></comment></comment><comment><USER22> Started project 13 days ago and has 4k+ GitHub Stars

"This might come in handy; better star it."

</comment><comment><USER23> Okay, who pressed the star button? [https://github.com/zenorocha/clipboard.js/issues/56](https://github.com/zenorocha/clipboard.js/issues/56)

</comment><comment><USER24> And for the truly adventurous, there's the html5 clipboard api.

<comment><USER25> Isn't the "problem" of the HTML5 clipboard API that you can only access the clipboard during clipboard events, so you can't put stuff in the clipboard on a button click, which is exactly the user-case for this library?

<comment><USER24> This is very true, though I was mostly commenting on the "Modern copy to clipboard" description.

</comment></comment></comment><comment><USER26> The whole point of flash-based clipboard libraries are to support legacy browsers. This attempt at removing those falls flat and does not support ie8, sigh...

<comment> nevi-me IE8 is holding the rest of us back. Reality is that most usage of IE8 comes from corporate users who are stuck on such IE8. We can't keep supporting legacy browsers forever. The solution that has been working is to warn the user that their browser is not officially supported, and that things will break.

One of the banks that I consult with allow users to install the corporately limited version of Chrome, it's sometimes 10 or more versions behind latest stable, but it's better than IE8.

Bootstrap, Font-Awesome, jQuery are dropping IEx because it's a burden to maintain that for a small subset of users. We should all do the same.

<comment><USER27> I personally follow Google in what I support unless the project I am working on is specifically for older browsers.

My idea is if Google who owns a huge market share of the web can drop it, it's safe to say so can I.

</comment></comment></comment><comment><USER28> Another trick using clipboards (in Mac OS X): use pbcopy/pbpaste to copy and paste to clipboard

</comment><comment><USER29> So modern!

Isn't any library released today, which improves on libraries from yesterday, by definition "modern"?

Whenever I hear "modern" I just picture the developer thumbing his nose at the old fuddy-duddies who don't know how to code and spew out garbage. Yeah yeah, I get it -- the older stuff sucks. But maybe I'm imagining it. If people here tell me that the word "modern" is truly free of derision, I'm open to that viewpoint.

<comment><USER11> Well, in this case it relies on the Selection API which is currently still a Working Draft - that sounds quite "modern" to me. It also eschews an older technology (Flash) in favour of newer ones, so that also seems to be a forgivable usage of the word.

I think it would be fair to describe this as a modern and quite reasonable alternative to the venerable (see what I did there?) ZeroClipboard.

</comment></comment><comment><USER30> Browsers support clipboard events for a long time. Except safari. Because Safari, we have to use that "flash & bloated frameworks".

Because "the people using safari" is still a thing (i dont know why)

<comment><USER31> I use Safari because Firefox and Chrome are both bloated pieces of crap. I can open 30 tabs simultaneously in Safari with minimal impact on my CPU, while Chrome locks up for close to a minute as it tries to cope. Much better smooth scrolling performance, integration with Safari on iOS, etc.

The only thing I wish Safari had that Chrome does is profiles.

<comment><USER32> I find Chrome to be a CPU hog. This may have to do with each tab running in its own process. Can this be disabled, or can processes be suspended if they are not the top tab?

I don't open a lot of tabs on Chrome because of this.

<comment><USER33> [https://groups.google.com/a/chromium.org/forum/#!msg/chromiu...](https://groups.google.com/a/chromium.org/forum/#!msg/chromium- dev/EUqoIz2iFU4/kPZ5ZK0K3gEJ)</comment></comment></comment></comment>


ODroid Magazine - <AUTHOR> https://magazine.odroid.com/issue/201810/ <comment><USER2> I find the DIY projects featured in this mag to be brilliant. Someone has made a touchscreen infotainment system using an Odroid C2!!!</comment>


Programmable 6,000-Part Drawing Boy Automaton Built 240 Years Ago - <AUTHOR> http://www.thisiscolossal.com/2013/11/the-writer-automata/ <comment><USER2> Article/media is a bit sensationalistic in describing this as an early computer. The programming language (a list of letters) doesn't support arbitrary logic, branching and doesn't have a notion of memory. In that way, it's like an intricate player piano or HTML. Still a marvel of mechanical engineering but conceptually far from what we consider a computer today.

<comment><USER3> I logged in to say this exact thing. This is not a programmable computer any more than a printing press was a programmable computer.

This is an EXTREMELY clever and complicated printer.

<comment><USER4> It's a bit oversimplistic to equate this with movable type. It's also sensationalist to describe it as an early computer, of course. But I'd be perfectly happy with characterizing it as a very early precursor to the computer.

It's common to trace the lineage of modern computers back through the Analytical Engine and the Difference engine to the Jacquard loom. And what the Jacquard loom is, is an apparatus which takes input in the form of a digital code and performs a distinct action for each possible value in the input. The Writer is also an apparatus which takes input in the form of a digital code and performs a distinct action for each possible value in the input.

The key difference is that the Jacquard loom's input is composite: It's a series of bits, each of which indicates whether a corresponding harness should be raised or lowered. That's enough of an incremental difference that I think it'd be totally fair to treat The Writer as another step back in the modern computer's lineage.

<comment><USER5> It would be fun to create a loom that could weave holes, and then feed its output into it as input.

<comment><USER6> A knitted quine?

</comment></comment></comment></comment></comment><comment><USER7> There is also a way older "computer" from 100BC called the Antikythera mechanism which was used by ancient Greek astronomers and sailors [http://en.wikipedia.org/wiki/Antikythera_mechanism](http://en.wikipedia.org/wiki/Antikythera_mechanism)

</comment><comment><USER8> A similar machine formed a central plot point in the children's movie (and book) Hugo: [http://en.wikipedia.org/wiki/Hugo_(film)](http://en.wikipedia.org/wiki/Hugo_\(film\)).

<comment><USER9> Makes me think of Doctor Who: [https://en.wikipedia.org/wiki/The_Girl_in_the_Fireplace](https://en.wikipedia.org/wiki/The_Girl_in_the_Fireplace)

I wonder if this mechanism was the inspiration for that episode, seems close enough.

</comment></comment><comment><USER10> This reminded me of this TED talk:

[http://www.ted.com/talks/john_graham_cumming_the_greatest_ma...](http://www.ted.com/talks/john_graham_cumming_the_greatest_machine_that_never_was.html)

Does anybody know about software for simulation of this kind of mechanical devices? Is it even possible, for example to simulate all parts of mechanical clock? Add 3D printing to that and ... :-)

<comment><USER11> If you are in the mood for video (though not necessarily on this topic) Prof. Simon Schaffer's documentaries are invariably very interesting:

[http://www.imdb.com/name/nm2406693/](http://www.imdb.com/name/nm2406693/)

Absolute Zero and Light Fantastic stick out in my mind as particularly good.

<comment><USER10> Thank for the link, will check it out.

</comment></comment></comment><comment><USER12> There is also DaVinci's programmable robot from 1495 - [http://www.ncbi.nlm.nih.gov/pubmed/17206888](http://www.ncbi.nlm.nih.gov/pubmed/17206888)

</comment><comment><USER13> BBC Four had a documentary on it a month ago [1], that I watched an hour ago. It's on TPB (but I don't think I can link to it here?). Make sure you have energy when you start watching this, the presenter is not the most enthusiastic.

[http://www.bbc.co.uk/programmes/b0229pbp](http://www.bbc.co.uk/programmes/b0229pbp)

</comment><comment><USER14> This was featured on HackaDay a while back. Some cool documentaries in the comments:

[http://hackaday.com/2013/10/29/retrotechtacular- automata/](http://hackaday.com/2013/10/29/retrotechtacular-automata/)

</comment><comment><AUTHOR> There are two other, though not as complex: [http://en.wikipedia.org/wiki/Jaquet- Droz_automata](http://en.wikipedia.org/wiki/Jaquet-Droz_automata)

</comment><comment> v-yadli About the second video in the page: I finally know where the name of Amazon MTurk comes from, now. :-D

</comment><comment> v-yadli This is a machine that works on fixed length strings with fixed alphabet, and accepts all strings. Thus it's not a computer. Of course, the side effect of execution is amazing.

</comment><comment><USER15> If you are impressed with intricate work, than you'd be more impressed with virtually any piece of software written today. 6000 parts? Programs routinely have that many object instances - and usually far more. Indeed, many programs have that many _classes_. When you include runtime dependencies like the OS or database, you're _average_ application is several orders of magnitude more intricate and complex.

That is not to say that The Writer is not a unique accomplishment. It truly is - to succeed in creating something that complex in physical objects, and without the helping constraints that modern programers labor under is remarkable and laudable. But if you are a fan of intricate work, you can find much better examples in modern software (not to mention CPU design).

EDIT: down-vote me all you want, assholes.

<comment><USER16> This sort of comment that makes me uncomfortable. I don't understand the insistence that everything you do has to be harder than everything else, or that this doesn't have any meaning because modern engineering is harder.

You see, you should be bragging that modern programming is _easier_. The whole point of evolving engineering is to make things easier -- what would be modern science and engineering good for if we had to build things this complicated from scratch?

People (rational people, that is) don't choose a framework or method of doing things because it's _hard_ ; in fact, quite the opposite, they obviously go for the easier ones! The merit should be -- look what I can achieve with such beautiful simplicity. It is trivial to conceive a harder way of doing almost everything.

In fact, if you look at the automata shown here, almost anyone with a tiny bit of programming knowledge and some hardware knowledge could build something similar with an arduino and some stepper motors.

<comment><USER15> _> I don't understand the insistence that everything you do has to be harder than everything else_

Where did I say that? Someone who appreciates intricacy should know that modern automata (e.g. software) is actually more intricate, by far, than antique automata.

It's just you'd never see a documentary piece like this on a piece of software, and I think it's _only_ because software is a) not physical, and b) difficult to visualize.</comment></comment></comment>


Sorting a Billion Numbers with Julia - <AUTHOR> http://mikeinnes.github.io/2016/03/21/sorting.html <comment><USER3> I'm working on a dataframe for Java for large datasets: [https://github.com/lwhite1/tablesaw](https://github.com/lwhite1/tablesaw).

It's not ready for prime time, but:

Time to sum 1,000,000,000 floats: 1.5 seconds

Time to sort 1,000,000,000 floats: 30.5 seconds

The code to fill a column and sort it:

        FloatColumn fc = new FloatColumn("test", 1_000_000_000);
        for (int i = 0; i < 1_000_000_000; i++) {
          fc.add((float) Math.random());
        }
        fc.sortAscending();

<comment><USER4> Neither as fast nor out-of-core but, for fun, parallel sort of 2^30 floats (i5-6260U, two cores running at 2.7GHz):

    	  $ go run billion.go
    	  1m32.471498831s

Where billion.go goes:

            package main

            import (
    		"fmt"
    		"math/rand"
    		"time"

    		"github.com/twotwotwo/sorts/sortutil"
    	)

    	func main() {
    		floats := make([]float64, 1<<30)
    		for i := range floats {
    		        floats[i] = rand.Float64()
    		}

    		t := time.Now()
    		sortutil.Float64s(floats)
    		fmt.Println(time.Now().Sub(t))
    	}

Out-of-core stuff's cool, and sometimes seems a shame it's not available directly (rather than by exporting data to some other program) and widely used in more programming environments.

Glad original poster's experimenting and got something about external sorting up here.

<comment><USER3> This is cool, I'm kinda fascinated by golang.

Tablesaw's not out of core either: too much of a pain (for me) to work with variable width columns like text that way.

</comment></comment></comment><comment><USER5> As part of Spark 2.0, we are introducing some new neat optimizations to make a general engine as efficient as specialized code.

I just tried on Spark master branch (i.e. the work-in-progress code for Spark 2.0). It takes about 1.5 secs to sum up 1 billion 64-bit integers using a single thread, and about 1 secs using 2 threads. This was done on my laptop (Early 2015 Macbook Pro 13, 3.1GHz Intel Core i7).

We haven't optimized integer sorting yet, so that's probably not going to be super fast, but the aggregation performance has been pretty good.

      scala> val start = System.nanoTime
      start: Long = 56832659265590
      scala> sqlContext.range(0, 1000L * 1000 * 1000, 1, 2).count()
      res8: Long = 1000000000
      scala> val end = System.nanoTime
      end: Long = 56833605100948
      scala> (end - start) / 1000 / 1000
      res9: Long = 945

Part of the time are actually spent analyzing the query plan, optimizing it, and generating bytecode for it. If we run this on 10 billion integers, the time is about 5 secs.

</comment><comment><USER6> I was initially skeptical of the 2.5s for the SUM base case but after a bit of experimenting I concluded the following. Note this is for the SUM base case only. The whole file is loaded into memory for my tests.

MMAP'd from HDD 1st run - <USER2>

MMAP'd from HDD 2nd run - 1.35s (OS caches pages in memory)

MMAP'd from NVMe 1st run - 6.5s (OS Caches dropped)

MMAp'd from NVMe 2nd run 1.35s (OS cached again)

</comment><comment><USER7> in a purely functional language like Haskell, you can sort a billion numbers in nanoseconds, all with no performance crushing side effects. Yes, it's true, that's the kind of results you get with lazy evaluation!

When you start searching the resultant sorted list, it might be somewhat slower than if you sorted using other techniques, but--silver lining--search times will only improve after that!

I'd give you the actual stats but so far I've only lazy evaluated them.

<comment><USER8> The complexity of fully iterating over a lazily-sorted range is more than 'somewhat slower'. At a guess I'd say it's O(n^2) rather than O(n log n).

<comment><USER9> Finding sorted(array)[k] is possible in linear (even worst case!) time. Using that n times would as you note take n^2 time. But we could employ a trick. Once we have been asked for log n elements, and have thus already spent n log n time, we could then sort the array.

</comment></comment></comment><comment><USER10> 1 billion numbers (let's say DWORDs) doesn't even use full 4GB worth of space. Load it up into memory and sort. With QWORDS it grows to about 8GB. A modern laptop still can load twice the amount and sort everything in memory. That is not a large dataset.</comment>


Advertiser-friendly content guidelines - <AUTHOR> https://support.google.com/youtube/answer/6162278?hl=en <comment><AUTHOR> > Content that is considered "not advertiser-friendly" includes, but is not > limited to:

> Controversial or sensitive subjects and events, including subjects related > to war, political conflicts, natural disasters and tragedies, even if > graphic imagery is not shown

</comment><comment><USER2> Hopefully we'll go back to a early era of free content creation.</comment>


Distributed SQL vs. NewSQL - <AUTHOR> https://blog.yugabyte.com/distributedsql-vs-newsql/ <comment><USER2> I am one of the Maintainers of Vitess and I just wanted to pipe in on a few points.

The comparison chart lists vitess as not having "High Performance" due to what they are saying is a single coordinator node ( vtgate ). You can actually have as many of those as you want to scale Vitess far beyond what any other system ( including yugabyte ) offers in terms of performance. We recently did a benchmark in partnership with AWS showing how far you could push AWS Aurora using Vitess ( [https://planetscale.com/news/planetscale-aws- benchmark](https://planetscale.com/news/planetscale-aws-benchmark) )

Secondly is the claim that there are no distributed transactions or cross shard joins. There are in fact distributed transactions, and cross shard joins. You can in fact do full cross shard ACID transactions. We do recommend that you are aware of the sharding mechanism as we will not trigger a cross shard transaction unless we need to.

Finally they do say there is no Native Failover/Repair which is only technically accurate. We use a third party tool called orchistrator to do the fail overs, and we recommend you run that along side the cluster so yes its not "built in", but it will launch as part of our Helm Chart fully configured to automatically do native fail overs.

<comment><USER3> Vitess is great. Although I am wondering whether the Planetscale achievement is partially attributable to the fantastic work put in the closed-source Aurora.

In particular, it is an order of magnitude better than the displayed CockroachDB benchmark[0], with 81 c5d.9xlarge nodes instead of Vitess' 64 r4.16xlarge.

Can you still beat that with MySQL or MariaDB?

[0]: [https://www.cockroachlabs.com/docs/v19.2/performance.html](https://www.cockroachlabs.com/docs/v19.2/performance.html)

<comment><USER4> I would imagine this is some form of AWS partnership to showcase Aurora here. I do not think results with MySQL would be substantially different. If anything I would expect better price performance.

<comment><USER2> That is correct, this was done in partnership with AWS to show off Aurora, however we have achieved similar results with stock MySQL. We are pretty confident that with standard MySQL using MyRocks and some high end storage devices we will be able to beat those numbers with fewer resources.

<comment><USER5> Don't mean to be crass... but why not do what you just describe and publish it to promote Vitess? Would lay the argument to rest :)

<comment><USER2> The cost to run those benchmarks with AWS was close to $50,000 just in infrastructure cost ( that was the main reason we jumped from 16 shards to 64 shards at the larger instance size, we wanted to show the top end, but we didn't have the resources to do all the sizes in between ), that doesn't even account for the engineering time to put together the solution and run the tests. We would love to have more funding to run those kinds of tests, but Vitess doesn't have a big sponsoring company to bankroll it the way some other projects do.

</comment></comment></comment></comment></comment></comment><comment><USER6> Re VoltDB:

>In VoltDB, all replicas for a given shard are updated synchronously by the client application. This is where VoltDB pays a significant performance penalty for write operations when compared with Raft/Paxos-powered distributed SQL databases. Distributed consensus protocols like Raft and Paxos require writes to be sent to all replicas but commit as soon as a majority of replicas have acknowledged the request. Waiting for all replicas to respond is not necessary since the consensus can be established with a majority. Additionally, VoltDB does not detect network partitions by default and requires a special network-fault-protection mode to be configured. When a node of the cluster gets partitioned, the network-fault-protection mode comes into play. It negatively impacts cluster performance by increasing cluster recovery time for not only accepting writes on the shards whose replica was lost in the node partition but also for repopulating the data on the partitioned node when it joins back the cluster.

Wow. Basically none of that is true.

<comment><USER7> Ya. The stuff on NuoDB is basically all untrue or misleading as well (source: I work there). I wouldn't put too much stock in this self-promotion article.

</comment></comment><comment><USER8> I tried Yugabyte a few weeks back, the TServer kept crashing while my backend was creating some very simple tables. I'm stuck with CockroachDB for the time being.

<comment><USER9> > I'm stuck with CockroachDB for the time being.

Would you care to share any negative experiences you've had with CockroachDB? What is motivating your desire to look elsewhere?

<comment><USER8> A few weeks back after my attempt to switch to Yugabyte I decided to commit to CockroachDB in a dedicated cluster with TLS, my backend runs migrations including creating the database but only admins and root are able to create databases but because TLS was enabled my root user didn't work anymore. I was not able to create more admin users because that's part of the RBAC Enterprise offering. The database was there but the authorization system triggers before IF NOT EXISTS condition, so I had to make changes to make it work in production. I honestly thought that if I stayed away from Enterprise features then I would be fine, but no: it seems like if I wanted to deploy my backend on-premise, the deployment process would have to include an step to run the CREATE DATABASE script out of band either before enabling TLS or via localhost(?) in a database node, otherwise I'm fucked unless the client buys CRDB Enterprise to smooth out the deployment process. That's a very extreme case of Open-Core money-squeezing shit I would like to stay away from.

CockroachDB Core is fine as a product, the docs are up to date and it's not hard to operate, all that is appreciated it. I just don't think I'm the target user, unless you make a shit ton of money as you scale it will be hard to operate without:

\- Distributed Backups(Enterprise)

\- Follower Reads(Enterprise) (no, follow-the-workload doesn't cut it)

\- RBAC(Enterprise)

Yugabyte offers all that in their open-source license, which is why I'm rooting for them, that in addition to Change Data capture(Enterprise offering in CockroachDB) is also included.

<comment><USER10> Hello bithavoc, I happen to be currently working on that area of CockroachDB and I would like to help.

To start with, any scenario that you find where a secure deployment without an Enterprise license makes you unable to operate your apps using the core features, would be a serious bug on our side and we'd give it high priority. There is no intent to do bait-and-switch here.

If I read between your lines I can see the following:

\- you need an admin user to create databases;

\- the only admin user is root;

\- it appears that you cannot log in with root on a secure cluster (TLS enabled). Is that understanding correct?

I think I understand the following: your SQL client is attempting to connect using TLS but without presenting the root client cert to the server. The root client cert is a cert that you can generate using the cluster's CA key (using the `cockroach cert` command). Without a valid client cert, the connection is refused.

You may think this seems to work for other users than root. This is because for other users, when cert validation fails, the server allows the client to use a password instead. This choice to use a password is not available for the root user up to and including 19.2.

Here is your way forward:

\- either you configure your SQL client that creates database to present the root client cert to the server. This will enable you to use the root account on a secure server and create databases (and other users, and then assign privileges to these users over the new database).

\- or, you wait until crdb 20.1, where it will be possible to configure a password for the root user and let root clients use password authentication instead of presenting client TLS certs.

Does this help?

If it doesn't please create an issue on github and describe your situation in more details and cc me (@knz). Thank you

<comment><USER8> Correct, I'm using TLS with password authentication, I don't want to generate certs for my backend user, so yes, root password authorization solves the issue. I can wait for 20.1, should be coming up this month right?

<comment><USER11> v20.1 is slated for release in April.

</comment></comment></comment></comment></comment></comment><comment><USER12> Yugabyte has great articles in general and seems almost to be good to be true what I have read.

It is just missing a bit of independent third party reviews but it does look interesting.

Anyone with hands on experience?

<comment><USER13> Yugabyte v. Cockroach analysis: [https://www.cockroachlabs.com/blog/unpacking- competitive-ben...](https://www.cockroachlabs.com/blog/unpacking-competitive- benchmarks/)

disclaimer: I work at cockroach

<comment><USER5> Nice. Seems a bit disingenuous on yugabyte to claim they're better performant in light of this.

Just want to say that I love your docs. Haven't used cockroachDB just yet but the docs are clean, easy to follow, devoid of marketing and focused on facts.

<comment><USER8> Second that, the CockroachDB documentation is excellent. All the DDL and DML statements are clearly documented and with examples, that's really cool.

</comment></comment></comment></comment><comment><USER14> I know this is basically an ad for yugabyte, but it's a nice summary nonetheless.

</comment><comment><USER15> Does anyone have good experiences with seamless sharding at the database layer? I'm old school and I'd be strongly inclined to look for a simple cut that allowed me to shard at the app layer, but I'd love to hear an alternative view.

<comment><USER16> Vitess is awesome if you're using MySQL.</comment></comment>


Should I put single skill or multiple skills on my Resume - <AUTHOR> I know multiple technologies like iOS, Android, Ruby on rails and somewhat python. I have side projects also and worked on official projects. I was wondering should I mention multiple skill or single skills on resume. My primary technology is iOS.
How recruiters choose resume.  I have some very bad experience in past as I have shown some of great achievements of stackoverflow and github which are very good but none of big companies choose my resume instead they choose my colleagues resumes which are just ordinary and have nothing special.
I want to know If I mention multiple skills on resume is it good point or bad point. How recruiters select resumes. <comment><USER2> If I were a hiring manager (I used to be one, but no longer am), seeing multiple skills would indicate:

\- An ability to learn new things, which is probably more important than any particular skill.

\- Knowledge that's not limited to a specific, narrow area. For example, someone who has spent their entire life doing nothing but coding mobile apps in iOS would probably have less ability to deal with issues that interact with the server side of the software stack.

So I'd recommend listing all the skills you have significant experience in.

However, your resume is a marketing document, not some kind of official biography, and as such can be tailored to the kind of job you're applying for. If you're applying for iOS jobs, you can spend more space presenting your iOS skills.

</comment><comment><USER3> I have hired dozens of devs and seen hundreds, maybe thousands, of resumes and I don't remember ever seeing one with just one skill on it.

I expect a list in order from most skilled to least, with perhaps some qualifier (proficient in:, familiar with:, etc). The skill should also show up in the areas below (projects, jobs, etc). If I look at a resume and see iOS listed first on the skills, and it is used in the current job, and it's in the first github repo -- I know I'm looking at an iOS resume primarily. The other skills just fill out the picture.

In your case, I would concentrate on improving your english proficiency (or getting editing help). There are many (small) grammatical mistakes in your question above, and if your cover letter or resume are similar, they could get rejected on just that. I'm not saying that that's a good thing (I personally have not rejected on that), but the folklore in resume screening values grammar as a proxy for "attention to details".

</comment><comment><USER4> I've seen and judged more resume's then I can count. My advise; list what you're good at, don't list what you've only little expertise in.

Some companies prefer polyglots over 1 language specialist, but others prefer someone that focussed on exactly the language (and version and frameworks) they use as well.

In general; you can't be good at 25 things. For me its a red flag if you list every technology you've briefly touched or read about as skill.

If you're interested in more than just iOS, find a place where they for instance value server-side programming skills. If they purely and only want to see iOS, odds are you'll end up in a job with very narrow responsibilities and focus. Which is okay if you just want iOS and nothing else. But perhaps not in your case.

</comment><comment><USER5> What I'd do is list as many as I can without making it look like random name dropping. Basically try to relate the skills to past jobs, projects, experiences etc. If iOS is your primary skill don't be afraid to call yourself an "iOS developer". You're much more than that, but if somebody is looking for an iOS developer they want an iOS developer.

Is the difference between you and your colleagues just the skills and experiences listed? Same school, age , nationality, ethnicity etc? It could be that you're being filtered out by something that seems completely arbitrary.</comment>


Lessons Learned Launching My Startup on My Final Day at Wired - <AUTHOR> http://contextly.com/blog/2012/11/startup-launch-lessons/

<comment><USER2> I am working for a startup right now and all these things are spot on. I am just glad I work for a company with great healthcare benefits.</comment>


Dropbox (YC S08) Now Effortlessly Syncing Files For 1 Million Members - <AUTHOR> http://www.techcrunch.com/2009/04/24/dropbox-now-effortlessly-syncing-files-for-1-million-members/ <comment><USER2> I know that Dropbox is a darling of HN, but I just can't help but point out the following.

As someone how developed and grew (somewhat) similar service to 3 million users, I am guessing there's a _lot_ of abandoned and idle accounts in this million. Anywhere between 75% and 95%.

But even if they are actively synch'ing files for just 100K users instead of a million, it is still a pretty big achievement. Also the growth of the active users tends to follow the curve of the cumulative number, so the exponent nature of it is quite impressive as well. It would help to keep the title of the PR bit factual though, especially when it's geared towards the technical community. As they are _not_ synch'ing files for 1 million users.

<comment><USER3> You're right that accounts are abandoned or go idle. It's substantially less than 75% though =)

Also, well over 100K users actively sync a file each and every day.

<comment><USER4> So, if you define 'active user' as the user who did a file sync today, 10% of your users are active users?

<comment><USER3> I think your question is phrased in a somewhat misleading way. For starters, I gave 10% as a loose lower bound, not as the actual number. More importantly, the question implies that a user is active _if and only if he/she syncs a file today_.

There are a variety of things that can be used to define "active user". Logically, the point of defining "active user" is to differentiate between users who have a good chance of using a product/service again or on a regular basis or something like that (depends on the nature of your service), and users who almost certainly won't and are just taking up space in your user registry (or in our case, space on our servers as well). I won't go into any further details, but the number of people who do a file sync at least once a week is much higher than the number of people who do a file sync every day, and the number of people who do a file sync at least once a month is much higher than that. In addition, there are people who don't add or edit new files very often, but do frequently access their existing files, who aren't being counted in the approximate numbers I gave above. Anyway, going off of this qualitative "definition" of activity, I consider much, much more than 10% of our users to be active users.

</comment></comment></comment></comment><comment><USER5> I like dropbox a lot, but have two problems.

The major one is that their price point just seems too high. $99 a year just isn't a price that I'm willing to pay without seriously considering the value I'm getting. (Then when I consider the value, $99 can buy a pretty big portable hard drive or flash drive. So I buy the drives and just use the free 2 gig Dropbox for synchronizing smaller files.)

On the other hand, I had no problem paying $30/year for Flickr Pro. That's an easy price point, even for a cheap bastard like me.

The minor one is that they don't seem to own dropbox.com. I don't know how that didn't get done before settling on the name, but they gotta get that cleared up.

<comment><USER6> Yeah, it's not the $99 that is the problem, it is that it buys you 50GB. The most valuable digital data I have is baby/kid photos, video etc. They are truly irreplaceable (to me, anyway), and I wouldn't mind some convenient way to do off-site backup rather than lugging rsynced disks around, but 50GB is not enough to back it up.

As for syncing and sharing project files, that is what the git repository is for.

But quite obviously there is a market for what they sell. I'm just not in that market. Not until I can get half a TB for my $99, anyway.

<comment><USER5> I have the same problem. I have almost 100 gigs of photos which are the majority of my backup needs.

Flickr's been an somewhat easy way to back them up and share them at the same time. But I have growing frustration with Flickr because I'm really using it for three different purposes now: backing up all my photos, sharing family snapshots with friends and family, and sharing more artsy photography with mostly strangers (which is really Flickr's sweet spot).

Using Flickr for backup is incredibly annoying and iffy because their uploaders are so fragile and downloading the entire collection depends on third-party software that Flickr could break or disable at any moment.

I'd love to move my photo backups off of Flickr and am close to using just some rotating USB hard drives, but I'd much rather have something automated. Dropbox could definitely have my business there and be price competitive if they offered say 250 gigs for under $100 a year. Backing up my photos should be a good profit for them over time, because once I get past the initial backup, I really only upload a gig or two a month and rarely download. Unlike my other documents, I don't need my photo backup replicated between my different computers running Dropbox.

Sharing photos with friends and family is something that Dropbox could be good at or at least acceptable. The Flickr, Picasa, and Facebook uploaders all suck compared to the ease of Dropbox. If they added some better privacy settings, RSS feeds, and some more viewing/downloading options, they could be player in the photo sharing space.

<comment><USER7> _Dropbox could definitely have my business there and be price competitive if they offered say 250 gigs for under $100 a year._

Amazon S3 (some consider it wholesale) would cost you 3 times that, and that's not even considering in/out bandwidth charges or Dropbox's nifty client and version history (which lets you, for instance, restore "deleted" files).

<comment><USER6> The question is not whether dropbox is overcharging (I am quite happy to concede they are not) compared to some other service. The point is purely made from the user end. When I can own outright a 1TB drive for $99, what is the psychological price point for which I am willing to pay not to maintain it? Twice as much, sure. Three times, okay it hurts but maybe. Eight times? Asbolutely not.

All I am saying is adoption will rise if the price point drops. Right now it is high enough to turn off price-sensitive individuals.

</comment></comment></comment></comment></comment><comment><USER8> Great for them. As was said in prior threads and this one, it's a fantastic product that I use on a daily basis.

I even noticed the subtle changes on the web interface some days ago. Nice stuff and looking forward to the P2P system ahead.

<comment><USER9> Apropos P2P: Ever heard of Wuala?

</comment></comment><comment><USER10> I've used allot of similar things, and I can say that Dropbox has been the one I've stuck with the longest.

I've even managed to loose data using USB drives (mistakenly wiped the wrong one, or lost it) but I haven't pulled an "unrecoverable boner" yet with DB.

Working with the same code across multiple OS platforms (rails code mostly) it's been a lifesaver always having the latest copy (even offline).

I'm considering a paid account (reaching my limit) but like the others have said, it's just beyond the "impulse buy" price-point for me these days...

</comment><comment><USER11> compare to the half dozen failed attempts by microsoft and other big co's to solve this problem

<comment> old-gregg Yeah, Linus rocks.

</comment></comment><comment><USER12> I use Dropbox at my business school for every group project. I've probably signed up at least 20 other people.

Knowing that you'll never have to email another attachment is an incredible feeling.

</comment><comment><USER13> Absolutely inspiring. It is going to keep me up tonight. Crazy easy to use.

</comment><comment><USER14> Such a great product, such a great team!

</comment><comment><USER15> i wonder if amazon will stay on the "wholesale" side of this equation or eventually move into the retail side...seems gdrive or whatever it is google is planning might upset the amazon/N-front-ends racket</comment>