Ask HN: How do you version control your neural nets? - <AUTHOR> When I started working with neural nets I instinctively started using git.
Soon I realised that git isn't working for me. Working with neural nets seems way more empirical than working with a 'regular' project where you have a very specific feature (e.g. login feature): you create a branch where you implement this feature. Once the feature is implemented you merge with your develop branch and you can move to another feature.
The same approach doesn't work with neural nets for me. There's 'only' one feature you want to implement - you want your neural net to generalise better/generate better images/etc (depends on the type of problem you are solving). This is very abstract though. One often doesn't even know what's the solution until you empirically try to tweak several hyper parameters and see the loss function and accuracy. This makes the branch model impossible to use I think.
Consider this: you create a branch where you want to use convolutional layers for example. Then you find out that your neural net is performing worse. What should you do know? You can't merge this branch to your develop branch since it's a basically 'dead end' branch. On the other hand when you delete this branch you lose information that you've already tried this model of your net. This also produce huge amount of branches since you have enormous number of combinations for your model (e.g. convolutional layers may yield better accuracy when used with different loss function).
I've ended up with a single branch and a text file where I manually log all models I have tried so far and their performance. This creates nontrivial overhead though. <comment><USER2> If your neural net config is in a relatively standalone file, or you can mark it with a special comment block, you could have your test runner actually read the source file, regex it out, and concat the source block, date, current git SHA, and performance metrics into a "neural_runs.txt" file. If something else about your data pipeline is changing as well, e.g. filter settings on your image preprocessing, you can throw that in there too.

If you check this in, then every commit will include the diff of everything you tried to get there alongside the final source file, and additionally that file will serve as a single historical record for everything you tried for all time. Asking yourself a month later "did I ever try cross entropy" is as easy as grepping the file.

Heck, you could insert into a database as well if you really wanted to, and visualize your performance changes over time a la [http://isfiberreadyyet.com/](http://isfiberreadyyet.com/) . Sky's the limit.

</comment><comment><USER3> I am very interested to see what people's answers are for this, because I pine for a version control system designed for the twists and turns of experimental investigation rather than the needs of engineering implementation. I very much suspect that some sort of structured approach to one's commit messages might be key, and a careful mapping of DAG concepts to experimental ones--branching as the modification of an independent variable, with a base commit selected as the control point of comparison? Would one want to be able to rebase in order to compare against a different point? What would the semantics of merges represent?

</comment><comment><USER4> I've been trying to do this better recently after having some non-reproducible results. I've settled on taking all hyperparameters (including booleans like whether to use batch norm) from a global dict. Instead of commenting and uncommenting lines, I look up a key with a default value, adding the default to the dict if it wasn't there. Then I print and log the dict with the results.

I end up with a bunch of code like:

        if get_param('use_convnet_for_thing1', True):
            convnet1_params = get_param('convnet1_params', None)
            thing1 = build_convnet(thing1_input, convnet1_params)
        elif ...

By logging the hyperparameter dict, source checkpoint, and rand seed, results should be reproducible.

This works well for rapid iteration like in jupyter notebooks. For models that take days to train, you might as well use source control for your scripts.

</comment><comment><USER5> Great questions and discussions. I'm definitely passionate about versioning in the context of models and data science for both data and code. I work full time on the open source Pachyderm project (pachyderm.io), and we have users versioning their data and models in our system. Basically, you can output checkpoints, weights, etc. from your modeling and have that data versioned automatically in Pachyderm. Then if you utilize that persisted model in a data pipeline for inference, you can have total provenance over which versions of which models created which results (and which training data was used to create that version of the model, etc.).

</comment><comment><USER6> Shameless plug: [https://hyperdash.io](https://hyperdash.io)

I got tired of maintaining one-off scripts to do recording, so I started working with friends on a dedicated solution. Today it lets you stream logs via a small Python library, then view individual training runs on an iOS/Android app. Takes less than a minute to get setup.

We're planning on expanding to model versioning in the next few weeks. Interesting to see how others are thinking about it. If you have model versioning thoughts you dont feel like posting here, drop me a note at email@example.com

</comment><comment><USER7> Maybe write a shell macro to pull accuracy and error into the commit message along with your comment on the changes. You could also add some automation to automatically branch if your test results are worse than before, so if you hit a dead end on that branch and realize the experiment didn't go well down the line, you can head back to where you branched, or if the end result works, you can merge back into your starting branch.

</comment><comment><USER8> Is there any value to the code in failed attempts or do you just want a log of things you have tried?

If the former, you could try a single experiment branch and use tags to denote different experiments. Add a tag when you finish an experiment then overwrite with your changes for next experiment and repeat. This would keep all the changes while not have having a huge number of dead branches and the branch could be merged when necessary.

If the latter, why not an experiment log that is checked in which has a similar form to a change log? Or maybe create an issue and branch for each experiment then update the issue with results and delete the branch?

</comment><comment><USER9> Why would you manually log your models? In my NN experiments, I automatically write the list of all hyperparameter values and the corresponding performance to a file. In addition, I automatically generate and save graphs showing the results, typically one graph per a nested 'for' loop.

</comment><comment><USER10> Why is it so bad if you have many branches?

<comment><AUTHOR> It has several disadvantages: (1) it creates a nontrivial overhead for your workflow. You're basically creating git branches every few lines of code (I guess part of it could be somehow automated). (2) it kind of feels like an overkill to create a whole new branch just to for example change a single variable. (3) a lot of those branches would remain "unfinished". By that I mean they would simply exist just to inform you that you have tried this model of neural net. You couldn't merge them to anything since those changes would make your net perform worse. (4) if you would want to see the code of some specific model you had tried before you would need to always switch to a different branch. This creates a bulky workflow.

<comment><USER11> There's data, and then there's metadata. Consider how you want to shift information between your files of data (variables), and the tree-like structure in a version control timeline.

You shouldn't need to record _how_ the data changes directly alongside the data. That would be like commenting out old code for every change instead of making new commits. It just defeats the purpose of using version control.

Branching is metadata about _how_ you have based changes off a given starting point, and committing records the actual, linear changes. Nobody ever said a branch _must_ be merged into another branch--that's just typical of how bugs get fixed--it's not important to the tree data structure.

I would think that switching branches to see the code of some specific model would simplify my workflow; I would only have to manage a single set of data, instead of accommodating multiple model versions in my working directory. If you are going to be recording version/parent information every few lines of code anyway, then you might as well do it in a tree/timeline data structure with lots of tooling available.

You're right that branching could be a lot of overhead done manually as you are, but automating git (for example) to create or switch branch should not be difficult nor particularly slow. There are also ways you can avoid being overwhelmed by noise such as searching branches from only specific commits, limit the depth of results, etc.

</comment></comment></comment><comment><USER12> Not just for neural nets - balancing experimentation against building reusable tools is probably the biggest logistical challenge in scientific programming in general.

I've converged to a workflow where I maintain a library with a main project pipeline and reusable tools for the project, and do all scripting with jupyter (all notebooks version controlled).

I've found that machine learning projects can be pretty effectively parametrized with config dicts for data, training and the model. Each type of config gets it's own pipelined method that does all of the library calls - pipeline_batch_gen, pipeline_train, pipeline_build_model.

Example of a poorly organized config from a project:

model_config = { 'optimizer': optimizer, 'clip_grad': clip_grads, 'name': model_name, 'residual': residual, 'n_conv_filters': n_conv_filters, 'n_output_hus': n_output_hus, 'activation': activation, 'batch_norm': batch_norm, 'output_bn': output_bn, 'generation': generation, 'data_spec': { 'uniform_frac': uniform_frac, 'include_augment': True, 'batch_size': batch_size, 'bulk_chunk_size' : bulk_chunk_size, 'max_bulk_chunk_size': max_bulk_chunk_size, 'loss_weighter': loss_weight }, 'train_spec': { 'early_stopping_patience': early_stopping_patience, 'lr_plateau_patience': lr_plateau_patience, 'learning_rate': init_lr, 'clip_grads': clip_grads, 'partial_weight': partial_weight } }

I've wanted to give Sacred a try [https://github.com/IDSIA/sacred](https://github.com/IDSIA/sacred) \- looks promising but haven't tried yet so can't comment.

I still tend to keep track of model performance by hand though. But I have always have the notebooks I can go back to for reference. This is something sacred could help a lot with.

Another very non-trivial aspect of this kind of work is the compute/storage infrastructure you need to scale beyond a single workstation.

We have a nice system here where $HOME lives on NFS and gets mounted when you log into any machine on the network - I can hardcode paths in my code and count on every worker having the same filesystem. I can't imagine how we would do distributed jobs without NFS. That's not a very realistic solution for homegamers though - you need a very fast network and expensive commodity hardware. And sys admins.

Does anyone have a solution for that half of the problem? I've seen a number of merkle-tree based data version control solutions recently...</comment>


US Military Could Lose Space Force Trademark to Netflix Series - <AUTHOR> https://www.cbr.com/us-military-lose-space-force-trademark-netflix/ <comment><USER2> You can use the same trademark in two different industries without any violation. Trademarks are industry specific.

There's no battle here and really no story given this very basic foundation of trademark law. Space Force (military) have no grounds for a violation against Space Force (streaming) or vice versa.

<comment><USER3> [https://en.wikipedia.org/wiki/Apple_Corps_v_Apple_Computer](https://en.wikipedia.org/wiki/Apple_Corps_v_Apple_Computer)

Doesn't seem to prevent a lot of lawyers from achieving many billable hours, however.

<comment><USER4> Ah that reminds me... [https://en.wikipedia.org/wiki/Sosumi](https://en.wikipedia.org/wiki/Sosumi)

</comment></comment></comment><comment><USER5> " The show's name has no relation to the newest organization of the U.S. military..." Is this a joke? It's more bothersome than the subpar legal analysis.

<comment><USER6> Seriously, the show was named after the branch of the military. The creators have been totally open about that. The US announced the new Space Force branch, and the creators thought "that would be a funny show". Steve Carell explaining it: [https://youtu.be/TQggDFcXy88?t=322](https://youtu.be/TQggDFcXy88?t=322)

</comment></comment><comment><USER7> I thought anything the US made was in the public domain? Are trademarks exempt from that?

<comment><USER8> There are federal laws protecting the use of federal seals, not specifically copyright or trademark laws

</comment></comment><comment><USER9> This kind of reads like an advertisement.

</comment><comment><USER10> Submarine advertising?

</comment><comment><USER11> Follow the Force</comment>


Restaurant chain bans mobile phones - <AUTHOR> https://www.bbc.com/news/business-46369736 <comment><USER2> A somewhat misleading title:

'No one will be forced to take part, the firm stressed, but staff will be "actively encouraged" to get diners to participate.'</comment>


Fiz: A Component Framework for Web Applications - <AUTHOR> http://www.stanford.edu/~ouster/fiz.pdf

<comment><USER2> It is a great application. Do you know if it provides an API?</comment>


Ask HN: Where are optimized solutions to leetcode problems being used? - <AUTHOR> Of-course sorting problems are being used everywhere but I am trying to understand other applications which require that level of algorithmic knowledge as part of the day to day engineering work. <comment><USER2> \- 3D rendering in game engine

\- geo-spatial related problem, path finding etc.

\- geo positioning

\- low level networking

\- shaders code

\- physics and fluid simulation

\- GPU stuff

I mean everywhere there is a problem to solve, there is depth to explore.</comment>


Hands-on with Remodel: A New Python ODM for RethinkDB - <AUTHOR> http://rethinkdb.com/blog/remodel/

<comment><USER2> I don't know much about databases and which way to best do things but in the past 6 months I have been building lots of things in flask and do it's a problem I'm increasingly becoming acquainted with.

I wonder: -is it useful to not have a defined data structure? I would k it makes it slightly more verbose when you are trying to update elements to have to refer to the key value pair rather than just the foot notation style -does this mean that you don't need to redefine a data structure/model if you add additional elements? -apart from what looks looks an elegant syntactical structure for data queries, are there any specific advantages to this approach?

Thanks for your help in increasing my understanding

</comment><comment><USER3> RethinkDB keeps looking better and better. That aside, I have recently been convinced of the huge advantages of a strict schema on your data. It's just very very useful to _know_ that your data will be a specific type when loaded.

In cases where we didn't enforce that (e.g. allowing nulls when we shouldn't have, or using a JSON blob), it came back to bite us in the ass.

</comment><comment><USER4> Nice, I have been holding off on using rethinkdb for my web apps until there was a nice odm, i'll certainly give it a look although I must admit that postgres has been pretty great for us.</comment>


Speed Bumps on the Road to Virtual Cash - <AUTHOR> http://www.nytimes.com/2011/07/04/business/media/04link.html

<comment><USER2> If you're interested in Bitcoin, the absolute best thing you can do right now is provide viable, interesting, valuable economic exchange to the system.

Work for Bitcoin, offer to pay for work in bitcoin, sell your farmshare for bitcoin, etc. Bootstrapping an economy is hard, but can be rewarding as the article mentions.

</comment><comment><USER3> The thing that will keep bitcoin alive is black market transactions. As long as there is a single officially supported currency that can be traded into bitcoin there will be a huge market for it.

I don't see every nation in the world banning bitcoins so that pretty much means it's guaranteed to remain viable (and more likely grow.)

<comment><USER4> But can black-market transactions support the current $100M market cap? The world black market is probably tens of billions of dollars, but what fraction will adopt Bitcoin?

<comment><USER3> Check this site out: <http://bitcoinwatch.com/>

The velocity of bitcoin is 730k/24h or $11M/24h, which is pretty insanely high and suggests that if anything the current value of bitcoin is low (due to perceived risk.)

Lest you think this is all trading volume, the total trading volume on all exchanges for the last 24h is under 20k bitcoin. So that remaining 710k in transactions is coming from somewhere, and I'd bet good money that it's not ISPs or any of the other merchants listed who accept bitcoin. If I had to venture a guess I'd say drugs and money laundering.

Think about it, this is a money launderer's dream come true. A completely untraceable currency with 0 fees. You can pump in a few million in USD, trade it into bitcoin, pull it out of some random exchange, and dump it in any random part of the world you feel like. Then hide it in some Swiss bank account. There's not a thing that anyone can do to trace it.</comment></comment></comment>


Hey programmers, we need to talk - <AUTHOR> http://sealedabstract.com/rants/hey-programmers-we-need-to-talk/ <comment><USER2> Two things:

1) This is based on a false premise that the time I'm spending writing comments is taking away from time I'd otherwise spend programming. I don't know about anyone else but I'm one of those programmers who likes to internalize a problem, absorb it and digest it subconsciously while doing other things and then write code. Guess where internet comments fit into this? Additionally, reading and writing on places like HN has contributed immeasurably to my skills as a programmer over the years. Not so much by teaching me specific things but by keeping me in touch with the community and aware of things I wouldn't otherwise be aware of.

2) There are already real points which we are awarded for programming, it's called dollar bills. Ok, you were talking about free open source projects that we wouldn't get paid for. True, no dollars directly there. But these are things you can cite on your CV/portfolio when job hunting or when review time comes around. Dollars.

Also, Github.

<comment><USER3> Based on your #1, I can say the same thing. Grep for my username in both these places:

[https://gist.github.com/paulmillr/2657075/](https://gist.github.com/paulmillr/2657075/)

[https://news.ycombinator.com/leaders](https://news.ycombinator.com/leaders)

Now, maybe my GitHub rank would be even higher without my HN rank...

<comment><USER4> Sorry man, that is only for people with followers over 187. It stands to reason that the people that are truly at the top are the ones that don't take time promoting themselves on Hacker News.

Not that I think you are wrong, but those things might be related :)

<comment><USER3> Of course, that's always possible. It's also only GitHub, there are many, many, many people who develop a _lot_ of open source software on other places as well. Furthermore, the long tail dictates that the mass of people with a few contributions contribute waaaaay more than the people who are at the top.

That said, there's nobody on that list who has an insignificant number of contributions, either, which is more important. #256 is ~3 commits/day.

</comment></comment></comment></comment><comment><USER5> I don't understand how you've concluded that the reward for writing code is lower than the reward for HN comments. I get paid lots and lots of money to write code. So far, I haven't seen much return on the time I've invested into HN comments, except in terms of the internet points you mentioned. The going conversion rate for those to dollars is around infinity:zero.

<comment><USER6> Exactly. And it's not just about being paid to write code for your job. My open source contributions have had orders of magnitude bigger effects than writing on Usenet or commenting on forums.

\- I made some of my best friends through working on projects in the same open source ecosystem.

\- I attribute some of my best jobs to having built some reputation from OSS. Would I have gotten through Google interviews if one of the interviewers hadn't been a user and a fan of one of my projects? I'm pretty sure it, and having a long term public track record must have helped some.

\- What little internet fame I have (and it is very little) is all purely attributable to coding something, releasing it, and then writing about it.

\- I still get a kick out of looking at some old code I've written every now and then (like old Perl Golf entries). Every now and then I'll get to tell a good debugging war story to friends, or swapping stories with them about cool things we've coded lately. Boasting about the cool forum posts... Yeah, not so much.

And Hacker News? I don't think I've have had one business lead (which didn't pan out) as a result of a comment, but haven't interacted with a poster outside of HN otherwise. There's no reason to believe my HN karma will ever matter for anything. I guess it's nice to cross the downvote threshold.

</comment></comment><comment><USER7> Ok, beautifully written article, I enjoyed reading it. But you really are downplaying and underestimating the kind of reward and benefit that one gets from writing articles and code. For example, I have written a number of articles and library tools on and in C++. Over time, these things received lots of links from places like stackoverflow. As a consequence, my stuff is #1 on the search engines for terms like "C++ rvalue references". That gets me an enormous amount of attention and exposure. And if someone, like a prospective employer, hasn't found me yet that way, I just tell them, "Hey, search for 'C++ rvalue references', you'll see." Now if someone were to say, "Ok, that's all fine and well, but you don't have a lot of Karma points on HN," then screw them, I don't want to work for them anyway.

<comment><USER8> That's been my experience, too. I get much more attention from colleagues and recruiter for my articles, papers and other publications than my HN or Reddit karma. If you want to get your name out there as an expert, you're much better off writing the occasional long form article, chunk of code, paper or conference talk than racking up HN karma in my opinion.

</comment></comment><comment><USER9> Starting from false premises makes the rest of your article likely worthless (I can only say _likely_ worthless because I stupidly kept skimming but fortunately stopped carefully reading).

 _This programming thing is a lot of fun, right? We have been entrusted with the unique responsibility of making pretty much the entire world go round. (There aren't a lot of industries that don't need software these days.) And not only that, but we have a lot of fun doing it._

Fun? It can be, sometimes, or at least was for some of us when we started. Making pretty much the entire world go round? No, most of us aren't anywhere close to that. We make happy little sites and apps for rich people to buy jewelry and make fun of their friends and mark posts 'liked' on social networks. I know people who work on the software that runs airplanes, and that's soul-crushing C tweaks and opimization at best. We have a lot of fun doing this? No, I think most people in any reasonable place to comment do this for _work_ , and "fun" is almost always a secondary consideration if it is one at all, at least for the non-independently-wealthy and personal-future-minded.

Yeah, I guess we do need to talk. Because you don't seem to understand what the vast majority of us _don 't_ do or what our priorities _aren 't_, let alone that we are pretty much _normal people_ that don't need hyperbole about which comments on which sites matter.

<comment><USER10> I am so sorry if programming stopped to be fun for you but please don't extrapolate that to all of us. I would hope that for most there is fun and less work.

In immortal words of Confucius "Choose a job you love, and you will never have to work a day in your life"

<comment><USER9> Very cute. I didn't extrapolate it to "all of us;" if it fits you, you're not "us" in the sense it was intended.

Choose a job you love, and you will never work a day in your life? Very petty sentiment from an industry that's swamped with demand.

<comment><USER10> Point is I am doing this for last 30 years , I choose this when computers were not 'cool' and 'hip' , internet was arpanet and BBS was well BBS. It wasn't demand or promise for richness what drove my choice of profession. Good question is why you choose/do this if it is not fun to you?

</comment></comment></comment></comment><comment><USER11> From [http://philip.greenspun.com/seia/writeup](http://philip.greenspun.com/seia/writeup)

 _Between March 2001 and April 2004 roughly 400,000 American jobs in information technology were eliminated. Many of those who had coded Java in obscurity ended up as cab drivers or greeters at Walmart. A personal professional reputation, by contrast, is a bit harder to build than the big salary but also harder to lose. If you don 't invest some time in writing (prose, not code), however, you'll never have any reputation outside your immediate circle of colleagues, who themselves may end up working at McDonald's and be unable to help you get an engineering job during a recession._

</comment><comment><USER12> Before there was Hacker News there was Slashdot. Before Slashdot there were NNTP servers (Usenet). Before Usenet there were BBSes. We've been chatting online for the past 30 years and yet have still been able to create operating systems, window managers, office productivity suites, databases, compilers and all sorts of other things.

Not saying our system is perfect, but it's working and besides developers need to let off some steam and just talk about geeky stuff.

</comment><comment><USER13> But then, let's look at the other side of the equation.

By having places where programmers hang out with each other and just chit chat a community arises. Experience, expertise, knowledge, and judgment is shared.

Developers learn many important things this way.

They learn about new technologies, new languages, new tools, they get a sense for new things to study. And new things to build too.

They learn about security vulnerabilities in frameworks and operating systems. Not everyone is signed up for all of the relevant security mailings that impact them, and sometimes there aren't security mailing lists for some products.

They learn about good practices and bad practices. Everything from how to work, how to bill and interact with clients, how to be a good boss, signs of bad bosses, what opportunities are out there in the world, all that stuff.

They learn about interesting and deep problems in programming and computer science and perhaps think about ways to tackle them.

It's easy to discount these things but they are important. As important as fundamental computer science stuff like algorithms, if not more so. Sure, it also comes with a lot of baggage and bloviation and uselessness, but sometimes it can be more difficult to tell the difference between useful and useless than one might think.

<comment><USER14> Certified newbie-programmer here: without HN, r/programming on reddit, and a few other places, there's no way I'd know about half the things I know now. When it comes to choose tools for a project, that's really helpful.

That all said, I think it's reasonable to say "Hey, we should think about motivating people to contribute code too!", but both elements can be complimentary.

</comment></comment><comment><USER15> I'm perplexed by this. On HN, karma's purpose is to promote civil discussion. It attempts to do this by providing both positive and negative feedback. If you confuse those points with some universally valuable thing... well, don't.

I will note that I have had many pleasant interactions with people on HN outside of HN itself. Some of these interactions included offers to interview for a position. But this did not happen because of my karma points itself. They happened because of how I conduct myself here.

</comment><comment><USER16> This is only true for people who care about "Internet points".

As others have pointed out, there are many other metrics. Number of lives you positively affect, for those of us who work in large-scale projects. Dollar bills for those who choose to get paid. Recognition by peers for those who choose to publish papers/reviewed articles.

And to be completely trite, "you get back what you put in". If all HN is to you is a place to collect Internet points, yeah, that's pretty shallow. If you think it's a place where you can have interesting discussions with people with similar interests, it's a completely different kettle of fish. I'm not here for my karma points, I'm here because I occasionally enjoy the discussions. (And I comment because I wait for my compiler ;)

</comment><comment><USER17> I find that many of the best programmers, and many of the programmers on my Twitter feed, do not post very much if ever here on HN, or on Reddit, or anywhere else really. They're too busy doing what they love, which is programming and not talking about programming.

The same goes for musicians. Name the last Daft Punk comment you saw on Reddit? Oh it's not there? Probably because they're too busy spending 6 weeks on the perfect kick drum sound.

<comment><USER18> Sooo true. And although programming is fun, it's not necessarily how a person might chose to spend their leisure time.

As tomphoolery alluded to, maybe the problem is that our society as a whole gives too much credit to how we live our leisure life vs our work life? This isn't a problem related to programming, it's a general critique.

You could easily write the same article from the point of view of a 4.0 GPA college student:

Guys, we need to talk. This whole college thing is awesome. Most of you will only get 4 years of this. Why do we give so much credit to the guy who just won the case race but not the engineer who's developing the next rocket? WHY?!?! We should be studying!

I'm going to keep commenting and coding more than the average person.

Great article though.

</comment></comment><comment><USER19> If the intention is to reward in terms of karma, judging one liners is a lot easier (and quicker) than judging the quality of a software patch. The feedback is instantaneous, thereby explaining the skewed ratio of one-liners to useful patches. Regardless, those who currently exhibit a superhuman level of programming expertise will continue to do so while us mere mortals seek out cat pictures on Reddit.

</comment><comment><USER20> I really dislike the phrase "we need to talk"

 _shudder_

</comment><comment><USER21> The good news is I suspect the people able and motivated to make real contributions to our field aren't too worried about Internet points.

At least, I hope that's the case, because the Internet seems to reward easy to understand and seemingly witty over real contribution. It happens even when we're talking about actual code—I can't count the number of "X, but in JavaScript! (Because I ran X through Emscripten, which anyone could've done, but hey! I published the output first!)" projects I've seen on the front page here.

At least the person who actually wrote Emscripten got a few Internet points for the effort too.

It's worth remembering though that to some people some things matter more than money. Internet points are a cheapened version of peer recognition, and as such they might matter to some more than they logically should.

</comment><comment><USER22> I think this is more perception than reality. People _do_ get Internet points for code, it's just that they get very little feedback that they're getting Internet points for the code. But everybody knows (or can easily find out) if you, say, invented Django or Redis or memcached or sold a company or make lots of money selling Bingo Cards or redesigned the Google search page or various other technical accomplishments. And that does affect how your comments are perceived. You don't _see_ that effect because it's tacky to argue from authority, but people know who the authorities are.

</comment><comment><USER23> > There's no real reason for this comment to exist.

Sure, there is. Someone wanted to state his opinion. And as the internet most likely won't get full anytime soon I don't see why that would be bad.

</comment><comment><USER24> Why don't we create an HN like website for writing code. Essentially a member of an open source community would come and submit a need for a function. It tell us what goes in the function, what it should return, and the language necessary. In the "comments" solutions are put into place critiqued and updated, and everything is up voted.

This lowers the bar of entry, increases the points for doing important stuff, and helps people learn. Email me if you want talk more. jonathan.barnes11 @ gmail dot com

</comment><comment><USER25> I'm glad I read this article instead of actually working. I'm even more glad that this article made me want to write a response to it on HN instead of working. Wait a second...

</comment><comment><USER26> I think the article raises some interesting questions: Can we create a better incentive system for open source software contribution? Why do the rewards have to be imaginary?

</comment><comment><USER27> I feel the writer wants me to feel bad about not working 100% of the time. What's so bad about procrastination, as long as we don't overindulge?

</comment><comment><USER28> I want to comment, but then I'd feel bad...

</comment><comment><USER29> linking to content is absolutely a high-effort, reward-worthy endeavour. perhaps not as much so as writing the linked-to content, but the fact is there is a torrent of bad articles and webpages out there, and anyone who wants to take the time and trouble to sift through it and selectively amplify the good stuff is providing a valuable service.

</comment><comment><USER30> hot air blog posts get too much attention here

</comment><comment><USER31> You know that the "top" comment you see in a post is not the highest-rated one, right?

</comment><comment><USER32> Too many people mistake one-liners for wisdom and/or wit.

... see what I did there?

</comment><comment><USER33> Where are you getting your data from for these graphs?

</comment><comment><USER34> lol. OP is bitching about internet karma. 1st rule of internet karma is it's irrelevant.

</comment><comment><USER35> We still need karma for linking because Google failed to provide us with the long term value it demonstrated to society when it was established. Had Google been successful, (at something other than Ad Words) then these other forums wouldn't be so popular.

</comment><comment><USER36> I agree with the author entirely.

</comment><comment><USER37> Wasn't this blog linked to on here recently? Anyone know what it was about, is he buthurt about comments in that thread?</comment>


Bring on the female superheroes - <AUTHOR> https://www.ted.com/speakers/christopher_bell <comment><USER2> Honestly enough superheroes in movies, as a wife I'm way too tired of seeing 2-3 every year and also just stop the endless spidermen movies.

</comment><comment><USER3> Don't. Care.

Give me a good story. I could care less if the hero is male or female, black or white. Just make a good movie dammit.

<comment><AUTHOR> I agree with both of you (@MollyR, @joeblow9999). I thought the most compelling theme from the whole presentation was the concept of "Public Pedagogy" and the almost Kafkaesque way that the big media companies erase women superheroes from movie merchandising. Lets accept the premise that the motivation is to protect their investment in the manufactured identities created in genres targeted towards girls. All these characters are created so that we can identify with with them... its that affinity that makes us want to buy merchandise. Making a good movie dammit is probably not the priority (but it helps).</comment></comment>


Supersonic free fall from 120,000 feet live at 8:30AM EST - <AUTHOR> http://www.redbullstratos.com/live/ <comment><USER3> Maybe a little off topic:

Red Bull's marketing is impeccable. They started out with all the actionsport athletes and moved upstream to more mainstream, and I guess, more expensive sports (two Formula 1 teams, more soccer teams than I can count), while still doing these crazy things. From Parcours in Santorini, to Air Races all over the world. Seems like every crazy sporting event is sponsored by Red Bull.

I think these have allowed them to be still seen as "hip" and not just selling sugar water like Coca Cola.

<comment><USER4> and like all good things, it started by accident. (i got this story from one of their marketing managers) once upon a time someone at the formula 1 won a race. and the masterchief (or however the bosses of the racing teams are called) drank a red bull instead of champagne during the victory interview.

next day red bull was sold out in austria (which was their main market at that time)

they just took it from there.

<comment><USER5> "It was an accident," quoth their marketing manager. Yeah, riiiiiiiiight.

<comment><USER6> It's more likely that a marketing manager would take personal credit for an accident (i.e. boosting their resume) than to attribute a brilliant marketing to an accident.

<comment><USER5> You are neglecting one issue: advertising and lawsuits. "Oh look, we have smuggled our product into the tightest-controlled advertising event, and we did it on purpose; sure, we'll happily pay you the $$$ for this priceless publicity boost." vs "Oh, it was an accident, there's no way we'll pay you anything."

I'm completely certain that the F1 organizers would sue the heck out of Red Bull for their unauthorized ad, if they only could prove RB did it on purpose (and RB isn't so foolish to provide them with ammunition).

<comment><USER6> What exactly do you think F1 would sue Red Bull for?

Without RB having a contract of some type with F1, it is unlikely there is any way for them to sue RB. It may be possible for F1 to sue the driver, but at this point that would seem ridiculous since RB is now a huge F1 sponsor.

Accidents also don't eliminate lawsuits. Even if there was a way for F1 to sue RB, they should still be able to sue them for accidents. For example, if RB had a contract with F1 which stipulated when and how RB could be shown, and RB showed their product at a different time by accident F1 could still sue them.

</comment></comment></comment></comment></comment></comment><comment><USER7> Nitpick re "edge of space": Low earth orbit starts at around 160 km and orbits decay rapidly below 200 km. This is 36 km altitude. But what I think most people miss is that to "be in space" the way most people think about space you'll need to enter orbit which means that once you get up to 160+ km altitude you need to go sideways at roughly 24 times the speed of sound - <USER2> then you're in orbit. Then when you're all done having your fun you need to take all that orbital energy and do something with it. The best idea so far is to use it to burn up a heat shield. This business of going vertical a little higher than jets and a lot lower than satellites is a little pretentious. But it's fun and it's Red Bull so who cares!!

<comment><USER8> While there isn't anything special marking the edge of space, the generally accepted definition is 100km <https://en.wikipedia.org/wiki/Edge_of_space>

<comment><USER9> <http://en.wikipedia.org/wiki/Air_sovereignty>

Makes me interested to see what the treaties say as I'd be surprised it no one thought to limit the control of countries to allow satellites to pass without permission or allow space vehicles in general.

Though as you point out there is the Karman line convention.

</comment></comment></comment><comment><USER10> There are precedents, see <https://en.wikipedia.org/wiki/Joseph_Kittinger> for other crazy skydives

<comment><USER11> He's working on the project as a consultant

"The record is currently held by Col. Joe Kittinger, who in 1960 jumped from 102,800 feet as part of a U.S. Air Force mission. On this attempt, 52 years later, Kittinger is a consultant and mentor.

He has also been giving Baumgartner advice on what to expect. For example, he described what it feels like to fall through space when there is so little air: "There's no way you can tell how fast you're going, because there's no visual cues."

But Kittinger rejects any suggestion that he is jealous that Baumgartner is poised to beat his record.

"Oh no. I'm delighted," he told CNN recently. "He's advancing science, and he'll do a great job."

[http://www.cnn.com/2012/10/09/us/skydiver-record- attempt/ind...](http://www.cnn.com/2012/10/09/us/skydiver-record- attempt/index.html?hpt=hp_t1)

<comment><USER12> You have to be cut from a particular bitter length of cloth to be jealous over _only_ keeping your world record for 52 years.

<comment><USER13> No kidding. Breaking Kittinger's record is a tribute to Kittinger.

</comment></comment></comment></comment><comment><USER14> The tech to shoot and transmit this whole event live is almost as incredible as the event itself

[http://www.redbullstratos.com/technology/cameras- communicati...](http://www.redbullstratos.com/technology/cameras- communications/)

</comment><comment><USER15> Just aborted due to high winds. They are going to retry tomorrow.

<comment><USER16> tomorrow also looks unlikely...no confirmation yet about tomorrow's jump as of now.

</comment></comment><comment><USER17> Is the web site overwhelmed? I'm getting "Loading Javascript" messages, but nothing is happening.

        We're just trying to load the data for you. If you
        don't have JavaScript enabled in your browser for
        this site, please switch it on and refresh the page
        in order to view the Live Jump experience.

It would be nice to know if it's working for other people.

 _Added in edit: OK - on weather hold - due off an hour from now._

<comment><USER18> The YouTube live view seems like it's struggling to cope with the demand, keeps stalling or failing to load for me.

<comment><USER19> <http://youtube.com/embed/vkJ5ItzE>

<comment><USER20> I believe you're missing a few chars in the hash. Should be <http://youtube.com/embed/vkJ5ItzEq3M>, or for the full youtube page: <http://youtube.com/watch?v=vkJ5ItzEq3M>

</comment></comment></comment></comment><comment><USER21> Direct link to the live YouTube stream

[http://www.youtube.com/watch?&v=vkJ5ItzEq3M#](http://www.youtube.com/watch?&v=vkJ5ItzEq3M#)!

</comment><comment><USER22> This year has been really great for awesome space events. Mars, 2 SpaceX missions, now this. I hope this is just a taste of things to come.

<comment><AUTHOR> We went to the moon, what, 40 years ago? It's about time we started living in the space age.

</comment></comment><comment><USER23> Can someone explain the physics behind this?

<comment><USER18> Lots of the science was broken down over at Wired: [http://www.wired.com/wiredscience/2012/02/stratos-space- jump...](http://www.wired.com/wiredscience/2012/02/stratos-space-jump-can-you- fall-faster-than-the-speed-of-sound/)

<comment><USER24> Another silly question might be: won't he "burn up on reentry"? Clearly, he's falling from a different altitude than a rocket, and starting from 0, but what are the actual numbers like?

<comment><USER25> > Clearly, he's falling from a different altitude than a rocket, and starting > from 0, but what are the actual numbers like?

Without an atmosphere, the result would be deterministic and trivial to compute:

acceleration = f(t) = g

velocity = ∫ f(t) dt = g * t

position = ∫∫ f(t) dt = 1/2 g t^2

(The value of g changes slightly for this problem, in this altitude domain.)

But because of the atmosphere, the calculation of velocity as a function of altitude is much more difficult. And it turns out that the atmospheric pressure as a function of altitude is not trivially characterized. And at high altitudes, it's not even constant -- it depends on temperature, the position of the sun in the sky, even the time in the 11-year sunspot cycle.

The air resistance of a falling object is some constant k (based on the object's size and surface roughness) times the square of the velocity times the air pressure. But the air pressure is changing as the descent unfolds, so such a computation ultimately relies on a numerical solution to a differential equation.

This is why one doesn't see a trivial equation describing descent velocity for a skydiver. I have worked out skydiver velocity profiles for constant air pressure nearer the surface:

<http://arachnoid.com/sage/terminal_velocity.html>

But this equation, only an approximation at lower altitudes, is of no use at all for a problem like the Baumgartner jump.

<comment><USER26> The air pressure is proportional to either v or v² depending on the speed and atmosphere.

For a friction force Fr, the terminal velocity is reached when the acceleration becomes null:

m _g + Fr = 0

m_g - k* Vterm² = 0

k, the friction coefficient depends on different parameters: <http://en.wikipedia.org/wiki/Drag_equation>

So in the end, the terminal velocity will depend on:

\- The guy's weight (contrary to a free-fall's acceleration which is independent of mass)

\- The "contact surface" between the body and the atmosphere.

\- The atmosphere's density (which is lower than on Earth's surface).

The guy aims at reaching Mach 1. Note that due to the lower atmospheric pressure at this altitude, Mach 1 is a bit smaller than it is on the Earth's surface ( 301 m/s at 29 000 meters and -48 degrees C compared to 340 m/s at sea level ).

Finally, the term "free fall" is not appropriate as the definition of a free fall is "any motion of a body where its weight is the only force acting upon it."( <http://en.wikipedia.org/wiki/Free_fall> ) This does not take into account the drag which is not negligible here.

<comment><USER25> > The air pressure is proportional to either v or v² depending on the speed > and atmosphere.

1\. I think you mean "air resistance". Yes?

2\. If so, then no, air resistance transitions from (linear) Stokes drag at low velocities to (aptly named) quadratic drag at higher velocities, as a function of velocity, but not a function of air pressure. So not "either v or v^2", but a combination of the two factors.

<http://en.wikipedia.org/wiki/Drag_(physics)>

<comment><USER24> It made my day to have my question answered by an honest-to-goodness rocket scientist. Thank you!

I think the numbers that best sum up the situation are provided by yardie and blaze33 though. I knew that stuff coming from space was obviously going to be moving faster than the guy's 0 velocity, but had no idea about the order of magnitude.

</comment></comment></comment></comment></comment></comment></comment><comment><USER27> While we're waiting for the weather to clear, you can watch nicely edited footage of Kittinger's 1960 record of 100,000 feet; it's at the start of the Boards of Canada video for Dayvan Cowboy: <http://www.youtube.com/watch?v=A2zKARkpDW4>

</comment><comment><USER28> And it's aborted; 'Gusty winds'.

</comment><comment><USER29> Looks like they've got a weather hold until 8am Mountain Time (10am Eastern)

</comment><comment><USER30> Aborted for today, due to too much wind.

</comment><comment><USER31> Actually, its at 9:30AM EST...

</comment><comment><USER32> Speechless ...

</comment><comment><USER33> Aborted.</comment>


Bavarian raids - <AUTHOR> https://riseup.net/en/about-us/press/zwiebelfreunde <comment><USER2> Might be a good time to donate. Especially Zwiebelfreunde will need money to cover their lawyers. Note that if you donate via IBAN, your account data will likely be in the hands of the police. At least when the next raid happens.

RiseUp: [https://riseup.net/donate](https://riseup.net/donate)

Zwiebelfreunde: [https://www.torservers.net/donate.html](https://www.torservers.net/donate.html)

<comment><USER3> I'm against the raiding of servers, but want to point out that that Riseup are radical leftists, just so you know who you're donating to.

<comment><USER4> They are radical anarchists. If you're going to try to split this along ideological lines, at least get the ideologies right.

<comment><USER3> IMO, Anarchy definitions are not consistent enough to use the term univocally, definitions sometimes contradict each other.

They are leftists, because they see capitalism (and hence private property) as oppression and want to abolish it (instead of the moderate leftists who just want to regulate it in 'favor' of poor people), that makes them radical leftists.

<comment><USER5> Most people on the left want regulation to provide a universal safety net, if you want to regulate capitalism to favor poor people you are essentially for full on wealth redistribution which does put you on the radical side.

</comment></comment></comment></comment></comment><comment><USER6> To translate what happened into HN terms and analogy: the Open Source Initiative is an umbrella organization which advocates for Mozilla, Linux, Debian, Python, Apache and other open source projects.

Riseup.net is similar. It is not a group with a very specific ideology, it is an umbrella organization which many organizations are associated with, inside and outside of Germany. In the case of Riseup.net they are an umbrella for left-leaning groups. As their web site says, they provide tech services like email, web etc.

The raids on five locations were on Riseup.net, this umbrella tech organization which provides tech resources for much of the left.

Look at Riseup.net's mailing lists. Thousands of left-leaning groups have mailing lists there. Riseup is not an organization with a narrow focus. It is like a GitHub or Gmail of the left.

What probably happened is the German police were focusing on one person or group using the service, and raided not the members of that one mailing list, but the overarching umbrella organization that has 999 mailing lists on feminism and such.

</comment><comment><USER7> Discussed at [https://news.ycombinator.com/item?id=17456289](https://news.ycombinator.com/item?id=17456289)

</comment><comment><USER8> There are several comments here to the effect that Riseup is "communist" (although I cannot tell whether that's intended as a slur or compliment). Reading their "About us" and even "Political Principles" page, there's nothing terribly Communisty. Anyone have a pointer to backstory?

<comment><USER6> You are correct. Their about page says "We oppose all forms of prejudice, authoritarianism and vanguardism".

Opposing authoritarianism is implicitly saying they are not communist, saying they are against vanguardism (a word used by Lenin) means they are explicitly saying they are not communist.

Also as their web page says, they are not really a group with an ideology. They are a bunch of left-leaning techs who provide mailing lists, web site service etc. for left-leaning groups who "believe it is vital that essential communication infrastructure be controlled by movement organizations and not corporations".

My guess for what happened - the police were looking for one person or group that uses Riseup. As so many progressive organizations use Riseup, the targeting of one user or group of the service became a targeting of five locations, of people who provide the technological infrastructure of much of the left in Germany.

<comment><USER9> "we oppose all forms of oppression blah blah". I find they stop being interested in your oppression as soon as your views differ from theirs.

</comment></comment></comment><comment><USER10> while I have upvoted this story and deeply sympathize with the victims of what appears to be a wide 'overreach of power' by the police, I also had a look at the website that was the 'cause' of this raid.

!!! EDIT: I was under the impression, that this was their website. This is absolutely NOT proven and changes my viewpoint drastically !!!

While I deeply sympathize with the political views and what they are protesting (a far right-wing party in germany), they have published/written tutorials on different forms/tools of protesting ranging from 'peaceful' tools like grafitti and the use of colors up to how to use stones, nails in boards (wood boards? similar to this: [https://bit.ly/2tTF0kp](https://bit.ly/2tTF0kp)) in order to stop cars or even how to burn cars.

Now I'm not a lawyer, but while tutorials/information like this in itself probably isn't illegal, I assume there are laws that prohibit spurring people to protest using 'force'/violence. The context of this website and having those tutorials on it (including use gloves if you are using stones, because there could be fingerprints on it) combined with locations of AFD (super right-wing party) friendly locations can easily be seen/interpreted as a 'call to "violence"'.

It's a super interesting debate, but it's not as black and white as those CCC members make it appear. Again, I deeply sympathize with their cause - I am probably not perfectly OK with their methods, though.

edit: the website (in german) for those who want to judge themselves: [https://augsburgfuerkrawalltouristen.noblogs.org/](https://augsburgfuerkrawalltouristen.noblogs.org/) Loosely translated: "Augsburg (city) for riot tourists"

<comment><USER11> Is is worth mentioning that the german police has stated, that the only connection between this website and the people raided is, that they process donations for riseup. They have nothing to do with this website. And the police never thought, that they have.

<comment><USER10> you're right. I reread the article on Spiegel, and got the impression they were.

</comment></comment></comment><comment><USER12> Standard: communists calling for violence reaction in here without any of he commenters supplying any sort of links that demonstrate that.

Thought and discussion termination isn't cool.

</comment><comment><USER13> >What caused the police-state to raise up its ugly head? In this case, the justification was a website created to organize against a rally of an extreme right political party. It seems in Bavaria, you cannot make a website that tries to get people to come protest neo-fascists without also offending the police.

This is simply wrong. The website called for violence.

Also riseup is a communist organization.

<comment><USER14> Who's downvoting this? I don't see any unfactual or inappropriate statements.

<comment><USER15> Possibly the person who created an account literally named "communist" to comment on this.

<comment><USER16> New users can't downvote.</comment></comment></comment></comment>


Career Advice NeededJob offer from company using Coldfusion and no version ctrl - <AUTHOR>

I have a junior developer job offer for a company that promises career training, challenging projects and development autonomy. They are a leader in their field and have over 200 employees. They are in a desirable downtown location and hire people around my age (just out of school).<p>The only thing is that they don't use version control and their stack is Coldfusion. The company doesn't have a test database either. Would you sign on to get the immediate programming autonomy or pass for a more stable company? <comment><USER2> IMO, if you're a junior developer you don't want autonomy, you want guidance and mentoring from fantastic senior programmers. I didn't realize it at the time, but that was a big win when I started my career.

Also, I managed a ColdFusion team. I didn't want to use it, but the company chose it before I took over because it was easy for non-programmers to pick up. Not coincidentally, I was managing developers with way more experience because I had more good practices in me after 2 years at JPL than most in that team. Smart, wonderful people! Just not serious coders.

Bottom line, find a company who's development team impresses you and learn from them.

</comment><comment><USER3> _I have a junior developer job offer for a company that promises ... development autonomy._

That is clearly a lie, because they're still using CF with no version control.

</comment><comment><USER4> CF - No.

No version control - No.

What were their justifications for all that?</comment>


Bitbucket is down - <AUTHOR> https://bitbucket.org/site/master/issues

<comment><USER2> This is a more useful link to see what's happen(ing/ed) : [http://status.bitbucket.org/](http://status.bitbucket.org/)

</comment><comment><USER3> 'Major outage' according to their status page:

[http://status.bitbucket.org/#day](http://status.bitbucket.org/#day)

</comment><comment><USER4> [http://status.bitbucket.org/](http://status.bitbucket.org/)

Major Outage Apparently.

</comment><comment><USER5> They seem to be down quite regularly recently. Some deeper underlying issue, I'd say.

</comment><comment><USER6> Nothing on the page right now .. Seems up ..

</comment><comment><USER7> Just as I wanted to commit a private repo :(</comment>


RawGit is now in a sunset phase and will soon shut down - <AUTHOR> https://rawgit.com/ <comment><USER2> Kudos:

"Unfortunately, RawGit has also become an attractive distribution mechanism for malware. RawGit was meant to improve people's lives, but jerks are increasingly using it to hurt people.

Since I have almost no time to devote to fighting malware and abuse on RawGit (and since it would be no fun even if I did have the time), I feel the responsible thing to do is to shut it down. I would rather kill RawGit than watch it be used to hurt people."

<comment><USER3> Where does one draw the line? The internet is used for that, so should we shut down the internet? Is the content of the internet the responsibility of its creators, or operators?

I completely sympathize with this position for a one-person service, but I think the real problem is that we still haven't figured out how to fix incentives so we can distribute the work of keeping the jerks in line. On the internet, the jerks are winning.

<comment><USER4> > On the internet, the jerks are winning.

Arguably, it's not just on the internet. But I try to be optimistic.

<comment><USER3> Sure. I guess I'm old enough I still see the internet as a thing that we could hypothetically choose to shut down (without killing half the population) -- unlike, say, agriculture and civilization. Maybe that's not realistic, though.

</comment></comment></comment></comment><comment><USER5> I figured this day was coming. Excellent writeup. Sorry to see it go. Thanks for all the raw fish.

Sounds like the cost of hosting this were not insignificant. Curious how was this financed.

EDIT:

To answer my own question, I found the FAQ page, which is also great:

[https://github.com/rgrove/rawgit/blob/master/FAQ.md](https://github.com/rgrove/rawgit/blob/master/FAQ.md)

Sounds like donations helped (maybe). Oh and Stackpath. Kudos to them, too.

<comment><USER6> Hi! I'm the guy behind RawGit.

I've actually never accepted donations. I paid the meager cost of the $10/month DigitalOcean droplet for the origin server, and StackPath (formerly MaxCDN) sponsored the CDN, which handled the majority of RawGit's traffic.

Once, I worked out how much the CDN bill would have been if StackPath hadn't sponsored me and my head nearly exploded. So yeah, I definitely couldn't have done it without them!

<comment><USER7> How much would the bill have been? Rough estimate: 176TB per month: $3.500 to $4.200.

<comment><USER6> It was a while ago now so my memory is hazy, but that sounds about right.

</comment></comment></comment></comment><comment><USER8> RawGit was an amazing service, it's truly the end of an era for when you want to render the HTML page for source code you're viewing by just changing the domain.

</comment><comment><USER9> There are ~1M RawGit URLs embedded in open source projects on GitHub: [https://github.com/search?q=rawgit.com&type=Code](https://github.com/search?q=rawgit.com&type=Code) I'm curious to see how many there are a year from now.

<comment><USER10> How many of them would have SRI hashes added to them. If this domain eventually expires, and whoever buys it next can make trillions!

<comment><USER6> I'm not going to let the domain lapse for exactly this reason.

</comment></comment></comment><comment><USER11> > _" Unfortunately, RawGit has also become an attractive distribution > mechanism for malware. RawGit was meant to improve people's lives, but jerks > are increasingly using it to hurt people."_

This has been my experience too, and I'm now convinced that hosting any online services can only be done by corporations whose business model entirely relies on it, and that a small full-time staff is needed to combat malicious users effectively and keep the service usable for everyone else.

<comment><USER12> This is something the decentralization fans need to take into account too: if successful, ipfs will be full of malware being distributed from compromised routers etc.

</comment></comment><comment><USER13> It was a really cool project and I think the first to offer this kind of features. Sad to see it go.

But right now jsDelivr supports both GitHub and npm as source for CDN files. So here is an easy tool for migration [https://www.jsdelivr.com/rawgit](https://www.jsdelivr.com/rawgit)

<comment><USER14> Not really a full replacement, unfortunately:

> For security reasons, we serve HTML files with Content-Type: text/plain. We > recommend using GitHub Pages if this is a problem.

</comment></comment><comment><USER15> This will be interesting... I just did a grep of my local codebases and found 11 instances where CSS or JS resources were being downloaded from rawgit.com.

I have a feeling when it's turned off, a lot of sites are going to break.

<comment><USER6> Yeah, I really, really don't want to break things, so I've tried to be very careful about how I'm doing this.

In the shutdown announcement I committed to keeping the site running in sunset mode for at least a year. Hopefully that's plenty of time for everyone who's aware of the shutdown to migrate, but I expect there will be stragglers.

My unofficial, subject-to-change plan for dealing with that is that at the end of the sunset year, if there's still a significant amount of traffic, I'll start throttling requests to make RawGit slower. Hopefully people will notice their websites are slow and will investigate. I'll also try to notify stragglers individually by filing issues against their GitHub repos if possible.

<comment><USER16> Instead of throttling, you could also consider doing incremental brownouts where you drop requests for the first ten minutes every hour. PyPI did this recently when they phased out TLS 1.0, which worked really well IMO.

<comment><USER6> Thanks for the suggestion! This does seem potentially more effective.

<comment><USER17> IIRC Google also uses (or used) a similar system for deprecating old APIs.

Start with failing 1% of requests randomly and slowly ramp up from there.

</comment></comment></comment></comment></comment><comment><USER18> I've never used rawgit for my own projects, but it did make life a lot easier to check out demos and test run examples in JS libraries hosted on github.

Totally understand that the effort required to fight malware is tedious. But instead of shutting it down you may want to try a $5/mo plan. The money is not for making money but kind of like how Google chrome store required a $5 payment to publish your first chrome extension.

Spammers hate paying it and I think as soon as money changes hand there is verification and a trail which makes them nervous too (many forum owners do this as well).

<comment><USER19> I got the impression the author would rather not deal with the headache anymore, and charging money is trading one set of headaches for another. Building things is fun. Operating things is stressful. I don't blame them.

<comment><USER18> The stats he has quoted on the FAQ it wouldn't be surprising if it starts making $10K/mo (just a wild guess) and the malware reduces as a side-effect too.

But I guess you're right. Not everyone is driven by money and it would certainly turn a hobby project into a full time job. Sounds fair to shut it down.

</comment></comment></comment><comment><USER20> It was great while it lasted. I currently have a small extension that depends on it. Oh well, I knew the risks of using a free service so no complaints. Thanks for the free service!

</comment><comment><USER21> By now you can configure GitHub to publish your master branch as github pages. So no need to use rawgit either.

The only sad part, is all the links that are breaking.

</comment><comment><USER22> Is there a service that can replace RawGit for Gists, specifically serving them with the correct MIME type? This was a convenient feature for users of [https://reviewable.io](https://reviewable.io) to hack up a quick custom stylesheet or keyboard shortcuts override. It's not clear to me whether any of the suggested replacements support Gists.

<comment><USER23> try replacing "rawgit.com" with "rawgit2.com"

</comment></comment><comment><USER24> Congratulations, it was a great gift for the web. Nothing lasts forever anyway.

</comment><comment><USER25> Precisely why I never use free CDNs or stuff like this to serve static content. Just host your own thing. It maybe costs a little bit more but at the end you'll have websites that will run for decades.

Rawgit was useful, I wish it stayed up for a little longer, perhaps someone like Cloudflare could sponsor this?

<comment><USER10> CDNJS is from CloudFlare, running with what I believe is better DNS and more POPs.

</comment></comment><comment><USER26> Maybe Red Hat (who use Rawgit a lot) or, for that matter, StackPath want to take over?

</comment><comment><USER27> Thanks for everything! Are you going to open source the Rawgit codebase? I'd be interested just for educational purposes.

<comment><USER6> RawGit has been open source from day one: [https://github.com/rgrove/rawgit](https://github.com/rgrove/rawgit)

</comment></comment><comment><USER28> oh well, a few hundred of the working answers I posted on stackoverflow will stop working. Not sure how I can fix them

<comment><USER14> Thankfully, it's pretty easy to find the GitHub page for the linked source with a bit of URL surgery.

<comment><USER29> And SO will probably fix this automatically for all answers.

<comment><USER14> They might, depending on how widespread it is. It certainly wouldn't be without precedent. Alternately, there might just be a manual effort to fix them all.

</comment></comment></comment></comment><comment><USER30> "You either die a hero, or live long enough to see yourself become a villain"

</comment><comment><USER31> Why not sell or give away the project to someone who has time to fight the abuse?

<comment><USER6> I did consider this, and several people have offered to take over, but ultimately I feel that the thing that was most useful about RawGit — it made serving HTML at scale dead simple for anyone with a GitHub repo — is also the thing that made it most prone to abuse.

It came down to a simple equation: fighting abuse on RawGit will _always_ take more time and effort than spreading abuse via RawGit. One persistent jerk working a few hours a day could do so much damage so quickly that mitigating it would require multiple people working full time.

There's just no good way to scale that and retain the functionality that actually made RawGit useful.

I talked about this a little more here: [https://github.com/rgrove/rawgit/pull/191#issuecomment-42831...](https://github.com/rgrove/rawgit/pull/191#issuecomment-428310526)

<comment><USER32> Look at how enthusiastic people are about moderating forums. Would it not be possible for someone to add a web application where volunteers could go to manage blacklists or something? Like prescreened volunteers.

<comment><USER33> It's not free. You will have to now spend time managing people.

</comment></comment></comment></comment><comment><USER23> For the time being I threw this up.

You should be able to replace rawgit.com, with rawgit2.com.

<comment><USER34> Are you going to self fund the $50,000/year in hosting costs and devote time to fixing the malware problem?

<comment><USER23> If the time comes, sure.

Also it can be hosted for much less.

</comment></comment></comment><comment><USER35> I used this for a lot of Codepen stuff - sad to see it go.

<comment><USER6> You might find this useful: [https://github.com/mallendeo/cdp-rawgit- fix](https://github.com/mallendeo/cdp-rawgit-fix)

It'll scan all your pens and try to suggest jsDelivr URLs wherever you've used RawGit.

</comment></comment><comment><USER36> I loved your product and enjoyed it. Thanks always!

</comment><comment><USER37> This was very useful service, sad to see it go.

</comment><comment><USER38> > 176 terabytes of bandwidth

How did he pay for this?

<comment><USER39> It looks like he didn't, for the most part: [https://github.com/rgrove/rawgit/blob/master/FAQ.md#can-i- do...](https://github.com/rgrove/rawgit/blob/master/FAQ.md#can-i-donate- moneybitcoinpie-to-help-you-out)

"It's super nice of you to offer, but I don't need any donations at this time. RawGit's server costs are minimal, and the lovely people at StackPath provide RawGit's CDN service free of charge. Thank you though!"

</comment></comment><comment><USER40> Yet another trendy hip service folds with little to no notice. Meanwhile sourceforge keeps on going.

<comment><USER41> ... what?</comment></comment>


Javascript Birthday "Card" - <AUTHOR> http://dbattaglia.heroku.com/birthday

<comment><AUTHOR> A little javascript/canvas birthday "card" I made for my girlfriend. Thought it would be more fun and thoughtful than going to Hallmark. The code is not the cleanest I've written but it gets the job done.</comment>


10 Uses of RFID Tags - <AUTHOR> http://www.wired.com/techbiz/it/magazine/17-03/st_best

<comment><USER2> They've missed my favourite: Tagging groceries for automated billing.

I'm going to love walking out of a store without needing to wait in line for a cashier first.

<comment><USER3> Yes, but I would hate to be the guy forced to tag all the oranges. or steaks. or heads of lettuce.

Obviously, I'm being a little facetious. I love the idea of walking in and out on credit (although the privacy implications bother me), but I still havent figured out a way that they can do fresh produce without completely revamping how it is sold. I suppose you could go to an honor system where people bag and then print label/rfit sticker.

<comment><USER4> The chips could be inserted automatically by machine when the product is being assembled at the manufacturers expense. I could see Home Depot or a similar big box company, who have quite a bit of influence over manufacturers, making that a requirement.

The cost might be a trade-off if customer satisfaction is boosted.

Although that might be impractical for grocery stores or smaller items (small fruit, nails, etc).

<comment><USER5> Maybe the cost savings of tagging the more expensive items can make paying for the cheaper items part of an honor system. I.e. pay $X monthly to have unlimited access to item Y.

Or, untagged items are locked. They are unlocked with a credit card, and the weight difference is billed to the card.

Those are just a couple off the top of my head. I suspect there are many creative ways to deal with untagged items.

</comment></comment></comment></comment><comment><USER5> The problem with ideas like RFID is that without a global method of tracking people a global government is unworkable. However, once we can electronically track everyone, then global government becomes much more feasible.

That being said, I doubt RFID actually can allow global tracking. The infrastructure cost and maintenance to make that happen is way too high, and not worthwhile.

</comment><comment><USER6> And when you want to go against those 10 uses and any other possible uses...

[https://events.ccc.de/congress/2005/static/r/f/i/RFID- Zapper...](https://events.ccc.de/congress/2005/static/r/f/i/RFID- Zapper\(EN\)_77f3.html)

</comment><comment><USER7> Why does it not surprise me that Britain wants to tag prisoners?

</comment><comment><AUTHOR> One thing that surprised me was that so many of these uses are tagging living beings, rather than inanimate objects.

<comment><USER8> Being chipped so the club can automatically charge your drinks? God no. Being chipped so the police can track you after your are kidnapped? I dont think I want to live there. Chipping sponges so the surgeons can retrieve them? That makes sense.</comment></comment>


AWS Uses Photos of Google's Data Centers - <AUTHOR> https://plus.google.com/+UrsH%C3%B6lzle/posts/aJC7P2JF2f8 <comment><USER2> Ben Sloss' comment is pretty funny. But seriously, the presentations given by Amin Vahdat (who gives the keynote at ONS pretty much every year), and Urs are pretty amazing. It does make one wonder how much AWS and Microsoft are doing in this area as well. We just saw Microsoft adopt DC on board, so they could be pretty far behind, but they also have significant market share, so is google just behind in explaining their advantages?

</comment><comment><USER3> I don't see the story here. Somebody at AWS did a google image search for some stock looking photos.

<comment><USER4> the story is something mildly humorous happened.</comment></comment>


Why not just create a user curated search engine? - <AUTHOR>

I've been reading a lot of the Google-related search engine complaints. Who hasn't? They're everywhere for whatever reason. A common theme that I seem to see in most of them is that people seem to agree that some sort of user interaction is one of the few ways to eliminate spam and SEO content farm results.
So, why not just create a user curated search engine? Web pages would be indexed. Users would filter them, using queries. On a per-query basis, indexed pages would be subject to a reddit-esque voting system for relevance to that query. There would be no visible up voting, though. There would only be a visible down vote option. Up voting would be done automatically when a user clicks a result. Down voting would be done if a user visits a page and the page is not relevant to the query, so they come back to the search results and down vote the result (it would remove the automated up vote + 1 more to net a subtraction). Pages indexed more recently would be given priority over older pages. So what would eventually happen is the most viewed and newest pages for a query would bubble to the top, and older posts would be in the middle, and down rated posts would be at the end or maybe even excluded all together.
Obviously there are holes in this, such as people scamming the up-vote clicking mechanisms but I'm sure that could be prevented with more thought.
I think the key is that all of this voting is done on a per query basis, for relevance to that query. <comment><USER2> _Obviously there are holes in this, such as people scamming the up-vote clicking mechanisms_

I think you put your finger on it there. There are folks who have a financial incentive for their pages to rank high. A "user-curated" system just opens up additional avenues for "spam and SEO content farms" to game the system.

<comment><AUTHOR> Yes, but apply the Wikipedia phenomenon here. With more users using it for good, perhaps it'd outweigh the bad? This, combined with actual measures to prevent abuse of the system.

<comment><USER2> Nope, sorry. There are users with a financial incentive to game the system. The "actual measures to prevent abuse" would have to be pretty damned sophisticated. Algorithmic, in fact. Which brings us full circle...

<comment><AUTHOR> Then perhaps we could apply today's startup quote!

When you innovate, you've got to be prepared for people telling you that you are nuts. - Larry Ellison</comment></comment></comment></comment>


Gallery of 404 error pages - <AUTHOR> http://oops.re/ <comment><USER2> [https://www.lego.com/404](https://www.lego.com/404)</comment>


Luxembourg challenges EU order to recover tax from Amazon - <AUTHOR> https://www.reuters.com/article/us-eu-amazon-luxembourg-taxavoidance/luxembourg-challenges-eu-order-to-recover-tax-from-amazon-idUSKBN1E917S <comment><USER2> I got one am very pleased to see the EU doing good work against tax havens. It's no wonder the British press hate the EU so much... the proprietors, in general, have an interesting record when it comes to paying tax

<comment><USER3> But this race to the bottom for tax rates is clearly bad. Luxembourg could next make their rate 1%. Then Eire could come back with 0.5%. There's no good that comes of this - even the countries with the low rates are hurting their ability to raise enough money from tax. It only works in the short term when you have companies that cheat and book revenue from outside into that country. It's wrong and I would strongly support both (1) trying to stop the low rates, and (2) book money in other countries than you earn it. It's all just a bs tax dodge.

<comment><USER4> The EU governments could just stop taxing corporate income entirely, and only tax personal income, sales, and property. There's no reason in principle why corporations must be income taxed.

<comment><USER5> And then everyone stops being an employee to become a tax free one person contracting company, with car, home office, and anything else they can think of, considered business expenses.

<comment><USER6> If they get personal benefit from the car, that's taxable income. If they get personal benefit from the home, that's taxable income.

<comment><USER5> Sure, but unless you have tax inspectors following the entire population around and security cameras in their home, you're never going to be able to enforce that, as demonstrated by people already doing this sort of scheme. It's easier to just disincentivize it in the first place.

</comment></comment></comment></comment></comment></comment><comment><USER7> As an individual person without an army of lawyers and accountants try having EUR 1000 of unregulated taxes and you'll stir the wrath of the Olympus.

<comment><USER8> Plus corporate taxes are already very low. The highest nominal tax rates in Europe are around 30%, but that is misleading. The actual real tax rates are somewhere from 5% to 10%, because every country has these "ifs and buts" clauses, that they use to lower corporate taxes, and effective act as a tax haven, while pretending they are not.

Coincidentally, I just checked this statistic on OECD: corporate tax revenue is just around 4% to 8% of total tax revenue. I wonder where these corporations would be if the normal people weren't contributing ~90% of tax revenue for upkeep of state infrastructure, that enables the existence of these businesses in the first place.

[https://data.oecd.org/tax/tax-on-corporate- profits.htm](https://data.oecd.org/tax/tax-on-corporate-profits.htm)

<comment><USER9> That's true, but it's worth mentioning that income tax also doesn't seem to be a large portion of most country's GDP. It seems to range between 1% to 14%, except Denmark (25%) which is an outlier. See: [https://data.oecd.org/tax/tax- on-personal-income.htm#indicat...](https://data.oecd.org/tax/tax-on-personal- income.htm#indicator-chart). The global average is only 8%!

Another interesting fact is that in the graph for corporate profit, Luxembourg actually has the highest rates, which must be because of its small size and huge number of companies that are registered there to benefit from the countries low tax rate.

Despite this I think corporate should be higher judging how much large multinationals earn. Otherwise the income inequality is only going to grow further.

<comment><USER8> Actually, I looked into it more, in my country, the Netherlands, and over all it is split: 55% individual contributions, 45% business contributions.

Income taxes are just plain income taxes. There are also social security contributions, part of which is paid by the employer. I know it is not necessarily appropriate to say that that is a tax the employer has to pay, but I counted it as such. Transaction taxes, I counted as a "business tax". VAT, I counted as individual taxation, mostly because it is regressive and hits the poor, i.e. non business owners disproportionately.

So my earlier claim was incorrect. It is more of a 50-50 split.

[http://www.oecd-ilibrary.org/taxation/data/revenue- statistic...](http://www.oecd-ilibrary.org/taxation/data/revenue- statistics/netherlands_data-00247-en)

<comment><USER10> "social security contributions.... tax the employer has to pay": that's a great lie, that employers pay those, not employees.

For employers, it's one amount of money paid for hiring an employee: employers don't really care what's salary, and what's taxes. In France, employees don't even get 50% of their gross salary: there are more social security contributions than salary.

Employees should really want to reduce social security contributions, instead of thinking "my boss pays for it".

</comment></comment></comment></comment></comment><comment><USER11> Eliminate corporate income taxes, and make the change revenue neutral by raising taxes on individual corporate investors and high-income employees. The money eventually flows to them anyway. By eliminating corporate income tax we could cut out all the waste and non-productive tax avoidance financial engineering. We wouldn't need corporate tax accountants at all. This would boost growth rates and economic efficiency.

<comment><USER12> Wouldn't those high income employees just become tax exiles and move away? That's what happens with athletes like Lewis Hamilton living in Monaco.

<comment><USER11> It doesn't matter where they live. Foreign investors can still be required to pay income and capital gains taxes on money earned in a particular country. The government can always block or seize international payments to tax evaders.

In theory the countries where Lewis Hamilton races and earns his income could tax him there. But generally they let him slip through loopholes to maintain good relations with F1.

<comment><USER13> So Lewis Hamilton needs to fill tax returns in the 40 countries he works in at the end of the season

<comment><USER14> To be fair: I'd expect Mr. Hamilton to be payed for his 40 races by exactly one company which is based in one location, rather than for each race by one of 40 companies.

</comment></comment></comment></comment></comment><comment><USER15> Half of which belongs to Italy[0], considering they evaded taxes for years.

[0]([https://www.wired.it/economia/business/2017/12/15/amazon- acc...](https://www.wired.it/economia/business/2017/12/15/amazon- accordo-100-milioni/))

<comment><USER16> That is unrelated, the things contested to Luxemburg is that there was a special agreement in place for Amazon, which is forbidden by EU regulations.

The thing contested from the Italian tax office is actual tax evasion.

Edit: to clarify, I mean that Luxemburg can have had illicit behaviour independently from Amazon acting legally in Italy (which in theory they did since 2015).

<comment><USER17> JC Junker, current President of the Commission, was also Prime Minister of Luxembourg just before... Is it me or does it sound just awful ?

<comment><USER18> the EU has a tendency to be where clapped out/disgraced national politicians that are no longer electable in their home countries end up

it's a nice place to retire, and pays more than the national office too

there's peter mandelson, jean-claude juncker and guy verhofstadt, to name a few

<comment><USER19> Durão Barroso too.

<comment><USER20> He was the President of the European Commission which requires approval of European Parliament (elected directly) and European Council (the heads of countries).

What the fuck is EU supposed to do with him? Throw him out and tell leaders and directly elected MEPs from 20+ countries to shove it? And who exactly is the EU in that case? Who and how could challenge these decisions while the majority of actual people in Europe don't give enough of a shit about Barroso and elect (if they even go to the elections..) these governments, heads of state and MEPs that put him where he was?

</comment></comment></comment></comment></comment></comment><comment><USER21> Someone should just step up and review all of this mess.</comment>


3 hours to teach Blockain and Cryptocurrencies - <AUTHOR> Hi !
I'm a French student in 5th year of Engineering school (INSA Rennes), and I have to present Blockain & Cryptocurrencies to my schoolmates in 3 hours. I can divide this time as I want, for example : 1 hour lecture, 2 hours practicals.
I have some ideas of what to say during the lecture. HN was really helpful to find articles and videos (especially https://anders.com/blockchain/) but now I'm more struggling to find what to do during the practicals. If it is possible, I'd like to create a tiny currency, just for exercise, so that they could actually play with it, study the blockchain etc.
Do you have any idea how to do that, or do you have any other good idea ?
Thank you :) <comment><USER2> How about having some students be the nodes

Their job is to write down the transactions on a piece of paper.

Then have some students be miners.

They write down unconfirmed transactions and roll 2 dice. If they get a pair of 1's they can create a block and give it to the node students to copy. (Could use a photocopier to speed things up)

Then have some students be users creating transactions and such like.

<comment><USER3> Use physical objects like game tokens and a tray to represent blocks. Faster than handwriting or photocopying. Simple playing cards can also work (this "block" has 4 transactions, the AH, 2S, 3C, 10H).

Someone shuffles a handful of decks together, this creates the feed of transactions going into the network. "Blocks" are created by grabbing a random set of recent open transactions and rolling dice, like you suggest. Pair of ones and that "block" can be passed on to all the other nodes. Others can continue "mining" if they want and try to create a fork, other miners may choose to base their work on the newest block.

Have some sorted stacks of cards or tokens available to quickly assemble copies of the blocks and transactions.

</comment></comment><comment><USER4> No one is going to (or should) do your homework for you.

<comment><AUTHOR> That is not what I ask :) I'm only gathering ideas, from a competent community ; that fits completely in this homework frame. Moreover, I don't ask you to write this lesson for me. I'm more interested in "if it were for you, what would you like to do ?" What could be a good or efficient or fun way to learn the Blockchain ?

<comment><USER4> Fun? Maybe an illustration into the Byzantine General's problem.

But efficient? I think it's a complicated enough idea that it'll require a bit to outline the problem it solves and then longer how it goes about solving it.

</comment></comment></comment><comment><USER5> Have them invent it.</comment>


How Calls for Privacy May Upend Business for Facebook and Google - <AUTHOR> https://www.nytimes.com/2018/03/24/technology/google-facebook-data-privacy.html <comment><USER2> >"But some trade group executives also warned that any attempt to curb the use of consumer data would put the business model of the ad-supported internet at risk."

Yes the "ad-supported internet" may be at risk. So what? Is it unreasonable to think that this model might not actually be sustainable long term?

Why should this business model be any less vulnerable than any other business model? Note to Big Tech - public opinion also has the power to "disrupt."

<comment><USER3> Furthermore, it is possible to run ads without collecting creepy amounts of data on everybody.

Ads themselves are not so problematic. But the idea that companies have to collect amounts of data that make the Stasi look like a bunch of privacy activists is very much problematic.

<comment><USER2> Agreed. It's funny you mention the Stasi. When touring the old Stasi headquarters in Berlin its tough not to think about how 25 years later a company has amassed much more data on private citizens than Stasi could ever imagined.

</comment></comment></comment><comment><USER4> >The United States does not have a consumer privacy law like the General Data Protection Regulation. But after years of pushing for similar legislation, privacy groups said that recent events were giving them new momentum — and they were looking to Europe for inspiration.

>"With the new European law, regulators for the first time have real enforcement tools," said Jeffrey Chester, the executive director of the Center for Digital Democracy, a nonprofit group in Washington. "We now have a way to hold these companies accountable."

I'm super excited about the prospect of GDPR in America.

<comment><USER5> Effectively, you will have it, because it applies to European residents, and it is difficult for companies to know for sure that you aren't a European resident.

<comment><USER6> Well, not so much if the companies are really willing to.

They can block traffic coming from European IP addresses and require a legal document saying that the customer is not European.

<comment><USER7> So, for all intents and purposes, shut themselves out of the European market?

<comment><USER6> Yes. "Obey our rules or stay out of our market" seems like a reasonable thing.

</comment></comment></comment></comment></comment><comment> shady-lady Hopefully it will also change the "move fast & proper security is an afterthought" model of startups.

Years ago(in CS degree before professional work), I used to wonder & be amazed by how fast new companies(mainly U.S.) moved so quickly.

It wasn't because they had some secret sauce or 10x devs, it was because all they did was focus solely on features & take shortcuts. Their customers data security was an afterthought - e.g. production DBs open to anyone in company(even the world in some cases(mongo)), no auditing or logging of data access.

Maybe YC(& other accelerators) will make a security expert available to these people...because young adults with 0 professional experience will probably not consider security & rely on the defaults of whatever stack they're using to be good enough. Even less consideration will be given when they're fed & surrounded by a grow fast at all costs mentality.

Good security in their products needs thought & critical analysis put into it & I'm not sure new devs appreciate it.

<comment><USER8> > how fast new companies(mainly U.S.) moved so quickly.

I've worked in startup/mid size companies outside U.S and from what i've seen its totally inaccurate to say that startups outside U.S move slowly because they take privacy/security more seriously.

<comment> shady-lady The reason I mentioned US specifically was because VC funding to support hyper growth was historically & (still is) way more common there than elsewhere in the world(Western Europe specifically). A condition which seems to almost mandate the grow fast at all costs mentality.

It doesn't preclude anywhere else in the world from being lax about security. I also have worked for multiple small/mid companies in Western Europe and, anecdotally for me, data & thoughts to security are held in much higher regard.

</comment></comment></comment><comment><USER9> The Equifax thing didn't result in regulation and that was an actual scandal with real ramification not a pile-on by a press seeking to vilify an ad financed competitor.

Funny how the Times is using silly _metafilter_ vernacular now:

"the consumer surveillance model" \-- it's the same business model that's been supporting news publication for more than a century.

As for the GDPR it's similar to the recent digital tax proposal and numerous other initiatives by the EU intended to attack US businesses, the US government doesn't share that motivation and it must condemn and push against these EU actions.

<comment><USER10> >It's the same business model that's been supporting publication for more than a century.

I think there's an enormous difference between advertising in print, advertising on the internet, and what is currently the status quo, which is active surveillance of all information consumed and every movement that individuals make across the internet and in the real world by data linked through GPS/ cell phone location, credit card purchases, etc.

I'm aware that this is the new reality, but comparing it to print advertising (or even pop up ads of the early internet) is disingenuous at best.

<comment><USER11> > comparing it to print advertising is disingenuous at best.

Right. First, print advertising could only target the entire subscriber base. Second, they had a limited amount of data on you based on voluntary surveys and census tracts, not a list of every single thing you read, and where you are every minute of every day. Finally, print publications actually cared about reuse/misuse of their address lists. The list they sold you would include some addresses they controlled, so if you resold or reused it, they would catch you.

There are ways to deliver effective ads with a whole lot less surveillance.

<comment><USER12> Also credit card purchase records.

</comment></comment></comment></comment><comment><USER13> I just wrote a second draft of my essay on this exact topic:

[http://yuhongbao.blogspot.ca/2018/03/google-doubleclick- mozi...](http://yuhongbao.blogspot.ca/2018/03/google-doubleclick-mozilla- essay-second.html)

Among other things, I wrote about the sharing of remarketing lists.

</comment><comment><USER14> Guys, seriously. Why is this a surprise for people? Facebook is a corporation that literally lives from analysing and selling data about everyone, even people who have not signed up for Facebook, WhatsUp or Instagram accounts.

Facebook android application asks you for the following permissions when you install it:

This app has access to: Device & app history

        retrieve running apps

Identity

        find accounts on the device
        add or remove accounts
        read your own contact card

Calendar

        read calendar events plus confidential information
        add or modify calendar events and send email to guests without owners' knowledge

Contacts

        find accounts on the device
        read your contacts
        modify your contacts

Location

        approximate location (network-based)
        precise location (GPS and network-based)

SMS

        read your text messages (SMS or MMS)

Phone

        directly call phone numbers
        read call log
        read phone status and identity
        write call log

Photos/Media/Files

        read the contents of your USB storage
        modify or delete the contents of your USB storage

Storage

        read the contents of your USB storage
        modify or delete the contents of your USB storage

Camera

        take pictures and videos

Microphone

        record audio

Wi-Fi connection information

        view Wi-Fi connections

Device ID & call information

        read phone status and identity

Other

        download files without notification
        adjust your wallpaper size
        receive data from Internet
        view network connections
        create accounts and set passwords
        read battery statistics
        send sticky broadcast
        change network connectivity
        connect and disconnect from Wi-Fi
        expand/collapse status bar
        full network access
        change your audio settings
        read sync settings
        run at startup
        reorder running apps
        set wallpaper
        draw over other apps
        control vibration
        prevent device from sleeping
        toggle sync on and off
        install shortcuts
        read Google service configuration

It does literally ask you to grant access to your text messages and call logs. Why would anyone expect Facebook NOT to use that data? You all cant be such native, right?

<comment><USER15> Most people aren't that naïve.

The problem is not allowing a company access to your data. The problem is _how_ it's used.

Just because you allowed another company to handle your data should not give them carte blanche to do whatever with it they want.

This is exactly why Europe created the GDPR.. to prevent your personal data from being processed by companies in unintended ways, or sharing your information with 3rd parties without your consent, and heavy fines for breaking these rules.

Just allowing a company access to your data and giving up at that point is the attitude that people are trying to change, because that's not the way the world should work.

</comment></comment><comment><USER16> OT: is this domevent account a new bot on HN? The name, frequency of posts, closeness to article published times...

</comment><comment><USER17> Maybe we should pause and think what value does Facebook provide to the users? Is that value worth the $400B? Given how much we are connected through all kinds of channels, the answer is a clear no. Sure you can say their advertising business is worth that, but if they don't provide their users that much value it's not going to last.

<comment><USER18> The value can be argued either way. But remember the old adage: The market can remain irrational longer than you can stay solvent.

</comment></comment><comment><USER19> Data breaches can take down a business, but privacy concerns are there historical examples of that?

My thought is management gets fired and the company changes course by the time the bottom line gets measurably impacted.

Companies are much more likely to fail by not innovating fast enough.

<comment><USER20> Cambridge Analytica, in about three weeks.

</comment></comment><comment><USER21> One can only hope.

<comment><USER22> If we continue demanding and building the tools that enable privacy, we will be able to get there. More cuz of the building than the demanding.

<comment><USER23> This is definitely the right attitude, and I'd add the critical step of talking about these issues with people who don't know about them. It's frustrating, it can be alienating and it definitely takes time, but it can pay off. Nothing beats building a useful tool (like Signal), but even useful tools require publicity.</comment></comment></comment>