Building bots to mend badges, or how to get your GitHub account suspended - <AUTHOR> http://movermeyer.com/2018-03-08-building-bots-to-mend-badges/ <comment><USER2> I did this 3 years ago.

I created a bot that would scan for private SSH keys to connect to AWS and other services, it also warned about leaked software licenses for SublimeText and other popular programs at the time. While many people appreciated the initiative, it was not taken the best way by others. Ultimately, GitHub suspended my account and I had to explain what was all about.

One year later, through my employer, I created another bot to scan for security vulnerabilities in projects written in Ruby, Python, PHP and Node.js; this time I already knew that I would need to contact GitHub beforehand to make sure what were the limits of the _" automation"_. They simply stated that — at the time — no automation was allowed, which was quite surprising because CI is automation. Travis and other services are allowed to do things there so I didn't understand why my bot was different.

I reported to my employer that we would need to shutdown that project and move on to something different. One year later, I find that GitHub implemented a (semi) vulnerability scanner for a selected group of programming languages, warning the repository owners about problems with their software dependencies. I cannot be mad about this, it's their service, but it still made me a bit angry.

<comment><USER3> Assuming you're talking about a bot similar to the OP's that scans random projects you don't own, that's very different from CI which is explicitly configured by projects.

<comment><USER4> But if they are maintaining "no automation" as a rule, CI should not really allowed.

<comment><USER3> This really depends on how you define automation. CI on GitHub depends on webhooks which is an officially supported part of the API. So there isn't anything unsolicited happening.

I just don't think there's any meaningful claim that CI explicitly configured by maintainers using officially supported channels should be lumped in the same category as automated scraping of repositories for creating PRs.

Sure, there are many definitions of automation that would include both of these things, but I think it's obvious what GitHub intends in practice.

<comment><USER4> The automation wouldn't be doing anything unsolicited either, since it's only doing things that have been defined in the UI.

Perhaps they didn't mean automation per se, but CI is certainly automation, automatic merging, as in not manually merged by a human.

Perhaps a better rule would have been, no automation on repos not controlled by yourself.

<comment><AUTHOR> The actual rule is no "excessive automated bulk activity".

[https://help.github.com/articles/github-terms-of- service/](https://help.github.com/articles/github-terms-of-service/)

<comment><USER4> That being the case, seems like OP's bot probably should have been allowed, especially since it seemed like they were only automating their repos.

<comment><AUTHOR> This bot was exclusively applied to repos that were not mine.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER5> The answer to this is to do human rate limiting.

Build your pipeline so that a human has to approve each automated action being taken. The difference between a bot making 5000 network requests in a day and a human making 100-200 semi-automated requests isn't a whole lot in terms of throughput, but makes an enormous difference in terms of quality control and not stepping on toes.

I really wish more companies would do this. Fully automated business procedures that make demands on human attention are just plain awful. Humans should be interacting with humans, machines should interact with machines. Machines can help the human, and the human can help the machines, but interaction points should only be between two of the same types of entity.

Every time I've ever suggested human rate limiting though, I get looked at like I'm a moron. Even when I build the entire workflow myself and tune it so it only takes a relatively tiny amount of time to clean massive amounts of data, people just don't want to do it. It's beneath them. Even when it creates a massive difference in the quality of the product / service you're offering.

<comment><USER6> That's an interesting point. I'm not sure I would trust myself to do a better job than automated tests after a (short) while. It's not really a matter of being beneath me; for example, I don't mind doing repetitive manual labor (once in a while).

Just out of curiosity, what's the biggest job of this type you've personally handled?

<comment><USER5> I made a brief, aborted attempt at a restaurant recommendation service. We wanted to hydrate our data with existing pictures of dishes from the restaurants sourced from Yelp and/or Google Image Search. After looking at that data, I realized that a human touch to picking the right images would make a huge difference in the service.

We're talking thousands of restaurants that we wanted pictures of food from, each of the restaurants had dozens of images we could pull. So tens of thousands of images needed to be sifted through, I figured with the right tooling, myself and my cofounder could put together something really nice that would only need an hour or so of maintenance a day to keep up.

So I built a pipeline that used very basic and easy to build and maintain 'dumb' Rails asset pipeline pages to present data for sifting. Go to the endpoint, it shows you the name of the restaurant and a bunch of images, you select one, type in a name for the dish, and it saves it to the database and puts up another page of images.

It took me bitching up a storm to get him to even look at it. He complained about how long he thought it would take, while I just got to work. Took maybe three weeks to prototype our app. One thing I learned in the process is that if you're looking at a bunch of Southern food, for some reason the picture of shrimp and grits always looks the most appetizing.

I was well on my way to classifying and figuring out novel ways to present the data when I had to make the determination that there wasn't good cofounder fit. So now I work with CNN.

But now all my side projects revolve around ways to get human attention to improve automated tasks. I suppose one of these days I'll get the right idea and/or the right cofounder and I'll give it another go.

There's a wealth of usable information out there on the web that one can build businesses on top of if one only wants to apply a little elbow grease to clean it and turn it into data. It's far easier to scrape data with a regular web browser with a custom browser extension than to try to build out headless infrastructure. But no one wants to do it.

<comment><USER7> Your story reminds me of a tool I wrote for helping lawyers classify a feed texts as a test set for a project.

Our main initiative was creating a heuristic based classifier (think lots of regex). At my own initiative, I trained ML classifiers while we worked on it. As development went on, the ML classifiers were rapidly catching up with the heuristic based one. Unfortunately it was kind of a one off data processing task, and when time ran out the regex machine was still in the lead.

I was modestly proud of the legalese DSL generator I wrote up. The lawyers didn't even know they were writing coffeescript as they typed out what documents were, what key dates were, etc. :D

That coffeescript formed the basis of our accuracy testing suite. It was as fundamental as it was huge. That team ended up creating a couple thousand tests in less than a month.

<comment><USER8> I'd be interested in hearing more about this. Fancy dropping me an email (address is in profile)?

</comment></comment></comment></comment></comment><comment><USER9> I actually really like this idea. There are so many random things on github that get broken over time. The implementation though clearly is problematic and github has no choice but to block this behaviour.

I could image though a system where there was some sort of community managed github bot. Developers could submit pull request to the community service to fix common issues. Github would then run the service nightly themselves. Developers could opt-out of the service if they wanted. Something like this could be very handy for many things - security issues, typos, broken links, etc.

<comment><USER10> Github Applications exist for that. It's basically a bot you specifically opt into.

The main issue is one of discovery.

Though I imagine you could build an application to notify users of new fixer applications. Maintainers would opt into that for their accounts/repositories, it would then match repositories & applications submitted to it and ping submitters when an application looks… applicable.

> Github would then run the service nightly themselves. Developers could opt- > out of the service if they wanted.

It would be just as bad as TFA's.

</comment></comment><comment><USER11> Maybe Github should provide a checkbox somewhere in my account setting where I can generally consent to bots making PRs in my repositories.

</comment><comment><USER12> Great write up. I am a maintainer of a project you opened a PR on, but the diff is large and seems to include unrelated changes. Maybe you based of a fork and the opened the PR to upstream? If the bot had worked as intended I would have thought it was cool, but given that it opened a large PR with an incorrect description I found it harmful. I guess the problem with these bots is that it is easy for you to make a mistake which takes a lot longer for people to deal with than it took for you to make, so github needs to ban these to prevent this.

<comment><AUTHOR> Thanks for reporting this. Firstly, I am sorry for the inconvenience of receiving a broken pull request.

I have looked into the pull request and discovered that this is a variant of "Bug #4" from the blog post. It happens when the third-party renames their forked repo. At this point, the names don't line up and my bot doesn't realize that the two repos fork to the same location.

I have manually fixed my merge request for your repo and will be writing a script to look for others that might have had a similar experience.

Sorry once again.

</comment></comment><comment><USER13> I can see why this would end in suspension - you even mention yourself that when launching the bot (Even on a limited number of repos) you had bugs which messed up README's....

This sounds like a terrible idea! I wouldn't want an automated bot trying to auto-correct my work in this manner

<comment><USER14> I would actually really appreciate a bot that does that. There are so many bandges which are broken which is just annoying and many maintainers do not care about the readme once they have written it. They do accept pull requests but don't update it manually.

And since the bot is only creating pull requests, I don't see any harm: worst case for my repo, it would brake the readme but I double check it just like any other pull request, realize that it messed up, and fix it myself (but I would be thankful the bot noticed the broken link and I have a motivation to fix it).

What is a bit problematic about this bot would be that, due to a bug, it starts spamming (creating 1000 of pull requests) flagging false positives, etc).

Furthermore, it is also important where to draw the line. A bot that notices something is broken and offers me a fix is ok. A bot that notices I use a working service X and offers a pull requests to use service Y could be problematic because it might be useful but might also be annoying (because it is advertising, and service X might be good enough for me.

<comment><USER15> The issue with bots is never a particular bot. GitHub is just trying to avoid a situation where a large percentage of pull requests are coming from bots. This would drive people off of the platform even if every individual bot were reasonable and justified.

</comment></comment></comment><comment><USER16> If you still are interested in solving the badge problem, you could also register the domain pypip.in again and redirect the URLs to the other service you found.(looks like it's available)

Maybe the other service would do it if you emailed them?

<comment><AUTHOR> That's a neat idea. Looks like someone beat me to it though: [https://www.namecheap.com/domains/whois/results.aspx?domain=...](https://www.namecheap.com/domains/whois/results.aspx?domain=pypip.in)

I've reached out to them to try and see whether they want help.

[UPDATE: they're benevolent and I'll be working with them on this. Cheers]

<comment><USER17> If they are benevolent, and not going to serve spam or malware over those badges.

<comment><USER18> Well, if they are not benevolent, at least there will be a strong motivation to finally solve those broken badges then. ;)

I must say this made my day... It's one of those occurences when one uses many many many hours for a project that in the end could be solvable by a small amount of money ($10) and a few e-mails. Must admit I didn't think of it either when I was reading the blog post. :)

</comment></comment></comment></comment><comment><USER19> Removing already-merged pull requests from display seems adjacent to data integrity issues. GitHub should probably be doing more QA in this area. They've done a lot to improve their security and protect from DDOS so I'm sure they can. I hope someone there will notice it (through this comment or through the blog post) and this will be a wake up call.

</comment><comment><USER20> We let Greenkeeper update our JavaScript repos. We still have to accept and merge its pull requests. So there are ways to do this where GitHub won't shut you down (if you are Greenkeeper)...

Maybe a solution would be for someone to create an app like Greenkeeper, but which promised to start by doing five things, but to add more over time, informing you of each new thing and letting you opt out at any time: a list of checkboxes.

</comment><comment><USER21> This is exactly why GitHub will never be the monorepo (as described in Google's paper [https://cacm.acm.org/magazines/2016/7/204032-why-google- stor...](https://cacm.acm.org/magazines/2016/7/204032-why-google-stores- billions-of-lines-of-code-in-a-single-repository/fulltext)). The world needs an ability to do code repairs and refactoring at GitHub-scale and yet probably some people just blocked this person's bot for spam.

GitHub doesn't seem to have an ambition as for being the world's monorepo either, the features they've been building is not usually in line with that. I think GitHub should consider creating a team of people that think about the next 10-20 years of open source development and how THEY can carry the flag in terms of innovation in being a world-scale code repository.

</comment><comment><USER22> The guy running the show at the Appimage organization routinely runs a bot that automatically creates thousands of issues asking maintainers of random repositories to create Appimage builds, or if there already is an Appimage build it asks them to comply with good Appimage practices, or to fix paths or icons, and so on. Why can they do that with no problems while this less known but more useful bot can't? According to the description here, the bot was useful because it was fixing minor issues in an automated, easy to integrate way with no reasonable downsides to maintainers (other than having to eyeball the pull request and merge it).

<comment><USER23> Is there any information on this?

Whilst fixing problems is fine, spamming developers is not. It would be interesting to find out more about this 'bot'.

<comment><USER22> [https://github.com/probonopd](https://github.com/probonopd)

Check out the few times he has over 200 contributions in a single day, most of those are issues opened with a bot and so on.

</comment></comment></comment><comment><USER24> Honest and noble intentions to be sure. But the author should have foreseen the consequences, or at least investigated similar services to see how they behave (and what Github and users typically consider acceptable). Greenkeeper.io for example, provides a very similar service for Node.js package dependencies - but it's opt-in, as Github support was quoted as mentioning in the article.

All one needs to do is take a step back and take stock of how many real-life situations we find unsolicited anything acceptable, and the real potential for pitfall would've been clear.

</comment><comment><USER25> This is a great opportunity to remind everyone that [http://shields.io/](http://shields.io/) has been providing high quality SVG badges for all your repo metadata needs for 5 years now.

Even if Shields doesn't support the specific third-party service integration you're looking for you can generate a badge using the incredibly simple image API:

\- [https://img.shields.io/badge/hacker-news- orange.svg](https://img.shields.io/badge/hacker-news-orange.svg)

\- [https://img.shields.io/badge/rate-limited- red.svg](https://img.shields.io/badge/rate-limited-red.svg)

The API is open source: [https://github.com/badges/shields](https://github.com/badges/shields)

I started the project although it's maintained by Thaddée Tyl and Paul Melnikow these days. Here's a bit of backstory: [http://olivierlacan.com/posts/an-open-source-rage- diamond/](http://olivierlacan.com/posts/an-open-source-rage-diamond/)

</comment><comment><USER26> Did you investigate deploying the bot as a Github App?

</comment><comment><USER27> That's why you should never ever use your personal account for botting...

</comment><comment><USER28> At Google they use a system called Rosie to do something similar: [https://m.cacm.acm.org/magazines/2016/7/204032-why-google- st...](https://m.cacm.acm.org/magazines/2016/7/204032-why-google-stores- billions-of-lines-of-code-in-a-single-repository/fulltext)

</comment><comment><USER29> Would this also violate the terms of services to create an issue automatically, to explain what is broken. And include a link in the issue text to automatically create a PR via the bot? Then it would be an opt-in situation. Or is it just frowned upon to even do that?

<comment><USER30> IMHO it wouldn't make sense to allow this if PRs are off-limits, since it creates at least the same amount of noise.

</comment></comment><comment><USER31> The link to your personal site is broken, it points to [http://movermeyer.com/2018-03-08-building-bots-to-mend- badge...](http://movermeyer.com/2018-03-08-building-bots-to-mend- badges/movermeyer.com)

<comment><AUTHOR> Thanks. I've fixed it.

</comment></comment><comment><USER32> Hey Michael, your blog post isn't finished, what is the "much more complicated (and useful) bot that I was working on" supposed to be, if it's not a secret will you let me know, it's like watching a movie with no ending.

</comment><comment><USER33> I wrote my own alternative in Python when noticed pypi.in: [https://github.com/SavandBros/badge](https://github.com/SavandBros/badge)

</comment><comment><USER34> You absolutely deserve to be banned, regardless of ToS. You're wasting thousands of other people's man-hours to demand, however politely, that they fix a miniscule error that likely only bothers you. Even if you provide the fix with the request, it will still take a colossal amount of time to read and review the PRs. This is mechanised pedantry on a despicable scale.

<comment><AUTHOR> The total amount of time to review one of these pull requests can be estimated at < 30 seconds per repo. 30s * 1000 repos = 8h20m. Hardly thousands of hours.

But yes, this was a particularly trivial problem to tackle. It was meant to be a stepping stone to a truly useful bot that I was working on. However, that has been put on hold.

FWIW, the maintainers themselves gave lots of positive feedback on the project.

</comment></comment><comment><USER35> I think that the author is taking away the wrong lesson, because I think they started with some wrong ideas about communication. If you read the GitHub ToS, the relevant policy statement says "excessive automated bulk activity" not just "automated bulk activity". I bet someone complained about them, and I think it happened because they thought it would be bad to make the bot act like a person.

If your bot has output, always make your bot act like a person. That means messaging, and that means timing. Even in the best case, if your bot uses few resources and always perfectly does the right thing, people don't like bots.

> _There are four very important things that any automated message needs to do > in order to help avoid aggravating people: Be Accurate and Useful, Be Honest > /Open about being a bot, Have a mechanism for feedback, Be Friendly_

No. God no. There are two important things that you need in a PR: Don't act entitled (op did a great job there) and don't waste my time (op failed hard at this). Everything else is bad. Telling someone that your pull request is coming from a bot only hurts your goal. In the absolute best case, they treat your PR like any other. In many cases, though, knowing that a message was automated will get you instantly reported for spamming regardless of how helpful you were.

> _Automated messages should describe themselves as such._

This is off topic and therefore violates the "don't waste my time" principle. It also has a tendency to engage the gag reflex.

> _It should be the opening line._

Having multiple lines for something so small violates the "don't waste my time" principle. And definitely don't start your message with something that is off topic.

> _announcing it as automated helps explain why they are receiving the pull > request_

They are receiving the pull request because something is broken and you are fixing it for them.

> _Have a mechanism for feedback_

They can put feedback on the PR. This violates the "don't waste my time" principle.

> _I ended up settling on the following message for the pull requests:..._

Holy crapballs that's verbose. This definitely violates the "don't waste my time" principle. "Fix broken badge by pointing to working URL foo [see: bug_report_link]". Boom. Done. It's easy to read, easy to understand, and easy to approve.

> _Note that the last paragraph is only included in the message if the README > includes the "download count" badges. I debated working out a system to > delete these badges automatically_

You should have either skipped them or maybe filed an issue instead. "The download count badge in README is broken because the foo API no longer exists". Not a whole paragraph.

> _Do not make automatic unsolicited pull requests._

Most pull requests are unsolicited, and GitHub has an automation API for pull requests, and their ToS doesn't prohibit unsolicited automation, just "excessive" such, so this is probably the wrong takeaway.</comment>


Antibody finds, wipes out prostate cancer in mice - <AUTHOR> http://www.physorg.com/news181245375.html

<comment><USER2> "If I had a nickel for every time someone claimed to cure cancer in mice, I'd be a rich man."

First thing that comes to mind when reading titles like this. Mice is a model for human biomedical research, but far from a perfect one, especially in the cancer field. There are substantial differences that make it hard to extrapolate these results in mice to humans.

As a medical biotechnology grad student I read these claims often and it's a shame that most of the time it's impossible to find a reference to the original article. Hard to know than if it's an exaggeration generating false hope or a genuinely promising cure with consequences for treatments in humans.

<comment><USER3> I think that the 20th and the 21st centuries will go down in history as the best time to be a mouse on this planet.

<comment><USER4> This is both funny and sadly true -- if we tolerated as much failure in people as we did in mice who knows where medicine would be today.

</comment></comment></comment><comment><USER5> Any bio-nerds want to express opinions about this? Sounded super intriguing to me.

<comment><USER6> Recomment(?) from a reddit thread on the same story from someone who claims to work in cancer research:

 _The chances that this antibody will work in humans is EXTREMELY small.

The model that would have to be used here is injecting human prostate cancer cells into nude mice (this is a strain that is totally lacking in its own immune system so foreign cancer cells can actually survive and not be killed within minutes), allowing the tumor to grow, and then injecting the antibody or even ust the antibody fragment int the tail vein. You then palpate the tumor over the next few weeks. You'd also want to sample the tumor and maybe do some TUNEL to check for actual rates of apoptosis.

Chances are very slim that an antibody that fights this specific strain of lab grown cancer (lab grown cancers have generally been grown for so long that they are heavily mutated to say the least) will be effective in humans. Humans have many, many, many different strains of prostate cancer and unless these researchers have accidentally stumbled upon the holy grail of surface markers that somehow indicate internal genetic expression...I am skeptical.

Additionally, you've linked to an article that is so vacuous that it doesn't even feel the need to cite the actual results which it is based upon. VPNed through my department account and I can't even find the article._

[http://www.reddit.com/r/science/comments/ajcaq/an_antibody_h...](http://www.reddit.com/r/science/comments/ajcaq/an_antibody_has_been_discovered_f77_that_finds/)

<comment><USER7> Yeah, this is from a cancer researcher. As a former cancer researcher, I recognize the symptoms. ;)

All of this is basically true, though I hesitate to be so harsh without actually reading the article...</comment></comment></comment>


YC World – Explore YC Alumni by Country - <AUTHOR> http://world.ycombinator.com/ <comment><USER2> Hi everyone,

We put together a map that shows where YC companies are from. It's based on the location of the founders before applying for YC. As a founder and previous YC applicant, I know how daunting the application process can be, especially, when you are far away from San Francisco. Now that I work for YC, I wanted to help build it.

Let us know if you have any feedback

Thanks

<comment><USER3> This is great! Thanks.

Would it be possible to break down the US startups by state? It'd be great to connect with local people, if possible.

<comment><USER2> Yeah, we talked about it. We'll probably include it in a future iteration

<comment><USER4> If you do US States, could you do Canadian provinces as well?

</comment></comment></comment></comment><comment><USER5> Looks good. One suggestion if I may:

When I click on a country in map, the text listing should show the name of the country and not just the flag. Flags are great but I would rather read "China" than seeing the flag since a lot of us may not know every flag in the world.

The top 5 are :

        United States: 1062

        Canada: 62

        UK: 42

        India: 29

        France: 16

Rest are all in single digits so far.

<comment><USER6> The map really brings into light Japan's lack of presence in the tech world.. They barely register on the map.

I'm surprised India has more presence than China considering how much the news talks about places like Shenzhen.

<comment><USER7> Chinese is in general not as fluent in English as people in India (English is one of its official languages) or other European countries (easily multilingual). For most educated Chinese who don't have overseas experience, reading is probably okay, but speaking / listening is challenging -- we are trained to take exams growing up (e.g., doing multiple choices on paper).

If you can't pitch in fluent English and you can't hire someone to help you (due to the resource constraint of your early stage startup), how can you be properly evaluated by YC or other Silicon Valley VC firms?

Another possible reason may be easy to access local capital in China.

<comment><USER8> Innovation Works with Kaifu Lee (the guy who used to run google china) is like a Chinese YC. The Chinese market is also very inward facing at this point due to protectionism and the GFW.

I'm sure a lot of founders are still Chinese, just that they are coming from the USA (or other countries) before applying to YC.

</comment></comment></comment></comment><comment><USER9> Interesting map! German here, surprising to see only 3 German companies have made the list. I don't know what I expected, but 3 YC companies seems like a _really_ low number.

(Micro-nitpick: The color scale is a little bit confusing. Darker = more startups; but USA [1,062 startups] and Canada [62] both have #597b9f as color, even though there's a one-thousand-company difference)

</comment><comment><USER10> Good stuff.

I noticed an error: Flutterwave and New Incentives are supposed to be in Nigeria not Niger.

Niger is above Nigeria (the darkest country in Africa).

</comment><comment><USER11> Weird, this doesn't seem to have Stripe listed despite it being the first startup listed on ycombinator's homepage.

</comment><comment><USER12> Probably incorporation affects result by a wide margin.

E.g. our company (not YC alumni) is incorporated in Delaware and based in Mountain View, but both founders are Polish and half of the employees are in Poland. On such list, we would come up as USA company. Probably quite a few companies have the same effect.

<comment><USER2> It is based on the application, where were you located before applying?

</comment></comment><comment><USER13> Very cool to see companies mapped out.

Slight nitpick - I really wish click to drag panning and mousewheel/pinch zooming was supported on the map. Navigating the map was a bit of a chore - mostly for smaller geographically sized countries.

</comment><comment><USER14> Pretty cool. One nitpick: I wish the darkness was proportional to the number of companies. The U.S. is at 1062 and is as dark as countries with 7 or 29. I wish there were more contrast.

</comment><comment><USER15> Sadly completely unusable on mobile.

</comment><comment><USER16> The link to the company zipphone in India goes to a AliExpress link. Does anyone else see that ?

<comment><USER17> ZipPhone founder here.

For some reason my YC profile had a non-existent website listed (awaaz.mobi). I have changed it now to zipphone.com (not hosted, since the company was acquired), and changed the email to email@example.com.

</comment></comment><comment><USER18> the link of the only Polish company goes to a Chinese website

<comment><USER2> Credictive - s2012 is no longer active

</comment></comment><comment><USER19> Need one from Myanmar

</comment><comment><USER20> How comes Sixa is not listed? It is Ukrainian, S16

<comment><USER2> The list is not exhaustive. A company may not be included for several different reasons i.e they may choose to remain off the record

<comment><USER21> Would love to see Awesound listed. I see it's not in US, UK or Ireland lists. (I was in Ireland before YCF2). Thanks!

</comment></comment></comment><comment> bra-ket can't see Israel!

I think the founder of Greplin/Cue was from Jerusalem

<comment><USER2> You need to zoom in to find Israel. There are several companies from Israel</comment></comment>


Twitter looking to shed 183,000 square feet of office space - <AUTHOR> http://www.sfgate.com/realestate/article/Twitter-looking-to-shed-headquarters-office-space-9130070.php <comment><USER2> Cleaning up assets to sell?</comment>


NSA to cut system administrators by 90% to limit data access - <AUTHOR> http://www.reuters.com/article/2013/08/09/us-usa-security-nsa-leaks-idUSBRE97801020130809 <comment><USER2> Apparently, the reason for excessive surveillance isn't because of government and NSA overreach, it's because of the administrators of the systems used to do the surveillance.

Incredible! At least they'll be monitoring everyone, but doing it badly now.

</comment><comment><USER3> Wonder what percent of those system administrators given the axe are going to walk out the door with tons of information.

Any would be whistleblowers among those that are likely to be cut now have more incentive to produce smoking gun evidence.</comment>


Bad UI: macOS 10.14's Software Update Release Notes - <AUTHOR> https://daringfireball.net/2019/04/bad_ui_mojave_release_notes <comment><USER2> Fluent resizing _used_ to be non-trivial to implement but that hasn't been true for about a _decade_ so I think they're out of excuses. Especially since there is nothing else on screen requiring space. They would have almost had to go out of their way to break this!

I used to see some atrocious UIs (seemingly always in "enterprise software") that didn't make scrolling regions resize either. In great irony, these super- abbreviated lists usually appeared in screens that otherwise displayed _every conceivable thing in the app_ , cramming the screen with content but not using any of that space for what I wanted. My list was crammed into 1-3 items in a _tiny scrolling box_ with about 7 characters shown per item.

</comment><comment><USER3> How long can Apple coast on its reputation? Their hardware is terrible (laptop keyboards, out of date technology inside), their software is getting much worse (discoveryd, Marizipan apps, re-written apps with less functionality like Disk Utility, this article, just to name a few), and their services have yet to be proven.

It's lucky for Apple that most of their users have an outdated idea of what current versions of Windows and Linux are like, otherwise I think they'd be in trouble.

<comment><USER4> Come on. This is such a minor quibble and it dilutes the list of more important issues like keyboards and the T2 audio bug.

They got the most important information - restart required - passed to the user, I'm never able to see that in Windows Update on my work computer. When macOS updates delete the user's documents directory we can start comparing it to Windows.

<comment><USER3> Apple used to put effort into the details, that was the main reason their products were great. That attention to detail has been lost.

>When macOS updates delete the user's documents directory we can start comparing it to Windows.

[https://tidbits.com/2009/10/13/apple-acknowledges-guest- acco...](https://tidbits.com/2009/10/13/apple-acknowledges-guest-account-data- loss-bug/)

>When logging into the guest account, if the computer hangs, it is possible that, upon returning to your primary account, you'll find that all of the files and folders in your user account have been erased and that your account has been reset to default settings. Your account's path still exists on the hard drive, but everything has been erased from within it.

[https://www.macrumors.com/2018/02/19/apfs-bug-macos-data- los...](https://www.macrumors.com/2018/02/19/apfs-bug-macos-data-loss-disk- images/)

>Two related problems are identified by Bombich, above. The first is that the free space on the APFS-formatted sparse disk image doesn't update as it should when the free space on the underlying physical host disk is reduced. The second problem is the lack of error reports when write requests fail to dynamically grow the disk image, resulting in data being "written" into a void.

[https://macperformanceguide.com/blog/2019/20190305_1246-macO...](https://macperformanceguide.com/blog/2019/20190305_1246-macOS- AppleCoreRot-Finder-FailsToReplace.html)

>Copy a folder with dot files to another destination (I used ".icj"). Change one of the dot files. Repeat the copy. When the Finder offers Cancel or Replace, choose Replace.

>PROBLEM: the copied folder contains old data in the invisible ".icj" file.

<comment><USER5> _Apple used to put effort into the details, that was the main reason their products were great. That attention to detail has been lost._

You probably would have had a better reception of your original post if you had opened with that. Is the issue at hand a minor quibble? Yup. Would the issue at hand have shipped seven, eight years ago? Ehhh, probably not. And when I run into such things now, it sends a cold, "Windows" feel down my spine, like someone knew it was wrong but "shipping is a feature, too". (Nothing personal, Windows, you've served many of us well, but attention to detail has never been your strong suit.)

Is it the beginning of the end, where the world throws down their MBPs in disgust? Nah, don't be silly. It does, however, give the feeling that Apple might be slipping a little of late.

</comment></comment></comment></comment><comment><USER6> On 10.14.5 beta I can select and copy the text in the dialog. Still not resizable though.</comment>


Git Internal Architecture - <AUTHOR> https://dev.to/anuj_bansal_/git-internal-architecture-o1j <comment><USER2> This is fantastic. Perhaps there are other posts like it and I find these would be a much better introduction to Git coming from other another VCS than so many of the git tutorials.

Just title it "How Git (Really) Works" or something. This would have saved me so much time at the start of being lost.</comment>


Why Use Nginx? - <AUTHOR> http://wiki.nginx.org/WhyUseIt <comment><USER2> Every single project, open source or not, needs to have a "Why Use It" page.

(Now this is more of a "Testimonials" page, but for server tech it will do.)

> Apache is like Microsoft Word, it has a million options but you only need > six. Nginx does those six things, and it does five of them 50 times faster > than Apache.

This is exactly how I felt. I'm a pea-brained dolt in the server sphere, and when I was remaking my server I went with nginx over apache on the advice of a friend because "the config file is easier to understand."

He was right. And instead of being frustrated and bumbling through apache until it worked the way I wanted, I was able to configure nginx (for the first time!) in mere minutes. With nginx I was able to move on to the "get frustrated by Wordpress" phase of server setup much sooner!

<comment><USER3> > Every single project, open source or not, needs to have a "Why Use It" page.

Not only that, but a mention of all the major competitors with an honest factual comparison with them is really handy as well. In the FLOSS world, naming competitors won't necessarily cause a major problem and does a huge amount for trust.

<comment><USER4> yeah but you can't ask Product X to tell you about Product Y because they honestly don't know Product Y well enough to speak to it.

<comment><USER5> You have to ask yourself, if your building a product and you know nothing about it's potential competitors, what are you really doing? Research is key my opinion. It avoids cases of "Hey guys, look what I invented! I call it the wheel.."

</comment></comment></comment></comment><comment><USER6> One thing I still don't understand is why one would use a proxy server at all?

Why not just have your load balancers (which can operate cheaply at the TCP layer) throw traffic directly at your application servers?

If you need caching, that's cheap to do, too. If you need static file serving, can't you another load balancer end-point that points directly at static content servers, or make your application servers faster?

Is nginx primarily useful for slow application server runtimes that can't keep up with what nginx can do?

<comment><USER7> It's a good question. We (a large-scale website serving 250,000 pages/day) use Python+CherryPy for our "application server", but that's sitting behind an nginx reverse proxy.

The main reason is that nginx is much better and faster at handling certain things than Python:

* handling HTTPS and serving plain old HTTP to the application server so Python doesn't have to worry about it

* doing the gzipping of content before it goes out

* routing requests to different places/ports based on various elements matched in the URL or HTTP headers

* virtual hosts, i.e., "Host" header matching and routing things to the right place based on that

* various request sanitization, like setting client_max_body_size, ignore_invalid_headers, timeouts, etc.

Historically we've also had multiple types of application servers, some Python and some C++, and nginx routes requests to the right app server (based mainly on URL prefix).

We also use nginx to do GeoIP with the GeoIP nginx module (though arguably that would be just as simple in Python).

Edit: Note that we don't use it because our "application server is slow" (it's not). Also, I know some people use nginx to serve static content, because it's usually much faster/better than say Python at doing that -- we serve static content via Amazon S3 and a CDN, so that's a non-issue for us.

<comment><USER8> I'm curious to know - did you consider Varnish? It's much faster as a rev proxy caching server.

<comment><USER9> No, it isn't. It claims to be faster than squid, not nginx. But it doesn't even meet that claim.

</comment></comment></comment></comment><comment><USER10> A big gotcha with nginx is if you have an app server behind it and you foolishly have a long running web request which runs longer than the proxy timeout, nginx will retry the original web request. Better make sure everything is idempotent and don't have long running web processes. It is bad design, but we ran into this. Code that used to run in a few seconds started taking longer and then ran infinitely long without an error because it kept getting resent to the server.

<comment><USER11> Oh really, does it do it for POSTs? It should retry for GET, but not POSTs (which is exactly why there is a difference).

If you have non-idempotent GETs in your app, then that's the app's fault. If Nginx is retrying POSTs then it's Nginx's fault.

<comment><USER12> That's a pretty broad statement to make... You are assuming that out there only "your app" exists.

Actually, many times "your app" is somebody else's app that you bought or an app that somebody else develops and you don't have any control over it. Sometimes those apps are just bad (well, most enterprise apps are) and your only hope is that the infrastructure that you do happen to control doesn't make it worse.

I've never used nginx, so I don't know if the parent has a point or not. However, having a reverse proxy retrying _any_ requests to backends by default seems very bad form to me. Do you want your routers resending packets? It's the same thing.

<comment><USER13> > However, having a reverse proxy retrying _any_ requests to backends by > default seems very bad form to me. Do you want your routers resending > packets? It's the same thing.

No: HTTP GET is explicitly idempotent and cachable. As an HTTP client, you are supposed to be able to send the same GET all day, and it's up to the server to not screw that up.

<comment><USER12> I'm not saying that it's a standards violation or anything like that. I'm not even arguing against GET retries, but only against them being default.

Bad behavior from apps you don't control is a fact of life. Ignoring it doesn't make it go away, and behaving like it doesn't exist can make it worse.

<comment><USER13> I would argue that _relying_ on the behavior allowed by a standard is the only way to make it actually _be_ allowed. Otherwise, people code to your unwritten, stricter standard, relying on nobody else ever sending them something that _is_ allowed. Then, when something allowed by the standard _does_ happen, they blow up.

But once you introduce other software in the ecosystem that is guaranteed to send you these sorts of things, you'll damn well better release a new version of your package that works with them.

</comment></comment></comment></comment></comment></comment><comment><USER14> Though I do use nginx and am very happy with it, I am somewhat put off by the fact that I reported a bug five months ago (including a working patch for the problem) and no one seems to have as much as looked at it [1]. (Granted, this is in a module, not in the core server, but in general the community process for the project seems messy and vague.)

[1]: <http://trac.nginx.org/nginx/ticket/242>

<comment><USER15> Yes, I agree - nginx's community feels a little strange, at least to someone who hasn't tried to really get involved. Apache, by contrast, is a huge, loud, unruly crowd. Whereas the nginx author seems to be one of those quiet, aloof, l337 h4x0r types. Which is consistent with software that is a) really fast and b) not responsive to change requests. :)

</comment></comment><comment><USER16> I _think_ ngninx is likely faster and more stable than apache, but I have yet to see anything close to a trustworthy benchmark.

I come from the php world, and people always say how much lower-memory nginx+php-fpm is than apache+mod_php. Well no doubt! If you understand how the architecture actually works, it's clear this isn't a fair comparison. mod_php means php is fully-loaded for serving statics, not to mention having a smaller pool of php-fpm processes will take less memory and also be faster (due to less context-switching) than the larger number of mod_php processes.

However the real comparison should be between nginx+php-fpm and apache+mpm_event+php-fpm. Nginx is an evented server, so at least try to compare apples-to-apples. I've seen very few comparisons of nginx with mpm_event.

Also, apache's default tunings are much more geared towards modest server usage whereas nginx's seem more geared towards high scalability. An argument could be made that apache should have "better" defaults, but since at scale you need to start tweaking your OS/rlimit/etc to prevent bad things from happening you can see why apache might stick with more modest tunings ootb.

Our app has a lot of apache custom config and so I was a hesitant to try to switch to nginx due to the risk of getting things wrong porting the configs. We did move from apache/mod_php to apache+mpm_worker and php-fpm and we've been able to improve throughput (especially on statics) at a far lower memory footprint. Key to success in lowering the memory footprint was dropping ThreadStackSize (from 8M default to 1M). What a difference!

Other than that, the competition is good for everyone. I am sure nginx pushed apache to work on mpm_event much harder.

<comment><USER17> You may want to take a look at [http://www.eschrade.com/page/why-is-fastcgi-w- nginx-so-much-...](http://www.eschrade.com/page/why-is-fastcgi-w-nginx-so- much-faster-than-apache-w-mod_php/). The answer might surprise you.

As for ThreadStackSize: it impacts virtual memory but not actual memory usage. Actual memory usage stays the same. You should never use the 'vm size' as a good measurement of memory usage. Unfortunately memory management on modern OSes is complicated and people don't understand the numbers, so they arbitrarily pick a column in 'ps' and conclude that X is bloating memory... :(

<comment><USER16> That link about AllowOverride is true. The reason I didn't mention it is that one of the benchmarks I saw did turn off AllowOverride so I figured at least that part was fair :) But it is a very good point. I think I'll do a talk soon about tuning apache and make sure that's in there. Optimizing with strace is always really fun. I used it pretty heavily when I was researching php/apc and require/require_once. It's amazing how much faster you can get if you implement things to not talk to the disk 20x on every request :)

I definitely know VIRT is complicated. I couldn't find any kind of clarity on it. If you know of a good guide I'd love to see it.

That said, virtual memory still likely affects some kernel decision-making. For instance the oom-killer was kicking in on a daily basis until I made these changes. With mpm_worker using 250+ threads, I was able to reduce the "committed" by several gigs. The system overall seems more stable and the oom- killer hasn't reared its head in days.

I can imagine that the stack is treated differently since it'd be a terrible idea to page out stack. I couldn't find proof, but if I were a kernel I wouldn't page out stack :)

</comment></comment></comment><comment><USER18> I dropped Apache in favor of Nginx about two years ago. Haven't looked back since then. It's so much easier to configure and it uses far less memory.

<comment><USER19> I am using lighty for that reasons. Can anyone compare lighttpd with nginx performance on small servers like Pis?

<comment><USER17> I dropped Lighttpd years ago because on some occasions it used 100% CPU for no reason. It didn't make the server crash, and Lighttpd itself appeared to run fine otherwise, but still... the CPU usage was there for no reason. This was never solved, and development also seemed stalled. So I switched away from Lighttpd to Nginx. Nginx just kept working and working, never broke once.

</comment></comment></comment><comment><USER20> One major drawback is you can't control output buffering and gzip with phpfpm (LAMP stack equivalent) You cannot flush head, the user has to wait until the whole page loads before rendering.

<comment><USER21> Latest version got much better at handling streams if I remember correctly

<comment><USER20> True. Correct me if I am wrong but I am not sure if streams work for general text pages (like blogs, eCommerce, etc). It is better suited for chat-style applications (comet).

</comment></comment></comment><comment><USER22> I'm a newbie - if I install Passanger to be able to run Rails apps on on Nginx, are these benefits lost?

Better yet: What exact is Passanger? (Explain it like I'm five)

Their site says, "Phusion Passenger is an application server for Ruby (Rack) and Python (WSGI) apps." - so it's something that runs below Nginx and run Ruby code?

Or is it an extension for Nginx/Apache?

Thanks!

<comment><USER17> Phusion Passenger extends Nginx and turns it into an application server. An application server is a program that runs application code, so in this case it allows Nginx to run run Ruby/Python code. Likewise, the Apache version of Phusion Passenger turns Apache into an app server that can run Ruby/Python code.

The benefits are not lost. Phusion Passenger integrates into Nginx to give you the benefits of both. For example one of the tasks of Nginx is to buffer HTTP requests and responses in order to protect apps from slow HTTP connections. Phusion Passenger fully makes use of this Nginx feature and even extends it.

</comment></comment><comment><USER23> Nginx is great, but before you get down and start using it, make certain that you'll never, ever use any features it doesn't support. I was bitten by this when I found out Nginx has no equivalent to Apache's mpm_itk_module.

<comment><USER24> What does that mod do out of curiosity?

<comment><USER25> "mpm-itk allows you to run each of your vhost under a separate uid and gid—in short, the scripts and configuration files for one vhost no longer have to be readable for all the other vhosts"

\- <http://mpm-itk.sesse.net/>

<comment><USER13> I don't quite understand why that would be necessary. Nginx has no business accessing other users' files in the first place.

I mean, I understand why Apache needs to do it: with Apache, you have things like mod_php running in-process, so it makes sense to restrict Apache, running one of Bob's scripts, from accessing Alice's files.

But with Nginx, anything with "intelligence" runs out-of-process. What Nginx expects you to do is to run _it_ as one user, but run each _app server_ (in PHP terms, each FCGI socket daemon) as the user whose files that server should access. (Or, better yet, run the app server in an LXC container along with a bind mount to only the files it needs to access. Very Plan9y.)

<comment><USER17> It may still be a good idea for security reasons. Suppose that an Nginx process is exploited. If it runs under a certain user ID then the exploit likely does not travel past that user. You can make the Nginx user ID different from the actual user ID so that it only has read access.

</comment></comment></comment></comment></comment><comment><USER26> I don't care whether it's for anti-wrinkle cream or a web server, I find testimonials about as useful as that ball of lint in my belly button and trustworthy as a used car salesman.

It's true many of benchmarks out there use flawed methodologies, so lets try to fix that. When you benchmark something as complex as Apache, people are going to find faults with your initial run, no matter how careful you are. This is why you need to be completely transparent with you setup configuration and should be prepared for a followup run with user suggested settings.

</comment><comment><USER27> Can someone give me a small comparison between nginx and HAProxy? It seems like they're starting to overlap a lot. I'm really excited that nginx added Websocket (including SSL termination) support.

<comment><USER28> I don't think you can compare them

</comment></comment><comment><USER29> Nginx is the most reliable workhorse I've used in nearly any tech stack. Doesn't matter how much traffic we throw at it, it just keeps on kicking. It's very much the 'Redis' of proxy servers.

<comment><USER30> Given that nginx is almost twice as old as redis, shouldn't that analogy rather be "redis is the nginx of databases"? ;)

</comment></comment><comment><USER31> Benchs: [http://blog.inetu.net/2013/01/nginx-vs- apache%E2%80%94which-...](http://blog.inetu.net/2013/01/nginx-vs- apache%E2%80%94which-web-server-is-right-for-your-project/)

[http://blog.celingest.com/en/2013/02/25/nginx-vs-apache- in-a...](http://blog.celingest.com/en/2013/02/25/nginx-vs-apache-in-aws/)

[http://readystate4.com/2012/07/08/nginx-the-non-blocking- mod...](http://readystate4.com/2012/07/08/nginx-the-non-blocking-model-and- why-apache-sucks/)

</comment><comment><USER32> I've used Apache exclusively for the last 3 years until just a few months ago when I set up another server to host a number of sites I wanted to move off Apache and on to Nginx. After having used both, and trying real hard here not to start a religious war as is what happens often in these kinds of discussions, I have to say neither is "better" overall or in general in my experience.

If you're familiar with Apache configuration then you should have no problem with Nginx because the way both servers structure their config files is very similar. I prefer Nginx config files however because it feels more like writing JSON whereas Apache config files are like writing XML, especially in the area of virtual hosts. That said, neither is better, its really more about what you're comfortable with and prefer. Nginx had most of the same configuration options and the tough part was figuring out what Nginx calls the corresponding Apache option.

For me there was a barely noticeable performance difference with Nginx being faster. The caveat here is that in my case I started moving all of my static sites and sites with "simple" php script type apps over to Nginx and used the apache server for a very few apps that were running more memory and CPU intensive apps. The Nginx server was also new and clean while the apache server had been in use for a great many more things including non-web applications and managing private got repos for about 20 code bases.

Nginx did use about 25% less memory in my case than apache even while serving up more sites.

I love being able to host multiple SSL sites on a single IP with no hoops to jump through with Nginx. On Apache your options are to acquire more IPs or set up SNI which for me was more hassle than it was worth.

So since we're on HN I'm assuming most people are serving Ruby or Python based apps with Nginx or using it as a reverse proxy. That's cool and all but there's still an enormous cross section of the developer community using it to serve php and as I have a lot of sites that were originally built in php my Nginx server needed Php-fastcgi. I mentioned earlier that configuring Nginx was a breeze since most of my apache knowledge transferred over but setting up fastcgi was not a breeze. It's easy to get set up and working but actually understanding what it's doing and if you really do want to configure it in the way whatever online guide shows you is the tough part. On apache you'd just install mod_php5, 'a2enmod' it, and then all you need to worry about is your php.ini file. On Nginx you have the added step of adding a config block to each server block for php. That's easy enough to get the gist of but then you start wondering if you've made the right decision after you read those warnings about improperly setting it up leading to security holes with file uploads and then you start wondering what other options should I know about, should I implement them, etc. Maybe I'm totally misguided here but with mod_php you didn't worry about security. You only worried about the security of your actual code, the server itself (firewalls, ssh, port blocking and all that), and your .ini's. So that was a downside for me but not insurmountable by any means.

It's also far easier to find information on apache than Nginx. Nginx has a ton of available support and articles and tutorials out there but most of them cover the same narrow section of topics and are contradictory sometimes. The Nginx wiki itself even has warnings about getting advice from outside the wiki in the Pitfalls section. Of course you need to be careful when sourcing information from the web no matter what the topic but I felt more secure in searching for Apache information than Nginx information.

I really love Nginx though. It can really take quite a beating without even batting an eyelash as I've seen. That said, I'll still be using Apache as my "workhorse" server for some time until I can get more Nginx experience under my belt. So I'd say take these testimonials for what they are: just testimonials. True or not, any piece of software worth using can get people to rave about it. What's important is if they're situations, expectations, and needs align with your own.

<comment><USER33> > I love being able to host multiple SSL sites on a single IP with no hoops to > jump through with Nginx. On Apache your options are to acquire more IPs or > set up SNI which for me was more hassle than it was worth.

What? Neither Apache nor nginx can serve multiple SSL sites off a single IP without a UCC certificate, SNI, or multiple IPs. SSL requests have their Host header encrypted, which means the server doesn't know which SSL certificate to present until after encryption.

<comment><USER32> My mistake. I was always told Nginx can serve multiple SSL sites with no extra work required besides the usual configuration you'd change for a single SSL site.

<comment><USER34> So you said you "love being able" to do something that you have not tried (since you didn't know it was impossible you must not have tried it)? I think you should evaluate systems based on how they work for your use case, not by how they might work if you wanted to do something later.

</comment></comment></comment></comment><comment><USER35> I came across Nginx, for the first time today, when I was trying to figure how to make 'cleaner' URLs for the wiki I'm making. Totally going over my head and out of my comfort zone…what a coincidence this article pops up on HN the same day…maybe it's a sign I need to figure this Nginx thing out.

Thanks for the link!

</comment><comment><USER36> nginx has data structures that scale sub-linearly with the number of requests. this is desirable. others usually don't do things this way.

</comment><comment><USER37> Fake, these guys are corrupted and benchmarks are rigged :-P

</joke>

Long live to Nginx !

</comment><comment><USER28> Why not?

</comment><comment><USER38> yawn!

nginx's ssi capability is so bare currently. however, it is an excellent reverse proxy for me.</comment>


WebGL - A New Dimension for Browser Exploitation - <AUTHOR> http://www.contextis.com/resources/blog/webgl/

<comment><USER2> Response from Mozilla: <http://blog.jprosevear.org/2011/05/13/webgl-security>

<comment><USER3> I think their response is very good. Basically Context is claiming that there is a possibility of WebGL being used for kernel level attacks but has not shown any proof. And as for crashing users machines, my boss reported that my app crashed his computer a few times, now he's more careful to save all his work before launching my app ;p But it hasn't happened in a few months now so, I think I fixed it or his browser got updated. :)

<comment><USER4> The standard for security mitigation in architecture is now ... proof of exploit? Seriously?

This issue is very real. Anyone who has ever done significant 3D work is aware that these drivers crash machines regularly when exercised with edge cases. Those crashes are all kernel exploits waiting to happen.

<comment><USER5> Javascript, IE, and Skype are all very vulnerable to all sorts of security and privacy problems, yet people still leave them on all the time.

I'd like to see some exploits first.

<comment><USER3> That's sort of the way I see, yes gfx card drivers are terribly brittle but this is something the card manufacturers need to start looking into. We can't blame this all on the people trying to bring something awesome like WebGL which I think is crucial for the future of things like chromeOS. The browser vendor are aware of the issues and do a lot of checking (at least chrome does) of shaders.

<comment><USER5> Checking is still obviously not the answer, due to the halting problem.

</comment></comment></comment></comment></comment></comment><comment><USER6> The old 3d driver model relied on trust. If you ran a 3d program, you have probably installed it and you trust it.

3d program writers spend a lot of time fixing bugs to try and make sure their program does not crash users machines. I've been writing 3d code for over 15 years now, and have yet seen a machine that I don't accidentally crash whilst writing code against it.

This has all now changed. WebGL allows anyone to access a much greater section of the 3d hardware than before. The 3d driver writers have not accounted for this in writing their software.

Good for 3d drivers, because now there will be much wider testing with more easily repeatable bug reports. Since it is such a nice attack vector, we should also see more 3rd party testing. Also, this can only increase the amount of people using 3d - so driver writers will definitely need to spend more work on making their code safer against evil input.

I don't think it is as bad with WebGL compared to what you can do with opengl. There's less pointer passing of shared memory regions. Also, all the browsers have black lists for 3d hardware/OS versions already. So if a particular card/driver combination is found to be buggy, they can black list it easily for all users.

tl;dr it definitely is unsafe, but the sky isn't necessarily falling... yet.

</comment><comment><USER7> I can't count the number of different contexts where I've seen an OpenGL application show a patchwork of chunks of different applications mixed with crap... I wonder if a third type of exploit could send sensitive screenshots back to the server by intentionally triggering such an uninitialized memory bug.</comment>


Ask HN: Any ideas how to approach intercepting my own IoT device SSL traffic? - <AUTHOR> I have a Jibo robot. Last year its cloud servers were shut down as the parent company bankrupted, and the robot lost ability to recognize speech (all recognition was happening in the cloud). Recently I decided to turn it on and realized that it suddenly started working again. No news from new owners. Perhaps it is nothing to worry about, but nevertheless I wanted to look into what it is sending and where. I was able to find out domains it uses - these looks legitimate, but I can't figure out how to peek into HTTPS traffic there. I already tried self-signed certificate in hopes it will accept it, no luck.<p>Any ideas if there are some vulnerabilities available from up to 2015-2017 in common SSL stack? I would expect it to run Linux, and probably with openssl, but I have no clue which version. <comment><USER2> Look for Heartbleed:

[https://heartbleed.com/](https://heartbleed.com/)

<comment><AUTHOR> Did not realize it can be exploited against clients as well. Thanks! Unfortunately, looks like it is not vulnerable to it. Perhaps at some point they upgraded libraries.</comment></comment>


I'm Betting My Startup on Windows 8, Here's Why - <AUTHOR> http://www.tarekayna.com/post/34452526773/bettingonwindows8

<comment><USER2> To be fair I don't see corporate America throwing out their PCs running Windows any time soon. Another thing with Microsoft is that they're brilliant at being the underdog, and add to that they tend to get it right after a few versions (Windows 3 anybody?). I also wouldn't bet on Google, Samsung and Apple to always get it right — so I wouldn't write off Microsoft just yet. Lastly one thing that Microsoft does well: They value their developers.

</comment><comment><USER3> "Windows 7 sold more 600 million copies in under three years. That is, on average, over 200 million additional potential customers every year. No other platform offer this."

This is wrong. Android should ship to 500 million _new_ devices in the next 12 months alone: [http://www.phonearena.com/news/500-million-Android- devices-a...](http://www.phonearena.com/news/500-million-Android-devices- activated-around-the-world_id34344) If you are a developer who wants to focus on the fastest growing platform, Android is the one, not Windows 8.

<comment><USER4> However, getting your app noticed on Android is way more difficult than on Windows 8.

<comment><USER5> But android is notorious for piracy, what would I do with 500 Mil. clients who - by a great %, pirate apps and do not pay?

I'd go the iOS way hoping to make some money back to stay alive, and maybe make it to the top 100 one shiny day.

My 2 cents and I mean NO disrespect to Android, Android users or devs.

Peace all.</comment></comment></comment>


I/O Thoughts - <AUTHOR> http://rustyshelf.org/2014/07/01/io-thoughts/ <comment><USER2> One of the things that I've always seen as an interesting difference between how Google and Apple run their developer relations teams is the title that they give their employees. Apple has _Technology Evangelists_ , suggesting that they promote the use of certain technologies (typically Apple's own technology or Apple-supported web technology). Google, on the other hand, has _Developer Advocates_ \- these are people who actually represent non-Googler developers internally at Google.

While they may just be titles, my experience has been consistent with their titles. Apple representatives you see once a year at WWDC, while Google representatives you see at Google I/O and also at a number of Google-hosted events year-round, including the regular videos they put out (and the "office- hours" type Hangouts they have so that you can actually talk to them and get your problems resolved) - just look at all of the events they have scheduled on the Google Developer Events page[0] or all of the videos they have on their YouTube channel[1]. I've also seen a lot of them give talks at tech conferences, while Apple employees are rarely seen at conferences. And you can see that Google's Developer Advocates really do talk directly to the actual teams directly and file bugs against them when necessary on behalf of developers everywhere. It's a system that has worked pretty well in my experience.

I will also agree that Google's Developer Advocates don't care too much about what platform you're using and just want to help you. I've been to a number of Google-run technical workshops and codelabs, and the people there are willing to help you with whatever problems you run into, no matter what it is.

[0]: [https://developers.google.com/events/](https://developers.google.com/events/)

[1]: [https://www.youtube.com/user/GoogleDevelopers](https://www.youtube.com/user/GoogleDevelopers)

<comment><USER3> The difference between a developer advocate and a developer evangelist is inbound vs outbound.

Developer advocates are focused on helping address developers' problems. Developer evangelists are focused on increasing product use and adoption.

That said, the responsibility of these positions mostly overlap. In my personal experience as a developer evangelist at Marketo, my day-to-day work has encompassed the function of both titles. My goal is to create a great developer experience [0] in order to drive platform adoption [1].

Anecdotally, I have noticed a shift towards the developer advocate title. At Facebook, Dropbox, and Twitter, the role is called this.

[0] [http://murtza.org/creating-a-great-developer- experience/](http://murtza.org/creating-a-great-developer-experience/)

[1] [http://murtza.org/driving-the-api-adoption- funnel/](http://murtza.org/driving-the-api-adoption-funnel/)

<comment><USER4> Which category Microsoft will sit on? _Evangelist_ or _Advocates_?

<comment><USER5> Definitely evangelist, promoting the company product.

</comment></comment></comment></comment><comment><USER6> > _ever since we've had success on the Android platform [Our Apple developer > rep has] made it very clear that his services are no longer available to > us._

Wow. That is shocking to me - is that typical of Apple? I know they're very much about having people their ecosystem, but I didn't realize that there was this kind of animosity towards people who stepped out of bounds.

<comment><USER7> > Wow. That is shocking to me - is that typical of Apple?

Yes it is. We have a pretty successful app in the iTunes store and as a consequence, we have privileged access to the entire team, including engineers and executives who advise us how to word the description of our app and telling us all kinds of tricks to increase downloads.

A few months ago, we released an Android app and since then, we have fallen into a complete Apple black hole. The Apple people we used to exchange daily emails with are not even responding to us any more.

Apple is very, very serious about exclusivity.

And probably feeling very, very threatened too.

<comment><USER6> That's...well, bizarre seems too kind. As an outsider who hasn't ever dealt with Apple in that kind of capacity, it seems downright childish.

I'm not sure that I was in danger of developing iOS apps any time soon, but these kinds of tales make me want to run screaming.

<comment><USER8> How is it childish ? It's business. Apple has finite resources and there are thousands of amazing, exclusive apps that don't get any direct assistance. Why would they invest their time if it doesn't benefit their ecosystem.

Also as someone who has worked on some of the most popular apps there is always the problem of feature equality. Companies don't want to annoy their customers by only offering features on one platform. So they don't. In which case you end up with a lowest common denominator of features.

<comment><USER7> > Why would they invest their time if it doesn't benefit their ecosystem.

The problem with this approach is that it only works if you're #1 with a comfortable lead. Apple is #2 and fast losing mind share, and this kind of policy is only going to accelerate that trend.

<comment><USER9> It's #1 in the market it cares about, which is "people who spend money". And that doesn't appear to be changing anytime soon.

<comment><USER5> It's #1 in the market it cares about, which is "people who spend money". And that doesn't appear to be changing anytime soon.

That sounds like hubris and arrogance to me. Are you so sure that won't be changing anytime soon?

<comment><USER9> No, I'm not sure, it's just a guess. What do you think?

<comment><USER6> The data from the last 8 quarters or so shows that Apple's app store revenue lead over Google is shrinking rapidly. Given Google's rapid gains in both marketshare and app store revenues, it isn't crazy to think that Apple could be the underdog before too long.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> "...is that it highlighted just how insular and superior a lot of Apple developers act and feel. If you don't believe me, just join a group of them at WWDC and whip out your Android phone. Within moments, you'll wish you had whipped out something less offensive, like your genitalia instead."

Can relate to this a little bit...A coworker who is completely in the Apple camp argued fiercely how consistent the iOS keyboard was when I mentioned to him that the Android one changes case to match the Caps lock setting.

<comment><USER11> As a die-hard iOS user... that was a stupid argument to be in. There is a reason Apple does what it does (supposedly to match a real keyboard) but it's not strong and doesn't matter.

And the iOS 7 shift key symbol stupidity should automatically lose him the argument.

<comment><USER12> > There is a reason Apple does what it does (supposedly to match a real > keyboard) but it's not strong and doesn't matter.

Wow. Really? That a silly reason for an otherwise indefensible UI decision. I really wish the iOS keyboard changed case. Ever since iOS 7, I can't tell whether shift is on or off. Maybe I'm just an idiot, but sometimes I fumble with it a half dozen times before getting it right. It's worse than trying to plug in a USB cable.

<comment><USER11> I see nothing wrong with only showing uppercase letters on the keyboard as they seem easier to read.

The major flailing is the shift key. Right now it's white with a black arrow. That means the sentence started with an initial cap. At any other theme, say mid sentence, it's different again. Mid sentence it's a white arrow on darker gray.

So, in short: about to type cap? Black on White. Mid sentence? white on dark grey. Want to do all caps? Double tap it and get a new version.

I'm not sure how these semantics are different enough from iOS 6 to cause confusion but it does ALL THE TIME.

<comment><USER13> They're not easier to read

[http://en.wikipedia.org/wiki/All_caps](http://en.wikipedia.org/wiki/All_caps)

<comment><USER14> Are you sure that prose and single-character button labels have the same readability characteristics?

The same property that makes "BIG TEXT" more difficult to read than "small text" might make each individual character _easier_ to read. I'm not sure that it does, I just don't think your wikipedia page discusses this at all.

</comment></comment></comment></comment></comment></comment><comment><USER15> > Android is now clearly their platform of choice, it runs on TVs, cars, > phones, tablets, watches and in your home. The same OS, different screens > was their message.

This was one of the big surprising take aways for me. The minute Pichai took over I had the feeling that Android's future was limited - they would keep investing in it for sure but it would be on a long term road map for convergence and eventual transition to ChromeOS. I expected to first see Chrome apps running as first class citizens on Android, then a new SDK for Android that was all about ChromeOS and then finally a deprecating of the Java SDK. It always felt to me like internally Google preferred ChromeOS and the web as a platform and Android was a sort of accidental success that they were just running with until they could somehow meld it into their pure vision of computing, as represented by ChromeOS.

So now I'm having to significantly revisit all that thinking, as if anything Google seems to be pushing Android everywhere and little was said about ChromeOS.

<comment><USER16> I got the distinct impression Sundar has changed his tune now he's not threatened by Android, but instead runs it.

Anyone that had thrown their lot in with Chrome Apps or similar probably felt this year how Android devs did last year.

<comment><USER17> This is not some single person dictatorship where Sundar can just call all the shots. They are so many people vested in Android at Google. I don't think Sundar has a choice.

I really dislike how everyone thinks the too management can sweepingly do anything they want. It does not work like that.

</comment></comment></comment><comment><USER18> Not directly related to his post, but if you haven't tried Pocket Casts yet, I highly recommend it. Best podcast app I've ever used, hands down.

<comment><USER19> Do however be wary if you use it in a car. I've found it quite flaky. That includes randomly stopping for no apparent reason (on content that was fully downloaded), random crashes, forgetting things (like it was playing a list of episodes not just one), starting to play 2 hours after I pressed play, forgetting position and the list goes on.

They were responsive the first time I contacted them (before buying) but ignored my emails after I had bought and found issues like the above.

If you aren't in a car then you can resolve most of those issues by pressing the right buttons in the app. However while driving you can't, hence it mattering a lot more.

<comment><USER20> Agree 100%. Also, it has serious streaming problems. Don't bother starting a podcast unless it's 100% downloaded.

</comment></comment></comment><comment><USER21> "Having met a lot of Apple employees, I can't say the same is true on the other side. They seem overly obsessed with Google, and are insanely sensitive should you bring the topic up, or deity of choice forbid, you actually develop on a Google platform."

It used to be exactly like that in Microsoft around 2010. Microsofties were extremely sensitive about non-WP devices other employees used. Hopefully things have changed there since then.

<comment><USER22> I was trying to enjoy the company of a friend and his friend turned out to be an Apple executive and had a lot of passive things to say about the fact that I had an android phone. the gist was it was too big, not many apps, and other nitpicking stuff. I couldn't help but think... well gee just try another smaller android phone, but I wanted to just eat dinner and ignored his snide comments. They're definitely obsessed from the top down.

IMO, they need to stop whining and give users what they want, a cheaper phone. That could have stopped android years ago, instead they just sued and became indignant. As if that helps. You want to talk about a bubble. I fear the day when apple's revenue drops. I sort of want them to stop bitching and do something, because that day won't be pretty.

</comment></comment><comment><USER23> > None of them care what phone you use, what laptop you choose or which > platforms you develop for.

Benedict Evans has a very interesting take on why Google has this sentiment. Basically, it comes down to Apple being about hardware and considering the cloud dumb storage and Google being about cloud services and the hardware being dumb glass (not Google Glass, but hardware interfaces). It seems like Google as a company is in sync, all they want to do is get their cloud services on as many devices as possible because the real value of Google is in the cloud. I agree with Benedict's hypothesis and I can't wait to see how these two companies unfold over the next decade. They currently seem to have two different trajectories.

</comment><comment><USER24> meh...I've been in the "Apple Camp" since they released the SDK in 2008 and I've yet to work with anyone who would find it offensive that someone has different tastes than them when it comes to their device choices. it really doesnt matter that much.

That said I'll probably be looking to do more non-iOS work as well in the near future, I need to expand my horizons again.

</comment><comment><USER25> You have to remember, Apple is selling all Apple stuff. Google is selling the web.

<comment><USER26> Google is actually selling the Ads, IMHO.

</comment></comment><comment><USER27> Late one night I began watching the room sessions, and after I kinda wrote off the Keynote I began to see the same level of organization the writer spoke about. Unification across all applications, platforms, screen sizes, and use cases was a huge priority and that is a rather radical shift from what I've seen. They have focus.

</comment><comment><USER28> How sad it is we are so easily seduced to take sides on a false dichotomy. How easily we forget our work becomes just a cog on someone else's platform. Relations centralized, us with the big player, very one-sided.

</comment><comment><USER29> I agree with everything you wrote -- thanks for taking the time to put it to paper! Two other things: I've been to five I/Os and this year's keynote was absolutely the worst in terms of quality of presentation. It honestly seems like they didn't plan to announce L until it was leaked a couple weeks ago. I'm glad they did and I think, given the lack of prep, it went pretty darn well. Secondly, before they started charging $900/ticket, I'm absolutely sure a lot of locals applied just for swag. I don't think that happens anymore, and you can tell from what they handed out this year that there will be no more free phones and tablets. If there is hardware swag, it will be _new_ tech, not the stuff everyone already has.

Additionally, I was truly surprised at the heavy focus on Android at the expense of nearly everything else. I think with Sundar in charge we're seeing a sea change at Google in terms of platform focus, and this is -- while a little confusing sometimes -- a good thing. It also means I/O is turning into truly a hardcore developer conference with a unified theme (or several big themes), rather than a sort of dog & pony show ("Hey, look at this shiny, useless crap we've been working on!") like in the past (Nexus Q anyone? Android @Home? Google TV?). There was no mention this year of anything besides Android platform products & services, except the expected general purpose dev platform stuff (Compute Engine, App Engine, etc). I was surprised at the lack of mention of anything enterprisey, nothing about Google+ or identity management, anything about ChromeOS (just a small plug in the keynote about schools adopting Chromebooks), or any of their X projects. I at least thought they'd have some robots there to ogle.

If the keynote was subpar this year, the 'box talks were a highlight. The speakers were almost universally awesome, the open fora allowed high quality engagement, and if one topic didn't interest, it was easy to stroll to the next whiteboard/screen and see if that one did. Jake Archibald's talk on browser quirks was my favorite.

[http://www.html5rocks.com/en/tutorials/speed/script- loading/](http://www.html5rocks.com/en/tutorials/speed/script-loading/)

[http://vimeo.com/77497239](http://vimeo.com/77497239)

Btw, I was fully expecting Android Wear to be a novelty I had no use for, but I very quickly came around and already find myself talking to my watch, James Bond style. I completely agree with this Ars post: [http://arstechnica.com/gadgets/2014/06/android-wear- smartwat...](http://arstechnica.com/gadgets/2014/06/android-wear-smartwatches- make-google-glass-obsolete/). My wife & I have a "no phones at the dinner table" policy but even my kids have been telling me to "ask Google" for answers to questions. Having the knowledge graph so handy is awesome.

</comment><comment><USER30> Are google glass completely dead now?

</comment><comment><USER31> Could we please not appropriate the generic term I/O for the gimmicky marketing travesty that is Google I/O? Some people expect an insightful article about system programming.

</comment><comment><USER32> At the end of the day, Apple has always delivered far superior tools, frameworks, and ecosystem (development-wise). That to me says a lot more than how friendly the staff is.

<comment><USER33> >>Apple has always delivered far superior tools, frameworks, and ecosystem

Its easier to make good tools when they need to work on 5% market share (Mac only)?

Google Tools are free. Apple tools start with you buying their $2000 machine.

<comment><USER34> You can get a Mac Mini that more than suffices for around $600-$800. But honestly, if paying $2000 was the only way to get a set of Android SDKs and development tools of the same caliber as Apple's developer offerings, I would immediately do so. I like the idea of the platform and I believe in anything that makes it easier for developers to build better applications.

<comment><USER35> Excellent. Now I've bought my mac mini, I can plug it in and listen to it consume power. I wonder how I can see what it's doing?

Hrm... well, it can come with a monitor... for an extra thousand. Okay, scratch that, I'll get a cheap DisplayPort monitor... add a couple of hundred (or if I feel like splurging, get two). Oh, and a keyboard, another $50 (apple-branded, of course, because normal keyboards don't have apple keys). Mouse? Apple mouse at $50+? Maybe I'll go with a cheap logitech or something... 4GB memory standard? for a dev machine? OSX has trouble with that little memory. Up to 8GB for another hundred (or 16GB for three hundred). Spinning rust will do, but if you did want an SSD, throw on a couple hundred more.

The idea that mac minis are 'cheap development boxes' is nonsense. It's cheaper than a decent macbook pro, but it's not a cheap workstation.

<comment><USER34> You can buy the Apple-branded stuff. Or you can buy a KVM switch, a SSD, and two sticks of RAM off Newegg yourself. Monoprice has HDMI to DVI converters for less than $3. Upgrading the internals is about as challenging as upgrading the internals of a non-Apple laptop. It's honestly not that hard.

On the off chance you don't have a mouse, a keyboard, or a display, you can get a decent mouse for around $5, a decent non-mechanical keyboard for $30, and a high-quality display for $150. All three of these will improve your productivity on your laptop if you're not traveling. If you don't have a laptop or a desktop...how are you developing for Android to begin with?

<comment><USER35> _how are you developing for Android to begin with?_

Well, this requirement wasn't in the original brief. And in any case, you're trading your own labour and domain knowledge to shave a few dollars off. The scrounging you've mentioned is only a little cheaper than what I've mentioned - my point is that it costs more to set up a mac mini for a developer (rather than a headless server) than the throwaway line presented. Even with your cheaper, more-effort-given-to-scrounging pricing, the monitor + HIDs alone add 1/3 to 1/4 the price, never mind the ram or ssd.

</comment></comment></comment></comment></comment></comment><comment><USER36> Great piece, very interesting.

> Overall they just seem more balanced, you can tell that what Apple is doing > doesn't keep them up at night.

Eh, I have found the opposite, whereas Android developers will be very hardcore, almost akin to .Net developers who only use bing. They will only use Android, and Google, and Open Source, and never pay for anything ever. Even apps. And beware if you have an iPhone.

Not all, and maybe I am sheltered from the Apple critics because I leave my Android phone and table at home. And it really depends who you meet.

I certainly see Apple as a very closed company compared to Google, but it looks like that is changing with the WWDC 3,000 API's announcement, really opening up the OS to developers. iOS devs can start to do cool things that Android devs have been doing for a while.

As for the Google anouncements, is TV any different than the last few years? And they still don't seem to have wearables right yet.

<comment><USER37> I read the sentence you quote as referring directly to the engineers who work at google, not android developers.</comment></comment>


Show HN: Golang queue abstraction, working redis impl., amqp on the way - <AUTHOR> https://github.com/phonkee/ergoq

<comment><AUTHOR> I needed this for one of my sideprojects. Just simple queue (with pubsub) where driver can be chosen by dsn. API should be almost stable and I am working on improvements (drivers). Also the test coverage should be high in master branch. Any comments?</comment>


Show HN: Create your own 3D printable phone case - <AUTHOR> http://zetoff.com

<comment><USER2> Please don't do astroturfing and promotional voting on Hacker News.

<comment><AUTHOR> Sorry about that we've deleted the comments made by ourselves.

Is there anything else we should avoid doing in the future when submitting to hn?

<comment><USER2> > Is there anything else we should avoid doing

The main other thing is that we want people to vote for stories because they find them interesting, not because they or their friends are promoting something.

A lot of work has gone into making the HN software good at distinguishing promotional votes a.k.a. voting rings. If a submission gets many such votes, it is likely to be penalized, as are the accounts doing the voting and (in repeat cases) the site being promoted.</comment></comment></comment>


Ask HN: Switching a business to PHP - <AUTHOR>

I'm currently negotiating with a potential co-founder. He's a senior PHP dev with incredible experience and a long list of solid experience.
My fear (unfounded or not) is that by rebuilding the current site in php it will pose hiring challenges down the road. It feels like most younger developers are avoiding php.
Is this a legitimate concern?
I'm not looking for a flame war here so please refrain leave those comments for somewhere else. <comment><USER2> I know this doesn't answer your question specifically, but if you have a site and a product already launched, and if it is working, why change to just to accommodate a single person. Not to knock him or his skills, but that seems like spinning wheels when you could be out selling and improving.

Now if the site is not developed and you do not have a product, I would say do whatever you are most comfortable in to get it launched. There will always be the next great language out there and to second guess yourself now is just wasting valuable time. PHP, while it has its moments, is not a bad language to build a product with. There are plenty of big name sites that got their start as PHP or are still built on PHP. I highly doubt that in the next few years that PHP developers are going to be a thing of the past.

<comment><AUTHOR> I've been developing for about 18mths and built the MVP in Django. The site has decent traction and user feedback has been incredible. But, in order to get the product to the next level I need to take a backseat when it comes to the development and focus on marketing (my primary skill set).

Migrating the site to php would be trivial, the codebase is pretty basic. The technical requirements of V2 of the product will far exceed my abilities as a developer.

He also has extensive CTO experience and just all around is the kind of developer that can take the company to the next level...

</comment></comment><comment><USER3> PHP is going through a technical renaissance and does not deserve (some of) the bad rep it has: [http://radar.oreilly.com/2014/03/the-new- php.html](http://radar.oreilly.com/2014/03/the-new-php.html)

The community is huge and there are plenty of mature software and components to use. I would not be shy of using PHP and wouldn't worry about hiring challenges in the next few years.

That said, I would not hang my business on a single technology. Building with a combination of PHP and Node.js should be a good bet as these compliment each other and will provide developers chances to learn new things.

<comment><AUTHOR> Interesting, I know next to nothing about the php community. Appreciate your advice!

</comment></comment><comment><USER4> I can't talk for others but according to my experience I found that getting high quality Python/Django developers is much easier than PHP. Someone would expect the situation to be the other way around due to the larger PHP pool, but it's not.

<comment><AUTHOR> Funny you say that because I built the current site in Django. This is definitely my fear!</comment></comment>