Show HN: A REST API for Calculating VAT in the EU - <AUTHOR> https://vatmossapi.com <comment><USER2> The EU administration itself or the national tax authorities in the EU should have provided an official service like this to go with the VAT changes for digital sales from day one.

I appreciate someone else stepping up and doing it, though I'd want to see more information about the source(s) they were using for the data and how frequently they were checked before I'd hook a real business up to a third party service like this.

<comment><USER3> They did, it's called the VATMOSS portal: [https://www.ros.ie/vatmoss- web/vatmoss.html](https://www.ros.ie/vatmoss-web/vatmoss.html)

The problem is that it is a pain to use and too labor intensive to be practical.

<comment><USER2> I'm painfully aware of the EU VAT MOSS system and how it "works"[1], but as far as I can see, that doesn't explain the source for the near-real-time tax rate data being offered here.

[1] Obviously it doesn't really work very well at all, and multiple national tax authorities have already been caught sending unjustified threats to many businesses for money they never owed, among other screw-ups.

<comment><USER3> This is just a lookup table. An actual service should go a lot further to be useful. Note how it does not matter what country the seller is in so that parameter can be ignored. What remains is a - hopefully always up-to-date - lookup table that converts a country code to the highest VAT Rate.

That's hardly worth an API.

<comment><USER2> It's the "hopefully always up-to-date" that is my concern.

There are, at minimum, 28 different rates to monitor here. They can be changed arbitrarily by their national governments, and in at least one case already such a change has been made with just a week's notice.

Even if you're only checking those rates against your existing look-up table manually, the effort to do so at least every week is going to add up.

There should have been an authoritative, machine-readable source for this data from day one, and vendors should have been able to rely on that data and protected from consequences if the authorities provided incorrect information and vendors acted on the incorrect information in good faith.

</comment></comment></comment></comment></comment><comment><USER4> How about different rates for different goods / services? This looks like a gross oversimplification. Doing VAT correctly is quite hard - there is a reason why there are dedicated VAT specialists working in larger consulting firms.

<comment><AUTHOR> It's only for digital goods/services.

<comment><USER4> Ok, sorry, now I see it :)

</comment></comment></comment><comment><USER5> The barrier of running an EU B2C business is so high if you need to sell all over EU that most of the time it doesn't even make sense. It's a shame because if you're in the EU and you want to bootstrap a small B2C business you have no choice other than sticking to the country you're in.

<comment><USER6> Or you just outsource it to someone like Avangate.

<comment><USER5> I don't know how their service works. Do they pose as a 'gateway company' between you and your customers or are they just a payment gateway? If it's the latter, then the hardest problems still stand since accounting and invoicing is still something that you have to do.

It would be great to have a service that resells your goods/services and takes care of all that stuff so you can send a single invoice to them every month and that's it, but frankly, I'm not sure of the legal part of that.

<comment><USER6> They are buying the product from you and selling it on to the customer. So you only have one customer to deal with and you just need to invoice them once a month.

</comment></comment></comment></comment><comment><USER7> Is there any reason to ask for the country of the business/seller? As far as I know it's irrelevant and the VAT only depends on the buyer's country (for digital goods/services).

<comment><USER8> You need it for sure. Example:

I have my business in Germany. If I write an invoice to a german company I add 19% VAT. If I sell to a company inside the EU but not in Germany I add 0%, since it's a reverse charge.

That means that the company in the EU country directly pays the taxes to their country and I don't have to pay it to my country. This solves a lot of bureaucracy hassle.

Exception is where I am selling to no business, but a private person within the EU, I will charge VAT there and pay it to my country which redirects it to the EU country (that's called MOSS).

<comment><USER7> I'm aware of that. However since the service in question only seems to handle the business to consumer case, I don't see a need for asking for seller's country.

</comment></comment></comment><comment><USER9> While the effort in general should be appreciated (especially open sourcing the code!), I'm doubtful that the REST API has practical use.

"I calculated the VAT for the products I'm selling using that one website" is just too much of a liability risk for a business to take. Who do you hold liable in the event that the calculation was wrong?

<comment><AUTHOR> The data for the API comes from the most widely used open source library on the subject [1]. It has 418 stars, is used by lots of people and is continually receiving pull requests / input from users. I bet you that it is not easy to find another entity (be it a person, an organisation or a piece of software) that knows as much about the problem.

Of course, you are right about liability. You can sue your accountant. But I assume it will be more difficult for an open source project.

[1]: [https://github.com/mpociot/vat- calculator](https://github.com/mpociot/vat-calculator)

<comment><USER9> That's the problem: even with top-notch data, unless you continuously audit the source code, you're still blindly trusting a third-party website regarding tax advice. In the event that something goes wrong, that could be construed as negligent.

Hence why I appreciate that yours and the other project are open source! Because even though I might not be able to use the REST API, I can certainly download the code and integrate locally after auditing it, or having it audited by my accountant, or tax attorney, or whatever.

</comment></comment></comment><comment><USER10> Something similar, but not the same: [http://ec.europa.eu/taxation_customs/vies/technicalInformati...](http://ec.europa.eu/taxation_customs/vies/technicalInformation.html)

Official Europe VAT Verification Service endpoint ( docs )

Edit: for checking the VAT Id

<comment><AUTHOR> You mean VAT _ID_ verification service. That's needed for when you're selling to _businesses_ in the EU. Then, the VAT you have to charge is 0% and the businesses have to pay it themselves when they buy from you.

</comment></comment><comment><USER11> That won't work in Ireland. While 23% is the headline rate, it's not the only rate: [https://www.taxworld.ie/vat/summary/vat- rates/](https://www.taxworld.ie/vat/summary/vat-rates/)

<comment><USER2> There are variable rates almost everywhere, but this API is specifically about digital sales (where the MOSS system applies) so it will typically be that headline rate that you need anyway.

Of course, there's no guarantee that that will always be the case, and to know for sure you need to understand the tax systems of 28 different EU member states. This is (one of many reasons) why the new EU VAT rules have been so controversial: the burden for reliable, full compliance is ridiculous.

<comment><USER11> Sure, the headline rate is applicable for most things, but not everything, and those differences can be important. For instance, if Amazon are selling children's shoes into Ireland, they _must_ charge 0% VAT. That's still a digital sale.

<comment><USER2> That's a sale of goods, though, which isn't currently covered by the VAT rules and MOSS scheme that we're discussing here, and doesn't seem to be the intended use for the OP's API.

<comment><USER11> That still makes it rather useless as an API, when it could be reduced down to a conditional GET on a CSV file listing the countries and their headline VAT rates.

For this to be useful as an API, it would need to deal with the actual complicated bits of VATMOSS, which is dealing with multiple different rates for different product types.

</comment></comment></comment></comment></comment><comment><USER12> I was looking into this recently and found this for Elixir, it looks good: [https://github.com/xavier/european_vat](https://github.com/xavier/european_vat)

</comment><comment><USER13> You should specify a bit more clearly (it's on the bottom though) that this is for B2C. B2B different rules apply.

</comment><comment><USER14> While this is a great way for you to collect internal business numbers from other companies, I don't quite see how the benefit for those companies is so large that it warrants sending internal information to you.

By the way, do you plan to provide statistical summaries of your findings? Such as: How many companies from country X works with what set of countries Y? How many invoices do they (approximately) file to which country?

EDIT: Fixed typo.

<comment><AUTHOR> No bookkeeping numbers are being sent at all? Unless you count a pair of countries and a post code (without any amounts or anything!!) as bookkeeping.

<comment><USER14> Sorry, this was a typo. I meant business information, not bookkeeping information.

 _> Unless you count a pair of countries and a post code_

It is more than that. You also have the source IP address, which you can map not only to countries, but to relatively small regions and sometimes to individual companies.

If this service was meant to be "datensparsam", it would provide an updated table or dataset, and not require a single request for every small bit of information.

<comment><AUTHOR> Oh come on, how is that sensitive. Use a proxy if you're really worried about this.

<comment><USER14> Not really worried, more curious how the collected datasets will be analyzed.

<comment><AUTHOR> Nothing, honestly. That data on its own has no value to me.

<comment><USER3> Then you should say so on the page or in a separate privacy statement. Transparency about such details is very important in this space.

<comment><AUTHOR> I'll add it, thanks. This was just a weekend project.

<comment><USER3> You may also want to drop fields that you don't actually use in the lookup.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER15> In Greece, in some island in the Aegean Sea, the VAT rate stayed at 17% for 2017 (and did not rise to 24%).

</comment><comment><USER3> This does not make the distinction between consumers and corporate (VAT registered) customers nor does it make a distinction between services and goods (and various classes of goods, to make things even more complicated). Which makes me suspect this is a very broken implementation and you should not rely on it for your business needs.

<comment><USER2> Given that it's specifically for VAT MOSS, presumably it's specifically intended for B2C sales of digital products/services where the MOSS-related place of supply rules apply?

<comment><USER3> Yes, but the customer isn't always a consumer, so in that case you get a lot of extra burden: pull in the VAT number of the consumer, verify it and if it is correct make the sale without VAT, otherwise make the sale with VAT.

If you don't do that properly you're either leaving money on the table or you could be later charged with the difference. Note that under VATMOSS any of the tax adminstrators in the EU has the right to audit you so it really pays off to do this well.

And it's the 'place of consumption' rather than the 'place of supply'.

<comment><USER2> _Yes, but the customer isn 't always a consumer, so in that case you get a lot of extra burden: pull in the VAT number of the consumer, verify it and if it is correct make the sale without VAT, otherwise make the sale with VAT._

Yes, but such sales are outside the scope of the VAT MOSS scheme.

 _Note that under VATMOSS any of the tax adminstrators in the EU has the right to audit you so it really pays off to do this well._

We're well aware of this. We spent a great deal of time trying to implement a fully compliant system when the new rules came into effect, and we were still among the first businesses to get one of the fake demands for money never owed because apparently with our vast resources as a little family business we were better at getting it right than several national tax authorities with the resources of an entire government behind them. If that isn't a perfect illustration of the impracticality and excessive burden of the current system, I don't know what is.

 _And it 's the 'place of consumption' rather than the 'place of supply'._

No, for digital services and private consumers, it's the place of supply rules that changed:

[https://www.gov.uk/government/publications/vat-supplying- dig...](https://www.gov.uk/government/publications/vat-supplying-digital- services-to-private-consumers/vat-businesses-supplying-digital-services-to- private-consumers)

Of course the practical effect of the changes is typically that the place of supply is now treated as the customer's location where previously it was probably the business's location, so perhaps that's what you meant and we're really talking about the same thing here.

<comment><USER3> > Yes, but such sales are outside the scope of the VAT MOSS scheme.

So why ask for the country of the seller in the API?

> Of course the practical effect of the changes is typically that the place of > supply is now treated as the customer's location where previously it was > probably the business's location, so perhaps that's what you meant and we're > really talking about the same thing here.

Yes, that's what I meant.

BTW that 'demand for money' also happened to me, it appears that in the first few months of VATMOSS there was a massive cock-up at the Irish department tasked with this leading to a whole pile of nastygrams sent to lots of EU businesses, they never even apologized for that (or sent any follow up saying that they were in error).

<comment><USER2> _So why ask for the country of the seller in the API?_

I wondered the same thing. I'm not aware of any circumstances under which it would make a difference that are still affected by MOSS.

 _BTW that 'demand for money' also happened to me_

My sympathies. The Irish department reportedly wasn't the only one to mess up like this, either, though it's the only one I've encountered personally so far. In the UK, it was actually HMRC (our own national tax authority) that sent out messages to affected businesses confirming it was an error.

<comment><AUTHOR> > So why ask for the country of the seller in the API?

I must admit I'm not 100% sure either. I'm just exposing the API of the underlying open source project which says to supply the seller's country [1].

[1]: [https://github.com/mpociot/vat- calculator](https://github.com/mpociot/vat-calculator) At the bottom, where it says "Important"

<comment><USER3> If you're 'not sure' you should take this offline until you _are_ sure. Bookkeeping is an extremely bad area to go 'move fast and break stuff' because what you're breaking is other people's administrations and you won't have a way to fix it or even to reach them to inform them of an issue.

<comment><AUTHOR> You're being extremely critical in this entire thread. Tell me: How is having this field, unnecessary or not, a risk? You want me to take the site offline. But it's providing value to my own business, and apparently other people find it interesting as well. It's merely a REST API for an existing open source project. And whether people use the library or the REST API is the same from a risk perspective from their point of view. I'm providing value here, and am happy to run the risk you're advising against. So I will not take it offline.

<comment><USER3> > You're being extremely critical in this entire thread.

Yes.

> Tell me: How is having this field, unnecessary or not, a risk?

\- an unnecessary field implies that that field is either being collected or being used in the calculations, if it isn't then it simply should not be there.

> But it's providing value to my own business, and apparently other people > find it interesting as well.

I'm sure it is, but it is _also_ a risk and it seems as though you have not done the homework required to field a service such as this. That's a dangerous hobby.

> It's merely a REST API for an existing open source project.

No, it is a piece of software that you field that has the risk of affecting businesses that use it in a critical and potentially negative way if you mess up and it does not appear that you treat this subject with the seriousness normally associated with such a project.

> And whether people use the library or the REST API is the same from a risk > perspective from their point of view.

That is so far from the truth that I again wonder if you know what you are doing.

> I'm providing value here, and am happy to run the risk you're advising > against. > So I will not take it offline.

Suit yourself.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment> mike-cardwell This shouldn't be an API. It should be a query library and a dataset you can download with a batch job.

Please consider releasing the source code.

<comment><AUTHOR> All it does is provide a REST API for an existing open source library. That's written on the home page.

You can use the existing open source library (if you're using PHP). But then you have to keep it updated. The API does that for you - as well as allowing you to use it from any language.

</comment></comment><comment><USER16> VAT just sounds like a huge, overcomplicated mistake by the EU.

Why do you guys continue to live with it instead of repealing or fixing it?

<comment><USER17> "VAT" (the idea) is certainly not a mistake, it's one of the best way to do taxes, because it's, if not painless, at least almost invisible to the taxpayer (but it's considered by many to be unjust, because the poor spend a much larger part of their income in taxable goods than the rich, and so pay a bigger share of VAT).

The complexity of VAT in Europe is recent.

(I think) it used to be the case that if you sold something from one European country to another, then you weren't subject to VAT because it was considered an "export". Foreign companies (esp. US-based ones like Amazon) exploited this loophole so much, it became a problem. All they had to do was set up shop in a small, low-tax country [1] (Luxembourg / Ireland) and sell in every other country tax-free.

So the EU created this rule that the VAT that should be paid, is the VAT of the country where the customer resides -- it had to be, because if it was the VAT of the seller's country then companies would have again clustered in the country with the lowest VAT in order to evade the new rule.

It's not a bad rule; if you're a huge corporation, well, maybe you don't like it but you can hire the necessary lawyers and accountants to implement it.

If you're an SMB that sells across Europe, it's absolutely horrible. I guess the bureaucrats who wrote the rule didn't care much about SMBs or thought there were few who sold internationally.

It would be simple to exonerate SMBs based on their revenue (and I think it's the case, but the triggers are very low and different in every country); it would be even simpler to have just one tax rate but that's politically impossible.

In practice it doesn't seem to be very much enforced for small companies.

\- - -

[1] Low _corporate_ tax, to optimize taxes even further; no need to be based in a low-tax country to free oneself from VAT; the smaller the country the better, because you want to have as few of your customers in the same country as you, as possible.

<comment><USER18> I am admittedly naive on these matters, but I always thought that VAT is more efficient than a sales tax from the vendor's perspective and transparent to the consumer (an item costs $1.25 and you pay $1.25).

<comment><USER19> The main difference between VAT and sales tax is that VAT is paid by each stage in the supply chain (every time someone "adds value"), and reclaimed by charging it to the next stage up in the chain. Sales tax is charged only once, at the final point of sale to the consumer.

The end effect is the same: the consumer ends up paying whatever % the tax is. The difference is in how it's collected. Sales tax is easier to evade because business-to-business sales usually don't charge sales tax. So you could wink at your customer and say they're a business and not charge them sales tax. With VAT everyone is supposed to pay it, so it's harder to evade (and consequently more expensive for governments to collect).

<comment><USER20> But on the other hand there is a catastrophic amount of VAT carousel fraud going on, which is even worse than just 'tax fraud' (ie: not paying tax) because the govt is paying out VAT losses but not then collecting the VAT gains in the carousel.

</comment></comment></comment></comment></comment><comment><USER21> Why use POST?

<comment><AUTHOR> It made the curl example easier to write.

</comment></comment><comment><USER22> Doesn't seem correct, at least for Holland.

As far as I'm aware, as a seller in The Netherlands, you only have to charge VAT for buyers that are also in The Netherlands.

<comment><USER23> VAT is technically paid by the customer, you just collect them to make the process slightly less ridiculous.

therefore, VAT is paid based on customer location.

this does not mean that you dont have to pay VAT at all if you sell to germany. in that case, you ahve to pay german VAT. if youre smalltime enough, its likely that nobody will go after you, but thats what it is.

<comment><USER22> Correct, but you're not required to pay the local (Dutch) VAT, so the seller doesn't have to collect it. This tool shows the Dutch VAT rate (0.21) for all buyer countries, which is not correct in that case.

<comment><AUTHOR> I think you're wrong. Just because you don't owe VAT to the Netherlands doesn't mean you don't owe it to another EU country.

[edit] Maybe I misunderstood. In what situation do you think you don't have to pay Dutch VAT, and what's the source of your information?

<comment><USER22> [https://www.belastingdienst.nl/wps/wcm/connect/bldcontentnl/...](https://www.belastingdienst.nl/wps/wcm/connect/bldcontentnl/belastingdienst/zakelijk/btw/zakendoen_met_het_buitenland/goederen_en_diensten_naar_andere_eu_landen/goederen_en_diensten_naar_andere_eu_landen)

>Over het algemeen brengt u geen btw in rekening aan ondernemers uit andere EU-landen. Bij goederen past u dan het 0%-tarief toe en bij diensten wordt de btw naar uw klant verlegd.

In general, you should not charge VAT to entrepreneurs from other EU- countries. For goods, the 0%-tariff applies. For services, the VAT is transferred to the client. (Which means they need to declare the transaction and pay VAT in their country, which has nothing to do with your transaction with them).

<comment><USER3> Goods and services are dealt with completely differently when targeted at consumers or corporate customers (depending on the country). So there are 4 different combinations times the number of EU countries. And there are more exceptions when it comes to goods depending on the class of goods (food for instance).</comment></comment></comment></comment></comment></comment>


Z Kombinator - <AUTHOR> http://zkombinator.com

<comment><USER2> The sad thing is that the Samwer brothers are real people who actually do this, and they're quite successful at it too: [http://www.businessweek.com/articles/2012-02-29/the- germany-...](http://www.businessweek.com/articles/2012-02-29/the-germany- website-copy-machine)

<comment><USER3> why is that sad? Silicon VAlley claims ideas are worthless and execution is everything, until someone actually copies their idea.

<comment><USER2> Well, I'm not Silicon Valley. I just don't like the concept of simply copying someone else's idea as a business model, because you're piggybacking on someone else's idea, and it's uncreative.

<comment><USER4> I wonder what is your opinion about barbers, shop-owners, car manufactures or basically any business outside of the internet. I think that there's big difference between copying the business model and the product/service itself.

<comment><USER2> It's more copying a company, its business model, idea, everything down to the fine details that bothers me. Not so much copying a business model or an idea.

</comment></comment></comment></comment></comment><comment><USER5> Obviously a parody, but it underscores a historical trend of overseas investors pouring money into clones of US companies without any substantial innovation or changes.

One question I'm struggling with... Is a clone a legitimate (abstract) business if the differentiator between clone and U.S. original is the experience working with European / Asian customers and catering to their needs specifically?

<comment><USER6> He's making fun of the Samwer brothers, which would mean he probably doesn't think clones are legitimate business.

<comment><USER5> That much is obvious. Not asking him. Asking the HN crowd.

</comment></comment></comment><comment><USER7> Haha, great little parody.

Timing of it may have something to do with our little stunt yesterday: <http://ncombinator.com>

<comment><USER8> do you regret calling it NCombinator? Seeing as you could have avoided slipping into this clone mindset or does it not really matter to you guys?

<comment><USER7> Doesn't really matter. Our goal was to get some early attention. Once we build the community up we can change the name and turn it into something real.

Right now it's just an idea and the only platform we had to generate the sufficient early interest was a ridiculous name and launch.

</comment></comment></comment><comment><USER9> If YCombinator thought globally, Z Kombinator wouldn't have a chance in the first place ;-)

</comment><comment><USER10> With a German American background, I hear that like someone with a strong accent mispronouncing the word "the" as "zee". It adds another dimension of humor for me.

<comment><USER11> As a German-American: you mean like "sie"?

<comment><USER10> Um, not really what I was thinking. To me that sounds less harsh/strong than what I had in mind.

I guess it depends. I am imagining a very harsh accent. Some folks in my family pronounce ich like "ish", others closer to "ick" and then some do a more gutteral sound like you find in loch (a la loch ness monster). So, uh, maybe.

</comment></comment></comment><comment><USER12> Where is Z Hacker News?

</comment><comment><USER13> what the ....</comment>


Panera is losing nearly 100% of its workers every year - <AUTHOR> https://www.cnbc.com/2019/08/29/fast-food-restaurants-in-america-are-losing-100percent-of-workers-every-year.html <comment><USER2> The restraunt where I work on the weekends looses nearly the same amount every 6 months.</comment>


Firesheep usage leads to Idiocy - <AUTHOR> http://jonty.co.uk/idiocy <comment><USER2> Idiocy indeed. While you may run this for altruistic reasons, and I think it's good to teach people about the dangers of unprotected browsing, running a script to automatically hijack and post to the Twitter accounts of everyone around officially makes you an asshole, IMHO.

The fact that someone isn't wearing a belt doesn't mean they deserve a pantsing.

<comment><USER3> _The fact that someone isn't wearing a belt doesn't mean they deserve a pantsing._

Perhaps not, but, to stretch your metaphor a bit, if someone's fly is open, they might appreciate having it pointed out to them.

<comment><USER4> And how do you point it out to them? As discreetly as possible, or by shouting it to all their friends?

<comment><USER5> If half of their friends also have their fly open, shouting it might be a better option.

<comment><USER2> I'm missing the logic here. If few people make a mistake, they should be informed privately, but if many people make a mistake, they should be shamed publicly? No.

And to be honest, I'm getting slightly tired of the community's tendency to stretch metaphors way beyond their applicability. Mine was mostly meant to add a touch of humor. Pants, flies, and underwear aside, auto-hijacking the Twitter accounts of everyone on your network is just dickish (no pun intended). Especially when the belt store is closed to everyone except the pants-manufacturers ;)

<comment><USER6> I do not think this can really be considered "shaming" as it's nothing to be embarrassed about.

Regarding private/public notification; Posting to your own account is an easily understandable visible demonstration of what's possible and why you should care - a DM from another account, or an @ just won't have the same effect on the user.

(I have said this elsewhere in the thread, so apologies for repetition)

I have been informed that it's possible to send a DM to yourself, so I'll add that as an option shortly as a nice alternative.

<comment><USER7> I understand the intention is to warn people. But IMO, there are plenty ways of warning or educating them, e.g. blogs, tweets etc. There's no need to 'prove' it just to make awareness.

I did tell my friends and family that a tool (Firesheep) has been released and what is the impact in layman language, and inform them to be careful when browsing through open wifi networks.

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> This sounds like an interesting idea, but is it legal to run? Sure it might be a good thing to educate people about the dangers of accessing the internet over open wireless, but what if you accidentally run it on the account of someone who is willing to sue? Do they have a legal basis to sue you?

Just because they are doing something stupid, doesn't give you the right to mess with their accounts.

<comment><USER9> Unless they're running a honey-pot machine... how do they tell? You're coming from the same IP, with the same session. At best they can fingerprint your browser, which is far from proof and easy to change. Or nab things from Flash, maybe - but people likely to use this exploit to educate are probably more likely to run Flash blockers.

<comment><USER8> While it may be unlikely for you to get caught in my mind that still doesn't make it a good idea to run Idiocy. In essence I consider it to be a form of cyber terrorism.

Terrorists use illegal or unsavory acts to gain attention and draw media coverage of their cause. Essentially idiocy is just cyber terrorism. It says "Look! I can take over your account. Now that you are scared let me show you what I want you to do."

I'm sure that if this takes off it will get media coverage, and may even cause people and websites to change their habits and protocols, but why should terrorism be used to accomplish a change to secure web protocols?

Then again I have never been a believer in "the end justifies the means" so even assuming that the end result of forcing people to use HTTPS is good, I don't think cyber terrorism is a good way to accomplish it.

<comment><USER10> By that definition, if I find a flaw in (as an example) the New York Times, and e-mail them explaining the flaw and how to fix it, I'm engaging in cyber terrorism.

Shit, I remember when people would throw around the words "white hat" and "black hat", instead of "terrorism".

<comment><USER11> Actually, "Idiocy" is more akin to finding a flaw in the New York Times, and then using that flaw to change the headline of their front page.

I'm pretty sure that would land you in jail if caught.

<comment><USER12> not all people in jail are terrorists.

</comment></comment></comment></comment></comment></comment><comment><USER13> 126 lines of python. This is a great demonstration of how simple it is to exploit the vulnerability. Be aware, people.

</comment><comment><USER14> Actually a good idea, but a MUCH better idea would be to automatically post a direct message to that person (from your own account or from central Idiocy account) about the problem, rather than hijacking the session and basically hacking the account, running into problems with law.

<comment><USER6> I've been bouncing this back and forth with people all day!

Posting to your own account is an easily understandable visible demonstration of what's possible and why they should care - a DM from another account, or an @ just won't have the same effect on the user.

I might see if it's possible to send a DM to yourself, as that would achieve the same result without the public exposure.

</comment></comment><comment><USER15> It's possible to mitigate HTTP session hijacking without resorting to SSL: [http://carcaddar.blogspot.com/2010/10/protect-flock-or- how-t...](http://carcaddar.blogspot.com/2010/10/protect-flock-or-how-to- mitigate.html)

It's not really viable to serve most things over HTTPS because there's no hope of caching anywhere but at the end-point (and that's usually disabled by default, too).

</comment><comment><USER16> _In the last few days a tool has been released_

That seems to imply it's a single tool, and once this is "stopped" you don't need to worry any more. Surely it should mention instead that a recently released tool is _widely publicised_? Firesheep didn't make this possible, it has been for a long time, it just made it more accessible.

<comment><USER6> I thought the rest of the paragraph covered that, but I'm happy to edit it if you can suggest a rephrasing?

It's just supposed to be a quick introduction stating that the chances of being exploited are significantly higher now that the entry barrier is practically non-existent.

<comment><USER16> I... how did I miss that. I just went back and it's right there in the next sentence, my apologies. wtf.

<comment><USER9> I get a slightly hard-to-read vibe from the page, personally. Maybe the contrast, maybe the harsher anti-aliasing in Windows (used to OSX), maybe the largish sans-serif font.

</comment></comment></comment></comment><comment><USER17> I don't like it that Firesheep is driving everyone to use HTTPS.

You cannot have multiple SSL certs on a single IP and you have to pay a significant amount to get a cert that won't pop up an annoying page which turns away most non-technical users.

There needs to be an alternative which just does key exchange and symmetric encryption without the identification.

<comment><USER18> Your alternative to SSL would allow MITM attacks. Thus it would be useless.

<comment><USER17> I don't see another compromise that wouldn't require the signature infrastructure.

MITM attacks are much less common than sniffing attacks as we can see here.

<comment><USER18> If people moved to your scheme MITM would become common and we'd be writing about FirePiggyInTheMiddle or similar. They can be automated. You need some trust infrastructure for encryption to work.

<comment><USER17> In order to do FirePiggyInTheMiddle, you need to control the router. You can't just sit in the subnet and start sending commands to random clients to hijack their connection, especially when it is symmetrically encrypted.

<comment><USER19> ARP poisoning doesn't require control of the router; it makes MITMs pretty trivial.

[http://www.windowsecurity.com/articles/Understanding-Man- in-...](http://www.windowsecurity.com/articles/Understanding-Man-in-the- Middle-Attacks-ARP-Part1.html)

<comment><USER17> Ok you guys can stop with the "but what about this" now.

We all realize there are 1000 different attacks with various levels of difficulty and all of which have appropriate countermeasures and are nowhere near the success rate nor ubiquity of simply receiving packets.

</comment></comment></comment></comment></comment></comment></comment><comment><USER20> Speaking of this, when will HN allow for HTTPS connections? :P

<comment><USER21> Post a feature request: <http://news.ycombinator.com/item?id=363>

<comment><USER20> thanks, I noticed user culix already suggested it, so I upvoted his suggestion: <http://news.ycombinator.com/item?id=499851>

</comment></comment></comment><comment><USER22> I wonder how this jives with security expert Bruce Scheiner's stance: [http://www.schneier.com/blog/archives/2008/01/my_open_wirele...](http://www.schneier.com/blog/archives/2008/01/my_open_wireles.html)

<comment><USER9> Doesn't seem to line up much... his stance is about running his own open wireless access point, and how much FUD there is around it, not SSL on websites.

edit: ah, now I see what you were getting at. Though things have changed now that it's easier, and there's nothing _inherently_ dangerous with open wifi - SSH to your server, and you're plenty secure.

</comment></comment><comment><USER23> "Given that most websites will not be making SSL the default any time soon, the only option is to educate people."

And this educates them how, exactly? Education thru vandalism is just vandalism.

<comment><USER6> It educates by providing information about the attack used and how to prevent it happening in the future. You can find the link provided to users here: <http://jonty.co.uk/idiocy-what>

The tweet can be instantly removed by the user. It's hardly vandalism as nothing is being damaged or destroyed.

As I've said in a message elsewhere in this thread, posting to your own account is an easily understandable visible demonstration of what's possible and why they should care - a DM from another account, or an @ just won't have the same effect on the user.

I've been informed that it's possible to send a DM to yourself, so I'm going to add that as an option shortly.

</comment></comment><comment><USER24> Ok, dumb question time: Outside of things like banks, does anyone actually run a 100% SSL web server?

I thought the point of Firesheep was more "don't use unsecured networks" than "don't use websites that aren't 10% SSL." If it's the former, then this only does any good if the "victims" are the people providing the service. How many people do you think are going to notice this tweet immediately, realize where they were when it happened, and complain to the wifi provider (whose response, of course, will be "use at your own risk").

It might do _some_ good for people running insecure networks at home, but the people that understand what happened and how to fix it would already be running secure networks at home.

</comment><comment><USER25> Has anyone gotten this thing to even work? Seems to rely on <http://code.google.com/p/pypcap/> and when I attempt install from source the install.py turns out to be horribly useless.

Without installing that pypcap thing and just using libpcap, I get this error:

      Traceback (most recent call last):
      File "idiocy.py", line 128, in <module>
        main()
      File "idiocy.py", line 20, in main
        cap = pcap.pcap(device)
      AttributeError: 'module' object has no attribute 'pcap'

</comment><comment><USER26> How does this educate users, who don't have the option of using SSL to begin with? It's the website operators that need the education. This just embarrasses users due to no fault of their own.

</comment><comment><USER9> A question:

What _does_ it tweet? So far I'm only seeing people tweeting _about_ Idiocy, nothing that appears to be coming _from_ Idiocy. Different tones / content of the message could have extremely differing responses, and as it's _intentionally_ high profile, should be extremely careful.

<comment><USER27> From the code:

      status = 'I browsed twitter insecurely on a public network and all I got was this lousy tweet. http://jonty.co.uk/idiocy-what'

<comment><USER9> Unfortunately, the link it includes doesn't contain the part mentioning this capability is nothing new, merely that there's a new tool for it: <http://jonty.co.uk/idiocy-what>

If the creator is browsing through these: _include that part_ , or people will associate the danger with the new tool, and nothing else.

<comment><USER6> Good point! If you have any other comments I'd love to hear them.

Let me know if you think I should make any other edits: <http://jonty.co.uk/idiocy-what>

<comment><USER9> I think I'd swap the first and second paragraphs. That way, it makes more sense after "What happened?!", and reassures people ASAP so they won't lose interest / be confused before they find out what happened.

</comment></comment></comment></comment></comment><comment><USER28> It's like if Microsoft starts disturbing virus and worms to alert you of new bulletin updates, which is what some security experts condemn to do (sorry I couldn't find the article about Conficker with experts saying this as an approach to fight conficker)

</comment><comment><USER29> Anyone have ideas on getting it running on OS X?

One downside to some (often interpreted) languages is that there isn't always an easy way to package them up into a single binary for user consumption.

<comment><USER30> Install Python, go to terminal, and type `python idiocy.py`

<comment><USER31> master>python idiocy.py Traceback (most recent call last): File "idiocy.py", line 2, in <module> import getopt, sys, pcap, dpkt, re, httplib, urllib ImportError: No module named pcap

Is there a python equivalent to RubyGems ?

<comment><USER32> Distribute. See <http://python-distribute.org/pip_distribute.png>

<comment><USER31> Yeah how do you use this? I see pcap here <http://pypi.python.org/pypi/pcap/1.1> but can't seem to install it.

master>pip install pcap Downloading/unpacking pcap Could not find any downloads that satisfy the requirement pcap No distributions at all found for pcap Storing complete log in /Users/bob/.pip/pip.log</comment></comment></comment></comment></comment>


New York college moves to strip gender markings from all bathrooms - <AUTHOR> http://www.theguardian.com/world/2016/mar/29/gender-bathrooms-cooper-union-college-new-york <comment><USER2> Good on them. It's not like this alone will cause wide-spread overnight change but it's certainly a step in the right direction. Any small thing that can be done to make men and women seem a little less 'alien' to one another should be heralded as progress.

I expect there to be backlash from both sides because of discomfort and prejudices held by both sides, but if they keep with it it'll blow over eventually and maybe for next year's class it'll be the new normal.

</comment><comment><USER3> When I was there, the University of Pennsylvania had coed dorms and coed shared bathrooms. It wasn't a big deal.

</comment><comment><USER4> I will be interested to see the results of this.

Especially when we live in a world where even looking at someone the wrong way constitutes 'harassment'.

Many more hetro men would love a chance to see more women and don't really care if they see their unmentionables.

Women? Not so much. I'm all for equality, but this seems a bit extreme.

<comment><USER5> I don't know what women you've been talking to, but any woman that's been on the internet in the last 15 years has seen enough dick pics that they probably won't care if they see another penis.</comment></comment>


Ask HN: Best Intro to Web Development Course? - <AUTHOR> Looking to help a friend get started with web development. I'm technical but on the DevOps and Backend side of things, so not sure what approaches/courses are best for getting started these days. <comment><USER2> I'm not sure if I'm helping much but I started in 1996 by making webpages. Then I got a job in the field and kept going.

My advice today is DON'T start with a framework, make the simplest web page with no logic or CSS. Then add a bit of CSS and a bit of JS, all "by hand", no frameworks. When your hands hurt from typing verbose javascript then look at the frameworks. Starting from high-level going towards the bottom is a bad idea, as you don't understand half of what's happening. Start at the bottom, understand the DOM and CSS properly, then manipulate it by hand. Only later look at the whole 1-page app thing, server side partial rendering, SASS, etc.</comment>


Hitwise acquired by Experian -- a great move! - <AUTHOR> http://www.centernetworks.com/hitwise-acquired-by-experian-a-great-move <comment><AUTHOR> Someone asked if the bubble was back a few days ago. I think it is. Companies are being bought left right and centre.</comment>


What would you say is the buggiest game of all time? - <AUTHOR> http://www.redbull.com/uk/en/games/stories/1331623553889/glitch-gaming-bugs

<comment><USER2> The recently released X: Rebirth sure gives these games a run for their money. Unlike some of the games listed, dev time for X was around 7 years. We can't say that the game was re-arranged in the last hour (like that Superman game). The general consensus is that the publisher said "enough is enough" and forced a release.

X is a bit of a niche series (Hyper-detailed, hyper-complex 4X), but X: Rebirth was supposed to be a much more accessible re-invention. Instead, it is/was an unplayable mess that has been universally reviewed like this: [http://www.ign.com/articles/2013/12/02/x-rebirth- review](http://www.ign.com/articles/2013/12/02/x-rebirth-review)

I'm not sure if it is buggy enough to be considered the buggiest game of all time, but it is definitely in the discussion for that distinction. Fortunately, the developers have spent years after release patching their previous X titles, so maybe some day Rebirth will be playable. In the meantime, stuff like flying around, loading saved games, the entire economy stopping randomly, the inability to complete storyline missions, friendly AI not working, random crashes, and a huge pile of other bugs make this game a mess of massive proportions.

</comment><comment><USER3> ET on the Atari 2600, if you played it you would agree.

[http://en.wikipedia.org/wiki/E.T._the_Extra- Terrestrial_(vid...](http://en.wikipedia.org/wiki/E.T._the_Extra- Terrestrial_\(video_game\))

</comment><comment><USER4> The Elder Scrolls II: Daggerfall perhaps?</comment>


Gow - The lightweight alternative to Cygwin - <AUTHOR> https://github.com/bmatzelle/gow <comment><USER2> This is a junk submission. Where to start? This project hasn't shown activity in 10 months.

Furthermore, the project is largely written in VBScript and batch files, which is insanity, even according to Microsoft themselves.

It was time to move to Powershell a long, long time ago. You even have solid projects like NuGet to jump off from for code and design. I can't take a project seriously when it's written in VBScript because so many of its constructs are shoddy.

But again, those last two paragraphs are meant as advice for living projects, which this one is not.

<comment><USER3> Plus it's been submitted multiple times in the last few months already: <http://news.ycombinator.com/item?id=3312009>, <http://news.ycombinator.com/item?id=4085022>

</comment></comment><comment><USER4> No, for many years the lightweight alternative has been mingw. <http://www.mingw.org/>

<comment><USER5> GOW seems to be more focused on a userland environment, whereas mingw is focused on enabling developers to link against unix libraries in a windows environment. Sure there's msggrep, and msgfilter as userland in mingw, but GOW gives you direct access to 'ls', 'gfind', 'grep', etc, and doesn't care a fig about giving access to libz.

</comment></comment><comment><USER6> The bigger purpose of cygwin is to be able to compile your unix c/c++ to run on Windows. The shell utils is a small part of it, and better covered by things like UnxUtils and the various sets of binaries of the actual unix shell utils that people have packaged already.

</comment><comment><USER7> My only problem with Cygwin is that installing and updating packages is a pain since you need to use the setup program. Apart from that it works great for what I use it for - a unixy shell with ports of a few tools I like using.

<comment><USER8> apt-cyg makes that much less painful:

<http://code.google.com/p/apt-cyg/>

</comment></comment><comment><USER9> I've been using UnxUtils as a lightweight alternative for a while now. Its available on sourceforge.

</comment><comment><USER10> I have been using this package for months and I'm happy with it. My setup includes console2 + mingw (git for windows) + gow and it meets my expectations. I don't care if it's written in VBScript as long as it works. I have OSX at home and I can always use a Linux box if I need something more.

I'm glad that this post actually pointed me to the brand new release which hopefully will provide a fix for a file permission bug that I recently hit.

UnxUtils is another similar package that I have successfully used in the past. I think I jumped to Gow because I saw it here by the time I was getting back to Windows.

</comment><comment><USER11> Apparently, since last time I looked, there's actually source code. It's a tarball containing a collection of zip files of different layouts, with no documentation of how to actually build things, some build trees, etc. Many of the packages in that tarball are years out of date.

It's not a project that anyone else can reasonably use and build on -- it would be easier just to replicate the work. (Which I would like to do, incidentally.)

</comment><comment><USER5> Personally, I think Gow is awesome. The few times that I need power unix stuff, it's been just there, and it just works. That's the most important thing.

For me, it's worth the 'price' of admission, just to get a usable instance of 'gfind', and avoiding the cognitive dissonance of typing 'ls' and not getting useful info.

And it's not the PITA that is cygwin.

</comment><comment><USER12> No ssh command?

<comment><USER5> it includes the command lines for putty: pscp and psftp at least.</comment></comment>


Google's Schmidt warns regulators against killing innovation - <AUTHOR> http://www.physorg.com/news/2012-02-google-schmidt.html

<comment><USER2> _Asked by an Iranian member of the audience if Google will stop blocking Android or Chrome downloads in his country, Schmidt said that the ban was due to US sanctions on Iran and that "we can't violate US law."_

Is this due to crypto within the software or what?

<comment><USER3> That has to be the reason, because any software(libraries or executables) that contains SSL encryption code is covered as a weapon under export rules.

<comment><USER4> That stopped being the case years ago.

</comment></comment></comment><comment><USER5> It is for the most part an unfortunate fact that regulators are a stifler of innovation. What has their recent push for SOPA just taught us? If they want to regulate it to truly protect people that is one thing. For example, children need to be protected from predators who would exploit them on the Internet. However,when regulators attempt to regulate out of greed (because their pockets are being laced by Hollywood or some other interest group) they ruin things for everybody.

</comment><comment><USER6> I can live pretty well without innovations in privacy intrusion systems. ;-) But I'm looking forward to their innovations in deleting or hiding interconnected data. That's still a huge problem, I'm wondering what they 'innovated' to delete my G+ account while not becoming totally inconsistent on related pages of people following me etc. (I intentionally deleted it yesterday)

<comment><USER4> I can assure you Google and any sane Internet company never truly deletes anything, such that it is unrecoverable. They just mark it as deleted so that, it doesn't get served up but nothing is invalidated.

</comment></comment><comment><USER7> Regulations kill innovation, yes? Software patents are regulations... let's start there.</comment>


The Unschooled. Does an education matter? - <AUTHOR> http://www.tnr.com/article/politics/magazine/111376/the-unschooled

<comment><AUTHOR> "Entrepreneurship is not a full human education, and living is never just succeeding, and the humanities are always pertinent. In pain or in sorrow, who needs a quant? There are enormities of experience, horrors, crimes, disasters, tragedies, which revive the appetite for wisdom, and for the old sources, however imprecise, of wisdom—a massacre of schoolchildren, for example."

I used to think that school/college was just an assembly line. You enter as a child and are filled with facts and formula until you are a productive member of society. Boy was I wrong. School is about forming relationships, exploring your interests, and developing an identity. I did well in k-12 without applying myself and being the class clown, so I never took the whole institution too seriously. However, after dropping out of college to start a company, which didn't go as I thought it would, I realized a few more semesters would not have killed me. I really just wanted to say to the world 'I made it without that piece of paper, defied the odds, and did what I set out to do.' I've been humbled quite a bit as I've seen college graduates make tenfold the money I do, have more social encounters, and still somehow pursue their side projects.

I'm a first generation American without a father figure, so my whole life I've tried to be a man even though I was only a boy. That definitely has had an impact on my world perspective. Today, as my prefrontal cortex has almost fully developed at age 23, the 'angry young lad' in me has begun to calm down into a simple, passionate member of civilization. I've decided to go back to school (Gwinnett Tech) and finish my CIS degree. I hope it can provide a steady income while I attempt my second startup. I have learned to avoid extremes and see the world in a more balanced way. As the old saying goes, you live and you learn. That is definitely true. All in all, I have learned a lot that could never be taught in a classroom but I find myself in need of something to fall back on for those rainy days that no one ever wants or expects.

I think Dale Stephens is a nice guy but seriously missing the point. His followers mostly think just like him too and need some "authority figure" to justify their experience to themselves. I hope he learns that the future of education is a hybrid of in-person and internet models, as it should be.

<comment><USER2> wise words. thanx for sharing.</comment></comment>


Announcing "jism": an x86 assembly interpreter in JavaScript - <AUTHOR> https://twitter.com/cmcbot/status/434418512933949441

<comment><USER2> How perfectly clueless. The word you've chosen has a definition that you need to become aware of:

Link: [http://en.wikipedia.org/wiki/Jism](http://en.wikipedia.org/wiki/Jism)

Quote: "A slang term used to describe semen."

I assume you didn't mean to turn your software project into a sleazy linkbait caricature.</comment>


Ever wondered how many open FTP servers there are? - <AUTHOR> https://github.com/massivedynamic/openftp4 <comment><USER2> I used to leave a file called README on my public ftp directory, which contained only:

cat: README: No such file or directory

I'd occasionally get email from frustrated people who had trouble trying to read the README file, so I'd tell them to simply run "emacs README", and emacs would solve all of their problems. I don't know if my passive aggressive emacs evangelism ever worked, because I never heard back from them.

<comment><USER3> Reminds me of the Abbott and Costello skit "Who's on first" [1]

[1] [https://youtu.be/kTcRRaXV-fg?t=63](https://youtu.be/kTcRRaXV-fg?t=63)

</comment></comment><comment><USER4> I got my first internet account on my high school's VAX in 1992 in order to download Dr. Dobb's code from ftp.mv.com. The VAX had a help file installed which was a list of thousands of anonymous FTP servers, most of which requested that you please not use anonymous FTP during business hours to avoid overloading them; shortly I was downloading all kinds of things from wuarchive.wustl.edu (which had a LOT of stuff) and WSMR-SIMTEL20.army.mil, which worried my dad. I bound some function keys on my terminal to different internet commands (FTP, TELNET) and internet sites (like those I mentioned) so I could FTP to wuarchive or TELNET to HPCwire with two keystrokes.

At some point I got hold of Scott Yanoff's list of interesting Internet services (capitalizing Internet was still justifiable at the time) and learned about the Weather Underground, Archie, HPCWire, and this new thing called the "World Wide Web" — I started telnetting to a server at the University of Kansas where I could use Lynx, and it seemed pretty clear that this was going to be a big deal, because of how enormously much easier it was than downloading text files over FTP.

So not only have I wondered how many open FTP servers there are, my exploration of the internet pretty much started with a list of them.

Nowadays I occasionally look for FTP servers because they tend to be less of a pain in the ass for downloading stuff than HTTP servers — you can usually get a full list of what they have, and they never interrupt you with CAPTCHAs. It's kind of like a real-world "shibboleet" — I guess sometimes assholes push mandates for CAPTCHAs and whatnot on a company's technical folk, but they leave FTP open because the assholes don't know about it.

If you're wondering how many open HTTP servers there are, Netcraft does a pretty good monthly survey.

<comment><USER5> "capitalizing Internet was still justifiable at the time"

Given the Internet is a particular thing, shouldn't it still be used as a proper noun?

<comment><USER6> AP Style Alert. Internet and web are no longer to be capitalized. Check the googles.

<comment><USER5> I don't work for the AP so I will continue to capitalize it.

</comment></comment></comment></comment><comment><USER7> I have a fun story about open FTP servers.

When we were undergrad students, a friend and I wondered exactly that same question about FTP servers. So we wrote a script that tested random IPv4 for FTP servers using nmap and then attempted to connect.

To our surprise we found quite a few. Most of them were small or not writable, but at least one of them was writable, and it had hundreds of GB of free space. So we connected to it, and then saw that it had huge data files containing what seemed like random junk.

Then, we tried to ssh to the box, but no ssh server was listening. So we tried to telnet in. That worked but we were prompted:

        Enter password:

Ah, too bad, we will never guess what the password is… Mh, let's try "123456" just in case… But then, instead of logging us in or telling us that we entered the wrong password, it simply said:

        Re-enter password:

Mh? So we re-entered "123456".

        The password has been set.

Haha! The owner must never have set this up. And then we were in. It was a very strange system with a few commands with standard names but non-standard behaviors, and a very minimalistic shell. By poking around and searching the web, we understood that we were actually connected to a surveillance camera and that the big files were probably parts of video.

Poking around more gave us access to other such cameras in the same network and more importantly to a web interface from which we could see the videos streaming live. We saw offices and people doing stuff like taking the garbage out (I don't know why but that is one of the more precise image I can recall ^^). The only distincive thing I remember was a big sign saying "Miami Fitness Club".

After that we never did anything about it as we had other things to do, but I kind of cherish this story as a nice souvenir of my first year at the ENS.

<comment><USER8> I remember doing something similar except it was with my cable provider, when they first came into town.

Since all clients would basically be connected to a LAN, as soon as I found I could port scan random users I started doing it.

Of course there were a lot of businesses on the network that apparently used FTP to move files around but were unsecured.

I spent the better part of a couple of weeks just going through the data (never actually downloaded anything since I knew it could land me in trouble with my parents).

Like you said, I also cherish this story since it was a rare peak at other peoples lives... without even knowing a 12 year old had access to all their business files.

<comment><USER9> @Home was wide open around 1999ish. My friend would send random (if you can call goatse.cx random) print jobs to various open SMB printer shares. Ah, to be young again.

</comment></comment></comment><comment><USER10>
      $ lz5
      -bash: lz5: command not found
      $ brew install lz5
      ==> Auto-updated Homebrew!
      Updated Homebrew from b5a6b4e to 7926114.
      Error: No available formula with the name "lz5"
      ==> Searching for similarly named formulae...
      Error: No similarly named formulae found.
      ==> Searching taps...
      Error: No formulae found in taps.

So, rather than complain about this, how do I do something to fix it? Can I add lz5 as a tap to Homebrew?

EDIT: Let me try this again, in a more productive way.

lz5 does not currently exist in Homebrew. I'd like to fix this. I've never done this before. Does anyone have advice? Is it as simple as forking lz5, then adding a tap to Homebrew?

Thanks. And as a note, this edit occurred after specialp's replies. They were right: this originally wasn't a productive comment.

<comment><USER11> [https://github.com/Homebrew/brew/blob/master/Library/Homebre...](https://github.com/Homebrew/brew/blob/master/Library/Homebrew/cask/CONTRIBUTING.md#adding- a-cask)

<comment><USER10> Ah, it'd be a cask? Cool, thanks.

<comment><USER12> [https://github.com/Homebrew/homebrew- core/blob/master/.githu...](https://github.com/Homebrew/homebrew- core/blob/master/.github/CONTRIBUTING.md) Is a better link. It would just be a new formula in homebrew-core

</comment></comment></comment></comment><comment><USER13> I wonder how many of those are just mirrors of Linux distros and other open- source software, how many have more interesting things (including software), and how many of _those_ were deliberately configured to be open for sharing. There is the somewhat-well-known filesearch.ru if you want to look for things on this non-HTTP part of the Internet. (If I remember correctly, Google used to index FTPs too and you'd get plenty of results with the right queries, but that seems to have mostly and silently disappeared...)

<comment><USER14> There's still a uprising number of niche ftp site around: MSX, Demoscene, etc. Mostly older scenes that pre-date the WWW and are still around. I think it might be useful to test every port on every ip to see what happens protocol- wise. Limiting to just common ports is probably missing lots of cool things.

<comment><USER15> How many ports are there?

<comment><USER16> According to Reserved IP addresses there are 588,514,304 reserved addresses and since there are 4,294,967,296 (2^32) IPv4 addressess in total, there are 3,706,452,992 public addresses. There are 65536 ports, so one would have to scan all those ports on all those addresses.

</comment></comment></comment></comment><comment><USER17> S/he talks about it as if it's something bad, something unfixed. The whole thing sounds like it but in particular "[to be excluded] go fix your shit".

I agree that if you don't want people to access it, you should secure it. Yet not all these servers are accidentally open: my ftp on 184.108.40.206 (I assume it's in there) hosts a copy of Damn Small Linux because all downloads were extremely slow or broken at the time.

<comment><USER18> Agreed. Having an open FTP server with read-only access is no more problematic than having an open HTTP server.

</comment></comment><comment><USER19> I remember back in the 90s, people would hunt for FTP servers that allowed anonymous writes. Companies that didn't know how to secure servers would suddenly be hosting a ton-o-warez.

<comment><USER20> > would suddenly be hosting a ton-o-warez.

which could then be found by searching for "index of"...

<comment><USER21> ... if only we had had search engines then. :)

<comment><USER22> I used to telnet archie.mcgill.ca to search the collected index of archive sites, back in 1991 or so.

[https://en.wikipedia.org/wiki/Archie_search_engine](https://en.wikipedia.org/wiki/Archie_search_engine)

</comment></comment></comment></comment><comment><USER23> Side note: "xz -9 -e" compresses the file to 3,296,864 where as "lz5 -15" only compresses the original file to 4,643,261 bytes. The xz compressed file is 29% smaller.

Even "gzip -9" compresses the file to 4,035,858.

So I wonder why lz5 was chosen for compression.

<comment><USER21> How long did that "xz -9 -e" take compared to "lz5 -15" or "gzip -9"?

<comment><USER24> Numbers on my machine, ordered by compression duration:

        Compressor    Duration    Size
        cat           0.0s        11385584 [*]
        gzip -1       0.2s        4918778 [*]
        bzip2 -1      0.9s        3334653
        bzip2 -9      0.9s        3122347 [*]
        xz -1         1.1s        4222016
        gzip -9       2.4s        4085818
        lz5 -15       6.9s        4643261
        xz -9 -e      10.7s       4033589
        zpaq -m5      34.2s       2655834 [*]

A [*] indicates that no better compressor was faster. Test methodology was `cat | $compressor > output` run 3-4 times to get an average.

I'm surprised by how bzip2 turned out.

</comment></comment></comment><comment><USER25> I recommend using Shodan for this kind of stuff.

[https://shodan.io](https://shodan.io)

</comment><comment><USER26> Scanning the IPv4 space. I know there are many different projects that do it. I was thinking about how I would do this today. I believe the first step would be to arrive at all the IPv4 blocks (/22, etc), then do a calculation to arrive at the address of each based on the prefix. Then in an array of threads or so, try to connect(2) to the address on some type of service in a timeout handler. If it succeeds, then consider that address as up. I would consider doing this in an async loop with epoll(7), so that many connections could be attempted at once and improve through put.

Anyway. nmap can probably do this and is a great tool

<comment><USER27> If you want to have your scan done in a reasonable amount of time, you probably want to use raw packets, and not connect.

If you don't mind all the flak you'll get, just send a SYN on the port you care about to each IP (maybe skip rfc1918, multicast and reserved addresses; or only send to addresses included in bgp announcements). If you send one packet to each addresses, including the addresses you should probably skip that's 4 Billion packets; if you do it at 1M pps (should fit on a 1Gbps ethernet connection), that's less than two hours.

<comment><USER28> That's what masscan automates. To fully utilize ms, one requires a friendly ISP. And even then, they can only be ms-friendly as long as their peers are. If you annoy the peers enough, they'll just drop you ([http://www.sudosecure.com/ecatels-harboring-of-spambots- and-...](http://www.sudosecure.com/ecatels-harboring-of-spambots-and-malware- causes-bgp-peers-to-stop-peering-with-them/)). Many datacenters classify port- scanning as an offensive action, even with low package thoughput.

Pulling data from research servers (such as Censys), reducing and _then_ scanning is always a good idea.

</comment></comment></comment><comment><USER29> The submission links to a blog post on how the data was retrieved: [http://255.wf/2016-09-18-mass-analyzing-a-chunk-of-the- inter...](http://255.wf/2016-09-18-mass-analyzing-a-chunk-of-the-internet/)

> For this little experiment, I've setup a single KVM instance, running a > single 2GHz vCore with 2GIB of RAM and 10GiB of HDD space. This is > sufficient. Probing for ftp access is an extremely CPU-intensive task. You > are going to hit bottlenecks in this order: > > CPU > Memory > a whole lot > of nothing > network > > While the rescan was running, only about 1 to 2kpps > were exchanged, while the CPU was pinned at 100%.

So this means his setup spent about 1-2 million clock cycles _per probe_. That's a lot!

I suppose this is because he runs the probe script once per IP address? I suspect that an implementation which would stay in-process would be at least an order of magnitued faster.

<comment><USER28> Sure. Faster even with a better scheduler. I just wanted to show how the simplest and most redneck way still finishes in a reasonable amount of time. :-)

<comment><USER30> I was amazed how fast that went. Was fully expecting the story to unfold with how you rented out 100 AWS servers to complete the task, instead it was just one computer and only took hours.

<comment><USER28> It's all about reducing data offline before throwing the kitchen sink at the internet.

</comment></comment></comment></comment><comment><USER31> Good old archie... [http://archie.icm.edu.pl/archie- adv_eng.html](http://archie.icm.edu.pl/archie-adv_eng.html)

<comment><USER13> > Search Type: Sub String Exact Regular Expression

Google has an advantage in terms of index size, but definitely lose in terms of precision. The way it munges queries is a bit unsettling at times, and even the "exact" option doesn't seem to always work the way it should.

<comment><USER32> Have you tried verbatim mode? (Thanks [https://news.ycombinator.com/item?id=12046056](https://news.ycombinator.com/item?id=12046056))

<comment><USER33> That is a huge improvement over the normal search, but unfortunately it's not truly verbatim. For example, searching for `self` with the backticks turns up results for just self with no backticks. It works with some symbols but not others, not sure why.

</comment></comment></comment></comment><comment><USER34> Are there any legal implications for doing this? Was going to do something similar with Redis and MongoDB.

<comment><USER35> This might be a useful starting point:

[https://blog.shodan.io/its-still-the-data- stupid/](https://blog.shodan.io/its-still-the-data-stupid/)

Shodan crawls for most nosql/ queueing software including mongodb and redistribution.

And related to OP: we also crawl for FTP and attempt anonymous as well as a few other things to better understand FTP deployments on the Internet.

<comment><USER28> Related to your related: Nice. Is this research public or commercial?

<comment><USER35> All the data is searchable for free on [https://www.shodan.io](https://www.shodan.io)

</comment></comment></comment></comment><comment><USER36> [http://ftpsearch.ntnu.no](http://ftpsearch.ntnu.no) used to index many of them

</comment><comment><USER37> That would be fun to use together with fun side project [RandomFtpGrabber]([https://github.com/albertz/RandomFtpGrabber](https://github.com/albertz/RandomFtpGrabber)) which will download random stuff from a list of FTPs.

</comment><comment><USER38> "All in all, there were exactly 18,454,087 things that responded to a banner fetch... The JSON file is about 4GiB." where can I download this JSON file? I only see the list of ips available.

<comment><USER28> I specifically left the mass _scanning_ part out. If you want to experiment with scan data without scanning yourself, a good place to start is [https://censys.io](https://censys.io).

</comment></comment><comment><USER39> > consider piping the list through shuf each time you try something new. You > know why.

No, I don't know why. Can someone explain? (without being condescending, preferably)

<comment><USER40> It's not entirely clear to me either, though I would assume it's so that the first server in the list doesn't get constantly hammered by a hundred people going through the list sequentially.

<comment><USER39> hmm. Yeah that makes sense.

Would be better if they simply stated that instead of playing insinuation based guessing games.

</comment></comment></comment><comment><USER41> How many of them are honeypots, intentionally left open?

</comment><comment><USER42> How many are there with write access?

And how do they protect against spam?

</comment><comment><USER38> any stats on how many servers are running Pure-FTPd vs. Paradise FTP vs. some other software?

<comment><USER35> There are ~300,000 Pure-FTPd instances: [https://www.shodan.io/search?query=product%3Apure- ftpd+port%...](https://www.shodan.io/search?query=product%3Apure- ftpd+port%3A21)

Not sure how to fingerprint Paradise FTP (searching for "Welcome to Paradise" also returned some non-relevant results) but there aren't many that contain "paradise" in their welcome banner.

Shodan also fingerprints lots of other FTP software (check out the "Top Products" section):

[https://www.shodan.io/report/WHJBsZqV](https://www.shodan.io/report/WHJBsZqV)

ProFTPD is by far the most popular choice at the moment.

Note: Doing a search that uses a filter (ex: "product") requires a free Shodan account. None of the above require paid access, you just need a free account.

</comment></comment><comment><USER43> I can see having an open anonymous FTP that was read-only as a way of serving some files that needed to be downloaded and didn't contain any sensitive information. It's no different from providing a URL to them.

Or even a write-only one so people could deposit data.

The only real problem is a read-write one where people can use it to exchange information.</comment>


Ask PG: Qn on the 'More' link at the bottom of News.YC's home page - <AUTHOR>

It changes with each page refresh now: from http://news.ycombinator.com/x?fnid=D5S0Tu4VK5 to http://news.ycombinator.com/x?fnid=ARW2OBvB6s... while pointing to posts 1-30. Any reason you are doing this?
Similarly, the More link at the bottom of Page 2 also changes  each time but still points to Posts 31-60. And so on....
I think that at one time, the URL used to be just /page2 or something easily guessable. <comment><USER2> Closures?</comment>


Perl 6 approved nickname is 'Raku', per Larry Wall - <AUTHOR> https://twitter.com/zoffix/status/1058796898235105280 <comment><USER2> More details from Larry at

[https://colabti.org/irclogger/irclogger_log/perl6?date=2018-...](https://colabti.org/irclogger/irclogger_log/perl6?date=2018-10-25#l584)</comment>


The Cinematography of "The Incredibles" - <AUTHOR> http://floobynooby.blogspot.co.uk/2013/12/the-cinematography-of-incredibles-part-1.html

<comment><USER2> Holy shit, look at this:

[http://4.bp.blogspot.com/-AQTRoWPB3n4/Tbn5O99q6lI/AAAAAAAARG...](http://4.bp.blogspot.com/-AQTRoWPB3n4/Tbn5O99q6lI/AAAAAAAARGY/MzV38NYcm6o/s1600/longshot4.png)

Though it wasn't mentioned in the movie, Brad Bird described Edna as "half- German and half-Japanese". The walls of this room resemble Japanese rice-paper walls -- but the divisions are arranged as in a Piet Mondrian painting. Mondrian was Dutch, not German, but there's still a sense of Asian and European art styles mixed here. I never noticed that before.

Fuck this movie is so good.

<comment><USER3> I thought Edna was modeled after Edith Head [1]

[1] [http://en.m.wikipedia.org/wiki/Edith_Head](http://en.m.wikipedia.org/wiki/Edith_Head)

<comment><USER4> I thought it was the boss from NCIS LA

<comment><USER5> Sorry... Pixar did not invent a time travel machine just to get a character reference. (NCIS LA came five years after.)

<comment><USER4> Surely Linda Hunt is older than Pixar...

</comment></comment></comment></comment></comment><comment><USER6> Interesting read, though sometimes it sounds a bit like over-analyzation. Especially the constant mention of "triangular composition" \- three points form a triangle (unless they form a line). You'd have to make a conscious effort NOT to have "triangular composition".

<comment><USER7> Completely agree. I enjoyed reading about framing, how positioning backgrounds to ensure the viewer can easily follow action, the far/medium/full shots, bird/worm's eye view and the like.

Mentioning triangular composition so often was strange. I guess it's a basic filmmaker thing? (don't line up talking characters in a line?) Moving the camera out-of-plane of the conversation allows the viewers to more easily understand the flow of conversation, who the characters are paying attention to, and who we should be paying attention to.

<comment><USER8> You'll notice that the triangular composition shots had three figures on three different levels in the shot, when most shots with a single figure in them had their head at basically the same level, right below the top of the frame. If the default placement for a character's head is near the top of the frame, it would indeed be a conscious choice when placing three head in a shot to make them at three different levels.

</comment></comment></comment><comment><USER9> That's cool, but I think if you want to study film, you should study the likes of Akira Kurosawa, Sergio Leone or Krzysztof Kieslowski [1] first. A lot of what Hollywood is doing has been first done by these guys.

Especially computer games and Pixar movies are often directly reusing famous scenes and camera angles.

[1] Disclaimer: I'm a big fan of "Three Colors", especially "Red".

<comment><USER10> Thanks for the reccomendations! I've heard so much about Akira Kurosawa. Gotta check him out.

<comment><USER11> Kurosawa has a wide range of styles - which is why I like to compare him to Kubrick, although Kurosawa usually stays within Edo period contexts while Kubrick also had a wide range of themes. My favorites so far are Seven Samurais, Yojimbo and Dreams. Don't be afraid of watching one of his films from the 50ies - because of their relatively quick pace, clear themes and modern looking cuts I find them highly enjoyable, not just as an educational experience, but with a genuinely high entertainment value even today. From an educational standpoint it's amazing to see the strong influences he had on Star Wars for example.

</comment></comment></comment><comment><USER10> This is great stuff that I think everybody should check out, whatever your field- it's just generally interesting. Insights can be useful metaphors for presentation, direction, writing, project management...

Relevant link: The use of light in cinema: [http://imgur.com/a/cpLno](http://imgur.com/a/cpLno)

<comment><USER12> One thing you don't get from that first photo is that they're sitting round a green baize table. Like poker players.

</comment></comment><comment><USER13> Somewhat related: A collection of the different types of chairs in The Incredibles

[http://jimunwin.com/extra/incrediblechairs/](http://jimunwin.com/extra/incrediblechairs/)

<comment><USER14> I think The Incredibles is at least partially responsible for the return of mid-century modern furnishings.

</comment></comment><comment><USER15> Check All "hayao miyazaki" films. He defined the animated films.

<comment><USER16> I know it is a mainstream movie and everyone has seen it. But try watching Disney's Snow White again, and remember that it was released in 1937. Your statement "defined the animated films" is more applicable to that movie than anything else.

<comment><USER17> My favourite Disney book is "The Illusion of Life" which is written by Frank Thomas and Ollie Johnson, 2 of Disney's "nine old men"

The first chapters deal with the creation of the Snow White film, and the startup culture within the Disney studios at that time.

Co-incidentally, both authors are referenced in the film The Incredibles (the original subject of the OP's post)

<comment><USER2> Not just referenced -- they appeared as themselves!

I can't help but wonder if this is a shout-out/jab to the perennial Stan Lee cameo in films based off Marvel characters, especially considering that _The Incredibles_ was often seen as competing with, and compared favorably against, _Fantastic Four_ (2005).

</comment></comment></comment></comment><comment><USER18> This is awesome it adds a whole other perspective to movies.</comment>


Vegan parents face jail for feeding children poor diet - <AUTHOR> http://www.thetimes.co.uk/article/vegan-parents-face-jail-for-feeding-children-poor-diet-k2w6mlmv0 <comment><USER2> A law like this seems misguided and designed to score political points rather than solving any real social issue.

People suffering from this type of malnourishment in most developed countries account for a tiny fraction of the population. But it's much easier to go after a few outlier cases than pointing out the elephant in the room (pun intended) with regards to public health and pointing out how basically the majority of people are now overweight or obese.

I'm not a huge fan of most people being vegan because it requires a much higher level of dedication to getting enough protein and other nutrients than most people are willing to devote to it. But there are in fact vegan athletes and even elite bodybuilders, so it's not impossible to have a productive vegan diet that gives a normal person a normal healthy life if you put effort into it. I say this as a dedicated meat eater.

</comment><comment><USER3> Meanwhile :

Obesity rates in Italian two-year-olds are the highest in Europe with a rate of 42%

[https://en.wikipedia.org/wiki/Obesity_in_Italy](https://en.wikipedia.org/wiki/Obesity_in_Italy)

</comment><comment><USER4> Is there a workaround for this paywall?</comment>


Workshape.io – Talent matching for tech startups - <AUTHOR> http://workshape.io/ <comment><USER2> I don't like the requirement to sign up for Github in order to sign up for your thing. Seems like an unnecessary hurdle considering how few real-world developers actually have profiles there.

Worse, I do actually have a Github account, but I'm not going to tell you about it in order to log in and learn what this thing does, given that I have no idea what you plan to do with it. I can imagine a scenario where I'll discover a page on the internet proudly proclaiming that "Jason Is Looking For A Job!" when the reality is simply that "Jason is curious what dimensions go into that shape thing".

Give me a user/pass. Or better still, no requirement at all to dig in further. Then I'll dig.

<comment><USER3> "few real-world developers actually have profiles there"

Is that really true? I work for a mega-corp and I still put as much work as I can on GitHub. You always have side projects you can put up there as well.

<comment><USER4> I work for one too and it's hard to even discuss Git itself with someone here; most people just know SVN. The stack in my industry is very stagnant and I think a lot of people just learned what they had to when they were still young and became set in their ways. They have families now and coding has become a 9-to-5 chore; they're not interested in keeping a portfolio up to date or even trying anything new.

I'd presume most people who come to HN are interested in evolving their skill set. The common worn-out office drone doesn't visit HN.

<comment><USER5> Git != Github.

I have always used git, privately (eg: Bitbucket). And if I want to run an open source project, I can very well do that with Bitbucket too. I don't know why employers ask to see a Github profile.

<comment><USER4> I never said Git == GitHub.

I meant that they don't even know Git, _let alone_ GitHub.

</comment></comment></comment></comment></comment><comment><USER6> This was a Show HN, last week:

[https://news.ycombinator.com/item?id=8652042](https://news.ycombinator.com/item?id=8652042)

<comment><AUTHOR> Oops, I missed that! Has HN stopped blocking duplicate submissions? Or is it because I submitted the naked domain?

</comment></comment><comment><USER7> Would be nice if they listed how many jobs are in their DB. Last time I tried I couldn't login due to a server error. Next time I tried they didn't list anything for me. Even when I went back and added a little of everything instead of just coding. So as a user, I would never go back because it is several screens of work for no listings back. Maybe they should move to an email model where they will email you something if they ever get some employers on board.

<comment><USER8> Hi lnanek,

This is the underlying model. We hope to find you matches on the spot, but if we can't then whenever a match appears we will notify you immediately via email that a match has been generated.

The majority of the jobs on the site are based in London right now, with some in New York and San Francisco. If you enter one of those locations you are more likely get a match.

On our end we are working to improve the way location is recorded so that we can cater for people with varying levels of specificity when it comes to where they want to work e.g. Anywhere, Europe, France, Paris

</comment></comment><comment><USER9> Interesting but you should have an option somewhere that says actively looking or not actively looking. Otherwise you run the risk of spamming people.

<comment><USER10> Hey dkarapetyan,

We're challenging the 'passive' vs 'active' paradigm. Most of the time we do exist in these binary states, rather somewhere along a spectrum between those two points. We think that if the opportunity is a good enough match - based on what you say you want - then you might be open to talk.

With regards to spam, no employer can message you with a notification trigger unless you make a clear choice to declare interest, after the match. So as a user you exist in two states - matched vs interested match. Only when you are an 'interested match' is your identity revealed to employers and conversation unlocked.

Ping me if you have any questions.

Best wishes

Hung

<comment><USER9> Ok, that makes sense but it is not clear from the basic interaction that is the mechanism you guys are going for.

</comment></comment></comment><comment><USER11> Interesting, but I don't see why I should give them a preferred job title. Interests, skills, job level -- of course, that's how they match people. But preferred job title and description? I don't want to end up in a catalog, I just want to see positions.

</comment><comment><USER12> I experienced a lot of lags and delays between pages with no feedback as to whether or not the page is loading. The tokenized skills input list had empty tokens when I clicked Confirm. The UI looks and feels very good in my opinion but the UX is broken.

</comment><comment><USER13> Devil's Advocate.

Upfront an engineer might specify that they'd like to spend 60% of their time on Data Science and 20% in Architecture but this doesn't mean that they are skilled appropriately for this.

What about companies that want to hire people to fill roles that require very specific experiences and education?

Is this app assuming that we when asked to describe what we want to do, we won't describe anything too far from what we already do?

Is the market on Engineers sides so much that they can describe what they want to do, not what they have done before and then get to do it at a cushy startup?

<comment><USER14> it first asks "what are you good at" then asks "what do you want to do".

the employer can reconcile the difference.

</comment></comment><comment><USER15> Did you just split all engineering jobs in front-end and back-end? I'm afraid I never did a lot of neither, and I have no place to put my experience in there.

</comment><comment><USER16> Server is down Edit: Up but no functionality

<comment><USER17> Does HN really drive enough traffic to a (relatively recently submitted) site to bring it down? Or is this a sign of poor scaling?

<comment><USER18> It drives enough traffic to kill a general web app. If you've just launched and have something like Rails / Django running on a single app server, it really doesn't that much to overload it. Wordpress seems to not handle it well either (unless you're caching).

</comment></comment></comment><comment><USER19> Very enjoyable user experience, 10/10, would do again.

<comment><USER20> And with rice ?</comment></comment>


Lattice Shapes of Rings of Integers of Number Fields: An Artist's Rendering [pdf] - <AUTHOR> http://www.theliberatedmathematician.com/wp-content/uploads/2015/11/PiperThesisPostPrint.pdf <comment><USER2> "Respected research math is dominated by men of a certain attitude. Even allowing for individual variation, there is still a tendency towards an oppressive atmosphere, which is carefully maintained and even championed by those who find it conducive to success."

This is def consistent my personal experience... Not so much sexism (nowadays at least) but a strange mix of ageism, cult of innate 'genius', and gymnastics/ballet-like, endurance/bravado-focused ultracompetitiveness [1,2,3]. I increasingly blame Hardy's bestseller --and his abiding influence on mathematics culture worldwide-- for it.

(Also I can't believe such a diss was accepted in the first place... Sign of changing times.)

[1] [https://mathbabe.org/2014/08/15/i-love-math-and-i-hate- the-f...](https://mathbabe.org/2014/08/15/i-love-math-and-i-hate-the-fields- medal/)

[2] [http://listeningtogolem.blogspot.com/2016/08/the- invisible-p...](http://listeningtogolem.blogspot.com/2016/08/the-invisible- pachyderm-ageism-in.html)

[3] [https://mathbabe.org/2015/01/16/representation-of-women- and-...](https://mathbabe.org/2015/01/16/representation-of-women-and-the- genius-myth/)

</comment><comment><USER3> Oh how I wish every math paper was written like this! Very few math papers are a fun read - this is one of them.</comment>