Grasshopper - <AUTHOR> https://wikileaks.org/vault7/?g#Grasshopper <comment><USER2> [https://wikileaks.org/vault7/document/DerStarke_v1_4_DOC/pag...](https://wikileaks.org/vault7/document/DerStarke_v1_4_DOC/page-1/#pagination)

\---

Der Starke is a diskless, EFI-persistent version of Triton. Once active on a target system, the implant executed within diskarbitrationd and typically performs network ommunications through a browser process so that PSPs like Little Snitch cannot easily detect it's presence. This Companion User Guide is meant to supplement the Triton User Guide.

\---

This is obvious to most people here but it might not be to others, if the attacker got physical access to the machine it's game over.

[https://wikileaks.org/vault7/document/Grasshopper-v2_0_2-Use...](https://wikileaks.org/vault7/document/Grasshopper-v2_0_2-UserGuide/page-16/#pagination)

\---

Grasshopper has the following system dependencies:

Python 3.4: The Grasshopper build system was developed and tested for Python 3.4

\---

Nice! A lot of government projects are stuck on older versions of Python tied to ancient OS versions.

I gotta say, the rule matcher syntax and the overall design is done very well. Hat tip to the authors if they are reading this.

<comment><USER3> They are reading everything.

</comment></comment><comment><USER4> So the CIA is using components from the malware known as Carberp, a suspected Russian organized crime rootkit, which mean instead of helping people protection their computer from organized crime they allow the vulnerability to continue. How is this moral?

<comment><USER5> It's not an either-or; this is malware, not vulnerabilities. For the most part it doesn't make sense to think in terms of using malware components to protect systems from malware.

This misunderstanding has come up on a bunch of other threads too; it seems to be pretty widespread. You can't really "burn" most persistence techniques, publishing them in a way that makes them easy to defend against. If you could, a whole lot of platform security engineering would be much easier.

<comment><USER4> What part of a "persistence method" are not using vulnerabilities? If you infect the boot loader, bios, firmware, or some other methods to create persistence then those are vulnerabilities. That a system can get infected in such way is not by any intended design.

Could give an example of a persistence techniques that is using a intended feature of a computer system in order to hide the malware in unintended places in such a way that the system can't remove it (which itself is a unintended functionality).

<comment><USER5> That's like suggesting a programmable page table is a vulnerability, because it can be used to hide a malware stub from processes looking for malware.

However you want to litigate this on a message board, nobody working in platform security or even malware defense thinks that way.

The difference between a persistence vector and a vulnerability is that you can patch a vulnerability, but you can't patch "programmable page table".

</comment></comment></comment></comment><comment><USER5> Backdoors from the 90s included actual purpose-built compilers. Whatever the CIA is doing, a good bet is that it's clownshoes stuff.

(I'll leave this comment here, but it responded to the original editorialized title of the story, which pointed out that this trojan had "its own language".)

<comment><USER6> Sub7 had an editor back in the day that would allow you to build a custom client. I don't remember the technical nature of it. I believe it was just packing configuration and the client into a single executable w/o recompiling, but this is the kind of stuff teenage script kiddies were doing back in the day with VBS and/or Delphi.

</comment></comment><comment><USER7> At this point I think Wikileaks is largely trash. Their agenda is blatantly anti-us motivated.

This leak and the previous one are examples. The CIA is a spy organization. Their job is to spy on foreign influencers. As such, they have spy tools... Just like Russia, China, the UK, etc. etc. If the headline here mattered, it would be, "Spy Agency has Spy Tool". There is no evidence _at all_ that this stuff has been used domestically. But wikileaks buries that headline.

Wikileaks continues to push the agenda that these leaks correlate with the legitimate PRISM leaks that we saw earlier. They push the narrative that these tools are being used on americans, should be ashamed, don't trust your intelligence agencies etc. Fueling the populist anti-intellectual bs we have going on in the world right now.

Don't get me wrong. The US government should be ashamed of PRISM and the actions of the NSA, but Wikileaks should be ashamed of these leaks and the way they're representing this information..

<comment><USER8> > There is no evidence at all that this stuff has been used domestically.

I read the article and they do not imply any connection to PRISM.

Anyway, why would you be fine with such tools being used on non-US targets? Would you not be outraged if an equivalent tool was used on US targets by a non-US government?

<comment><USER7> >I read the article and they do not imply any connection to PRISM.

These leaks attempt to piggy back off of the snowden media frenzy

> Anyway, why would you be fine with such tools being used on non-US targets? > Would you not be outraged if an equivalent tool was used on US targets by a > non-US government?

It is the duty of each government to look after the well being of its people with the resources it has, up to and including spying. Spying has gone on since the dawn of civilization. My opinion extends to foreign governments attempting to spy on the US.

Does it help the human race as a whole? No, but that's a much bigger philosophical debate.

<comment><USER9> The activities of CIA are not and have not ever been good for we "people" of USA. Sometimes if one squints hard enough, it seems that perhaps they have been _intended_ to be for our good, but usually that's not even the case.

<comment><USER7> You opinions about the CIA's actions are just that. I have negative opinions about the CIA's actions in many instances as well. Their possession of these types of tools however, clearly falls within their official stated goal, the standards we should judge them by.

<comment><USER9> It's interesting that 50-USC-46 doesn't seem to actually "officially state" _any_ goal for the agency. That seems too glaring an oversight to be unintentional, so your appeal to such standards seems unfounded. I suggest that by _any_ reasonable standard, CIA should have been shuttered decades ago. Even the tools seen here are indicted elsewhere in this thread by a knowledgeable person as "clownshoes".

<comment><USER7> I don't have time to read the entire law regarding the CIA, so I'll take your word on that, but from their website:

 _The function of the Central Intelligence Agency is to assist the Director of the Central Intelligence Agency in carrying out the responsibilities outlined above.

To accomplish its mission, the CIA engages in research, development, and deployment of high-leverage technology for intelligence purposes. As a separate agency, CIA serves as an independent source of analysis on topics of concern and also works closely with the other organizations in the Intelligence Community to ensure that the intelligence consumer—whether Washington policymaker or battlefield commander—receives the best intelligence possible._

Though I will agree with you that this is just a statement on their website, not law.

>That seems too glaring an oversight to be unintentional, so your appeal to such standards seems unfounded. I suggest that by any reasonable standard, CIA should have been shuttered decades ago. Even the tools seen here are indicted elsewhere in this thread by a knowledgeable person as "clownshoes".

Speculation and opinions are just that, speculations and opinions.

<comment><USER9> _Speculation and opinions are just that, speculations and opinions._

Hilarious! We're discussing CIA. The whole point is that they don't submit to the scrutiny of their employers. "You don't know what we're doing, so we _could_ be doing the right thing!"</comment></comment></comment></comment></comment></comment></comment></comment>


Removing your files from S3 can cost thousands of dollars - <AUTHOR> http://one.valeski.org/2010/05/take-2-amazon-s3-file-deletion-fail.html

<comment><USER2> This guy must have an incredibly large number of files if 2 XL instances running deletes for a month only cuts his S3 usage by 10%. I routinely see 700 DELETEs per second from a small EC2 instance; if performance scales linearly with CPU speed, he should be able to do 700 * 8 * 2 = 11200 DELETEs per second, or 29 billion DELETEs per month; if that's 10% of his objects, he must have 290 billion objects stored.

Except that, oops, S3 only passed 100 billion objects a couple of months ago, and at its current rate of growth is probably still under 150 billion, never mind 200 or 290 billion.

My guess is that the "FAIL" here is whatever process he's using for deleting files -- not in S3 itself.

<comment> brown9-2 He claims to need to delete "billions of objects": [http://one.valeski.org/2010/03/amazon-s3-file-deletion- fail....](http://one.valeski.org/2010/03/amazon-s3-file-deletion-fail.html)

Would it be naive of me to assume that no single person/company is making up a few percentage points of all S3 objects?

<comment><USER3> I checked to see how many files we have on our main CDN (not S3 hosted, but just to get an idea), about 500 million files, there are 6 of those boxes so that's 3 billion files.

We keep three copies of each, but since S3 does that for you I think we shouldn't multiply by 3.

3 billion files is a lot of files, but it isn't that extreme, he could very well be telling the truth.

</comment></comment></comment><comment><USER4> I have also had problems deleting large amounts of data from S3.

The problem that underlies the problem the author was having is that S3 is quite slow and you can't do batch deletes. On top of that, they limit the number of API requests you can make per second. In some case it has been faster to download an object and re-upload it again than do a copy.

<comment><USER5> That seems to make the case for not needing such a large machine to do the delete since you're being rate limited. Was your delete CPU intensive? What setup did you use?

</comment></comment><comment><USER6> $1500 would buy a really nice server or two on a colo rack.</comment>


Show HN: Flatfile – a drop-in CSV / XLS importer for web apps - <AUTHOR> https://flatfile.io <comment><AUTHOR> Hi HN! I'm one of the creators of Flatfile. At the core, it's a simple drop- in, browser-based component (React) that focuses on making the process of uploading CSV / XLS files into any web app nearly effortless. Has a column matching and error resolution experience, and a backend dashboard for supporting customers through import issues.

You can basically just drop this into your app, and with a little configuration get 100% valid and structured JSON data instead of messy, weirdly encoded, and difficult to parse CSV and XLS files.

This is my first time posting a Show HN - But don't go easy. I'd love to hear your thoughts / answer any questions! I wrote a blog post yesterday about my experience launching this as a side project the last couple years [https://medium.com/flatfile-io/flatfile-io-the-new- standard-...](https://medium.com/flatfile-io/flatfile-io-the-new-standard-for- data-import-5d00b6fada85)

Also, there's a developer playground + some docs here: [https://flatfile.io/developers](https://flatfile.io/developers)

<comment><USER2> Next step, make it so that the data can be seen on the website with another widget. I work with some do it yourself web building tools and some of them do not support tabular data. This would fit perfectly if it let you show the data as well.

cool product.

<comment><AUTHOR> Thanks! Yeah, we pitch and expect another system to catch. I would probably think of something like [https://www.npmjs.com/package/tableify](https://www.npmjs.com/package/tableify) for converting the output into a table

</comment></comment></comment><comment><USER3> This is a great example of a successful landing page.

I like that there was no cliche "pricing" menu item, which is on every cookie cutter SAAS website and is kind of a turn off.

They instead go through the sales pitch, listing all the features and examples and have the pricing at the end, which I prefer.

<comment><USER4> What? There is a pricing link at the top. And why would you prefer to have information hidden from you until someone else chooses to show it to you? Maybe a product is prohibitively expensive? I personally like to be able to view the pricing before I bother going through the sales pitch.

<comment><USER3> It seems on mobile, there's no pricing menu item at the top.

I don't think they are trying to hide it, a landing page is basically a sales pitch. And I like to know the features and service I'm looking at _before_ they ask for money from me.

Having the Pricing menu item at the top of a landing page that you have never seen, is like; _" ignore the stuff below, just pay us!"_ which is a bit insulting to me.

</comment></comment></comment><comment><USER5> Nice tool David! Your UI is great!

<comment><AUTHOR> Thanks man! Lots of love went into it.</comment></comment>


Yes the World Does Need a LinkedIn for Women - <AUTHOR> https://medium.com/1-one-infinity/why-we-need-a-linkedin-for-women-37a98a4dc5ed <comment><USER2> If you can flip the roles and it becomes sexist, it's sexist.

</comment><comment><USER3> sexism is not the answer to sexism. It seems we are moving toward a more segregated society, which is sad.</comment>


How to effectively evade the GDPR and the reach of the DPA - <AUTHOR> https://blog.zoller.lu/2020/05/how-to-effectively-evade-gdpr-and-reach.html <comment><USER3> We've actually been threatened with a lawsuit because RocketReach displayed some obviously inflated revenue for one of our customers. Luckily, we were able to prove that the numbers were changed recently and threatened to report them for fraud, which ended this pretty quickly.

Seriously shady company.

<comment><USER4> > threatened with a lawsuit

I don't understand, who threatened you with a lawsuit? Why did they care about RocketReach?

<comment><USER3> We used a product from a company (I'd prefer not to name them) and received an official letter from them that on of our customers had more than 10 million in revenue, which in turn would require us to buy a larger plan from them[0]. They cited the companies (inofficial) RocketReach page as a source and demanded 30k USD (iirc).

They only retracted the thread after we could prove (via Google Cache and archive.org) that the page was very recently modified to show such a big revenue and threatened to report them for fraud.

We probably could've deflected the case since the company was public and therefore its revenue was also public, but, as a very small company, we had neither time nor money to spare for an useless lawsuit. And we assume that this was their bet. We switched to a competitor after this, obviously.

[0] It later actually turned out that this AGB change was after our purchase and not yet affecting us, but we didn't know that at the time.

<comment><USER5> Sounds like that company was as shady as rocketreach... someone who threatens their own customer in bad faith (or negligence) just for 30k is likely to be more trouble and of less value to you in the future if that's their focus of increasing revenue.

Good call ditching them.

[edit] speculative aside...

What if they were intentionally feeding rocketreech miss-information? it might seem far fetched but these personal data collecting companies like rocketreach or even equifax obtain their information from a variety of untrustworthy sources.

I was a victim of this through my own foolishness a couple years ago:

I was using a car insurance comparison site and guessed one piece of the required information I couldn't completely remember - a speeding ticket date - I couldn't remember the exact year. Turns out I entered it exactly one year off, and it was so long ago anyway that it had no bearing on the quotes.

After continuing with my existing insurer, a few months later my insurer sent me a demand for a rather large quantity of money... that's right, they attempted to backcharge me for 5 years worth of insurance over an extra speeding ticket they had "discovered". Obviously there was no way I would pay them but it was extremely difficult to convince them to stop harassing me for this money even though they had no proof. Even after demanding they provide evidence of their discovery which they refused.

It's scary how easy this is to do, and I wasn't even trying.

<comment><USER3> > someone who threatens their own customer in bad faith (or negligence) just > for 30k is likely to be more trouble and of less value to you in the future > if that's their focus of increasing revenue.

Yes, that was really strange. We though they might need quick money, as this was right at the beginning of the corona crisis. Still, not acceptable.

> What if they were intentionally feeding rocketreech miss-information?

That's possible but quite strange. This information is public in our country, so there is a known reliable source and they should've known better.

> Obviously there was no way I would pay them but it was extremely difficult > to convince them to stop harassing me for this money even though they had no > proof.

Yes, this is quite usual. We were seriously lucky we discovered the change; but, given that they backed up so quick after calling it a fraud, I'm seriously assuming it was.

</comment></comment></comment></comment></comment><comment><USER6> Currently there's not much the data protection authorities in the EU can do about foreign companies abusing the data of users.

I assume that in the coming years (or decade?) there will be more efforts to ensure the enforcement of EU law for foreign companies that offer services to EU citizens as part of trade deals.

Right now there's e.g. a flourishing industry of data brokers in Israel that illegally collects data from EU (and US) citizens and sells it, a practice which is hard to stop as well since most of these companies don't have offices in the EU.

I think another possible strategy would be to go after the clients of these companies. If they can't legally sell their data to companies in the EU or US their business model would falter. The GDPR actually mandates that you as a data controller validate that companies which process data for you adhere to GDPR principles. Right now it seems this isn't being enforced much yet but I think it will be soon, which hopefully will have an effect on data brokers outside the EU as well.

<comment><USER7> It is enforced and viral in EU. Think of it like radioactive materials, any operation needs to be fully tracked.

While accessing any user personal details you need to have user consent to process their personal data. You can't simply buy the dataset and assume it has consent. When you buy data from data provider you need to make sure user gave consent to handle data by third-parties to that provider in accordance to GDPR. Users can revoke the consent, every party needs to be ready to handle that scenario. Any data export outside EU GDPR also needs consent. Moreover the dataset needs to be registered with local regulator.

<comment><USER8> _While accessing any user personal details you need to have user consent to process their personal data._

Consent is only one of the lawful bases for processing data under the GDPR. In practice, it's the one almost everyone tries not to rely on unless they can't avoid it, because it comes with extra obligations that other bases might not.

<comment><USER9> Could you list the others? Or at least provide some examples?

Basically all I know are based on either mandatory by law record keeping, or records used to fulfill whatever service/product/goods the user purchased, but even in these cases the processing must be described, right?

<comment><USER8> The GDPR itself is actually quite readable, so if you're interested in the details, you can got to the source. There's a neatly formatted version hosted here:

[https://gdpr-info.eu/art-6-gdpr/](https://gdpr-info.eu/art-6-gdpr/)

What the source material _won 't_ tell you, for better or worse, is how these are interpreted in reality by data controller, processors and regulators. The two main things to know in that respect are:

1\. Relying on the subject's consent is usually the last resort. It comes with lots of extra strings attached.

2\. The "legitimate interests" provision is open to interpretation. It is widely used as an excuse for processing that many of us might consider far from desirable. But it is also a risk for data processors doing things many of us might consider reasonable, because any regulator can take a different view and they get to win by default.

</comment></comment></comment></comment></comment><comment><USER10> I'm not sure how I feel about the screenshot at the end, showing that various policy makers also have their personal information being sold.

I guess the information is out there, and doing so also makes it definitively personal for the policy makers / enforcers involved.

That said, the policy makers / enforcers may be genuinely hamstrung. The US imposes its laws globally because of it's status as a global reserve currency (trading in USD requires the transaction to route via the US, thus making the entity subject to US law).

The EU doesn't have such status or power over US companies. The most it can do is try to prevent them from operating in the region.

As a person who almost certainly has his personal information being sold on this platform, I'm not pleased, and would love to see something done to prevent this kind of activity. Unfortunately, that depends on the US government to take action, and the last 12 years haven't been a flying endorsement of the effectiveness of the current government system. (This is not meant as an statement regarding the effectiveness of either President, but rather a regarding the low output from the system as a whole)

<comment><USER11> > trading in USD requires the transaction to route via the US

Is this correct? How's that enforced? Say, I have a company in Poland which sells some goods for a million dollars to another company in Poland. We both have USD accounts in Polish banks and the transfer is between these accounts. How does the money route via the US?

<comment><USER12> It's not _enforced_ but it's a de facto practical requirement.

If Polbank (forgive me for the bastardized names) wants to give 1M USD to Bankpolska, they either need to ship cash (which can be done but is expensive or tricky) or have a specific bilateral agreement betwene them (which can be done and is done sometimes, but linking every bank with every other bank bilaterally does not scale), or need some interbank settlement system that will do that, but there's no such system in which they can participate. E.g. there's Fedwire but neither Polbank or Bankpolska can be direct members as far as I understand (they generally are not members; I'm not certain if it's caused by some strict limitation or just practicalities and costs.)

So the standard means is to use 'correspondent banks' e.g. USA banks that do that for them. Polbank might have an USD account with Chase or Citi, and Polbank can ask Chase (via a SWIFT message usually) "hey transfer $1m from our account to Bankpolska, it's cover for a customer deal #1234" \- but this means that the transaction "goes through" USA.

Alternatively, multinational banks may have branches in both USA and Poland and so they can be direct participants and settle this directly, however, then it would involve a Fedwire transfer (in USA, subject to USA laws and limitations) between Polbank USA branch and Bankpolska USA branch.

That's standard practice for pretty much every currency. EUR settlement between two American banks usually (not always, there are various options) goes through EU, RUB settlement usually goes through Russia, etc.

If there's a sufficient need, Polish banks _could_ establish an interbank settlement system through which they could transfer USD directly (e.g. similar to the one they have for transfering Polish zloty), but it's a hassle and has costs, so currently they have not done so because for them it's generally not a problem to route all USD payments through USA.

<comment><USER13> Euro dollars are constantly traded without going through the US.

CLS currencies and any currency which is fully convertible can be used in transactions without any involvement of the jurisdiction that minted the currency in the first place.

The USD has a huge settlement infrastructure that is completely independent of the US.

<comment><USER14> Doesn't it still involve accounts in US banks though? Please see my direct reply PeterisP for explanation. I cannot see how could it work without Fed oversight as it would allow it to "print" dollars.

Also, could you please share more info? I'm very interested in financial settlement system, especially for USD and EUR, but sadly there's too little public resources.

</comment></comment></comment></comment></comment><comment><USER15> This same BS is perpetuated by YC backed Apollo.io by simply scraping public LinkedIn profiles & then masking asterisked emails & numbers(usually your company public numbers) & asking people to sign up.

And when you do request them to remove the same, they ask you to provide ID proof. As if one would provide the same to a company which didn't take your consent for the initial profile data either.

I somehow managed to get hold of the CEO's mail ID got mine removed. But I can only imagine what everyone else would have to do when they want to control their web-presence.

<comment><USER16> I've been in touch with a company called Acxiom, who shared my details on Facebook. I've never heard of it, so I submitted a Data subject request to see what they know about me.

They then asked me to provide my address to confirm my identity. Given that I moved quite frequently, and that I'm now asked to share more personal data with a company who's mishandling my data, I wasn't keen on it.

I mentioned that my full name is globally unique, but they refused. I tried to ask them to share some masked data that I can confirm in full (e.g. "give me a partial address and house number, I can give you the full address"). They refused.

They definitely try to make it hard for you, and to dodge responsibility.

<comment><USER17> Acxiom is one of the largest (and oldest, they started in the 1970s) data brokers in the world. I think they, like a lot of other creaky corporations, don't necessarily make things difficult _on purpose_ but they...don't go out of their way to make the bureaucracy any more navigable than it has to be.

In other words, it's not a bug, it's an accidental feature.

<comment><USER8> One good thing about the GDPR is that it was basically designed to allow the regulators to beat up businesses that do that. If you're too old or inflexible to live up to your obligations, congratulations, it's now a liability that could into substantial fines.

<comment><USER18> Has the EU actually shown any teeth to these outfits?

It's one thing to say something is illegal but if you don't enforce that these firms will be able to operate with impunity.

<comment><USER19> Note that in principle it's not up to the EU to enforce because the GPDR is a directive; it's up to the individual member states to enforce the directive as enshrined in their law.

<comment><USER8> GDPR isn't a directive, it's a regulation. It's literally what the R stands for.

The major difference between the two in terms of how the EU makes laws is that directives are the indirect one: individual member states are required to incorporate the provisions into their own legal systems to give them force of law. An EU regulation is the direct equivalent: it carries force of law across all member states immediately. In the case of the GDPR, the UK government has also stated that its provisions will continue here after Brexit and the related transition arrangements.

However, you're right that enforcement will normally be done by an individual member state, because it is typically the national data protection or privacy authority in each state that acts as regulator and has enforcement powers under the GDPR. In theory, there's supposed to be some coordination so one of those regulators will take the lead on any given investigation or enforcement action instead of 28 different organisations all diving in at once, but it doesn't seem to be clear yet how that aspect will work post-Brexit.

</comment></comment></comment></comment></comment></comment></comment><comment><USER16> Data sharing seems so prevalent, and I would dare say even with EU companies, the chances of getting caught (let alone fined) by the GDPR are pretty slim.

An interesting exercise: If you have a Facebook account, go to this page[0] or this one[1] and see if you even _recognize_ some of the companies that shared data about you. Not to mention gave explicit consent to sharing your data ...

My list includes companies I never gave consent to (e.g. Amazon, Uber), never signed up for or gave any details to (e.g. Robinhood, Triplebyte) and some I have zero clue about, but the name alone sounds dodgy (Opteo, Mindshare Biddable Digital ...).

[0] [https://www.facebook.com/ads/preferences/?entry_product=info...](https://www.facebook.com/ads/preferences/?entry_product=information_about_you&section_id=interacted#)

[1] [https://www.facebook.com/off_facebook_activity/activity_list](https://www.facebook.com/off_facebook_activity/activity_list)

</comment><comment><USER20> I had a very similar experience with Apollo.io. Somehow my professional data (business email, personal phone number, name, job title and my LinkedIn network and connections) ended up on this website without my consent. I'm assuming it was collected from several sources such as LinkedIn (Even though I had my privacy settings tight) and some conferences I attended in the past year. Either way I contacted them and they sent me a document to confirm my identity and then proceeded to remove my data from their website after I sent it back. I was a bit shocked as it's basically asking to confirm my identity and give them more information about me when I haven't even granted them permission in the first place. Such "data brokers" need to be regulated. The most annoying thing is that they only remove data under GDPR, CCPA if I am a resident of California, UK or EEA. Well what if I'm from a country that doesn't fall under one of those 2 regulations?

<comment><USER21> Vote for a better government that cares about its citizens digital rights?

</comment></comment><comment><USER22> The achilles heel of the GDPR is that you must act through a DPA. In the case of the Shrems he had to basically sue the Irish GPA in order for them to do their job. And instead of actually doing their job, the Irish DPA instead fought Shrems on behalf of Facebook.

As an EU citizen and resident, it's abundantly clear to me that getting a DPA to act in my best interest is mostly hopeless. I'm reminded of the CANSPAM Act where a US citizen can send their spam to the FTC and have them investigate it. Only they never will. All spam sent to the FTC just goes into blackhole, and next to no one is ever prosecuted. Even when it's clear who the spammer is.

I don't think many people realize this fact. That a politically motivated entity controls European's access to privacy restitution, and they're rarely motivated to actually do anything. This makes the GDPR is my eyes primarily a joke. It certainly isn't about securing my rights as an EU citizen. It seems more written to benefit lawyers and others who make money because things are complicated.

If the EU actually cared about my privacy rights they would allow all Europeans access to restitution without mediating it through national agencies. I want to be able to hire a lawyer and directly take abusive firms to court over GDPR violations. I shouldn't have to act via some pre-court mediator who gets to arbitrarily determine if my claims have merit.

<comment><USER23> What about Article 79, "Right to an effective judicial remedy against a controller or processor"? It reads:

> Without prejudice to any available administrative or non-judicial remedy, > including the right to lodge a complaint with a supervisory authority > pursuant to Article 77, each data subject shall have the right to an > effective judicial remedy where he or she considers that his or her rights > under this Regulation have been infringed as a result of the processing of > his or her personal data in non-compliance with this Regulation.

> Proceedings against a controller or a processor shall be brought before the > courts of the Member State where the controller or processor has an > establishment. Alternatively, such proceedings may be brought before the > courts of the Member State where the data subject has his or her habitual > residence, unless the controller or processor is a public authority of a > Member State acting in the exercise of its public powers.

</comment></comment><comment><USER24> Lusha in NY does this too except they claim the deletion magically happened automatically because of "algorithms".

I'd made a subject access request because they'd sold my personal email address linked to my business position to random spammers. That association didn't exist in any legitimately accessible data, only in the linkedin data breach.

</comment><comment><USER25> Looks like rocketreach is aggregating information that is public on fb,linkedin etc. He forgot to mention that the google search result he got is already selling those, but maybe we ve become blind to that? Rocketreach is packaging and selling it directly, google does it indirectly. Same thing though, are those illegal?

<comment><USER26> This information can also obtained from the chamber of commerce in the netherlands

</comment></comment><comment><USER27> Fundamentally the thing which everyone is missing is that the regulatory authorities can simply say that the data can not be used within the European Union by Rocket Reach. They may not be in the European Union but they can make their product useless in the European Union.

</comment><comment><USER28> Now watch the entire currently-EU based adtech industry relocate out of the EU...

<comment><USER29> It's like drug cartels relocating from Mexico: noone will feel sorry.

<comment><USER30> They would be relocating their corporation only - they'd still be operating in the EU on EU customers.

<comment><USER31> In that case they would still be subject to the GDPR.

<comment><USER30> ...yet since it's unenforceable, then they probably don't care.

<comment><USER32> Why not? If they have offices in EU, raid them. If they have customers in the EU, freeze their bank accounts or sanction their payment processors.

<comment><USER30> > If they have offices in EU, raid them.

They won't - that's what relocation means.

> If they have customers in the EU, freeze their bank accounts or sanction > their payment processors.

This is comical. The government isn't going to start shutting bank accounts for GDPR violations on small foreign corporations, as if they're smuggling nuclear fuel to Iran. Half the bank accounts in the world would be closed if we were so sensitive to regulations.

</comment></comment></comment></comment></comment></comment></comment><comment><USER33> Yes, it's hard for EU authorities to enforce its laws on a company that has no EU presence or revenues to threaten. At least the Luxembourg DPA is doing something about it, unlike the Irish DPA that deliberately does nothing (or worse, colludes with Facebook to help them skirt GDPR with highly dubious and most likely legally invalid semantic contortions).

</comment><comment><USER12> In this particular case, GDPR can get enforced for the _buyers_ of data.

Rocket Reach and similar companies may be outside the reach of GDPR, however, all the advertisers and global platforms who actually want to target EU customers _are_ within the reach of GDPR so it's illegal for them to buy data from Rocket Reach.

</comment><comment><USER34> Another company collecting and selling your personal data right there in Silicon Valley: [https://eightfold.ai](https://eightfold.ai)

</comment><comment><USER35> I don't know if there's another good example, but Poland fined an EU company under the GDPR for scraping profile data without giving proper notification: [https://news.ycombinator.com/item?id=19530087](https://news.ycombinator.com/item?id=19530087)

You shouldn't have to guess where your personal data is going, and how it's being used. When the GDPR was first coming into force, I remember getting bombarded with all these notification emails from all these companies coming out of the woodwork that I didn't recognize. But I don't think I've ever been notified by email, SMS, phone or smoke signal since then.

The biggest flaw of the GDPR in my opinion is that it leaves the definition of what's considered personal identifying information with too much wiggle-room for creative interpretation. Maybe it's hard to pin down exactly, but there's often too much emphasis on the word "identifying", as if it's otherwise OK to gather every intimate online detail and build a profile that is a unique identity in and of itself. It's even worse when real-world decisions can be based on it without your knowledge.

I recently had my own rude awakening learning about these data brokers and risk analysis services. The matter itself was relatively trivial, but I didn't realize the extent of this before and the scope of what personal information they're gathering. And it doesn't matter if you think it won't affect you, since you've done nothing wrong. From what I read elsewhere, even exercising fundamental consumer rights may be held against you. [https://news.ycombinator.com/item?id=21440526](https://news.ycombinator.com/item?id=21440526)

<comment><USER19> > The biggest flaw of the GDPR in my opinion is that it leaves the definition > of what's considered personal identifying information with too much wiggle- > room for creative interpretation.

Note that this is the totally normal approach for Civil Law systems: you define the general principles of what the menace is, and leave it down to the courts to determine whether or not those principles have been violated. In essence, you can view it as every case being decided on the basis of the mischief rule as exists in many Common Law systems.

</comment></comment><comment><USER36> > Instead of pursuing Rocketreach locally on that basis alone, the CNPD just > gives up arguing it has no jurisdiction in the US.

Which is true and obvious. Why anybody ever thought the GDPR would have teeth outside the EU is beyond me. It was always laughable to me that anybody believed that the EU had made a law that applied to every company in every country in the world.

</comment><comment><USER37> I had a similar experience with a company called RateSetter.

\- They email me some marketing

\- I respond with DSAR

\- They acknowledge receipt of DSAR

\- 6 months pass

\- I bump the email thread

\- They respond saying they have deleted my data as per my request (I requested access, not deletion)

\- I point this out

\- They apologise and offer £100 to drop the complaint

\- I refuse and complain to ICO

\- Obviously nothing happens

GDPR is toothless.

</comment><comment><USER38> The Privacy Shield framework that was just declared invalid by the EU included a requirement that US companies make themselves available for arbitration of disputes brought by EU data subjects. GDPR by itself doesn't include that concept. But if GDPR is going to be enforceable, the negotiation around a successor to Privacy Shield should probably include it.

</comment><comment><USER39> I've always wondered about the practical side of how GDPR is supposed to work for companies outside the EU.

If you've got actual _stuff_ in the EU, it's easy. You get fined under GDPR and if you never show up to argue your side in court or an administrative hearing or whatever, they seize your real estate or bank accounts or physical servers or whatever, and sell it to pay your fines.

If you're US-based, how does it work? Hmm, if you're a modern shop you probably have stuff hosted by big companies, like servers on Amazon's AWS or code on Microsoft's Github. Then the EU could presumably tell those companies to stop hosting your stuff, or they'll become liable for fines as an accessory to the violation. Microsoft and Amazon probably have a lot of bank accounts and physical stuff in the EU that could be seized and sold, so they couldn't simply ignore the fine. They'll probably drop you as a customer immediately once Europe starts making them pay fines, and maybe try to sue you in the US court system to try to recover those costs.

I've never heard of this happening though. So maybe this isn't actually a thing.

If all your stuff is on US soil, and you're careful not to use providers with any European presence, how would they do it? Does the EU have some way to order all European ISP's to blackhole traffic from your company's IP ranges? When your executives come to Europe for vacation or conferences or whatever, could they get hauled off the plane in handcuffs and taken to a European jail over your company's GDPR violations?

Again, I haven't heard of this actually happening. But it seems to me that would be how they'd do it, if they really wanted to prevent overseas companies from simply ignoring GDPR.

If there's no threat of enforcement, why bother with GDPR at all, unless you're planning on having seizable _stuff_ like real estate or bank accounts or physical servers in Europe someday?

</comment><comment><USER40> Is crunchbase/owler/cb insights and every other public data aggregator/lead generator service also illegal by the same logic?

<comment><USER41> define illegal.

and no, those aggregators don't process PII / Personal Data.

<comment><USER40> They typically have a person's different social media account links and work history. Isn't that personal data?

</comment></comment></comment><comment><USER42> I figured that the European Union would simply act to block such a website from being resolved in Europe by DNS resolvers?

</comment><comment><USER43> This constitutes denial of justus and you can sue them either in your country or in the European Court of Justus.

</comment><comment><USER44> For most, you can simply ignore it since it doesn't apply anyways

<comment><USER44> I'd to hear why I was downvoted. it is a fact people forget. Most have websites have that stupid cookie notification when 99 don't need to. Here is how to evade the gdpr, ignore it like it doesn't exist

</comment></comment><comment><USER45> It's always nice to not find yourself in one of those databases.

Data frugality ftw.

</comment><comment><USER46> GDPR compliance would be trivial if web browsers used a stateless request- response hypertext transfer protocol.

 _O Tempora O Mores_

</comment><comment><USER47> Does GDPR apply here? They might not be selling to the EU, and they aren't monitoring EU persons but just selling historic information. I don't read GDPR as applying globally to any and all trade in EU personal data. [https://gdpr.eu/companies-outside-of-europe/](https://gdpr.eu/companies- outside-of-europe/)

<comment><USER48> GDPR applies, it has worldwide scope for data on EU citizens. On the other hand, European courts lack jurisdiction to enforce their laws on companies without EU offices and assets.

FWIW I'm _really_ glad that EU courts lack this jurisdiction - any gain from privacy would more than be wiped out from losses to free speech, especially with the extensive history of libel tourism.

<comment><USER49> It's hard to make an argument for the EU courts having that jurisdiction without also granting the same to Saudi Arabia and China.

<comment><USER50> The way it works is that the EU fines their EU-based operations or stops them from operating in the EU. And if they don't have any, those of their customers who do could not legally acquire their data on EU citizens without the subject's informed consent anyway.

</comment></comment></comment></comment><comment><USER51> Typical of this kind of regulation: the real purpose is less about ensuring individual rights and more about giving bureaucrats more power. The GDPR is great in the latter sense. It's impossible to predict the outcome of a legal process even if you do your very best to comply, and you can be slapped with incredible fines... Cross the wrong bureaucrat and your days are numbered (in an economic sense).

<comment><USER51> Ops, that was obviously a controversial standpoint. Just to be clear: I'm all for individual rights. But laws need to have predictable consequences and be fairly and equally enforced, and my impression is that the GDPR is not. As an example I'm pretty sure the local court here in Malmö, Sweden has violated my rights under the GDPR. Do you think anybody would give a rats ass if I complained? I highly doubt it...

<comment><USER52> It might depend on the country, but Austrian authorities treat GDPR compliance very seriously. Even if the authorities got away, such cases make for embarassing press coverage and can threaten precarious coalitions, especially when elections are close.

</comment></comment></comment><comment><USER53> "Rocketreach has not met the requirement of the GDPR to name an EU representative (Art27) to account for the processing of European Personal Data. In their answer, the CNPD makes it sound like it is optional, it isn't. Instead of pursuing Rocketreach locally on that basis alone"

LOL, yes.

I'm sure they also do not meet the legal requirements of North Korea, Saudi Arabia, and many others.

Likewise, various EU corporations do not meet the legal requirements of non-EU places like those. Would he prefer that they did?

Even more interesting, since he expects the US to follow EU law, how does he feel about the EU following US law? The US has that Patriot Act, and lots of EU companies are not compliant. Maybe he should report a few EU companies to the FBI.

<comment><USER54> > I'm sure they also do not meet the legal requirements of North Korea, Saudi > Arabia, and many others.

China is the most straightforward example, companies cannot operate unless they basically do it through an - implicitly Chinese state controlled - partner company. China also has a literal Great Firewall monitoring, modifying or stopping all cross-border traffic. So yes, you have to play by their rules if you want access to the market.

US, EU and other western countries also require you to follow their - <USER2> more lenient - laws and rules for access to their market, but for now it's rarely enforced through blocking etc. Saudia Arabia, Russia, India, Turkey and other "second world" countries block a lot of services that don't follow their laws or government commands. Same thing: follow da rulez or our market is closed to you.

North Korea has their own exclusive "internet" and blocks all access to the regular internet except for a few highly monitored and controlled locations like universities and government institutes, which are not connected to the NK internet. Not comparable at all.

> Even more interesting, since he expects the US to follow EU law, how does he > feel about the EU following US law? The US has that Patriot Act, and lots of > EU companies are not compliant.

This is effectively already the case for a large part. All non-china global IaaS companies are US, so everyone has to play by US rules and law. I don't believe for a second that the NSA cannot get the data from the European Google/Amazon/Microsoft data centers.

<comment><USER55> I think your information may be a bit out of date, in China you can own and operate as a WFOE

[https://en.m.wikipedia.org/wiki/Wholly_foreign- owned_enterpr...](https://en.m.wikipedia.org/wiki/Wholly_foreign- owned_enterprise)

<comment><USER56> WFOEs indeed exist, but there are many restrictions on the types of business they can conduct, both directly[1] and indirectly because activities require licenses[2] that WFOEs can't get. The grandparent post was wrong in the details, but it's still a different world from the USA or EU.

1\. [https://www.fdichina.com/blog/china-company- registration/ftz...](https://www.fdichina.com/blog/china-company- registration/ftz-negative-lists/)

2\. [https://www.china-briefing.com/news/entry-strategy-chinas- on...](https://www.china-briefing.com/news/entry-strategy-chinas-online- gaming-market-opportunities-license-compliance/)

</comment></comment></comment></comment><comment><USER57> When are we going to admit that GDPR is a failure?

Asserting a bunch of rights around personal privacy is great, but I've yet to see any compelling evidence that the relevant courts and bureocracies are capable of enforcing the law effectively. EVERYBODY is cheating.

Every time this is brought up on HN, the response is to wait for when the big fines start coming.

It's been two years. They're not coming.

<comment><USER58> Some pretty big fines have been issued already. See:

[https://www.enforcementtracker.com/](https://www.enforcementtracker.com/)

Over time I expect them to go up further as companies can no longer claim they did not have enough time or were not aware of the law (that never was a defense anyway but DPAs tend to be lenient. So far).

Since the GDPR has come into effect I see in my practice that companies are a lot more aware of their responsibilities towards their users, have better processes and security in place. Is it perfect? Not by a long shot but the improvement is immense and as time goes by and more companies end up setting an example of how things should be done and those that don't end up getting find I expect this trend to continue.

What I like most about the GDPR is that it steers towards compliance, not towards making life of businesses unnecessary harder.

Contrary to you I think the GDPR is a resounding success, the only thing that would make it much better still is if other areas of the world would take up similar legislation so the playing field would level.

<comment><USER59> I dont know.

From what I can understand of German/Google translate, the third from top:

[https://www.enforcementtracker.com/](https://www.enforcementtracker.com/)

Link to .pdf:

[https://www.ris.bka.gv.at/Dokumente/Dsk/DSBT_20180927_DSB_D5...](https://www.ris.bka.gv.at/Dokumente/Dsk/DSBT_20180927_DSB_D550_084_0002_DSB_2018_00/DSBT_20180927_DSB_D550_084_0002_DSB_2018_00.pdf)

Is the Austrian Authorities making a 300 Euro fine to a "common citizen" making "illegal" use of a dashcam (it seems - but I am not sure about it - that the issue is that the car is not - how? - visibly marked as videorecording?).

Anyone more familiar with German (and legal German) can clear the matter/explain?

<comment><USER58> Why would you pick that example, rather than the 16 million fine an Italian company received?

<comment><USER59> As a counter example to the "success" you mentioned.

Again if I got it right a "common user" got stinged because of a dashcam.

The Italian example you refer to is actually a success, like most other ones, I was objecting not to the Law in itself (that is IMHO a good one) but rather on how it is applied, here and there, in spots and seemingly in a random way.</comment></comment></comment></comment></comment>


How can developers help fix the coronavirus issue? - <AUTHOR> I was wondering if there are any open problems to solve where software developers in helps in order to fix the coronavirus epidemic and the chaos created because of it. <comment><USER2> [https://www.linkedin.com/feed/update/urn:li:activity:6644298...](https://www.linkedin.com/feed/update/urn:li:activity:6644298397719371776?commentUrn=urn%3Ali%3Acomment%3A%28activity%3A6644298397719371776%2C6644356646657105920%29)</comment>


Google to build 1M square foot building in post-Brexit London - <AUTHOR> http://venturebeat.com/2016/11/15/google-to-build-1-million-square-foot-building-in-post-brexit-london/ <comment><USER2> > (Reuters) – Google delivered a vote of confidence in London's future as a > technological hub after the Brexit vote on Tuesday by announcing plans for a > new building in the King's Cross area of the city that will house thousands > of extra engineers.

There's something wrong with this story. I've been hearing about a Google building in King's Cross ever since I moved here, so that's been around 2 and a half years. They didn't just announce it.

<comment><USER3> They spent over a billion dollars buying the land for this HQ in 2013... I mean, I guess it's good they aren't pulling the plug but yeah, there's no new news here.

</comment></comment><comment><USER4> Maybe they're betting on the British pound to drop so they can hire engineers on the cheap. My understanding is that salaries (for software developers) in London are already pretty low compared to the US and even Canada.

<comment><USER5> Or Google is getting sick and tired of being raked over the coals by EU bureaucrats with manufactured problems and wants to move its talent and money into a country without annual witch trials and billion dollar fees?

[https://www.google.com/search?q=google+eu+fines&ie=utf-8&oe=...](https://www.google.com/search?q=google+eu+fines&ie=utf-8&oe=utf-8)

<comment><USER2> I don't mind that we demand that companies pay taxes.

I can assure you that if it was up to me they'd either pay or get blocked. I really don't understand this mentally whereby companies avoiding taxes is Ok. Who do you think pays the taxes that they don't?</comment></comment></comment>


Silicon Valley's Dumb Money - <AUTHOR> http://awealthofcommonsense.com/2016/05/silicon-valleys-dumb-money/ <comment><USER2> I would posit that it's not just money that's dumb, there simply aren't that many smart people in general. You can't escape the bell curve whether it's developers, investors, sales people and, worst of all, consumers.

</comment><comment><USER3> This presumes that VC money isn't also dumb money - often smart (or rather cunning) only in how deals are arranged to maximize leverage over founders.</comment>


Let's Encrypt the house – how I use LE for my internal homelab setup - <AUTHOR> https://whynot.fail/homelab/lets-encrypt-the-house/ <comment><USER2> Sounds neat but I couldn't follow what problem this was solving. Having an internal network that uses Lets Encrypt certs and updates automatically?

I solve that with Traefik in a docker container. It's setup with Cloudflare as my DNS provider using the DNS challenge and Lets Encrypt to issue certs. By slapping a few labels on any Docker container I get to use https and a subdomain on my real domain name. Everything is automatic and, while configuration was hard, I didn't have to custom roll anything special. I don't need an external VPS (but I do need Cloudflare to act as my domain provider, for free). Seems like that might be way easier for most people?

<comment><AUTHOR> I actually set it up and it's much better than my previous setup. Thank you! Editing the post that it's totally useless (but leaving it on the net anyway, just with a note on top).

I'm kinda disappointed as I was very proud for the setup but hey, let's not get emotional over bits n bytes :)</comment></comment>


Property: Giving superpowers to CSS variables - <AUTHOR> https://web.dev/at-property/ <comment><USER2> Exciting seeing more SASS features coming to CSS. Being able to edit CSS without a build step makes it much more flexible and clear.

</comment><comment><USER3> Wow this push the boundaries of what is possible! So sad that as usual, Firefox will slow down human progress.</comment>


What is wrong with new generation of programmers? - <AUTHOR> I have been interviewing programmers for last couple of weeks for full stack developer position,And really disappointed with quality of answers I've got so far .
I figured out a common trait among almost all whom I interviewed , They seems to know 'What it does' but dont know what it is or how it does.
For example, They do not know what database indexing is or how it works(They know what it does though, improve query performance, wow!)
Ask them few intermediate level sql queries to write and they would be like I did it using ORM of framework xyz.
Or at programming language level, I asked what singleton is, they replied correctly about what it does, but failed to answer how it works, and interestingly answer were different from 'experts' of different language or framework, ROR guys said it works probably by installing gem, PHP guy attempted to write class but couldnt , javascript guys usually like 'meh, I dont know'
Asked them about REST and all would correctly recite its verb and full form, but would fail miserably when you want to know from them what is purpose of doctype, Or how http works under the hood or sometimes they dont even know what is GET and POST.
I remember things were different around 10yrs back, People used to write lot of code from scratch and had less tools, I am not blaming tools or framework for it, but new gen programmer seems to be so dependent on tools that they dont even care to know about whats going on under the hood. they would just read few blogs, read tutorial of framework/tools and start calling themselves expert.
EDIT : Formatting <comment><USER2> You hear very often that the programmer industry is "age-ist", in other words, the older programmers have a hard time getting hired.

This post is the other side of the coin: older programmers think that when they entered the industry it was "the golden era", and everything since then has gone downhill. This is akin to someone in the late '90s asking all entry- level programmers "you mean, you don't regularly disassemble your object code to verify that the compiler did the right thing? What's wrong with you, why are you relying on your tools so much?"

The fundamental issue is that the industry is maturing, but the names of the positions for which you are hiring hadn't quite kept up with what you are looking for. People whose primary job is to make sure that data gets from the database to HTML and back really don't need to know what is under the covers. This position is called "product programmer". Their job is to translate whatever your UX guy cooked up into things that actually work. If you then need to make sure that whatever they made scales up to thousands of requests, you need to have a different person called a "performance developer" \- these people really hate translating UX wireframes into code, but once that part is done, they can optimize everything up and down the stack with their eyes closed.

Also, this is a personal pet peeve of mine: if you are asking what a singleton is in a job interview, you are interviewing for the wrong thing, so no wonder you are getting the wrong interviewees.

<comment><USER3> Pretty much everything the OP mentions is reasonably basic "core IT" concepts. Depending on what he means by "intermediate sql" that might or might not fit into that category. I don't think it's unreasonable to expect a "full stack" dev to have at least a basic understanding of things like DB indexing, singletons, http get vs post. Where I would agree with you is to become proficient with these concepts it does involve spending some time in the industry so the age thing might be the factor here as well. Personally I would strongly prefer these types of questions to the "recent CS graduate" type questions some companies are notorious for and not just because I can answer these :)

<comment><USER2> When people say "full stack", it means something very specific to me.

It means you can do both UI and server code, so you have a wide variety of knowledge in the many competing front-end and back-end frameworks out there, but you are not quite an expert in anything. At your last job you learned all the magic incantations of jQuery, Rails and MySQL, but but today you are expected to learn all the magic incantations of React, Node and Mongo. Specifically because you are expected to change your toolkit quickly, you do not dedicate any time to scratching the surface.

Certainly when hiring for "back-end developer", I expect deep knowledge of OS- and DB-level concepts like indexing and garbage collection.

Likewise, when hiring a "front-end developer", I expect deep knowledge of various mobile, desktop and tablet devices and the behavior various browsers exhibit on them.

<comment><USER3> "full stack" is definitely an overloaded term but then again I wouldn't consider the kinds of questions OP mentions to be something only the experts would (or should) be able to answer on the basic level.

Things like get vs post, encoding, basic headers (like accept etc) should be familiar to anyone dealing with the client side development regardless of the tools/frameworks.

And then on the backend some basic SQL, basic understanding of indexing, FKs, PKs, maybe simple inner joins is also something that someone doing anything DB-related on the backend should be familiar with (unless we're talking someone who is only familiar with nosql dbs) regardless of the tools/frameworks/orm etc.

</comment></comment></comment></comment><comment><USER4> How well do you understand an internal combustion engine?

Do you know it well enough to be able to diagnose an issue ad then fix it? Can you listen to it, identify when it's not performing well, and then make the tweaks to it in order to get the most efficiency? If you say yes, I would be very surprised, and yer every day, people all over the world rely on these things to get them to the place they need to go.

There is nothing wrong with these developers, they don't know these things because they simply do not need to. The tools have abstracted away those details, as they should. There is simply too much in the domain of writing software to know, and even in web development, there is too much to know. Most people focus on what is needed to get the job done because in most places, you are judged by your results and not your knowledge. What good is know what a singleton is if you never use it? Why do people need to know what the doctype does when <!DOCTYPE html> is good enough for the browser to interpret? By abstracting away these details, people can focus on higher level problems, like solving the actual requirements.

If people need to learn those things, they will do so.

Good developers seek to learn about their tools. Great developers learn to separate out the wheat from the chaff and focus only on the knowledge that is important.

<comment><USER5> I think that's fine for developers starting out. But once you start writing software professionally that is for "the enterprise", it's important to understand the underlying stack. Knowing how your kernel's plumbing of pipes actually works (for example) is invaluable in certain cases. Knowing how your framework's black boxes work is very useful when you run into performance problems or find bugs. Yes, abstractions are good. But the higher you go up the stack of abstractions, the less you know about what's actually happening and thus the less you'll be able to do to actually debug it.

EDIT: Sorry, I didn't read the OP. Yeah, his requirements don't make much sense (unless he can do the whole industry a favour and explain what he means by "full stack" developer).

<comment><AUTHOR> Hi, Thanks for comment. Personally speaking by "Full stack" developer, I expect someone who has enough knowledge of client as well as server technologies so S(h)e could understand (or better anticipate) problems or challenges usually s(H)e would face while working on it. It's not about working for "enterprise", even if it's an in-house product I expect a programmer to know what S(h)e is doing well. If someone do not know what tool/framework really doing while one using it,I wouldnt consider it ideal.

</comment></comment></comment><comment><USER6> So, your offering a job rewriting the internet...that's pretty cool. Oh, right it's just another CRUD job. Also, why not write all your interview questions down on a piece of paper and leave them for an hour with internet access. If they still can't answer your questions then you have a problem. And, if there a gaps maybe train them a little bit. I think if everyone stopped wasting so much time looking for ninjas, experts, and unicorns and started hiring people the world would be a lot better off.

<comment><AUTHOR> Hi, Agree that interview should be less academic, but academic knowledge does matter. it's not really about how much candidate can answer , for that written/technical test is enough, IMHO Personal interview meant to find out how candidate approaches to solution, And thats when its disappointing when candidate would say I know what "db indexing" does but do not know how. If one do not know how, it signifies candidate is not interested( or capable) enough to know problem and thus if selected chances of him/her writing erroneous code would be more. .

<comment><USER7> I understand where you're coming from, but the thing is asking :

\- What is DB indexing

\- Can you think of any impact when indexing in this situation

\- What are you general thoughts about indexing, can you think of an example when you used it and how it helped.

Will indeed give you insight into how he approaches problems and his maturity. But the difference with this and : tell me how indexing works behind the scenes, is that you focus on what really matters.

With that there is no right or wrong answers, there's just insightful answers.

I do not disagree on the fact that fundamental knowledge is important, but I disagree on the way you assess it. I disagree on the notion that there is an equivalence between fundamental knowledge and formal knowledge. One can be comfortable adding things up without knowing that it is called an addition.

If he does : great, if he doesn't it doesn't matter as long as he has the insight.

More generally the majority of our knowledge is not formal.

<comment><AUTHOR> > \- What is DB indexing

Yeah I asked exactly same question, And instead of answering what it i, candidtate told me why they used it.

> \- Can you think of any impact when indexing in this situation

That's Why I want to know if candidate understand what really index does, One cant tell a realistic answer of this question until S(h)e knows how index works.

> \- What are you general thoughts about indexing, can you think of an example > when you used it and how it helped.

Even if one used it in some situation in pastm and it helped, I would consider it not enough as it could be possible S(h)e used it by reading some blog/tutorial etc without understanding it fully, it's like ok switch it off and on, did it work? well forget about it then.

</comment></comment></comment></comment><comment><USER8> I've recently interviewed in 10+ companies (interviewed as went to the last part of the process) and just changed jobs.

I have a degree in computer engineer and 5 years of experience, what I have to say:

1\. In the interview process I always feel really hard to know how deep I should go in the answer... I had the same design an short url service in two different companies, one complained that I went too deep and didn't talked about the thing on a system level too much. Other said I didn't went too deep and looked like over heardish as I focused on the system level.

2\. I've studied a lot of algorithms and data structure as every company thinks they are google and asks for big o notations, algorithms optimisations, implement a tree, reverse a tree blablabla. This part was the easiest in all companies as I "gamed" the system by studying it weeks in advance. Does it make me a better developer? Probably not, I already knew that stuff but if you asked me to go deep in those questions when it wasn't fresh in my mind I wouldn't be able to develop an answer. I am pretty sure that 6 months from now I won't be able to answer them in a satisfactory manner

3\. I had the same singleton question and I didn't answer correctly, gave an answer like you said. Basically, I know what a singleton is, I've used this pattern in the past. Even in a C++ embedded systems app running in a touchscreen + ARM. But, please, don't ask me nuances about it. If I really wanted I could just game it as well by reading some cracking the interview questions on design patterns.

The best interviews I had were a CHAT where I talked about previous project, challenges and etc.

Not some question / answer quiz game where the interviewer is in a position of authority and I am always concerned that I am giving the answer he is expecting.

<comment><USER9> You're forgetting not everyone _could_ game the system like that. Not everyone is as motivated as you, either. These tests (even if they are contrived, unrepresentative of real work, and gameable) give desirable results.

A test who's only flaw is that motivated individuals with good problem solving skills could "game" if they worked hard enough?

That sounds like a good tool for finding strong candidates to me.

</comment></comment><comment><USER10> First you should look at the job advertisement and the money you offer. You might be missing the people you want and getting what you pay for. I don't think the number of people with deeper knowledge is smaller than 10 years ago. There are more jobs and more people after those jobs while the number of people who know their stuff stays the same.

If you pay less than 75% percentile for programmer job, you can't expect too much.

Occupational Employment and Wages, May 2014 15-1131 Computer Programmers [http://www.bls.gov/oes/current/oes151131.htm](http://www.bls.gov/oes/current/oes151131.htm)

<comment><USER11> Yes, would be curious to see his job posting and what he is paying. If you are paying average wage, you are going to find (at best) the average programmer.

Also you need to be aware of self bias when interviewing. You ask him things YOU think are simple, and are shocked when they don't know it. Of course the only sane way to do A singleton object in Java is to do a double checked lock with a volatile keyword, it says so right in "java concurrency in practice!"... But if a candidate has never needed to write a hand rolled singleton (say always worked in Spring), meh.. I bet they know things that you don't know. Ask a candidate you reject as dumb to ask you 10 hard questions he can prove he knows the answers to, and see how many you get right. If he stumps you on all 10, then you just proved that you each know things the other does not, which may mean you would be a good team working together ;)

P.S. I know how DB indexes work because I find it interesting. Does that actually help me? So far I have never used that knowledge, but hey - I sound cool in interviews!

<comment><USER12> I love your reverse interview idea. I am gonna start doing that (:

<comment><USER11> Hah let me know how it goes ;) Have never got around to it, but after being on both sides of super frustrating interviews I bet it could give some surprising results.

</comment></comment></comment></comment><comment><USER13> There's a lot of demand for programmers. In the marketplace, there are tool makers, and tool users. Some people make and use tools, of course, but for the most part workers will fall primarily into either camp. Naturally, tool makers are more expensive. It's possible the OP is paying a tool user rate and wants a tool maker. Due to the high demand for tool makers they are expensive.

</comment><comment><USER14> Ten years ago the stack was _much_ smaller than it is today.

If this is the "new generation", they probably have little experience. And you want "full stack" developers -- that's a really tall order, most people will work in a team where some people do one side and others do the other.

So you're looking for people who have little experience, and are still more about all aspects of this huge stack than just how to use it. It's not realistic.

Also, you're _both_ blaming them that they are dependent on tools, _and_ only asking them questions about various tools.

<comment><USER15> Exactly. I was concentrating on backend the last few years and didn't have much frontend tasks. I recently switched to a new project that was developed by a generally younger team, and it's way more complex.

Instead a JSP / ASP / Rails template files with a single CSS and JS page, along with JQuery and a few libs, there is a whole build process with dependency hell and a dozen tools to master - bower, grunt, npm, karma, etc.

Writing the frontend with promises and async patterns is actually more difficult than the backend which has been slimmed down to basic Rest -> Service - DB calls. Honestly, I'd probably do pretty poorly on an interview too, these days, if a 'Ninja Full Stack' dev. was desired.

</comment></comment><comment><USER16> I see the same with new guys. When they need to calculate the intersection area between a few rectangles they spend hours looking on GitHub instead of writing 50 lines of code. And they have no idea how even a simple linked list implementation could look like.

There seems to be something in the training that tells them it's not cool to implement things themselves, instead only libraries are OK.

<comment><USER17> I have had the same experience. Give them an API and they glue things together like there is no tomorrow, dealing with edge cases I probably couldn't have come up with on LSD.

Ask them to fix a bug in a procedure using call/cc, and in the best case they will rewrite without call/cc 3 times as long.

</comment></comment><comment><USER18> The people you want are still out there, but either you are not filtering for them, or some filter is keeping them out.

For junior candidates we use a written test followed by a phone screen to avoid inviting time-wasters and resume spammers into the office; We're sometimes flexible on that for candidates who clearly have relevant experience.

</comment><comment><USER19> I think this generation shift isn't new, and I think it's "standing on the shoulders of giants." It's what makes each generation more effective than the last.

Specifically, I've worked with very smart individuals who grew up working on ARM processors and when tried to transition to javascript had difficulty with something like Ajax. I being one generation younger, am somewhat oblivious of ASM, hard disk sectors, etc, but since the abstractions aren't leaky it actually let's me focus on more theoretical stuff.

Moreover, I think one generation from now things like caching will automatically be decided at the storage-engine level, and things that are now manual-work will become abstracted away allowing engineers even greater productivity.

edit: clarity

</comment><comment><USER20> I think they aren't being lazy, and they've adapted to a culture that wants things finished quickly at the expense of quality, or anything else really.

Edit: The same culture doesn't recognize that "done" implies a certain level of quality.

<comment><USER21> This. "Lean" and "agile" emphasize feedback and iteration over great engineering. For a large number of apps, that's acceptable.

<comment><USER22> "agile" and, _especially_ , "lean" emphasize feedback and iteration _as tools_ to achieve quality and fitness for purpose (i.e., "great engineering".)

They do not emphasize feedback and iteration _over_ quality.

<comment><USER21> Perhaps "complexity" should be used instead of "great engineering". The best engineered solutions are those that are simple, easy to understand, and fulfill business needs (profit, time to market, etc) vis-a-vis satisfying the programmer ego (In saying this, I'm pointing the finger back at myself as one who really loves to over-engineer)

</comment></comment></comment></comment><comment><USER23> Lately, instead of greeting people with "Hello" or "Hey man," I usually start off all my conversations with this speech.

I agree so much so I had to double check that I didn't actually write it.

</comment><comment><USER24> "How do you deliver software your users/clients want?" is really the only technical question you should need to ask the 10+ crowd.

The highly-experienced type of dev who could easily walk out of the interview and become your next competitor is the one you want to hire.

So, would you ask a competitor questions about one pattern, inner vs. outer joins, and naming conventions? Or would you use that time to actually learn/discover info about your (potential) competitor?

</comment><comment><USER25> I don't think you should limit this to the "new generation." I am at around 5 years experience in the dev workforce and believe I can answer those questions to a satisfactory degree. From interviewing and professional experience these are questions that seem to give everyone trouble. I interviewed a guy with 15-20 years of experience and he didn't understand B-Trees and indexing but his mentality was: why should I bother? that is the DBAs territory.

Obviously, my background is different. I think this is the root of your issue. Most people are learning what they need in order to get a job completed. Most of the time it doesn't involve them having to build the entire application, they are asked to work on a small portion.

All things considered, I agree that devs should understand more in depth the core components of web applications.

</comment><comment><USER26> Your last paragraph resonates with my experience learning. After two years of working with Rails I realized it's like the Mac of frameworks. It does so much to prevent me from thinking or looking under the hood that I don't actually know what's going on. So part of my reading this year is on topics like REST so I actually get it, and also building things without a framework (rather than use the devise gem, build my own login system, build own router, etc). It's pretty painful because I'm so used to effortless progress through gems, but very worth it.

I think people in my situation just get comfortable and forget that they need to keep digging deeper. I think this also differentiates someone being just a programmer vs an engineer.

</comment><comment><USER21> > I remember things were different around 10yrs back

You say you're hiring for a "full stack developer". I'd say that requires far more broad knowledge than a developer 10 years ago, when we tended to specialize back-end/front-end/ops.

</comment><comment><USER27> OP's question is a generalization over a whole generation and can thus hardly be true.

What might be true though is that knowledge in software programming is much more specialized than a couple of years ago. IMHO that is a sign of a growing industry.

That aside I have the feeling that some people did not have the pleasure of a good education in software development, which is the reason they miss some of the more fundamental bits (or they did not pay attention which is another issue). That may be due to the fact that it is easier for lesser-educated (sorry for my english) people to get into software development and that may be due to great demand for developers in general.

</comment><comment><USER28> What's wrong with them? Their magic line is too high :-D. You should probably read Steve Yegge's posts about it[1][2], he explains better than I could.

1: [https://sites.google.com/site/steveyegge2/practical- magic](https://sites.google.com/site/steveyegge2/practical-magic) 2: [https://sites.google.com/site/steveyegge2/age-racecar- driver](https://sites.google.com/site/steveyegge2/age-racecar-driver)

</comment><comment><USER29> Perhaps your interview process is suboptimized for finding talent.

Google famously struggled with this issue. They found the best predictor of how someone will perform in a job is a work sample test (29%).

This entails giving candidates a sample piece of work, similar to that which they would do in the job, and assessing their performance at it. Even this can't predict performance perfectly > [http://www.wired.com/2015/04/hire-like- google/](http://www.wired.com/2015/04/hire-like-google/)

</comment><comment><USER30> I would probably fail your interview but at the same time I could probably join your team and be one of the best members of that team. Funny how that is possible.

I would definitely fail the singleton question but if it came up in work (and it hasn't for me) then I would either ask someone or look it up and then I would know it.

EDIT: Although to contradict what I said above.. I would do intense study before interviewing for a job I wanted so I would probably do well..

</comment><comment><USER31> Programmers are expected to know about more than ever before. There are more abstractions than ever before. While I wouldn't let a senior developer get away with not knowing those things, I would give beginners and intermediate developers some leeway.

I would opt to learn how much code they've written and their general programming ability over trivia.

If they are smart and capable, they can dive into the details of any challenge that comes their way.

</comment><comment><USER32> Somewhat related to the general discussion I'm reading in the comments: [https://www.williamghelfi.com/blog/2015/12/26/william- versus...](https://www.williamghelfi.com/blog/2015/12/26/william-versus- technical-interviews/)

</comment><comment><USER33> This reminds me a lot of what mxcl went through with Google

[https://twitter.com/mxcl/status/608682016205344768](https://twitter.com/mxcl/status/608682016205344768)

</comment><comment><USER34> I do not believe you have to know what happens inside. And concept of blackbox in engineering actually comes from this insight. But the approach to solve a problem is more important. That is why debugging a code is blamed in testing.

<comment><AUTHOR> When I say, candidate should know what happens inside, I do not mean he should be knowing all minute detail of it, but a generic high level understanding of tools/framework being used is expected so if required one would not hesitate to go under the hood and try to find problem.

</comment></comment><comment><USER35> We've been taught by the "old generation of programmers".

Think about it.

</comment><comment><USER36> Sounds like you should find better candidates....

</comment><comment><USER37> Lets look at it from an evolution of technology angle.

10-15 years ago we had pain points with web apps and making them more fully featured/responsive and dealing with things breaking at scale. They were kind of clunky to build so, we did what all good engineers do and figured out how to make things easier to use for everyone. Now a days, the "kids" are learning how to build web apps that don't require a deep level of knowledge about the stack - we've done a fantastic job building out the tooling that abstracts a lot of these concerns away from the "average" developer. They know if I install framework X and use Library Y the two will communicate correctly. I might understand how they communicate, but don't need to know the intricacies at this time.

Technology is always evolving and what you need to know at a deep level is changing. 60s/70s/80s - The size of the program, how it was laid out on disk, disk sectors, memory layout. All that mattered to get the best performance. You needed to understand how the hardware worked intimately.

90s/ early 00s- Managed languages (Java etc.) become more of a thing and you need to understand less about all the gotchas the prior generations faced. The framework and language abstracted a lot of the prior generation's challenges away and increases in hardware capacity reduced the need to layout programs just so on the disk or in RAM.

Early 00s-Present - Hardware kept growing exponentially in terms of capacity and performance, and we started placing more emphasis on building out scalable, fully functioning web applications (thin clients) rather than thick clients running on a desktop. People figured out a lot of these applications were simply wrappers about DATABASE activities and wrote frameworks that made writing the backend for such apps KISS simple. RoR I'm looking at you! This freed up people to focus on the bells and whistles the user interacted with and reduced the need for anyone developer to know the intimate details of how their framework worked.

It's a changing world and the next generation of programmers is always going to be starting at our current level of abstraction, much like we started at our prior generation's and built our way up. Challenges we faced, overcame, and turned into libraries, become the starting assumption for the new kids that they'll build up from.

On the flip side, I gotta ask, what are you looking for? There are so many "full stack" combinations (language, server, SQL/NoSQL database, front end scripting library) now that the odds of finding a person with deep experience using everything you are using, unless you've reduced to the lowest possible common denominator, is low. Are you looking for someone you can train into the position and has enough knowledge not to blow their foot off, or do you want someone who you can plugin ready to go? Your expectations will influence how you're viewing the candidates, your expectations, and your relative measure of disappointment in them.

</comment><comment><USER38> Maybe you should accept that not everybody knows everything. Your questions are nice to impress your geek buddies but do they really help you to deliver quality?

So what they use ORM instead of raw SQL? Your role as interviewer is to figure out if they would be able to learn. I can almost see your ads: 5 years of OOP with design patterns, 5 years of MySQL database, 3 years of API, etc.

How about you try to figure out if they can learn? A few months ago, I interviewed a junior guy. We were looking at a simple code exercise he did before coming in. I asked "Why did you lose a List here?" "I don't know, I needed a collection" "Do you know how List is implemented?" "No". I spend a couple minute explaining about singly-linked Lists, and ask "Now that you know, would you use it again?" "No, cause it's terrible for append operations".

He didn't know, but I explained and he understood alone. Why don't you try to explain and see if they can come to conclusions?

<comment><AUTHOR> >Your questions are nice to impress your geek buddies but do they really help you to deliver quality?

I do not ask question to impress, it's to identify whether candidate is capable enough to understand problem domain and come up with efficient solution.Please read post, I am okay with learning part, and asked question for which candidate claims him/herself as expert. I have no issue with they use tools (ORM etc) , but if they dont know how tool works before using it, it's not ideal for me.

<comment><USER39> Compiler also is a tool. Do you disregard people who do not know how compilers work as well?</comment></comment></comment>