Inside the Largest US Voter Data Leak - <AUTHOR> https://www.upguard.com/breaches/the-rnc-files <comment><USER2> Speaking as a guy with a lot of experience with voter data ( I built the first "where do I vote" apps for Google and helped found the voting information project):

This is actually almost entirely public data. Yes, including addresses and phone numbers and political affiliation. There are some states that is not public as part of the voter file, but you can still get it other ways publicly. For example: USPS, etc. Some states/players would make you sign agreements not to use it for commercial purposes.

The modeling info included is not public.

Acquiring 50 state data can be a bit of a pain, but there are at least two major players that will sell it to you. (I remember one of them literally laughed when I told them we would want the databases without any personal info included, because we just wanted the address to various political precinct mapping.)

<comment><USER3> > This is actually almost entirely public data

Birthday is an included item. That's definitely private as it is often used to confirm identity.

"almost public" is meaningless. One data item, like credit card number, or birthday, can make this a dangerous leak.

<comment><USER4> Birthday is almost certainly public. You can get it easily from the DMV or from any of several dozen commercial providers that resell government data:

[https://www.dmv.org/public-records/](https://www.dmv.org/public-records/)

That it's used to confirm identity shows how weak identity-theft protections are at most institutions, not what's public information. (For that matter, mother's maiden name is basically public information as well: you can get it from genealogical records.)

<comment><USER5> The site you linked to is not affiliated with any state DMV. It links to a sketchy background check service that appears to be a scam.

Edit: spelling

<comment><USER6> The link was preceded by the words "from any of several dozen commercial providers that resell government data:" so that's how I took it. It would be at a .gov domain if it were state affiliated.

<comment><USER3> It said from the DMV. No evidence of that here.

Also, "reselling government data" implies that the government explicitly gave permission to a business selling your data. I doubt that's true. More likely, these entities gathered data from whatever entities they could, probably various private companies from which you've made online purchases.

</comment></comment></comment></comment></comment></comment><comment><USER7> As long as the CEO of an company (RNC) that gives data to an outsourcer (Deep Root Analytics) is not going to jail to give data to an unqualified company, nothing will change.

If the CEO goes to jail, things will change very rapidly (CEO will manage his CMO much tighter who will first want to see an security audit not older than 6 months).

At least CEOs I have reported to as CTO were very sensitive for implemention issues in areas that could land them in jail.

Same for every other hacking (e.g. Sony) or IT failure (e.g. British Airlines crashed DC).

<comment><USER8> What law did they break, exactly? These aren't medical or financial records.

A careless programmer makes a bad choice and the CEO has to go to jail? Come on.

<comment><USER9> >?A careless programmer makes a bad choice and the CEO has to go to jail? Come on

An institutional failure of review, testing and security that will lead to tens of billions of dollars of identity theft goes unpunished completely?

Come on.

A CEO is responsible for his organization. If you ruin lives, you have to pay the price.

Can't handle the heat?

Don't take the job.

I hate how CEO's get hundred million dollar parachutes because, the risk and danger and difficulty of such a position warrants such extravagant pay.

But, then, we ask them to be responsible, bear responsibility for the organization which paid them a hundred million dollars to be responsible,and we say "come on?"

Utterly ridiculous.

CEO's bear responsibility for their organizations, or the organization should not exist. There must be responsibility for private organizations, lest the concept of private organization be nothing more than a cheap trick to remove criminal and civil liability from wrong doing.

<comment><USER10> This isn't social security numbers.

This is all publicly available data scrapping stuff. Like your public Facebook profile.

If you don't want that stuff to be leaked, then don't put your info publicly on Facebook.

<comment><USER11> I didn't put mine on Facebook. What's step 2?

<comment><USER10> Are you registered to vote?

Then your info is publicly available for anyone to get. The government will just give it to you.

</comment></comment></comment></comment></comment></comment><comment><USER12> I hate to be in the position of defending a leak such as this. But if what they've done is "merely" compiling data that was available from our public profiles, are they obligated to secure that compilation? I'm asking -- I don't know for sure how the data was gathered, it just sounds like it was from scraping public records + public web sites.

Also, can someone ask Troy Hunt whether he has or can get access to this data so he can let us all know if we're on it? (But will it even matter if they don't have an email address field?)

<comment><USER13> Personally, while I understand that this is nothing illegal, I think it's terribly wrong.

We just handed everyone, including the 5% of society who tend towards sociopathy, a nicely tagged, collated (and yet probably slightly inaccurate) list of minorities.

Hate women? You have a nice list which includes the names, addresses, and telephone numbers of all those women.

Hate muslims? Boy do I have just the list for you. Blacks? Republicans? White muslim men who live in the same neighborhood as you?

Let's omit the sociopaths for just a moment, and let's look at the ad networks. Can you picture how much more accurate a picture those companies have of you now? They no longer have to guess at your age, ethnicity or religion - they now _know_. What could go wrong when _that_ list of "legally" collated data gets combined with the RNC leak, and is subsequently itself leaked?

So no. It's probably not illegal to compile these lists. It's probably not even illegal that it was released. But it was, for certain, a damned immoral thing to do, and there will be consequences.

<comment><USER4> I don't really disagree that this is a shitty data breach, but on the consequences, I'll point out that the 5% of society who tends to sociopathy already have much lower-effort means to target minorities:

Hate women? Look for boobs.

Hate muslims? Look for brown skin.

Hate Republicans? Look for MAGA hats.

That these are inaccurate signals is irrelevant: haters gonna hate, and I really don't think they care whether the brown-skinned person they're harassing is _actually_ a Muslim, they just want a target for their anger.

As for ad networks - they already have much more accurate models of age, ethnicity, and religion than the RNC has. There's a lot more money involved in targeting ads, and so they've put a lot more effort into it than political consultancies. Worrying about them is like closing the barn door after the horse is out.

<comment><USER13> > lower-effort means to target minorities

It doesn't get much lower effort than downloading a list, popping it into Excel, and sorting on a column (I'm willing to bet that some hate groups are already doing this and will release very specific lists to their membership). And with phone numbers and a couple of bucks, you don't even have to leave your house to send them hate messages by the thousands.

Haters gonna hate - far too flippant a phrase to describe those who emotionally and physically assault their targets.

As for the ad models - the RNC release has a very specific DOB, location, gender, and phone number. Some of these the ad network could guess at, but this provides concrete data.

<comment><USER4> It gets _a lot_ lower effort than downloading a list, popping it into Excel, and sorting on a column (really, how many non-techies would do that?). Get on the subway, find girls with brown skin, start screaming obscenities, pull knife when confronted, murder.

Look, I don't want to minimize the impact of hate or harassment on victims. It really is terrible, and should be challenged whenever possible. I _do_ want to inject some realism into the discussion of the likely consequences of this breach. The people who would go out and harm other people because of their ethnicity or religion don't particularly care if they get the ethnicity or religion right. (I'm reminded of a time when I was carrying my wife's purse while she went shopping in a nearby store, I walk by a pickup truck, and the guy inside is loudly muttering "Fucking faggots" over and over again. And then I met my wife in the parking lot, give her a kiss, hand her back her handbag, and he laughs a big "Ha-ha!" of relief and drives off.)

And the people who would do mass harassment over the phone have much easier ways to get this data, like e-mailing their state voter registry and asking for it.

</comment></comment></comment></comment></comment><comment><USER14> Wow, not good:

       "State", "Juriscode", "Jurisname", "CountyFIPS", "MCD", "CNTY", "Town", "Ward", "Precinct", "Ballotbox", "PrecinctName", "NamePrefix", "FirstName", "MiddleName", "LastName", "NameSuffix", "Sex", "BirthYear", "BirthMonth", "BirthDay", "OfficialParty", "StateCalcParty", "RNCCalcParty", "StateVoterID", "JurisdictionVoterID", "LastActiveDate", "RegistrationDate", "VoterStatus", "SelfReportedDemographic", "ModeledEthnicity", "ModeledReligion", "ModeledEthnicGroup", "RegistrationAddr1", "RegistrationAddr2", "RegHouseNum", "RegHouseSfx", "RegStPrefix", "RegStName", "RegStType", "RegstPost", "RegUnitType", "RegUnitNumber", "RegCity", "RegSta", "RegZip5", "RegZip4", "RegLatitude", "RegLongitude", "RegGeocodeLevel", "ChangeOfAddress", "COADate", "COAType", "MailingAddr1", "MailingAddr2", "MailHouseNum", "MailHouseSfx", "MailStPrefix", "MailStName", "MailStType", "MailStPost", "MailUnitType", "MailUnitNumber", "MailCity", "MailSta", "MailZip5", "MailZip4", "MailSortCodeRoute", "MailDeliveryPt", "MailDeliveryPtChkDigit", "MailLineOfTravel", "MailLineOfTravelOrder", "MailDPVStatus", "MADR_LastCleanse", "MADR_LastCOA", "AreaCode", "TelephoneNUm", "TelSourceCode", "TelMatchLevel", "TelReliability", "FTC_DoNotCall"

<comment><USER15> That does not even nearly fit on one line, so I broke it up. Yeah, it looks pretty bad. Here's hoping it's only sparsely filled out.

State

Juriscode

Jurisname

CountyFIPS

MCD

CNTY

Town

Ward

Precinct

Ballotbox

PrecinctName

NamePrefix

FirstName

MiddleName

LastName

NameSuffix

Sex

BirthYear

BirthMonth

BirthDay

OfficialParty

StateCalcParty

RNCCalcParty

StateVoterID

JurisdictionVoterID

LastActiveDate

RegistrationDate

VoterStatus

SelfReportedDemographic

ModeledEthnicity

ModeledReligion

ModeledEthnicGroup

RegistrationAddr1

RegistrationAddr2

RegHouseNum

RegHouseSfx

RegStPrefix

RegStName

RegStType

RegstPost

RegUnitType

RegUnitNumber

RegCity

RegSta

RegZip5

RegZip4

RegLatitude

RegLongitude

RegGeocodeLevel

ChangeOfAddress

COADate

COAType

MailingAddr1

MailingAddr2

MailHouseNum

MailHouseSfx

MailStPrefix

MailStName

MailStType

MailStPost

MailUnitType

MailUnitNumber

MailCity

MailSta

MailZip5

MailZip4

MailSortCodeRoute

MailDeliveryPt

MailDeliveryPtChkDigit

MailLineOfTravel

MailLineOfTravelOrder

MailDPVStatus

MADR_LastCleanse

MADR_LastCOA

AreaCode

TelephoneNUm

TelSourceCode

TelMatchLevel

TelReliability

FTC_DoNotCall

<comment><USER16> Plus the supposedly "startlingly accurate" preference and views modeling that is linked to all personal details and publically accessible. While someone could always deny it being correct it again raises questions about responsibility in data collection.

> RNC_RegID, State, 2012ObamaVoter_DRA_12_16, 2012RomneyVoter_DRA_12_16, > 2016ClintonVoter_DRA_12_16, 2016TrumpVoter_DRA_12_16, > AmericaFirstForeignPolicy_agree_DRA_12_16 > AmericaFirstForeignPolicy_disagree_DRA_12_16 > AutoCompaniesShipJobsOverseas_agree_DRA_12_16 > AutoCompaniesShipJobsOverseas_disagree_DRA_12_16 > CorpReputs_AmericanMakers_DRA_12_16, CorpReputs_DailyLives_DRA_12_16, > CorpReputs_Egalitarians_DRA_12_16, CorpReputs_EnviroConscious_DRA_12_16, > CorpReputs_OpportunitySeekers_DRA_12_16, > CorpReputs_STEMSupporters_DRA_12_16, CorpReputs_SupplyChainers_DRA_12_16, > CorpReputs_Unifers_DRA_12_16, DemLeadersStandUpToTrump_DRA_12_16, > DemLeadersWorkWithTrump_DRA_12_16, DParty_DRA_12_16, > FinancialServicesHarmful_agree_DRA_12_16 > FinancialServicesHarmful_disagree_DRA_12_16 > FinServicesCompany_Dreamers_DRA_12_16 > FinServicesCompany_RiskMitigators_DRA_12_16 > FossilFuelsImportantForUSEnergySecurity_DRA_12_16 > FossilFuelsNeedToMoveAwayFrom_DRA_12_16, > InvestInfrastructure_agree_DRA_12_16, > InvestInfrastructure_disagree_DRA_12_16, LowerTaxes_agree_DRA_12_16, > LowerTaxes_disagree_DRA_12_16, NonReluctantDJTVoter_DRA_12_16, > NonReluctantHRCVoter_DRA_12_16, PharmaCompsDoGreatDamage_agree_DRA_12_16, > PharmaCompsDoGreatDamage_disagree_DRA_12_16, > ReformGovtRegulations_agree_DRA_12_16, > ReformGovtRegulations_disagree_DRA_12_16, ReluctantDJT_Above.5_DRA_12_16, > ReluctantHRCVoter_DRA_12_16, RepealObamacare_agree_DRA_12_16, > RepealObamacare_disagree_DRA_12_16 RParty_DRA_12_16, > StopIllegalImmigration_agree_DRA_12_16, > StopIllegalImmigration_disagree_DRA_12_16, TrumpStandUpToDems_DRA_12_16, > TrumpWorkWithDems_DRA_12_16, USAFinancialSituation_Optimistic_DRA_12_16, > USAFinancialSituation_Pessimistic_DRA_12

<comment><USER17> Formatted:

        RNC_RegID

        State

        2012ObamaVoter_DRA_12_16

        2012RomneyVoter_DRA_12_16

        2016ClintonVoter_DRA_12_16

        2016TrumpVoter_DRA_12_16

        AmericaFirstForeignPolicy_agree_DRA_12_16

        AmericaFirstForeignPolicy_disagree_DRA_12_16

        AutoCompaniesShipJobsOverseas_agree_DRA_12_16

        AutoCompaniesShipJobsOverseas_disagree_DRA_12_16

        CorpReputs_AmericanMakers_DRA_12_16

        CorpReputs_DailyLives_DRA_12_16

        CorpReputs_Egalitarians_DRA_12_16

        CorpReputs_EnviroConscious_DRA_12_16

        CorpReputs_OpportunitySeekers_DRA_12_16

        CorpReputs_STEMSupporters_DRA_12_16

        CorpReputs_SupplyChainers_DRA_12_16

        CorpReputs_Unifers_DRA_12_16

        DemLeadersStandUpToTrump_DRA_12_16

        DemLeadersWorkWithTrump_DRA_12_16

        DParty_DRA_12_16

        FinancialServicesHarmful_agree_DRA_12_16

        FinancialServicesHarmful_disagree_DRA_12_16

        FinServicesCompany_Dreamers_DRA_12_16

        FinServicesCompany_RiskMitigators_DRA_12_16

        FossilFuelsImportantForUSEnergySecurity_DRA_12_16

        FossilFuelsNeedToMoveAwayFrom_DRA_12_16

        InvestInfrastructure_agree_DRA_12_16

        InvestInfrastructure_disagree_DRA_12_16

        LowerTaxes_agree_DRA_12_16

        LowerTaxes_disagree_DRA_12_16

        NonReluctantDJTVoter_DRA_12_16

        NonReluctantHRCVoter_DRA_12_16

        PharmaCompsDoGreatDamage_agree_DRA_12_16

        PharmaCompsDoGreatDamage_disagree_DRA_12_16

        ReformGovtRegulations_agree_DRA_12_16

        ReformGovtRegulations_disagree_DRA_12_16

        ReluctantDJT_Above.5_DRA_12_16

        ReluctantHRCVoter_DRA_12_16

        RepealObamacare_agree_DRA_12_16

        RepealObamacare_disagree_DRA_12_16

        RParty_DRA_12_16

        StopIllegalImmigration_agree_DRA_12_16

        StopIllegalImmigration_disagree_DRA_12_16

        TrumpStandUpToDems_DRA_12_16

        TrumpWorkWithDems_DRA_12_16

        USAFinancialSituation_Optimistic_DRA_12_16

        USAFinancialSituation_Pessimistic_DRA_12

<comment><USER18> How could one tell if a vote is "reluctant" or not, based on the available data?

<comment><USER19> Lots of ways. Perhaps you know they're a super-loyal Republican voter and you have things that correlate with that, such as willingness to vote when presented with an online poll. You might know they were reluctant because the person shifted between the available alternatives during primary season - perhaps supporting Jeb! Bush, then Marco rubio, then Ted Cruz, then John Kasich, before finally falling into line when Trump won the nomination. You could then infer that Trump was the absolute last choice but that they would still 'hold their nose' and vote for him in the general election, either due to loyalty to GOP on particular policy issues or because of some long-standing hatred of Hillary Clinton, or just a history of being very conformist in political matters (as many, many people are).

</comment></comment></comment></comment></comment></comment><comment><USER20> _"'Microtargeting is trying to unravel your political DNA,' [Gage] said. 'The more information I have about you, the better.' The more information [Gage] has, the better he can group people into "target clusters" with names such as 'Flag and Family Republicans' or 'Tax and Terrorism Moderates.' Once a person is defined, finding the right message from the campaign becomes fairly simple."_

Neal Stephenson wrote a book called _Interface_ which predicted a form of tech-enabled micro-targeted politics over 20 years ago. It was disturbing at the time; it's almost considered business-as-usual now.

I believe American democracy would benefit from including the study of such techniques in our educational curriculum. When I was in school, we studied advertising techniques to help us be skeptical. We need the same for targeted political messages now.

<comment><USER19> I agree, but citizen education should not imho be the only approach here. I'm for _much_ more muscular privacy laws and a slightly narrower tolerance on what's acceptable political speech.

Of course education is great, but look at the vast financial and operational asymmetries between even the most informed individual and well-resourced corporate actors like political parties. I have a super-strong political immune system but being politically engaged and navigating social media is exhausting. For the sake of objectivity I have to systematically expose myself to opinions I find disagreeable lest I retreat into a bubble and be surrounded by confirmation bias, but continuous exposure to countervailing political ideologies is intellectually and morally tiring, given the intense polarization and visceral rhetoric that prevails in today's political discourse.

Despite not liking programming, I've been seriously thinking about building a virtual assistant that I can train to pre-emptively tag people using my peculiar ideological criteria so that I can avoid or at least prepare for certain interactions that I know are going to be psychically difficult. By my value calculus, tuning out of politics is irresponsible at best and suicidal at worst; only communicating with people whose values you share exposes you to confirmation bias, and and inevitably exposes one to manipulation; observation of and argumentation with antagonists is psychically expensive and potentially dangerous.

so much as I agree with you on education, it's not something we can just put on the to-do list and wait a generation to benefit from. And that would be true even if we had a well-functioning educational sector rather than one that fails a large number of children and adults by leaving them only semi-literate and -numerate. People who can't read or reckon well are poorly positioned to identify fallacious political discourse.

<comment><USER21> I revel in my filter bubble and labor to improve it.

For policy wonks and activists such as myself, discourse, persuassion, marketing, are distractions from the real work of getting things done.

Firstly, because people vote their identity. Period. Almost no one votes on the facts, the issues, the policy, the platforms, whatever. There are no undecideds, no independents. Cite "Democracy for Realists".

Secondly, victory is achieved by mobilizing your supporters. You bring the heat, whoever is sitting in the chair will see the light.

The only distinction is if a voter is willing or unwilling to bother casting a ballot.

</comment></comment></comment><comment><USER22> This raises so many questions...

Why is U.S. voter registration made public at the individual name/address level?

Why do the states publish their voter registrations in the first place?

Why should private campaign operations (or anyone else) have access to this data?

Shouldn't voters' privacy be protected by the states?

Is there a privacy policy you can review when you register to vote?

<comment><USER23> So that the two parties who voted for this to be the case can have unfettered access to their potential voters, go to their houses, send them things, and know for sure they're only hitting up people in their party, so as not to mobilize the other side.

Nothing that would harm the 2-party system ever changes in the US, and nothing ever will.

<comment><USER24> But why does it need to be public?

In the US, you register with the party no? So the party you register with your information, and they can do what you said

> go to their houses, send them things, and know for sure they're only hitting > up people in their party, so as not to mobilize the other side.

<comment><USER25> Nope, in the US you register with the state and you can optionally tell the state that you are a member of some party.

You can register to vote and self-identify as 'Dem' 'Republican' etc while getting your driver's license.

Aside from the public voter file update, the Democratic Party doesn't get any special notification if you pick them, and you don't need to apply or get accepted in any way.

</comment></comment></comment></comment><comment><USER26> I don't typically don hats with this much tin foil, and I don't think this is _likely_ , but...

The real danger of data like this, in my opinion, illegal usage for voter fraud.

Find people who are likely to vote against you and likely to have poor voter registration documents, and remove them from the polls so they can't vote.

Find people who aren't likely to vote at all and vote on their behalf. In- person, the only verification required is name & address. By mail, the only requirement is a signature, which can be obtained from receipts (I assume this is available on black hat markets).

Leaving this S3 bucket as public-read allows for deniable coordination with illegal actors. I can't imagine they did this on purpose but that could be an explanation.

I don't know if it's possible, but I hope the FBI / Mueller team is able to get access logs.

<comment><USER27> No. The data that would be useful for wide-scale voter fraud is already widely available from public/free sources, including state Secretaries of State or Departments of Elections.

The loss here is all the very expensive extra modeling and demographic work that isn't included on those files. But having that doesn't massively alter the mechanics of the voter fraud effort you're describing.

<comment><USER26> The expensive modeling (and data collection) makes it much cheaper and more feasible.

I agree that it doesn't change the fundamental mechanics, or enable otherwise impossible attacks.

</comment></comment></comment><comment><USER28> I have this theory that the only way regular people will start caring about privacy breaches such as this one is to use that data against them in a malicious way. Tell the average Joe that the data of all US voters has been leaked, "Hmmm. That's bad." and they move on with their lives as if nothing happened. Instead, if this data is used to impersonate the average Joe on social media or if it's used to trick their mobile carrier into porting out their number, then they'll take notice. (I am _not_ suggesting people do this, it was just part of a thought experiment)

Unless the company involved is sued to bankruptcy and the people involved are prosecuted, sending a strong message to companies dealing with user data, nothing will change. But that's unlikely to happen as this company is backed by the RNC.

While we're on the topic of collecting personal data of people, there's a simple solution : just don't collect it unless it's absolutely necessary. Stop asking me to broadcast my address in my newsletter. Stop asking me to submit my billing address when I make payments online. Stop asking me for my mobile number when I visit a fast food restaurant. Most of the companies that collect this data are not competent enough to keep it secure. The reason companies ask for an address to broadcast in users' newsletters is some anti-spam act which does not prevent the spammers from doing their job. I imagine it's also a requirement for companies to collect a billing address for certain types of online payments. Change the law to remove these poorly thought out legislature.

More generally, we need regulations on how user data is used by companies. They should not be allowed to store user data indefinitely. If a user closes an account with a company, retain the data for a short period (eg- 1 year) and then delete the data automatically. Companies should not be allowed to build shadow profiles of users.

<comment><USER29> >I have this theory that the only way regular people will start caring about privacy breaches such as this one is to use that data against them in a malicious way

The other thought experiment is to adopt the opposite point of view that privacy is overrated. You can adopt various worldviews from 'everything is grey' to 'this is good, that is bad" but punishing someone to care about same things you care about is a pretty terrible approach. If you can't convince someone, it doesn't mean they're stupid, it could also mean you just aren't that good at communicating, or that perhaps what you think is important isn't all that important.

>Unless the company involved is sued to bankruptcy and the people involved are prosecuted, sending a strong message to companies dealing with user data, nothing will change.

Or we can give them tools that make it easier to secure data. I've always found that if you make it easy for someone, they almost always end up doing the right thing. As it stands the security products/services domain is a complicated maze where you have to be an expert to evaluate how various products work internally and which services, if any are worth purchasing.

<comment><USER30> There's a pretty compelling argument though, that people _aren 't_ good at making long-term assessments of diffuse, but potent, risks, and/or are willing to, or can be coerced into, arbitraging long-term interests with short-term exigency.

Gresham's law, availability hueristics, optimism bias, distribution of cognitive skills, various aspects of game theory, and more, strongly suggest this.

Examples: global warming, pollution risks, resource depletion, moral and morale hazard, just off the top of my head.

<comment><USER29> Yeah, and all those biases apply to people on HN too - You (the proverbial) aren't good at making assessments either. Also, flipping it for arguments sake, how come software companies are never penalized for introducing software bugs? How would you like it if an accountant wanted to send you to jail because you introduced a security bug and their critical data got wiped out and they went out of business? Or should we go back to blaming them for not having backups because we have an a-priori assumption that "shit happens" when it comes to software? Well, the other side could say 'shit happens' too.

<comment><USER30> Those are actually questions I'm actively exploring.

Some recent discussion (from myself and others) on this G+ thread:

[https://plus.google.com/+YonatanZunger/posts/267XsJzKM5B](https://plus.google.com/+YonatanZunger/posts/267XsJzKM5B)

I've discussed Gresham's Law dynamics numerous times at my subreddit/blog. See particularly:

[https://www.reddit.com/r/dredmorbius/comments/2h63fp/is_gres...](https://www.reddit.com/r/dredmorbius/comments/2h63fp/is_greshams_law_a_special_instance_of_a_more/)

I've been meaning to write up a bit expanding market price dynamics beyond the set of goods that Adam Smith defined: labour, capital, commodities, rents, and (indirectly) interest.

In particular, the question of _risk pricing_ , which is treated almost wholly as a _financial_ question rather than an economic one.

The question of pricing under duress is a key one -- the Backward-'S' bending supply curve is a curious economic anomaly:

[https://www.reddit.com/r/dredmorbius/comments/53mcxn/backwar...](https://www.reddit.com/r/dredmorbius/comments/53mcxn/backward_s_shaped_supply_curves/)

Also the behaviour of natural resource stocks under supplier pressure -- the price will fall to the lowest levels possible, and supplied volume will _increase_ , if possible, for a number of highly perverse reasons. The collapse of oil prices following the East Texas oilfield discovery, from ~$1/bbl to first $0.13/bbl, then $0.02/bbl, before wellhead production was siezed at force of arms by the Oklahoma and Texas national guard, and Texas rangers, comes to mind.

[https://www.reddit.com/r/dredmorbius/comments/6etqey/oil_is_...](https://www.reddit.com/r/dredmorbius/comments/6etqey/oil_is_not_a_drug_oil_dependence_is_not_an/)

</comment></comment></comment></comment></comment><comment><USER31> I think people are missing the big legal liability .... this information has been published to the world, it contains estimates of people's deeply held political beliefs - some of it will be wildly wrong and those people might consider that they have been libeled .... roll on the lawsuits

<comment><USER32> Libel, in the US, requires the publisher to either known the claim is false ormto publish it with reckless disregard for the truth when it is, in fact, false; I don't think that there is anyway that this could be construed to meet that, no matter what legal theory as to who the liable publisher is you use.

</comment></comment><comment><USER33> I still don't understand the concern.

Want the name, age, gender, home address, mailing address, party of registration, and voter history from every registered voter in North Carolina? Here is the "leak" on Amazon S3. [http://dl.ncsbe.gov/index.html?prefix=data/](http://dl.ncsbe.gov/index.html?prefix=data/)

Except, by leak, I mean, link I got from my state board of elections' homepage.

</comment><comment><USER34> Related: This legally mandated "leak" happened years ago, and it even included signatures. Most of the citizens pushing for the recall of a sitting republican governor were democrats, so it seemed like punishment for a Republican state assembly to pass this one-off policy. Especially for those who have zero internet-presence. The first thing many employers see when they search for politically active democrats is this information, paired with "quick searches" of their names on criminal, pedophile and dangerous persons databases. If you want to disinsentivize political action, this is how you do it. I won't link to that site, but can confirm it's still up.

[http://www.cnn.com/2012/02/01/politics/wisconsin-recall- peti...](http://www.cnn.com/2012/02/01/politics/wisconsin-recall- petition/index.html)

</comment><comment><USER35> Are the leaked files floating aroubd on the internet, or were they able to shut it down before anyone else got to it?

<comment><USER8> I've seen no evidence that anyone but the security researcher found this, but maybe that'll change.

</comment></comment><comment><USER36> Except this isn't a leak. This is publicly searchable information. I don't know why people are blowing this out of proportion.

<comment><USER37> Because technologists don't know everything about laws and society, even though they like to think they do. You might add that not only is it searchable, but the Freedom of Information Act and its derivative implementations in state laws mandate that it be made accessible in this way for no cost other than the expense in compiling the records.

People getting angry when "government transparency" is supposedly such a good thing no one questions? Go figure...

</comment></comment><comment><USER38> > It would ultimately take days, from June 12th to June 14th, for Vickery to > download 1.1 TB of publicly accessible files

Do security firms have special permission to do this? Because as a private citizen, I am pretty sure I would go to jail if I tried this.

<comment><USER39> IANAL, but I'm pretty sure if someone leaves a Top-Secret document on the ground, and you pick it up, you don't go to jail for that. The data is accessible to the public -- this isn't a hack, this is just downloading.

<comment><USER40> The only people who can go to jail for mishandling classified information are people with security clearances.

</comment></comment></comment><comment><USER41> Being discussed here:

[https://news.ycombinator.com/item?id=14586920](https://news.ycombinator.com/item?id=14586920)

<comment><USER12> Great to merge the discussion but IMO the Upguard article is original/superior details.

<comment><USER41> Agreed -- the Upguard article is a better topic link for this.

</comment></comment></comment><comment><USER42> If you're a glass half full person at least they care about the do not call list enough to give it its own column.

<comment><USER43> They take the DNC list and subsequent enforcement _very_ seriously. [https://www.ftc.gov/news-events/media-resources/do-not- call-...](https://www.ftc.gov/news-events/media-resources/do-not-call- registry/enforcement)

source: work at place with large call center. Avoiding DNC fines is among one of our top priorities.

secondary source: S.O. works at place with call center for a large bank. Same thing.

Also anecdotally: If you file a complaint with the FTC for an unknown number that keeps calling you back without them giving you a chance to "opt out" (this is most scammer numbers), you file a ticket with the FTC, and they usually respond to the ticket within 2-3 days. (Another funny thing -- they use Zendesk.). I stopped receiving the calls since filing the report.

As to why they take it so seriously? My guess is it's easy money for them. Kind of like traffic tickets for cops.

<comment><USER44> Political calls are exempt from the DNC list:

[https://www.donotcall.gov/faq/faqbusiness.aspx#who](https://www.donotcall.gov/faq/faqbusiness.aspx#who)

The parties take it seriously because they don't want to lose a vote by pissing someone off.

</comment></comment></comment><comment><USER45> Is this available online in a searchable way? I want to see what it has on me...

</comment><comment><USER46> What are the legal implications of a campaign using this leaked modelling data? Would it be breaking any existing laws to use data that was leaked to the public without the permission of the original company that did the modelling? (If nothing else, I suppose it's probably a copyright violation.)

Hypothetically, could one deliberately leak a trove of modelling data with some fake voters inserted, and then monitor the mailbox associated with that fake voter and sue any organization you don't like that sends campaign flyers for using the data without permission?

</comment><comment><USER47> TLDR. Was any of the information non-public?

<comment><USER48> The results of data science models are contained in the data dump, and those are non-public. The rest of the information is accessible via public records or registries maintained by states.

</comment></comment><comment><USER49> > Spreadsheets containing this accumulated data—last updated around the > January 2017 presidential inauguration—constitute a treasure trove of > political data and modeled preferences used by the Trump campaign

Genuinely curious: can you really have 198 million rows in a spreadsheet?

<comment><USER45> It's provably a CSV not meant to be used by excel.

<comment><USER50> Correct:

>Each file, formatted as a comma separated value (.csv), lists an internal, 32-character alphanumeric "RNC ID"—such as, for example, 530C2598-6EF4-4A56-9A7X-2FCA466FX2E2—used to uniquely identify every potential voter in the database.

</comment></comment></comment><comment><USER51> I'm just waiting to hear Trump tell Deep Root Analytics, "You're Fired!"

It would be good to see him make this a clear case of responsibility. Also, someone on the RNC side needs to get fired, too. I'm not sure who, but errors this big demand it.

<comment><USER48> Trump only has "signaling authority" over the entities with whom the RNC contracts; he can't actually force a contract termination, but he can express distrust in an entity and encourage the RNC to terminate a contract. I don't see that happening, for a myriad of reasons.

The people involved with the decision to start working with Deep Root are mostly not with the RNC anymore. Even if they were, that's simply not how the industry works.

</comment></comment><comment><USER52> Is there any way to access the Deep Root data to see what it says about me? Has anybody posted it online in a searchable format?

</comment><comment><USER50> Doesn't this kinda make many (flawed but still in use) "security measures" incredibly vulnerable to social engineering? That's _all_ the birth dates and phone numbers. That's crazy!

</comment><comment><USER53> Is StateVoterID the same thing as your Social Security Number (SSN)?

<comment><USER54> Not in Georgia, and I seriously doubt anywhere else either.

</comment></comment><comment><USER55> If I jokingly suggested to the gov that AWS S3 be categorized by all firewall vendors as "Anonymous File Sharing", would I get a yuge response?

</comment><comment><USER56> Has this been re-leaked onto the torrents yet?

<comment><USER57> Can you clarify, re-leaked? I've never seen this online.

Can you post some names, links? Thanks

Looking for research purposes.

</comment></comment><comment><USER58> Sounds like one heck of an AI training set.

DeepTrump?

</comment><comment><USER59> So you think it is okay to model ethnicity and religion of 200 milion citizens? This is how my country got all their jews killed in WWII [0]. The mere thought of compiling such information here would lead to some hefty jail time these days. I really cannot believe you can even defend such a thing.

[0] [https://en.wikipedia.org/wiki/Civil_registration#Netherlands](https://en.wikipedia.org/wiki/Civil_registration#Netherlands)

<comment><USER12> Whoa! We got to Godwin super fast this time.

Please re-read what I wrote and consider that I might have the best intentions to explore a discussion with HN. Note also that I discussed the nature of securing the data and not its compilation.

Note that IMO several European countries forfeit some freedoms that I consider valuable and critical for democracy. IMO it is your right to observe the world as it is and note what you observe. Yes, I understand that this right conflicts with privacy and find it an extremely unfortunate consequence, especially given the emerging power of AI. However I think that it's also possible find people culpable for evil intent of a compilation, though difficult to prove.

<comment><USER59> It's not an attack at you personally. The contrary. But I noticed that a lot of comments found this kind of data collection 'normal'. and apparently in the US this is very much legal. And I as a person find this very troubling, because in Europe these kind of decisions caused a Genocide, which is a result that should not be taken lightly. Do you really want the RNC to collect religious information about all 200 million voters? What happpens if this information falls into wrong hands (in our case it was Nazi's). You just handed someone a potential 200 million long hit list.

So yes it went Godwin quickly, but it is the painful truth.

<comment><USER60> I'm no expert on the subject, so feel free to set me straight. It seem to me, though, that this is a bit like cargo culting. An association between some part of a process and the result that doesn't factor in the big picture. More specifically, wouldn't the presence of such lists have proven a minor and nonessential part of the genocide? If the Nazis didn't have those lists, would they have just thrown up their hands and went away? It seems they'd have just gathered the information by the usual 'everyone accusing his neighbor' witch- hunt means, with people using it to settle grudges or other nefarious reasons. Maybe it would be less accurate, and maybe a little slower, but ultimately, you'd probably have about the same amount of bloodshed, I would expect. The real keys to these situations are a powerful group with ill intent and a population that is at least willing to condone it. If those are in place, the list doesn't really matter, and if they aren't in place, the list doesn't really matter. So why does the list matter?

Note, this is an invitation to enlighten me, not a firm belief on my part.

</comment></comment></comment></comment><comment><USER61> In other news, already known information. Every USA states has clear laws on releasing whose registered with which party, past affiliation and address.

Clickbait.

<comment><USER62> From the article:

> Along with home addresses, birthdates, and phone numbers, the records > include advanced sentiment analyses used by political groups to predict > where individual voters fall on hot-button issues such as gun ownership, > stem cell research, and the right to abortion, as well as suspected > religious affiliation and ethnicity.

In some Western countries, the mere existence of such a database would be illegal. Especially relating to religion and ethnicity. [http://www.newyorker.com/news/news-desk/can-the-french- talk-...](http://www.newyorker.com/news/news-desk/can-the-french-talk-about- race)

In the US, it is leaked online.

<comment><USER63> >In the US, it is leaked online.

It's not a "leak" when the data is entirely public already...

<comment><USER64> It is a "leak" when additional data that was previously private, such as modeling and analytic information, gets published against the will of its owners.

However, it isn't a "leak" in the traditional colloquial sense that someone stole the data and released it to the public. It's just a security leak.

</comment></comment></comment></comment><comment><USER65> The reddit posts are odd. What value do they have?

And why/how did they leak?

Fortunately, they don't hold much personal data, but given that they're looking to raise $$$, the fact that they had a security breach is interesting. Especially if they haven't disclosed it.</comment>


The most awesome startup I have ever seen. - <AUTHOR> http://www.danshapiro.com/blog/2012/03/the-most-awesome-startup-i-have-ever-seen/

<comment><USER2> It's a cool building, but I was a little less thrilled when I read about their tech. It's true that an isothermal process is a reversible process, and isothermal air compression is therefore a great way to get high efficiencies, but compressed air just doesn't seem like it will scale well. The work formula for isothermal compression of an ideal gas is:

        W = - ∫ P dV = - n R T ∫ dV / V =  P₀ V₀ ln(P₁ / P₀).

A shipping container holds 38.5 m³ according to the Wiki. At atmospheric pressure this would mean P₀ V₀ ~= 1 kWh. Cheap hardware might get you to 10 atm pressure or so -- but let's go crazy and suggest that you could get 100 atm, since it's only logarithmic in P₁ anyway. You would still store less than 5 kWh per shipping container, no? That could run a window-unit air conditioner for about a day, only. If you pay 15 cents per kilowatt hour, you could fill up the shipping container with under $1 of electricity, only. My cell phone battery stores an amp-hour at around 3.5 V, so if the above calculation is right, to store the same amount of energy in 100-atm compressed air you'd need 27 liters of air -- two backpacks or so. I'm saying this to guesstimate that it's about a factor of 1,000 less energy density than modern battery technology -- and that that's a fundamental limitation to the medium.

Given all that, I'm really interested to see how they'll scale compressed air up to handle the sheer amount of energy that they want to store.

(On the other hand, the factor of 1000 might not matter too much: it means that if they can build a shipping container air storage unit cheaper than electronics companies can build a battery the size of three backpacks, they could indeed be cheaper to store energy en masse.)

<comment><USER3> Dude, I like your analysis and eager attempts to calibrate but you messed up your calculation somehow.

I'm pretty sure your problem is that your formula misestimates the amount of gas stored. It is compressed, you know. You of off by about a factor of 200 - we use 200 atm air. It's 1.1 MWHr per high-boy shipping container

The energy stored is

efficiencyOutgoing * P_top * V_tank * (ln pressure ratio - 1 + 1/pressure ratio) -- you can look at our patents for more detail.

<comment><USER2> Sorry, you're pretty much right. The equation I wrote is correct, but I naively plugged in a bad number: yes, P₀ V₀ = n R T = P₁ V₁, but I used the known number for V₁ (the final air in the container) as V₀. The proper formula is indeed P₁ V₁ ln(P₁ / P₀).

You have my apologies for my confusion.

</comment></comment></comment><comment><USER4> _"middle school-dropout"_

Not the words that I'd use to describe someone who starts studying for a degree at the age of 12.

<comment><AUTHOR> Me either, but that's how she describes herself.

<http://daniellefong.com/about-the-author/>

</comment></comment><comment><USER5> "Danielle started her PhD at 17" -- Thanks for ruining my monday!

Just kidding, awesome idea. We're so used to IT startups that we often forget about other engineering. Very cool stuff.

</comment><comment><USER6> Wow. These guys rock.

<comment><USER3> Thx :-)

</comment></comment><comment><USER7> Found additional details here: [http://daniellefong.com/2012/01/22/green- dreams-life-in-the-...](http://daniellefong.com/2012/01/22/green-dreams-life- in-the-year-of-the-rabbit/)</comment>


Ever wonder why MP3s sell for $1.29? - <AUTHOR> https://medium.com/@binroot/the-1-29-hypothesis-82e31a6c5d2f

<comment><USER2> The second half is just wrong. There's no such thing as unused balance on an iTunes card. When you redeem a card, the entire balance is credited to your account. The credit will only go unused if you stop using the iTunes store, so that's at most $1.28 per user, not per card.</comment>


Amazon SageMaker Autopilot - <AUTHOR> https://aws.amazon.com/blogs/aws/amazon-sagemaker-autopilot-fully-managed-automatic-machine-learning/ <comment><USER2> Google's AutoML produces black box models that are only available over a network call. This services seems to produces downloadable models, and a notebook with Python code that creates the model. If that is the case, this is substantially better than GCP's offering.

AWS consistently releases similar products after GCP... but they are much more well-thought-out, as AWS has to support them indefinitely...

<comment><USER3> I'm not sure why the other post was marked dead, but AutoML models can be exported - you can export them to TFLite format[0] and then run them on edge devices, such as a SparkFun Edge[1] or a Coral SoC / board.

[0]: [https://cloud.google.com/vision/automl/docs/export- edge#expo...](https://cloud.google.com/vision/automl/docs/export- edge#export_to_devices) [1]: [https://www.sparkfun.com/categories/tags/tensorflow](https://www.sparkfun.com/categories/tags/tensorflow)

<comment><USER4> You can also export your model as a server within a Docker container:

[https://cloud.google.com/automl-tables/docs/model- export](https://cloud.google.com/automl-tables/docs/model-export)

</comment></comment></comment><comment><USER5> Would be useful to have a comparison to Google's AutoML Tables: [https://cloud.google.com/automl- tables/docs/features](https://cloud.google.com/automl-tables/docs/features)

<comment><USER6> This might be intresting:

[https://medium.com/analytics-vidhya/google-automl-tables- a-f...](https://medium.com/analytics-vidhya/google-automl-tables-a-first- taste-f25b3728e1a7)

</comment></comment><comment><USER7> Pretty neat, but unfortunately I cannot see a lot of business cases for this. I haven't worked with a ton of models, but especially if you are not dealing with pretty much solved problems like classification, the results won't be great.

First of all, which models are going to be used? How many combinations of hyperparameters are going to be tried? The combinatorial explosion is certain.

And then if you don't know how to prepare the right dataset everything is in vain.

Not really a critique to AWS, but to AutoML in general.

EDIT: After a deeper read it seems it's regressions on textual data only.

</comment><comment><USER8> Wait, really, I just upload tables of input and the expected output data and it tries various models for me?

Any other places do this?

<comment><USER9> You say that like that's the easy part of ML.

<comment><USER10> I think you're misunderstanding the comment.

I read "just" as surprise it's this easy from the end user perspective, as opposed to the actual technology.

</comment></comment></comment><comment><USER11> I assume this won't do things like add convolutional layers if you give it pixel or signal data, right?

Like is this just adding standard layers to a neural net, maybe trying a few activation functions, fiddling with the number of layers and just seeing which give the best results?

<comment><USER6> No. auto ml for DNN is a different ball game (Also known as Neural architecture search).

If I read it correctly this is using traditional "classical" ML models (e.g. XGBoost, GBM and even linear models).

</comment></comment><comment><USER12> In general, If you're interested in looking into AutoML landscape and its adoption here's a Kaggle kernel based on recent Kaggle Survey [https://www.kaggle.com/nulldata/carving-out-the-automl- niche...](https://www.kaggle.com/nulldata/carving-out-the-automl-niche-from- kaggle-survey)

</comment><comment><USER13> Do any of these autoML offerings have a way to use the generated model in JavaScript/nodejs? I know of [sklearn-porter]([https://github.com/nok/sklearn- porter](https://github.com/nok/sklearn-porter)) which transpiles scikit-learn models to JavaScript among other targets, but not sure if this nicely connects with any of the solutions discussed.

<comment><USER14> If you use GCP AutoML service, you can export AutoML Vision edge models (for image classification and object detection) directly for TensorFlow.js for use in browser or Node.js. Please see this: [https://cloud.google.com/vision/automl/docs/tensorflow-js- tu...](https://cloud.google.com/vision/automl/docs/tensorflow-js-tutorial)

</comment></comment><comment><USER15> I wonder how this compares features and price to similar products from H2O.ai's driverless ai, datarobot, bigsquid, etc.

<comment><USER12> H2O's main pitch is that except Driverless they're open source. Data robot seems to have got a strong Salesforce for each domain and driving sales. In terms of features, Google cloud AutoMl seems better as it makes the entire productionising part easy

<comment><USER15> Any experience with the datarobot tool? The few demos/YouTube's I've seen, it looks really slick. Lots of pre-built performance evaluation, model deployment/monitoring tools, etc. hard to find any pricing info on the web.

<comment><USER16> We recently purchased dataiku and I've found it to be really quite handy.

<comment><USER15> Any info on pricing? I can't find anything on their website. Don't like taking to sales goons.

It looks pretty slick. Going to install the free version to check it out.

</comment></comment></comment></comment></comment><comment><USER17> This is rather interesting development. Just last week I saw similar feature in IBM Watson being demoed on IBM Cloud. And now AWS Sagemaker has this capability.

Does this mean that going forward, for small-to-mid size IT companies and Corporates, the demand for Data scientists and ML developers would decrease?

<comment><USER18> My guess is it will, on average, increase demand.

ML is finicky; the model training pipeline itself isn't the hard part, compared to setting up for the right question and examples used to train the model.

For small-to-mid firms, data scientists are super expensive. And they might only deliver a valuable project every six weeks (or, at bigger firms, every year...).

If automl increases their productivity, suddenly they don't look so expensive.

<comment><USER6> Right. It will increase demand, since many area of the business will start using machine learning.

However, the job of the DS will move toward the business side (e.g. req gathering, data gathering and prep) and less about the modeling itself.

Also, there are a lot of data issues that are still in the releam of humans (e.g. imbalance data, correct labeling, etc).

</comment></comment></comment><comment><USER19> How does the algorithm analyze the results and look for overfitting?

<comment><USER20> Built-in regularization, probably, plus cross-validation. These techniques aren't new; they're included in a number of ML libraries already - just not at this level of automation.

</comment></comment><comment><USER21> This is the second science-fiction-level announcement from Amazon in as many days. Either they're about to take over the world with effective AGI and Quantum Computation, or they're being a bit silly.

<comment><USER22> What's science fiction about doing auto-machine learning? It's basically automation of a lot of the tasks that data scientists do manually to build simple ML models.

That is definitely a technically challenging problem but not an impossible one.

If anything AWS is late to this.

<comment><USER20> Right, AutoML != AGI.</comment></comment></comment>


Covid-19 Is Killing Small Businesses in Seattle - <AUTHOR> https://freshchalk.com/blog/covid-19-is-killing-small-businesses-in-seattle <comment><AUTHOR> This is so sad. From the survey we ran - Closure is looming for 35% of small businesses in Seattle. More in TFA</comment>


An Official Code of Conduct for PHP Proposed - <AUTHOR> http://news.php.net/php.internals/90072 <comment><USER2> Well, we know how this will go. A number of commenters will object, pointing out how the code of conduct reserves the right to police what people say in non-project spaces, relies on secret evidence and opaque processes, and does not protect people from being harassed for their political views. Then the project maintainers will ignore them and install the code anyway.</comment>


People Don't Fail, Processes Do - <AUTHOR> http://www.lean.org/LeanPost/Posting.cfm?LeanPostId=480 <comment><USER2> No, sometimes people do fail. People get jealous, they get mad, they sometimes do silly things. We put processes in place to prevent those malfunctions from negatively impacting our organizations. Unfortunately, even perfect processes can fail when humans are involved. That's why it is important to remember that people skills are as important to a manager as are project planning skills.

<comment><USER3> Jealousy==emotional self-regulation management didaction processes.

Anger==emotional self-regulation management didaction processes.</comment></comment>


Client-side load balancing - <AUTHOR> http://instantdomainsearch.com/articles/faster_domain_name_search

<comment><USER2> Code samples?

<comment><USER3> Yeah, I think the article should have a code sample.</comment></comment>


Chinese companies plan to be producing over 120GWH/year in battery cells by 2021 - <AUTHOR> https://www.bloomberg.com/news/articles/2017-06-28/china-is-about-to-bury-elon-musk-in-batteries <comment><USER3> Even though I'm dubious about government being too involved in business, a couple things stood out to me:

 _the Chinese government has launched a sweeping effort to increase the country's dominant market share._

 _"This is about industrial policy. The Chinese government sees lithium-ion batteries as a hugely important industry in the 2020s and beyond,"_

 _That's a huge opportunity, and China doesn't want to miss it._

While its entirely possible that people in the government make the totally wrong decision and end up wasting taxpayer dollars, this kind of forethought makes me feel that the US is going to be left in the dust in terms of renewable technology. I know there is a patchwork of federal and state money that probably goes into things like the gigafactory in Nevada, investment on this scale by the government seems like a total pipe dream here, it hurts to see how backwards our government is moving on energy. I can see how people will just say "let the market handle it" but when the best case scenario is you save the world for future generations and the worst case is you simply lose some money it seems like a no-brainer.

<comment><USER4> "the US is going to be left in the dust in terms of renewable technology"

China is already dominating the battery and solar panel industry. You really have to stop thinking of the US as being #1.

<comment><USER5> They have the cheapest panels, sure, but that's because they're being actively subsidized in an attempt to kill of competition. To my knowledge SunPower still has the highest efficiency production panels and First Solar still has the lowest demonstrated cost per watt for anything over moderate insolation.

</comment></comment></comment><comment><USER6> Just trying to work out how they claim Tesla will be "flooded". The forecasted demand in the article is going to outstrip both China's increase and Tesla's increase by 2024, and China's major market for their increase will be domestic, per the article.

       Current demand: ~75 Gigawatt Hours (GWh)
       2021 demand: ~160 GWh
       2022 demand: ~195 GWh
       2024 demand: ~290 GWh

So the market will need an additional 85GWh by 2021, 110GWh by 2022, and 215GWh by 2024.

Tesla is adding 35GWh by 2018. China is adding 120GWh by 2021. 120 + 35 = 155GWh < 215GWh.

Seems like Tesla will be in a really good spot. They'll have a > 10% share of a highly in demand product and likely be seen as one of the premium/high quality options.

<comment><USER7> Plus Elon has said he wants about 10 Gigafactory to meet demand. On another note, I would be interested to hear from anyone in the industry about the state of Battery Technology. Keep hearing about the demand for Cobalt and how Congo controls about 60% of the world's cobalt supplies and all the moral issues around mining there. Are there good alternatives to using cobalt in battery construction?

edit: Also curious about the impact of older batteries to the environment. I see all this talk about moving to EV because it's cleaner tech, but is that true if we have a billion batteries sitting in the landfills at some point?

</comment></comment><comment><USER8> Every single time I see the word battery, my mind involuntarily jumps to "toxic waste"

What is any country or company doing to deal with the massive toxic waste that will come from expired batteries?

Does tesla have a giga-recycle program?

How many duracells have you thrown out in your lifetime?

What is the sustainability of the minerals that are mined that go into battery production.

All these and more, should all have rock-solid ELI5 answers to how they are being dealt with.

If there is any startup that may want to have a positive impact on the future, figuring out how to deal with battery waste may be a good one...

<comment><USER9> Yeah, especially concerning given that no battery manufacturers are currently geared up to recycle old batteries.

</comment></comment><comment><USER10> Won't this be a good thing for everyone, even Tesla?

<comment><USER11> Elon's been saying this from the beginning, he understands that is what will happen. I imagine part of being able to move forward with big ambitions is you have to make peace with that fact - otherwise fear of the unknown, of competition, will easily overwhelm you.

<comment><USER12> It's more than that. That was the _whole point_. Tesla was built to electrify transport. _Not_ to be the dominant player forever, but to force other market players into this game.

</comment></comment></comment><comment><USER13> I have a related question: Does this mean the top lithium-producing countries[0] will experience an economic boom, similar to oil producers in the '70s, or is lithium abundant enough in the US and China to satisfy future demand?

[0] [https://en.wikipedia.org/wiki/List_of_countries_by_lithium_p...](https://en.wikipedia.org/wiki/List_of_countries_by_lithium_production)

<comment><USER14> The lithium prices may experience continue to increase, but lithium is highly abundant. It won't help the "countries" that much. The US and china has tons of lithium and so do many countries across the world. But most likely, lithium prices may start to decline if the US/Russia/China/etc who have most of the world's lithium decide to start mining seriously.

Certainly well connected mining companies might do well also. But lithium isn't oil. Oil is the basis of modern civilization. It is in pretty much every product you use and every industrial process. Even battery companies will use tons of oil and oil byproducts to create lithium batteries.

Oil and lithium batteries aren't comparable. One is foundational, the other is really a convenience.

</comment></comment><comment><USER15> Is the battery industry really going to become a zero-sum game between China and one guy? Shame on western economies for not paying attention to this stuff until the tech becomes dirt cheap (at which point, China is always already dominating).

<comment><USER16> Almost the same story with solar panels.

<comment><USER17> And REEs, and commercial genomics, and commodity SoCs, and greentech, and mass produced CFRPs and so many other things

China systematically seals all upcoming greenfield opportunities for tech companies and industry in West. A very smart play

</comment></comment></comment><comment><USER16> Before we know it, 1TWh worth of battery storage will be produced every year. That should make EV batteries and EVs in general quite cost-competitive.

</comment><comment><USER14> Battery production, recycling, etc and the batteries themselves are very toxic. Seems like we are trading one form of pollution ( polluting the atmostphere ) with poisoning our land. I remember it wasn't too long ago where we were told not to use too many batteries because they were toxic...

<comment><USER18> Nickel Cadmium (typical rechargeable) and Silver Oxide (watch batteries w/ mercury) are bad. Lead Acid (cars) are very recyclable, and considering there is no new lead production in the US, will command a premium for recycling. I don't think Carbon Zinc and most Lithiums are a problem.

</comment></comment><comment><USER19> Quality will be noticibly shitty enough that there will still be a market for Tesla.

<comment><USER20> Unlikely the last decade China has been pushing its industries to go from cheap and mass produced to better qualities at higher prices

</comment></comment><comment><USER21> As was already pointed out I believe that one of the most interesting opportunities will not lie in producing the battery but on ecologically manage those already produced.

</comment><comment><USER22> So an entire country of 1.3B people, with 8 factories under construction and a dozen or so more announced are going to outproduce one US factory. Okay, why is that news?

Seriously Bloomberg - this is what Elon has been saying publicly since day 1, the entire Tesla vision was built on batteries getting cheaper and more available.

The Gigfactory is important to Tesla not merely because of it's scale, but for the advantages of vertical integration for QC, capacity planning, supply chain stability, etc. The same reason auto manufacturers had their own steel plants back in the early days.

<comment><USER23> The article discusses how China is building 4 gigafactories worth of capacity by 2020.

It's unclear what your tone signifies, muddying your message. It's a couple commonly known truths about Tesla, with a tone of "take THAT Bloomberg!"

<comment><USER22> My point is the article doesn't make much sense - comparing a country's output of anything to a single factory is of little to no point. The entire article is fluff produced so Bloomberg could have a snazzy sounding headline: "China Is About to Bury Elon Musk in Batteries"

The fact hat the headline on HN was changed to be less clickbaity instead of matching the original pretty much says it all...

</comment></comment></comment><comment><USER24> I would worry about quality.

</comment><comment><USER24> quality.

I would expect many things to be prone to fires like both Samsung and Takata. I wouldn't trust them in a Prius, a Tesla, or a Galaxy Note. Gas tanks haven't spontaneously burned down cars for over a century, but they will do it this century. Put money on that. I expect the first examples will be shown in China, where manufacturers (21+ of them) can put carcinogenic insecticidal fire-retardant poison in infant milk and pet food for years with what is effectively impunity until it hits newspapers. This will translate to electric vehicles.

<comment><USER25> Lithium Ion is always prone to fire. If you wrap it in a hard-enough case so it can't be punctured, and then slap electronics on the end to "emergency cut power" when a catastrophe is detected... you can prevent the fires.

The issue with Samsung's "explosive" battery was that it had to be thin (so it had a soft case), and when you sat on it... it'd bend internal connections and then short them out.

\</comment></comment><comment></comment><comment>--

Honestly... I'd expect China to build the typical 18650-cell without any issues.

Unfortunately for China, Tesla isn't putting "typical" 18650 cells into their cars. They're putting in Silicon enhanced Lithium Ion with optimized chemistry for maximum power with minimum weight.

THAT's the problem. I don't think there's a very big marketplace for "typical" Lithium Ion cells. Its about the recent innovations that have made batteries lighter, more flexible, and deliver more power.

So the real question is: can China build Silicon-Lithium Ion Cells to the specifications of Tesla's Gigafactory? I have my doubts. There's a reason why Elon Musk built his own factory (with help from Panasonic)

<comment><USER17> China was making superior EV cells since around 2012. For some time, they banned export of unfinished cells and cell materials to favour makers of complete battery packs.

This is what led to EV and hybrid car makers outside of China having to use battery packs made out of consumer grade cylindrical cells for a while.

Tesla Motors guys have seem to have realized now how ridiculous was the decision to build battery packs from small cells, and announced that they will be transitioning to slightly bigger cylindrical cells for model S.

Still, current model S ships with commodity size cells. They may have decided that it does not worth effort changing to big cells when they have just got their factory running with tooling made for standard sized cells.

</comment></comment><comment><USER25> It depends on what the #1 usage of batteries will be.

Utility scale battery usage don't really care about the specifications. We just need a lot of them, cheaply. Shear size can make up for a lot of problems.

But Car batteries are highly sensitive to power, weight, cooling... issues that require innovation and improvement. Its unlikely for any US Car company to buy cheap batteries from China. They'll instead build batteries as a competitive advantage over each other.

\</comment><comment>---

I can't help but think of China's "Great Leap Forward". They believe that Steel is the future, but for large swaths of the country... the best "steel" that China makes is Pig-Iron (an intermediate product before Steel).

[https://en.wikipedia.org/wiki/Backyard_furnace](https://en.wikipedia.org/wiki/Backyard_furnace)

If the "best" Lithium Ion cell that China mass produces has poor weight and/or power density, then it has no use in a Car. Cells and Batteries are the #1 issue for an electric car. (More weight requires more batteries to lug around). Cutting back on weight while increasing the power is how you make a superior vehicle.

<comment><USER26> > Its unlikely for any US Car company to buy cheap batteries from China. > They'll instead build batteries as a competitive advantage over each other.

Just like US processor or SSD companies build things themselves?

I'm being ironic but my point is that when China cranks up production I don't see why US (or any other companies) wouldn't buy their cheap products as long as they meet the required specs.

Edit: Comparing Maoist China to post-Deng Xiaoping China is kind of silly :)

<comment><USER25> > Just like US processor or SSD companies build things themselves?

China doesn't deliver cheap, high-quality NAND Flash. Nor do they make Intel chips. Nor do they make Qualcomm chips. Nor do they make even AMD chips (I mean sorta, IIRC China has licensed an AMD design but they aren't making Ryzen).

Micron Flash is made in Virginia: [https://us.micron.com/micron-in-usa/us- jobs/college-recruiti...](https://us.micron.com/micron-in-usa/us-jobs/college- recruiting/manassas-virginia)

Intel Chips are made in Oregon: [http://www.infoworld.com/article/2672745/computer- hardware/i...](http://www.infoworld.com/article/2672745/computer- hardware/inside-intel-s-d1d-fab----through-the-looking-glass.html)

AMD Chips are made in New York (Subcontracted to GlobalFoundries): [http://www.eetimes.com/document.asp?doc_id=1327435](http://www.eetimes.com/document.asp?doc_id=1327435)

As far as foreign companies go... Taiwan (TSMC aka Nvidia's contractor), Japan (Toshiba), and Korea (Samsung) are the other countries with 14nm or 10nm fabrication labs. Those are the US's rivals when it comes to high-tech. NOT China. Sorry to burst your bubble.

I think some of those companies build plants in Israel. So Israel deserves an honorable mention. Singapore too.

\</comment></comment></comment><comment></comment><comment>--

Ermmmm... yeah. The best of the best is made in the USA. I'm not saying "China quality is bad"... but the fact of the matter is that China does NOT own any 10nm fabrication facilities.

China's manufacturing is a step behind the USA's technologically. Its a LOT bigger however, so that's a thing. China just has so many damn people that they manage to brute-force a lot of solutions. But unless they begin to push innovative designs, they aren't going to make the best.

Most likely, China's cells will be a shock for the utility-space companies, who DONT require high-quality batteries to do important things. Again, there are plenty of uses of lower-tech MASS PRODUCED cells.

Ex: stick thousands of them into shipping containers and then hook them up to the grid. That's damn useful. Its just not useful for cars.

<comment><USER27> High end NAND is made in China, but in factories owned by Korean, American, Japanese, and Taiwanese companies.

<comment><USER25> Erm, low-end NAND Flash is made in China. Lots of it, but its low-end.

Intel, Micron, Samsung, and Toshiba are making 3d-NAND, which has superior specifications... allowing controllers to read / write them as Triple-Level without very many endurance problems.

And the only locations for 3d-NAND are:

* Japan: [http://www.anandtech.com/show/10184/toshiba-to-build-a-new-f...](http://www.anandtech.com/show/10184/toshiba-to-build-a-new-fab-to-produce-bics-nand-flash)

* (South) Korea: [https://www.engadget.com/2013/08/05/samsung-ships-first-3d-v...](https://www.engadget.com/2013/08/05/samsung-ships-first-3d-vertical-nand-flash/)

* USA: Idaho for Micron's 3D Flash. IIRC they've partnered with Intel... which is primarily Israel and USA as far as manufacturing goes.

\</comment></comment></comment><comment>---

Now... name me a Chinese manufacturer of 32-layer (or more) 3D Flash. Toshiba hit 96-layers IIRC.

Hint: You can't. China doesn't make high-end 3d NAND Flash. At all. Maybe they will catch-up in a few years, but at the moment, they are not.

<comment><USER27> I'm not sure I'd call 15nm 2D NAND "low end." But when those Chinese fab lines were opened, they were cutting edge.

Nearly all the engineering is done somewhere that isn't in China, which was my point.

<comment><USER28> I am not aware of any 15nm NAND made by any Chinese companies, correct me if I am wrong.

The parents was right in regards to NAND and Fabs. At least in the next 2 - <USER2> years time frame. The plan for China, at least up to last year was to bump tens of billions of tax dollar and incentives, along with billions of investment from private sector ( Which isn't really private, but lets not go on with that for now ) to builds Fabs. Since China is now officially the largest Smartphone and electronics market, they want to source more components from domestic market rather then relying on foreign companies. In hope to build an Semi Con Industry that rivals Korea Japan and even US. Hence you see investment into SMIC, and a lot of people were poached from TSMC, Global F and Samsung.

In a typical fashion of Chinese Race to the bottom, they could sell the NAND at BOM cost and gain market shares. I am sceptical whether this works for NAND or Fabs in general. 3D NAND from other companies should have both quality and cost / GB advantage.</comment></comment></comment>


Show HN: Instagram Analytics - <AUTHOR> https://smartmetrics.co <comment><AUTHOR> Hello,

i have made Instagram Analytics for business customers. I will be grateful for any comments how to make it better. Also i'd like to get some advice on how to promote it.</comment>


Computer Science Interview Questions with C++ Solutions - <AUTHOR> http://www.grokit.ca/spc/computer_science_review/ <comment><USER2> I really wonder if this sort of rote learning has anything to do with being a decent programmer, and if sensible companies that I want to work for are really asking these sorts of questions.

Personally, I could probably tell you the uses for heaps, trees, tries, and hashes before I wanted to check something in a book or look something up on the net.

I would think that I am better placed to write sensible code because I have an understanding of complexity and I'm prepared to go and think about the problem, find or invent an algorithm, and then prove the complexity of that algorithm. Rather than rote learning 12 different types of trees.

<comment><USER3> There is little use in being able to integrate functions, or to solve trigonometric equations, or to rearrange equations. However, being able to do these sorts of things without thinking too hard then releases the mind to think about other things with freedom.

Similarly, effortless recall of the pros and cons of different algorithms and data-structures means that you don't have to think twice about many of the things that should be purely mechanical. It then lets you recognize things when they show up in disguise, and to concentrate on the higher level work, where the real work begins.

<comment><USER4> I treated college math and physics as symbolic manipulation games, and graduated cum laude with majors in both subjects. I can still fly through those exercises today, a quarter century later.

My only misgiving is that the emphasis on closed form solutions, at the time, was exclusive. It distorts the choice of problems that students are given.

Not being a CSist, I don't know if a similar distortion exists in CS.

<comment><USER3>
        > I treated college math and physics as symbolic manipulation
        > games, and graduated cum laude with majors in both subjects.

Interesting. I'm not saying this about you, but I've met several people who treated math as largely just symbol manipulation, and indeed, they ended up being great at solving the sorts of problems set, but I've sometimes had a feeling that they're sort of "barking at print". I've heard people reading perfectly well, but with no sense they they understand what they've read, despite the perfect production.

Similarly, some people can "do the math," but sometimes there's no sense of any real understanding. It's hard to explain.

This is one of the problems with interviewing. Some people can really talk the talk. They know all the words, all the phrases, can solve the problems, can produce the code for the set problems. And yet, after conversation, you start to feel that there's something just not quite right.

This is just one reason why interviewing is hard.

<comment><USER4> Well, I certainly can't plead innocent. ;-)

I've realized for a long time that what's gotten me through school and life is a series of accommodations for the brain that I happen to possess.

Oddly enough, things "clicked" for me in terms of developing the deep understanding underlying the symbols, but not always concurrently with whatever course I was taking.

</comment></comment></comment></comment></comment><comment><USER5> This is a mess: lack of comments, inconsistent coding style, and outright wrong answers. If I got this from a candidate I would reject it outright as a bad example of uninformed copy & paste. [Edit]: not trying to sound mean, that's just my raw and unfiltered reaction.

</comment><comment><USER6> I have a question, I consider myself a great software developer with a proven track record. I have open source code, lots of completed and shipped websites and software.

Do interviewers really care about this sort of interview knowledge outside of the San Fransisco start up bubble? I've been programming and making a great living for more than 8 years now, and not once did I have to implement a b-list tree or something CS-y like that.

<comment><USER7> Personally, I'd walk out of an interview if they started asking me these types of questions.

Unless of course, if it's a job where it'd be very algorithm-heavy to begin with, but that isn't where my interest lies.

<comment><USER3> That's your right (of course) and I have no argument with it. But let me offer another point of view.

To my mind, the point isn't whether or not the job is algorithm-heavy. If you don't have this stuff under your belt, there are limits to what you can do. If you have to concentrate on working out which algorithm to use, or which data structure to use, if you have to spend time researching these things, that doesn't make you a poor programmer, but it stops you from spending time on the real problem solving aspects of the job.

The programmers I hire don't often have to do major algorithm stuff, and some of them are not comfortable with algorithms and data structures, but that's OK. They have other strengths.

But even if the job doesn't require deep knowledge of algorithms and data structures, knowing them helps more often than you think. One needs to know these things to recognize them when they turn up unexpectedly and in disguise. Well, in my experience anyway. Your mileage may|will vary.

But when I hire, I interview for, and care most about people's abilities to get things done, to work constructively with others, and to extend their knowledge, skills, and abilities. To walk away when someone asks about something you don't (yet) know, says that we wouldn't be a good fit anyway, and not because you don't know about algorithms.

 _Edited to remove some unintentional snark._

 _Additional edit: I don 't know why you've been down-voted. Many people feel the way you do, and while I don't agree, you're entitled to your opinion, and I'm pleased you voiced it. I've up-voted you to try to offset the down- vote(s)._

<comment><USER8> So what do you think is an effective way to interview for a person's ability to get things done?

<comment><USER3> I don't know, but if you hunt through my comments on HN you'll see what I do. Mostly it seems to have been successful - the guys on my teams are pretty good. Mostly I've only gone wrong when I've put too much faith in paper qualifications.

I ask people to code something trivial, on a whiteboard, in any pseudo-code they care to invent on the spot. Then we talk about the strengths and weaknesses of the code they wrote. We end up talking about algorithms, data structures, project management, quick hacks, technical debt, comments, function and method size, functional programming, and more.

The idea is to find their strengths, to make sure their weaknesses are covered by people we already have, and to see if we could work together. Sometimes we don't even agree, and it's the way they discuss the points that matters.

I'd love to see it made into a science, but I'm nowhere near clever enough to do that.

</comment></comment></comment></comment></comment><comment><USER9> Each question seems to come with just an uncommented code dump. Not any discussion of the actual algorithms, why it works, what the trade-offs are, etc. If I happened to ask all these questions in an interview and got all these answers, I would reject the candidate.

<comment><USER10> Moreover, I think that some answers are not doing what we should expect from the subject. For example the "Detect Multiple Of Two" detects powers of two, not multiples.

<comment><USER11> Agree on needing comments.

My C++ is a bit rusty, but I think the code is in fact checking if the number is divisible by 2 (i.e. n % 2 == 0).

I think it's using the bitwise and operator (single &) to AND each bit in n and (n-1) and then checking if the least significant bit is 1 or 0. The code would need to shift (<< or >>) to check if the number were a power of 2.

I thought there was another bitwise NOT operator, not the !, but I think, and this is the part I'm hazy on after getting home, that the ! applied to an int is intended to flip each bit. Here's why:

\</comment></comment></comment><comment></comment><comment>

n = 6 = 110

n-1 = 5 = 101

n & (n-1) = 110 & 101 = 100

!(110) = 001 => true

\</comment><comment></comment><comment>

n = 5 = 101

n-1 = 100

n & (n-1) = 101 & 100 = 101

!(101) = 010 => false only if the least significant bit is used for logic decisions... this seems non-portable for some reason, just like using the ! as a bitwise NOT. It may actually be that ! only looks at the least significant bit, but I can't find c++ docs to say one way or another.

Whatever the code actually does, this is exactly the kind of example to use as a poster child for adding just a few comments.

<comment><USER3> The GP is correct, this is bit-twiddling code to test for a single bit being set in the int, and therefore being a power of 2, not simply a multiple of 2.

Broadly speaking, and ignoring edge cases, if N is a power of 2 then it has a single bit set. Subtracting one unsets that bit, so the AND of N and (N-1) is zero. On the other hand, if N is _not_ a power of two then subtracting 1 leaves the top bit still set, so the AND is non-zero. Therefore taking the boolean NOT gives the right answer.

        00010000  Power of 2
        00001111  N-1
        </comment></comment><comment>--
        00000000  -> 0

        00010110  Non power of 2
        00010101  N-1
        </comment><comment>--
        00010100  -> Non 0

Thus power of 2 is NOT( N & (N-1) ). Your examples are wrong because "!" is not bit-wise NOT, but boolean NOT. !0 = True, !N = False when N is non-zero.

It's actually wrong on the edge case of N=1, which is a power of 2.

<comment><USER11> Thanks, you're right. I don't know why I would have ever thought "!" was bitwise... only on a Saturday night.

It just goes to show that sleep is very important for interviews!

</comment></comment><comment><USER12> i don't know why but statistically the number of times i've written code to BFS on a binary-tree given the day-hour experience i've had, is zero.So why don't interview patterns change too.

If we expect the person to design,code,execute with you , why don't we check those aspects instead of pure-academic questions on integrity of a tree-search for high-performance solutions delivered in a 10-minute conversation.

Also may be am wrong & am the one with a green toe and rest of work-places do want only academicians with linked-list experience.

<comment><USER13> Since when is tree traversal "pure academic"? It's like one of the most mundane things, it's like sorting or hashing, it's something you run into on a regular basis, and the fact that it's wrapped into a library doesn't give you an excuse to not know it, because that knowledge is necessary to use the library correctly (i.e. answering questions like "do I use TreeSet or HashSet here?").

It irritates me because it's not even remotely "academic". Category theory is academic, this is mundane things. It probably is the bare minimum one has to know if they hope to work on anything interesting.

<comment><USER12> its the same point that am trying to make.my use of pure-academic was in the pun-sense of it.

Interviews shouldn't be checking bare minimum, when we spend a 10minute time talking to each other , i would rather prefer to spend it on something rather less mundane.To check for bare-minimum we could screen via their previous assignments.But again it depends on what level we are looking for.like the above comment , if we are looking for fresh grads it better be focused on basics.

</comment></comment></comment><comment><USER14> I really don't find these questions to be useful indicators of the things that make-up a good software engineer. I am using "software engineer" here on purpose and as something distinct from "programmer" or "coder".

One will design and develop good reliable products that are maintainable and extensible. The other will just crank out code and is likely to not really add value beyond that.

I want to know how a person thinks and reasons, how good he or she is about data representation or the formulation and communication of a strategy in creating a solution. I want to know how a person thinks at the abstract, code, project and product levels.

I don't really care to learn that he or she can memorize 150 CS interview tests. In fact, if someone aced the CS tests I'd pretty much assumed they memorized a whole pile of them for the interview. That, to me, is useless as it isn't an indicator of the kind of professional you might be hiring.

If what you are after is a coder, an assembly-line worker, then puzzles might work. If, on the other hand, you are looking for a true software engineer, you need a different approach.

</comment><comment><USER15> There's no harm in brushing up on CS 101 questions, but I wish more candidates spent time preparing for questions about code quality.

</comment><comment><USER16> As a side note: never interview in C++. For these types of problems java is just so much easier to work with, and it's easy to learn enough java to interview with in a weekend from a C++ base.

In my experience, interviewers always prefer a good easy solution (java) to a so-so difficult one (C++). Ruby and python are worth a try too, but often they make things _so_ easy that the interviewer disregards the answer--many common string manipulation interview questions are one-liners in ruby.

<comment><USER17> Java is your blub? Why not go for Haskell or ML?

<comment><USER18> Because neither is a language commonly used in interviews.

<comment><USER19> Maybe they're not commonly used in interviews because too few people refuse to take the sort of interviews where they wouldn't be allowed?

<comment><USER8> having trouble following your nested negative set logic sentence structure here.

<comment><USER20> Restructured: "Maybe they're not commonly used in interviews because too few people insist on being able to use them in interviews?"

</comment></comment></comment></comment></comment></comment><comment><USER21> Having been in the industry for a while, I've come to realize that the sad reality is that (IMHO) the vast majority of the companies would be quite happy if the candidate was able to regurgitate such answers in an interview. I know (acquaintances) who keep working on interview questions and, basically, keep interviewing. Hopping from job to job, they have done very well financially.

</comment><comment><USER4> I'm surprised, in an industry that is founded upon automating repetitive tasks, that the coding interview can't be automated. Isn't that what standardized testing is all about?

<comment><USER3> This comment nails it:

      > I really don't find these questions to be useful indicators
      > of the things that make-up a good software engineer.  I am
      > using "software engineer" here on purpose and as something
      > distinct from "programmer" or "coder".

      > One will design and develop good reliable products that are
      > maintainable and extensible.  The other will just crank out
      > code and is likely to not really add value beyond that.

      > I want to know how a person thinks and reasons, how good he
      > or she is about data representation or the formulation and
      > communication of a strategy in creating a solution.  I want
      > to know how a person thinks at the abstract, code, project
      > and product levels.

[https://news.ycombinator.com/item?id=6942548](https://news.ycombinator.com/item?id=6942548)

Standardized testing will quickly get subverted, with people "studying for the test." It's almost a theorem that any proxy for assessment will become subverted, with people passing the test without actually possessing the necessary underlying skill. In math people keep complaining about "teaching to the test" and that people pass the tests, and yet don't have the skills.

Same will happen if you try to standardize computer recruitment.

<comment><USER22> _Standardized testing will quickly get subverted, with people "studying for the test." It's almost a theorem that any proxy for assessment will become subverted, with people passing the test without actually possessing the necessary underlying skill. In math people keep complaining about "teaching to the test" and that people pass the tests, and yet don't have the skills.

Same will happen if you try to standardize computer recruitment._

Hasn't the industry effectively done that already? Doesn't the fact that almost anyone of reasonable ability[1] can pass an interview after studying something like _Cracking the Code Interview_ indicate that you can successfully "study for the test"?

[1] I am assuming that, even when "teaching to the test", a certain amount of baseline ability is still required to actually pass.

<comment><USER4> Here's my observation about "teaching to the test," having two kids who are now in middle school. Since every kid has to pass, the test is written to the lowest common denominator. Drilling the test over and over, to boost the school's average by a few points, short-changes the kids who have already mastered the material and need more challenge.

Something like "cracking the code interview" could become an upper bound.

</comment></comment></comment></comment><comment><USER23> > _Implement a simple linked list in C (not C++). It can be compiled on a C++ > compiler, but do not use any feature of C++._

Er, which standard of C?

<comment><USER24> You get the job if you can think of a non-trivial way in which in matters.

<comment><USER25> That's pretty easy. Pre-C89 did not have void pointers, thus a generic linked list would have to use char pointers.

Beyond the wrong-ness of using an incorrect dereferencer type you would also force any users of your linked list to make an explicit cast. Meanwhile a post-C89 linked list could return void *. Thus taking advantage of C's implicit cast of void pointers.

Meanwhile if you compiled this second void pointer returning linked list under a C++ compiler would have to make an explicit cast just like in the pre-C89 case.

<comment><USER23> Sorry I should have explained myself better:

I am referring to the C-standards, and I don't consider K&R to be one of them. I tried compiling the source with:

      gcc file -std=c89 -Wall -o out

&&

      gcc file -std=c99 -Wall -o out

I get a score of errors thrown at me.

Not one of them mentioned a void* or char _, they don 't appear in the code. The float which holds the data is located within the struct itself.

The first nine errors were all practically identical:

      error: unknown type name 'LL_Element'

I don't know much c99, but when it comes to c89 whenever an instance of a _struct some_type* is declared the keyword struct also has to precede the type: ie. _struct some_type some_instance;_. The only way I am aware of that overcomes this is the use of _typedef_ in the original declaration.

I'll have to check whether it is mentioned in the 1988 edition of _The C programming language_ or another text, but if I recall correctly the term _tag_ is used to refer to struct declarations.

Also the original code had the headers _< cstdlib>_ && _< cstdio>_. I think these headers have all their functions declared as _inline_ , a non-existent keyword with regards to c89. I had to change the headers to the traditional _.h_ style header in order to have a chance of getting the code to fit through the slot, so to speak.

I don't have a CS degree, I teach myself, so if I have missed something or made a _faux pas_ , please say so.

</comment></comment></comment></comment><comment><USER26> I am just a student but do interview questions come without any concrete problem to solve? Or are they really like "Which data structure would you use for repetitive look-ups?", "Implement FizzBuzz" and so on instead of something like "You need to look up a words in a English dictionary for a spelling program, how would you solve it?".

<comment><USER3> It depends. In many cases it's impossible to give sufficient context to provide a real-world problem that will let the candidate show their understanding of algorithms and data structures. Sometimes it's necessary to ask these sorts of questions in isolation, without a concrete problem. Not least, sometimes a real world problem is better solved in a different way anyway, which subverts the intent.

Understand, most real world programming jobs don't need in-depth knowledge of algorithms and data structures. Most real world programming jobs are implementing the algorithms devised by someone else, using data structures that are "obviously" the right structure for the job. When companies are asking these sorts of questions they usually have a somewhat inflated view of their own importance. Either that, or those interviewing have no real idea about the work done by their programmers.

The very best interviewers, when using these questions, don't worry too much about the detail of the solution, but in the discussion of the pros, cons, benefits, drawbacks, and possible development of the code in context. Even with FizzBuzz there is much to be explored once a coder has written it.

<comment><USER26> I see, thanks for your perspective on the matter.

</comment></comment></comment><comment><USER27> How much does one need to understand how a car's engine works to drive a car?

I believe there is some relevance to a deep understanding of algorithms, that relevance being a function of the job. But in situations where the relevance is CLEARLY low for the job, they serve as nothing more than projections of the egos of CS grads, and reduce the value of their company.

</comment><comment><USER28> 20 year veteran here (although experience does not really equal the 'years in business', it is a multiple of years in business and ability to 'accumulate' and 'apply' the experience).

Interview is a negatives filter, not a 'successful employee' indicator.

As a filter, it should be adjusted depending on the job markets, salary/benefit brackets and the type of effort a company/team is willing to put into a hire.

A retained hire is a hire who passed the negative filter and then passed the 'successful hire' assessment, which should be done in a 3 month period.

A company that is not willing to let a bad hire go after 3 month assessment, will eventually accumulate a horrible baggage, that will bring their productivity and morale.

No interview (negative filter) can be good enough to avoid the 3 month assessment. If you do not accept this thinking, and still assume that you can construct an interview that will be good enough to avoid the 3 month assessment -- then you are a likely looking to hire candidates that mimic you, and in general (regardless how average/special you are) -- you will hard time finding the candidates, and actually scaling up your hiring effort.

Back the actual interview.

Programmers need to know which algorithms are sensitive to data sets and in which way (memory consumption, disk consumption, run time cost)

Programmers need to be able to find good implementations for the specified constrained. this is important -- they need to be able to find the existing solutions, not build them themselves (unless you are hiring researches to work out things that have not been done before).

So a good test would be to present a problem, and ask a candidate to do a quick internent research, and find analogies/approaches for a given problem. Asses which one of the found on the net solutions fit better to the problem space. And then describe how they would incorporate what they have found into an implementation, and how much effort they think it should take.

Programmers need to be able to reason about maintainability and testability of the code base, using both previous experience, and analogies from open source projects. Having references to popular books on this topic is helpful is well. This is largely non-formal topic based on empirical evidence or personal experiences. So they do not need to match interviewer's expectations, but must be well argued.

Programmers need to be able to articulate how they react to business decisions that require weighting various trade offs.

Finally, important to see how they relate to the team members that they worked with, from the company they are coming from.

The interview questions posted by the OP address not more than 10% of the criteria I outlined above, so in my view these are poor examples of the CS interview questions.</comment>


Offer HN: Understanding SEO Ebook -50% - <AUTHOR> https://gumroad.com/l/understanding-seo/hacker-news <comment><AUTHOR> Hi, an experiment. My book, Understanding SEO 50% off for hacker news.

About the book: [https://www.fullstackoptimization.com/b/understanding- seo](https://www.fullstackoptimization.com/b/understanding-seo)

It's an experiment, it might get flagged. No one might read it. Maybe it gets ignored, maybe some people enjoy the book (and get rid of SEO agencies). I will see.</comment>