Ask HN: Databases, Scraping, and Copyright Question - <AUTHOR>

Hope you guys can settle an argument I am having with another HNer. I'm going to make up a hypothetical question because I don't want to discuss the real one in detail.
Let's say I have a site that is all about politics and stocks. I list the stock and the board of directors, and site members can submit their political opinions about how they think the corporate officers are acting. In addition, I write a program to go out and find the FaceBook, Twitter, etc accounts for these officers and I list them with the stocks.
Not wanting to do a lot of data entry, I find a database on an ftp site somewhere that has stocks and the corporate officers that are related to them. It is part of some huge system dedicated to do something completely different, like a brokerage system or a  system for investing. Although they provide the data openly, on each dump there is a notice about how you can't use the information for commercial purposes (but you are free to use for personal purposes and/or download, copy, and give to friends) Their price is $50K. For that amount of money, you might be able to use part of the data, depending on what your actual needs are (translated: we want to know how much we can get out of you, so we're not committing to anything)
Do I have to physically type in each stock and the directors? And then update it all the time? When there are multiple sources of publicly-available information that gives me the same thing? That sounds crazy.
I was speaking with another HNer just now. He said if I downloaded the database tables I needed directly that was bad. But if I crawled a site created from the database from the static information located on each stock's page, then that was different. I say that matters of fact -- which people are officers for which company -- cannot be copyrighted. It would be different if I were pulling down big hunks of the database and reusing/re-purposing them, but I'm only asking a simple question about information that is very widely distributed.
So who is more right here? If you were trying out an idea for a website, what would you do? <comment><USER2> Phonebooks can't be copyrighted, and databases can't be copyrighted (exactly) but if their terms of service state that usage is restricted, and they're the kind of company I would expect them to be, you can expect a lawyergram as soon as they find out. Unless your pockets are deep, this will be a Bad Thing.

Unfortunately, the _exact same thing_ applies to scraping their site for that data. If you scrape somebody _else's_ site generated from a paid use of that database, then ... maybe. Maybe. It's an awful risk.

It would be better if you could figure out a way to scrape other publicly available data sources - even using that database as a guide.

Don't get me wrong; there are really three questions here: the moral, the legal, and the practical. Morally, I consider scraping a site to be OK because it's really available to the public; just grabbing the database from a site that says, "don't do this" - not so much. Practically, I just went over that - it's a risk. Legally? I'm not a lawyer, but I suspect that the legality is more or less cognate with my moral feelings above - but a lawyer might be able to tell you.

If _I_ were trying out an _idea_ , though, which was your original question, I'd download the damn database and get it running. Then I'd try to find a more plausible data source, but that is a task that could be assigned a couple of months and be somewhat lower priority.

<comment><USER3> Actually my understanding is that your first sentence is a fallacy.

Each individual entry in a phonebook cannot be copyrighted, as that is public information.

However, the collection as a whole can have a copyright.

It's like not being able to copyright a music note, but being able to copyright how those pieces of information (notes) are put together into a song.

<comment><USER2> The Copyright Act specifically applies to "original works of authorship", and in 1991 the Supreme Court decided that phonebooks _specifically_ are not original works of authorship, although individual design elements in the book itself may be.

<http://www.ivanhoffman.com/database.html> is a good overview.

However, there is what's called a "compilation copyright", which is covered rather nicely at <http://www.bitlaw.com/copyright/database.html> \- the theory there is that the selection of data is original authorship. This is still based on the same Supreme Court case (Feist), where the Court stated that if the underlying data (phone numbers and names) is uncopyrightable (you can't copyright facts), then if that data is "selected, coordinated, or arranged in such a way that the resulting work as a whole constitutes an original work of authorship" the database may be copyrighted.

But the basic point of Feist is that phonebooks in particular are not subject to copyright. That's why I led off with that example, and it took me one Google search (on "database copyright", and these links were #3 and #4) to confirm my vague memory.

I'd actually say at this point that the database in the original post is probably _not_ copyrightable. Which is all fine and good, because it won't stop a company from suing you, and may the richest man win. So it changes nothing from the practical standpoint - copyrightable or not, the original poster would be better off finding independent sources of data.

<comment><USER4> Similar, In Australia, phone books were also recently ruled (I'm not a legal person, a judge made a ruling in a case I think) to be not covered by copyright.

</comment></comment></comment></comment><comment><USER5> My understanding is that most "public" data is not copyrightable, so scraping it is fine. For instance, scientific data is public, but access to databases can be charged for. What Macy's phone number is is public information, but Yellow Pages can charge for access to it. In these cases, scraping is fine unless they have a 'Terms of service' that states otherwise. Sites often have terms of service that you wouldn't notice unless you cared to write a scraper for them. Google local for instance has a terms of service that makes it clear it's not for scraping.

I have scraped a number of sites to 'repurpose' in other ways. If you're betting the farm on it, talk to a copyright lawyer. If there are lots of sites with the same data, I don't think anyone will notice or care.</comment>


After using both, I regretted switching from Terraform to CloudFormation - <AUTHOR> https://medium.com/@cep21/after-using-both-i-regretted-switching-from-terraform-to-cloudformation-8a6b043ad97a <comment><USER2> have you looked at CDK?</comment>


What Is Posix? RMS Explains - <AUTHOR> https://opensource.com/article/19/7/what-posix-richard-stallman-explains <comment><AUTHOR> "Open source discourse typically encourages certain practices for the sake of practical advantages, not as a moral imperative."

The line between practical and moral adoption of F/OSS continues to be interesting to me. Has the tide shifted toward practical at this point, or is that perspective still a minority?</comment>


Show HN: I made a social todo list, and just opened it up to the public - <AUTHOR> http://tasksmash.com

<comment><USER2> Though he'll probably shoot me for calling him a "Task Master," you might want to contact DanielBMarkham about task management tools.

[http://www.whattofix.com/blog/archives/2011/12/tyranny-of- th...](http://www.whattofix.com/blog/archives/2011/12/tyranny-of-the-1.php)

<http://news.ycombinator.com/user?id=DanielBMarkham>

</comment><comment><USER3> The wall of text is a pretty daunting landing page IMO. Why don't you try something a bit more inviting (e.g. spreedly.com)?

</comment><comment> saiko-chriskun my calendar links are broken (i.e. when I click on one of the calendar days.)

<http://tasksmash.com/saiko-chriskun/20111212></comment>


Raspberry Pi 4: Release Date, Specs, Price, Everything We Know - <AUTHOR> https://www.tomshardware.com/news/raspberry-pi-4-everything-we-know,38539.html <comment><USER2> He dislikes usb-c for ruggedness in challenging environments but its on MacBook pro and Lenovo X and t series being chucked into dusty backpacks and dirty cafe tables?

Sure you can get din mounts for a pi but if you want a pi in a rugged environment it has many many more points of weakness like it's bare gpio points and exposure to lots of surface mount parts</comment>


Japan's Silent Strategy for Driverless Cars - <AUTHOR> https://www.forbes.com/sites/tromero/2018/04/03/what-ubers-crash-tells-us-about-japans-silent-strategy-for-driverless-cars/#557ea1c23e39 <comment><USER2> The focus on autonomous buses is fantastic. This is a way of developing the AI without being a direct threat to taxi and bus drivers.

In the UK there are many places where it's not practical to run a bus service but at the same time it would be extremely useful to have one. The buses don't need to be huge just regular.

Buses in rural areas also tend to stop much earlier so if you go out drinking it's either designated driver, expensive taxi or as I'm sure many do just drink a bit less and take the risk of not getting caught.

I think bus and taxi drivers are the last people we should be replacing as they are professional drivers who are the safest. But if we can extend bus routes whilst keeping bus drivers on the busiest then it makes sense.

<comment><USER3> > The focus on autonomous buses is fantastic. This is a way of developing the > AI without being a direct threat to taxi and bus drivers.

Wouldn't autonomous buses be a threat to bus drivers?

Manhattan should be leading the way for autonomous buses and taxis. We have grid based streets for the most part. It is ideal for autonomous transportation systems.

<comment><USER4> How in the world would a grid of streets make autonomous vehicles easier?

<comment><USER5> Manhattan has bus lines that basically go in one direction: up/down the avenues or crosstown. Just worry about starting and stopping and less about turning.

</comment></comment></comment></comment><comment><USER6> Very interesting, it answered quite a few questions I had not dared to ask.</comment>


Show HN: Props to the Akinator App for creative crowdsourcing - I taught it pg - <AUTHOR>

The Akinator app, free on the Amazon App Store today, guesses a person you choose based on a series of questions. It knows a ridiculous number of people, seems pretty amazing - when I tried pg, I figured out how it worked. It asks you at the end to name your unknown figure, with a title - we're training it as we use it. Major props. <comment><USER2> The computer game 'Animal', dating back to the 1970s, used a similar mechanism to guess what animal you were thinking of – or learn from its mistakes with a new user-suggested question. (Seeing that game in elementary school was one of the first things making me interested in programming.)

There's a web version at <http://www.animalgame.com/>, and it's mentioned as one of the precursors to the similar '20Q' AI project/handheld-game at Wikipedia:

<http://en.wikipedia.org/wiki/20Q></comment>


Sending a Spacecraft to the Interstellar Asteroid - <AUTHOR> https://arxiv.org/abs/1711.03155 <comment><USER2> It's okay, we can wait for the next one. They come in threes anyway. ;)

Unfortunately the biggest hurdle to such a mission would be aligning political will and agency goals on a short time frame. NASA plans thingsnon 5 and 10 year horizons, so sending this mission means delaying something already in progress, which jeopardizes that project as everyone who was working on it goes off to do other things :( Can it be reached by a nearly empty payload on a Falcon Heavy or BFR certification flight? Getting a fully private mission might be the only option.

Edit: just realized SpaceX might not be able to recover any stages from such a launch, so maybe not so enticing. With cross fueling they might get he first set of ejected stages though.

<comment><USER3> It's a good reason to push for more space telescopes.

I believe some day (probably after we start having regular interplanetary travel) we'll have robotic light recorders in space recording every bit of incoming extraterrestrial light, especially on the planetary plane. Then we'll be able to back up to view where objects like these entered the system, not just after they are detected. And perhaps just detect them when the are entering the system to give us as much lead time as possible.

<comment><USER4> I am curious if anyone is actually thinking about this problem in terms of technology required that currently does or does not exist, particularity in tech needed to store such vasts amount of data, unimaginably high resolution sensors, etc. It sounds like a fun project to be part of really.

<comment><USER5> A space telescope isn't fast enough. That's why there is LSST.

</comment></comment></comment></comment><comment><USER6> We don't suspect that interstellar asteroids are particularly rare. The best course of action would be to beef up the Earth-based telescope observation network so that we can discover future ones further out. As a side effect we'd get much better forewarning of potentially life-endangering near-Earth adteorids as well.

Once you have better lead time on an asteroid, and catch it coming rather than going, it's easier to intercept.

<comment><USER2> What is interesting about this one is it's shape, otherwise I don't think there's be much interest in such a mission vs waiting for the next one (since there could be as many as 3 a day if our models are correct).

What process other than aliens creates a kilometers long rotating cylinder? It probably _IS_ a natural process, don't get me wrong, but we haven't a f$&@ing clue what that might be which makes it really interesting to study.

<comment><USER7> Passing at great speed very close to a star? Outer shell or entire thing becomes superheated and malleable. Extreme tidal forces as the object passes the star and gets churned through the high gravity stretch it out. What if it passed close to something super dense, like a black hole, allowing the stresses of tidal force as it whipped around it to melt the rock on their own leaving it to fly away, pulled long and cooling in the cold of dead space.

I don't know how likely such things are, but I don't imagine it takes intelligence to make a long space rock.

<comment><USER2> To make a long one without breaking it into multiple objects is very unlikely. This thing isn't composed of silly putty -- rocks and metals tend to be brittle and break rather than "stretch out".

<comment><USER8> Not if they are melted, or softened. It might have taken millions of close orbits around a hot body for that shape to form.

<comment><USER8> Which also might be why it was ejected and made it to our solar system. The energy in its orbit might have been really high on a close approach to a star and a slight deviation flung it out of that system.

A highly eccentric orbit would have a close approach around a star, and then have it rapidly cool as it traveled away from the star. Repeat millions of times. Would make for a nice orbital, thermal and materials science simulation.

</comment></comment></comment></comment></comment></comment><comment><USER9> Now this is what I'm talking about! Really cool stuff!

I think this was kind of a wake up call that we don't have any kind of rapid- response system ready and waiting to intercept interstellar objects. Even if we don't get this one, we should damn sure be ready for the next one.

<comment><USER10> I briefly worked with members of the planetary defense scientific community, the folk thinking up ideas to defend the Earth from asteroids. This would be great opportunity for them to demonstrate their technology, if they had ever received any funding.

</comment></comment><comment><USER11> I went thru all Reddit posts and major conclusion was that it cannot be a spaceship - not because of its shape but rather because it was going much too slow.

Someone calculated with its current speed it needs 50,000 years to get to another star (thank you for correction). So no alien with technology to build something like that would want to ride at a "bicycle" speed.

Unless of course they slowed down to take a turn (using Suns gravity pull), just like riding 500HP car you slow down in the corners to leave them safely.

Edit: corrected, thank you friend.

<comment><USER12> Could be a derelict, though. Arguably, spaceship wrecks should be more numerous than spaceships themselves. After all, assuming alien spaceships have existed for a long time and have a relatively short operational time, they exist for much longer as a wreckage and are as such much more likely to be found in this form.

Just consider our own near-Earth space : there are many more space debris there than there are operational satellites.

<comment><USER11> Few people argued back and forth that it could be a used alien rocket booster.

</comment></comment></comment><comment><USER13> Does it make any sense to place a series of landers in solar orbit, waiting for short-lived opportunities like this? I don't think we'd save much on fuel since the orbits wouldn't be optimized for the target. However, it would require much less planning when opportunities arise, and wouldn't be as reliant on the political winds of the day.

<comment><USER14> I'd guess not, as the various components that make up probes are constantly improving. While it would be nice to have a 1990's era probe on hand to investigate this asteroid, I bet we can learn a lot more from modern sensors, and the electronics and comms will be more powerful and lighter.

<comment><USER15> Don't knock 90s technology: Hubble is still sending us gorgeous pictures of the universe...

</comment></comment></comment><comment><USER16> You can ask questions to the authors of this paper here: [https://fermatslibrary.com/arxiv_comments?url=https%3A%2F%2F...](https://fermatslibrary.com/arxiv_comments?url=https%3A%2F%2Farxiv.org%2Fpdf%2F1711.03155.pdf)

</comment><comment><USER17> We maybe can't catch it or probe right now, but it wouldn't be too hard to target variety of EM transmissions at it

<comment><USER18> That's pointless, EM is an outdated technology, nobody listens to it anymore.

<comment><USER18> geez, it was a(n interstellar) joke guys! the aliens might be using neutrinos or something!

</comment></comment></comment><comment><USER3> "With the high approach speed a hyper-velocity impactor to produce a gas 'puff' to sample with a mass spectrometer could be the serious option to get in-situ data."

This seems to assume it could never be an artificially made object.

<comment><USER14> Plot for a new movie.

Live action "The Dig" anyone?

<comment><USER19> Fun fact: "The Dig" was originally planned to be a Spielberg movie, but the cost of special effects made them change it into a Lucasarts Point-and-click adventure game instead.

</comment></comment></comment><comment><USER20> Don't bother. It's just a decoy.

What we really need to do is start combing the databases for anomalous X-ray flares so we can find any nearby Oasa emitters.

</comment><comment><USER21> The problem is that unless you plan to crash into it you need to reach the same speed. Or use a rail-gun to shoot an anchor.

<comment><USER22> Or cast a net.

</comment></comment><comment><USER23> That was underwhelming. I'd much rather see NASA's or SpaceX's or Roskosmos's take on this. AKA the entities that actually have the means to pull this off.</comment>


Recruiters are, by and large, a waste of your time - <AUTHOR> http://blog.tutorspree.com/post/11909570564/recruiters-and-startups <comment><USER4> A few days ago I moderated a panel discussion at the Computer History Museum with the founders of Dropbox, Evernote, and Wildfire. I asked whether it was worth dealing with outside recruiters, and they all said no. They all had recruiters working for them in house. In fact between 3% and 5% of their employees were recruiters.

They said the best source of people initially were the friends of the founders and the first employees (which is one reason it's helpful to have gone to a good CS school) and that as soon as you've exhausted this source you should start hiring recruiters.

<comment><USER5> _as soon as you've exhausted this source you should start hiring recruiters._

Did they present any evidence to support this position, or does "should" in this case just mean that this is what they did?

There have been a few times when I've talked to technical people at startups and come away thinking "if it weren't for Tarsnap, I'd _totally_ go work for them". In contrast, every time I talk to a recruiter, I come away thinking "if this is the sort of people they have at X, there's no way in hell I'd ever want to work there".

I'm not sure what the best solution is here, but it seems that recruiters are far from perfect.

<comment><USER6> When a smart small company hires a recruiter, part of the goal is to have recruits never really realize they're talking to recruiters. Presumably most YC recruiters aren't randomly cold calling people; instead their job might involve a lot more of "setting up circumstances so that the right person in their company is likely to end up talking to Colin Percival".

<comment><USER5> Aha! Good to know that there are such things as good recruiters out there, even if I've never encountered them (knowingly, at least).

<comment><USER6> Most of the time, when people complain about recruiters, they're complaining about dialing-for-dollars cold-call transactional recruiters.

Recruiting is a special form of sales/marketing, and most businesses wouldn't expect randomly calling people on the phone to be their best early sales strategy. But that doesn't mean they don't do sales; they just execute on an early sales strategy.

Jeff Cox's _Selling The Wheel_ is a good, if somewhat patronizing, summary of the different sales strategies companies use at different stages of the adoption curve for their offerings. The same insight applies to recruiting.

</comment></comment></comment></comment></comment><comment><USER6> Here's my most recent recruiting cold call:

Him: _Hello, I need to speak to the person who manages your servers._

Me: _Uh, who is this?_

Him: _I need to report an issue with your servers. Are you the person who manages the servers?_

Me: _What kind of issue?_

Him: _Do you manage the servers?_

Me: _I manage the office. What's the issue you need to report?_

Him: _Oh! I'm calling because I understand that it's very hard to find good Linux and Ruby on Rails people in Chicago and-_

Me: _You're a recruiter?_

Him: _Yes, and-_

Click.

Our current recruiter policy, at least in the Chicago office: $50 to the person in the office who can keep a recruiter on the phone the longest using only Emacs Eliza.

In terms of actually recruiting people:

* It's one of the key jobs for the management team; for instance, it is the thing I spend the most time on after billable work.

* We do increasingly generous referral bonuses for good candidates.

* We're doing a number of outreach programs (free classes and the like) to meet candidates.

Ultimately, like SEO consultants and direct sales people, the problem with recruiters is adverse selection. There are good recruiters out there, but they can more or less print money; they aren't cold-calling you. Anyone who is cold-calling isn't a good recruiter. There aren't many good recruiters. The most useful shorthand is "all recruiters are terrible".

<comment><USER7> I usually respond to conversations like that with "you started this conversation with a blatant lie, why the hell should I trust you enough to do any sort of business with you?". They often call back almost immediately in the hope of someone else picking up the phone instead of me.

Other common lies are "I'm a personal friend of Mr Anderson" (Mr Anderson doesn't exist, we made him up and listed him in a couple of places as our resources/personnel manager) or "I was talking to <some person> earlier today" (that one is particularly fun when they state they were _just this minute_ talking to someone I has been sat opposite me talking to nobody for the last hour or few).

I've stopped being nice to them. I don't even make an effort to be professional.

Some companies will call repeatedly, day after day, sometimes several times a day. The only way to get rid of them I've found is immediately asking to talk to their boss to discuss being taken off their lists and demanding to talk to their boss until you get to or then hang up. And if that fails find the company's address and send them a letter by registered post stating that we do not want to deal with them, will not give them any more of of our time for free, and that any further calls will result in them being invoiced for our time at our consulting rates (we've gone as far as that twice, both times we heard nothing else from the company in question afterwards).

</comment></comment><comment><USER8> I need to write a post to discuss this at greater depth, but...

I am fucking astonished at how inept most recruiters are. I mean, just awful and useless at their "jobs."

If we concede the point that, especially in the Valley, technical talent is in high demand, it stands to reason that any overture to such talent needs to be persuasive, informative and intriguing. It should explain the role in reasonable detail, while making a sane case for how cool the company and opportunity will be.

Ideally, it should demonstrate that the recruiter has a basic understanding of a given prospect's skills and career trajectory. Super bonus points if it doesn't seem like a form letter.

Instead, what I get is stuff like

"Hey, I'm looking for developers for a role I'm filling. You need to be this, this, and this. Oh, and _this_ is a plus. Mmkay, lemme know!"

Well. Congratulations?

Hey, I _have a job_. And it doesn't involve proving to a random recruiter that I'm skilled at it. And from such pitches, how would I know that I'd even care? I know nothing about the company and only have the vaguest notion of what I'd be wanted for.

Recruiters _are_ a waste of time, though mostly their own. A founder out being social has a much better chance, I think, at successfully recruiting limited talent. Not that they have time, I know.

And don't get me started on the pitch that opened with "I'm looking for some iOS studs!" I had a friend at the company they were pitching. They weren't impressed when they heard about this one, especially since they had no formal relationship with the bozo.

<comment><USER9> I've worked with third-party recruiters for many jobs I had while contracting. The pitch you mentioned can work, but only in the high-turnover world of contract consulting for big companies. This is NOT going to work for startups, or anything even close.

Contract programmers often switch jobs once a year. That's driven my duration limits set by companies, layoffs and the wanderlust that contractors often have. Big companies don't seem to want to be bothered with in-house recruiters, so they pay third-party companies to do the work. In the midst of that shuffle, a job requirement is handed from a hiring manager (who may not know how to code) to a company HR rep (who may edit it) to the recruiter. What you end up with looks like this:

      Req: JAVA Developer
      REQUIRED:
      - 5-7 Years Java Experience
      - 3-5 Years J2EE Experience
      - 1 Year JDBC
      - 3-5 Years Oracle
      - MUST HAVE WEBLOGIC 10.3.5
      PLUSSES:
      - C/C++
      - <USER2>

Of course, this may or may not have anything to do with what the job actually requires. They may have a taped-together C/C++ and Perl system they're trying to move to JEE/Weblogic They probably haven't really ordered (or gotten approval for) Weblogic, but it's what all the other companies seem to use.

So, the recruiter goes around spamming folks, and manages to find someone who's just hit their 12-month limit at BigCo and is open to find another position. The recruiter gets paid, the contractor may or may not like the position (but gets paid, anyway), and the hiring manager gets bagels brought in, courtesy of the recruiter's firm. Everyone wins, so to speak. That's why the cycle persists.

I'm describing this as someone who played that game successfully for many years. To someone who's working in startup circles it may seem like madness, and it would be if applied to startups. But it's how Fortune 500 companies, for better or worse, find talent.

To put it in startup terms, the third-party recruiter model seems to work well for companies who've found their business model and need people to execute it. For companies who are in a race to iterate and find a product-market fit, it's the kiss of death. Skip the free bagels, delete the emails and find people on your own or with internal recruiters who really know your company well.

<comment><USER10> _I've worked with third-party recruiters for many jobs I had while contracting. The pitch you mentioned can work, but only in the high-turnover world of contract consulting for big companies._

Big companies have a specific reason for going with contract employees. It gives them (the company) a legal separation from the person who would otherwise have been an employee. That person can be unceremoniously dumped out onto the street if things aren't working out, and the company is willing to pay extra for that option.

Startups can't afford to pay a 30% premium on technical help just so it's less risky to fire the bad apples. And anyway they aren't big fat targets for lawyers like big companies are - good luck finding a lawyer to take your case on contingency when you're suing a company that's having trouble making payroll.

</comment></comment></comment><comment><USER11> _The roles that we fill at this stage are highly specialized and generally call for nearly contradictory attributes in candidates. You need incredible good engineers who are not already in golden handcuffs, are crazy enough to join a risky early stage company, do not want to found their own company right now, want to work at all hours, and are aligned with the culture you are trying to build._

It seems that this is indeed what startups are looking for. It also seems that they are generally unwilling to pay market salary. And startups don't seem willing to part with enough equity to make up the difference in the slightest if you weigh in how unlikely it is for employees to see any return on equity.

There may be a reason why these jobs are hard to fill.

<comment><USER12> Quite. There have been recent discussions on HN which point out the darkside of this: that early employees of startups often take as much risk and work as hard as the founders, yet get a tiny sliver of the upside. Co-founder, or nothing.

<comment><USER11> There has been quite a bit of discussion, going back almost 2 years: <http://news.ycombinator.com/item?id=1369039>

I'm personally interested in this because I know first hand about the bottom of EV-comp curve: I personally _enjoy_ working at really early-stage companies, have little in the way of expenses (27yo DINK), but don't have enough saved to start my own thing (payed off 50k in student loans last year).

It feels somewhat akin to working at game companies: Working in the industry is glamorized, so young and energetic people flock to it and get relatively shafted _on the margins_. My job is nowhere near as bad as working at somewhere like EA, but it's not unlike it either.

I won't be doing this for much longer though, as the market isn't changing fast enough. Next time it's co-founder or series C+. In-between I always feel like I'm being had, to a degree.

</comment></comment></comment><comment><USER13> Here's another side of the story: I just moved to SF, and I'm looking for jobs. Two friends sic'd a bunch of recruiters on me, and it is awesome! Here's the process:

       1. They email or call me, and I send my resume and short summary of what I want.
       2. They send me streams of awesome jobs.  (Two have sent bad jobs, but gmail has
          a hotkey for that :D )
       3. I say I want to talk to XYZ at one of these times, and they hustle to make it
          happen.
       4. They tell me little nuggets like "The last guy I placed here made $X"

I'm running my own processes in parallel, and I can tell you that they suck a lot more. Non-recruiters -- and even in-house recruiters -- are hard to herd.

<comment><USER14> Fair enough. That's the most use I've heard of anybody getting out of recruiters. Well done!

It's probably worth putting up a blog post somewhere about it :-)

</comment></comment><comment><USER15> Quote: _You need incredible good engineers who are not already in golden handcuffs, are crazy enough to join a risky early stage company, do not want to found their own company right now, want to work at all hours, and are aligned with the culture you are trying to build._

That really is asking for everything. I hope you have figured out a way to explain your value proposition to these dream candidates.

<comment><USER16> Agreed.

Are there people who really want to work at all hours? Maybe eager students right out of college.

Personally, I'm at the point where if I'm going to be working at all hours, I'd better be one of the primary beneficiaries of that work.

<comment><USER17> Basically, you want that rarest of birds - shy sparks who aren't nestled in a large company yet.

Too introverted to be really aware of their market value and have a life and smart enough that they're going to be obsessed with the work and work at it all the time.

Considering I'm having a hard time finding "reasonably competent" Rails devs, I can't imagine how you might hire for that kind of position.

<comment><USER18> maybe some combination of an IQ or programming test plus a Myers-Briggs analysis. That would probably be a great way to screen. Or just recruit at MIT/CMU/Caltech...

<comment><USER19> While I see what you mean with referencing Myers-Briggs, do you really believe that it is useful? Some reference reading: <http://www.skepdic.com/myersb.html>

</comment></comment></comment></comment></comment><comment><USER20> I had a conversation with a recruiter and she said how can you expect to get a job if you have ColdFusion on your resume instead of Cold Fusion?

I directed her to adobe.com so she could see the correct spelling. She went oh crap, I've been throwing away all the resumes with ColdFusion because the employer spelled it Cold Fusion.

No wonder the client is complaining to me about the low quality of candidates that I am sending them. She promised to get me an interview with them but never called back.

<comment><USER21> Wow. I wanted to LOL at that and claim you made it up, but it really does sound plausible.

I've considered putting together a training course for tech recruiters - something like a one day bootcamp to familiarize them with specific tech, so they know the diff between java and javascript, for example. but... need to find the time to do that on top of everything else!

<comment><USER22> Just last week I was sent a job requiring C++ and extensive Ball and Builder experience. It took me a couple of minutes to figure that one out ...

<comment><USER23> Borland Builder? I didn't know that still existed...

<comment><USER22> Yeah - the recruiter didn't know it had _ever_ existed!

</comment></comment></comment></comment></comment><comment><USER24> This is basically an argument that contingency recruiters are bad. You should differentiate this in the title from in-house or contract recruiters (retained search). Contingency recruiters, as you've noticed, have no incentive to filter out junk because they can just take a throw it at the wall and see what sticks approach.

Also, there are a ton of bad recruiters, just like there are a ton of bad engineers. There are awesome recruiters out there and the sample of those who have cold called you rather than going through trusted channels is likely strongly biased towards the bad.

</comment><comment><USER25> This cuts both ways: looking for work takes a substantial, non-zero amount of time, and it's extremely frustrating when startups decide to speak with you even though they know hiring you is "a long shot."

Searching for work at a startup has been a protracted, fruitless process. They have no problem conversing with you and checking out your background over coffee, lunch, whatever. After a few conversations and a technical interview, you find out there isn't a good fit.

Rinse and repeat for any number of tech startups, and you'll be waving goodbye to a month or two of your life.

Has anyone else had this experience? To me, the whole recruiting process is f'kd on both sides.

<comment><USER26> My opinion: if startups aren't serious about hiring you, they shouldn't go through the dance. Why are they wasting everybody's time?

<comment><USER27> I go through this dance a fair bit. At least for us, we're totally serious about hiring. It's just that we're very fussy.

To understand why, think about what we hope to happen. If we do things right, then anybody we hire today will be a de-facto leader two years from now, because they're one of the people who's been around since the early days. Any code they write will be in core parts of the system. And they're going to have a big effect on who we can hire next, because potential hires are going to judge our company partly by them.

We definitely try not to waste anybody's time. But we also don't want to miss a good candidate; some really solid programmers don't interview well. But as soon as we're sure, we politely stop the process and thank them for their time.

<comment><USER26> Fair point. I suppose it depends what you considerable a reasonable amount of time to evaluate a potential employee. All in all, it took me about two weeks to run the gauntlet at my current job (a startup), and I don't think that's unreasonable, given how busy everybody is.

If you keep getting passed over, however, I can see how that starts to really mount up.

</comment></comment></comment></comment><comment><USER28> I had expected this to be related to using recruiters to find jobs. I've found that to be primarily a complete waste of time as well, but I suspect the reasons are the same. To most recruiters, it's just a numbers game, as the article points out; they simply "throw engineers at companies in the hopes that one or two will stick and they'll get their commission". In my experience, it's quite rare to find a recruiter that will spend even a little bit of time actually matching candidates to the position in a meaningful way. It's also possible that, like the author, I've only ever dealt with really bad recruiters before.

<comment><USER29> I'll often get calls from recruiters telling me they've found a position for which I'm a perfect match. And then they explain it's a senior developer position requiring 5 years of experience, when it's very clearly stated on my resume that I'm a college student looking for an internship.

They just keyword search resumes.

<comment><USER30> I've gotten similar e-mails and calls recently, based on a resume I posted to a job site 5 years ago, trying to match me to positions with requirements I don't remotely match. Further, I'm now a full time stay-at-home parent, which is my dream career. What's sad is, when I tell recruiters this, on occasion they'll still try to convince me of what a great fit their position would be.

</comment></comment></comment><comment><USER31> Back during the housing boom, I landed two consecutive jobs through recruiters. Both were cases where the recruiter had a well established relationship with the hiring company (and needless to say were only recruiting me for that specific company).

But it took a lot of contacts with different recruiters to land each of those jobs. Among my favorite experiences was the guy who asked me, "Have you ever heard of _x, inc_?

"Uh, I worked for _x, inc._ and it is listed on my resume."

I have a couple of conclusions.

1\. All it takes is no regular job, a cellphone, a computer, and an account on a resume board to be a recruiter - and it shows.

2\. The vast majority of recruiters stand between you and the job because they have no relationship with the company hiring and therefor you are more expensive to hire and more of a PITA to hire because the recruiter is trying to get his cut (and if she doesn't get it, you will get the "company isn't interested" phone call from the recruiter).

3\. Really good recruiters are a valuable resource, know their industry and players, and won't put just any warm body forward in hopes of a score - <USER3> will screen the hell out of you.

I learned the first question to ask a recruiter is about their contract with the employer - the good one's have no hesitation describing their relationship. The bad one's waffle.

</comment><comment><USER32> I think the title needs to be changed to reflect the gist of the post: that for _startups_, recruiters are probably a waste of time. For a more established organizations, recruiters tend to work, I think.

<comment><USER33> Except large, established organizations usually have HR departments that perform the functions of recruiters anyway.

Or at least they do in my idealized fantasy world, where the role of HR is to acquire, foster, encourage, and compensate talent.

<comment><USER34> That's crazy talk. A hiring manager's job is to acquire talent. HR's job is to smooth the details of the process and keep it legal.

</comment></comment></comment><comment><USER35> It is outrageous when they take 20% of the billing rate (or more) hour after hour, every hour. All for sending a few emails, making a few phone calls, and arranging the interview. Among the middlemen in all industries, tech recruiters do the least work, and are least knowledgeable of the industry they work in - a real estate agent for example, does far more work than a recruiter, and could actually be very helpful.

There has to be a better way to do tech recruiting.

<comment><USER36> Actually, I would pay up to 25%... for the first (6 mo or less) contract. They got me that, how ever they did it. But _I_ got me the extension so why do they expect more than just a payrolling fee for the extentions?

</comment></comment><comment><USER37> As a technical recruiter I am not shocked by this post but I am a little frustrated with the large brush that is painted on the industry as a whole. I mean, I have worked with some BAD Developers, everything they write breaks the minute it is in production, you can never pry them away from the foosball table or lunch room, they smell and can't speak in complete sentences BUT I still recruit for technical talent because not everyone is alike. Just like every industry there are rock stars and other guys just trying to put food on the table.

I have been honored to survive this tough economical climate in the confines of a corporate recruiting gig for a technically forward high frequency trading firm in my hometown of Houston. Over the last few years I know more recruiters who have lost their jobs than I can count, some have lost their homes and their families. Does this sound familiar to anyone, maybe say the days of 2001 for the tech set?

If the stories I have heard our true (I got into the biz in '04) developers making 150/hr one day are banging on recruiters doors begging for interviews the next.

So ya, I get it, some recruiters have zero technical skills, know little to nothing about the job/client but at their core they are human beings. And just maybe, the client doesn't have a lot to sell at the moment, maybe they are still working out the kinks to their "value proposition" but they know they are onto something.

So ignore the calls, politely say you aren't interested but remember, the tides can turn…quickly. The recruiter who doesn't know squat about server level architectural may be next door neighbors to someone who has enough funding to throw you a bone.

<comment><USER38> Good developers are hardly ever on the market. The majority of them are happy at their current jobs, and they know they could go anywhere else they wanted because their skills are highly valued.

I'm sure it gets annoying when they frequently receive emails about "Technically Forward High Frequency Trading Firms" from recruiters who happened to find them through playing buzzword bingo.

These recruiters tout that this new "opportunity" will give more money, benefits, responsibility, good looks, and sexual prowess, but never do the due diligence to make sure that their client is actually a good fit.

It's the shotgun approach that most recruiters use that turns good developers off.

Maybe once these recruiters treat developers with more respect, so will they.

<comment><USER37> So great point, the best developers are never out of work. How would you recommend a recruiter network with said developers? How do we build mutually beneficial relationships? I tend not to cold call people, just because I think there are better ways to meet people but not all recruiters have that type of network.

I suppose I am interested in a solution oriented debate. I hear how you don't like be contacted, how can we do it better?

Oh, and a technically forward HFT means that we are using decently cutting edge stuff, HPC, low latency code, etc. As for offering you better looks and sexual prowess I think I have to opt out on that one. If you find that job, let me know I will send it to a few ex boyfriends. :)

<comment><USER39> "So great point, the best developers are never out of work. How would you recommend a recruiter network with said developers? How do we build mutually beneficial relationships? I tend not to cold call people, just because I think there are better ways to meet people but not all recruiters have that type of network."

As a recruiter myself, understanding the what it is like to be a developer and connecting on the "what makes them tick" level is important. Going to meetups and getting immersed in the startup scene is great, but going the extra mile will have long lasting impact. I am a non technical guy, but over the years, I have tried my best to learn code myself. HTML, CSS, Ruby. Just leveraging the vast amount of learning tools on the internet makes this possible. Although I might never be an employable programmer, I have had a lot of great chats with developers on my own experiences and that breaks the ice and creates trust immediately.

</comment></comment></comment></comment><comment><USER40> I had a call from a recruiter once, asking if I would be interested in a Ruby on Rails position. I said no, that's not what I'm looking for.

She then asked me to explain what Ruby on Rails was, how it related to other technologies and was it popular because a lot of people seemed to be asking for it.

Of course I explained the best I could, to make the world a better place and all that, but it seems (as other commenters have pointed out) that there could be a real niche for techies to run a recruiting firm that know what they're talking about.

<comment><USER41> "but it seems (as other commenters have pointed out) that there could be a real niche for techies to run a recruiting firm that know what they're talking about."

I'd love to do that. What would such a company look like?

</comment></comment><comment><USER34> Recruiters are marketplace makers, not matchmakers.

A recruiter is completely incapable of determining who is a good fit or of asssessing talent, and traditional compensation models (x% of annual salary, paid on hire) are not aligned with succession outcomes for business.

Recruiters are superior to staff at one thing: maintaineding a Rolodex. They have no ability to choose competently from that Rolodex, though.

If you have good PR and a good jobs page, you don't need a generic recruiter.

</comment><comment><USER39> As a third party executive recruiter, I definitely have to agree with many of the comments and frustrations that have been expressed. Unfortunately, at best the perception of recruiters is that they are a necessary evil sometimes in getting access to job opportunities. No recruiter is perfect and even the best retained ones do some cold calling during the week to establish relationships. However, I do agree the approach should not based on gimmicks and sales tricks, but have the intent to establish good rapport with those that are contacted.

But at the end of the day, a recruiter that is in tune with the market, cold calling should really make up less than 20% of their "deal" flow. Great hires are made through referrals, and great recruiters gets constant networking referrals to keep their pipelines full and their days busy. If the recruiter is doing their job, either they are getting warm leads from their networks when jobs are broadcasted, or their networks is referring strong talent to them.

Like a lawyer or accountant that desires to create a reputable brand in a particular marketplace, recruiters needs to see themselves as resources within this marketplace and not view recruiting as purely a transactional game. From my experience, just because I don't collect a fee does not make a relationship unsuccessful. Providing great customer service and building strong relationships will usually evolve into mutually beneficial interactions in subsequent years.

</comment><comment><USER42> What amazes me about a great deal of the recruiters I've dealt with is their inability to do basic word-matching.

Recruiter "I have a great role for you as a Ruby developer!"

Me: "Do you have a copy of my CV?"

Recruiter: "Yes indeed, I have it on my screen right now!"

The word "Ruby" does not appear on my CV, _at all_. This has happened to me more times than I can remember; with Ruby, C++, C# and a whole bunch of other things not mentioned _anywhere_ on my CV.

You don't even need to have _heard_ of a technology (never-mind understand it) to narrow candidates down more than this.

<comment><USER43> Indeed. I've been out of the job market since 2004 and today after a (thankfully) long dry spell I got a "cold email" asking about a .NET position. I was working on UNIX(TM) when .NET came to be and therefore it's never ever been on any of the resumes I left behind.

Then again, since wherever resumes of mine that are still floating around are so old, a shot in the dark like this has a higher chance of working than your situation, where I assume your resume is reasonably up to date.

</comment></comment><comment><USER44> There is a big difference between agency recruiters and recruiters within a company. I have not had any bad experiences with recruiters who work for a company (that is not an agency). In fact, even agency recruiters, there are definitely some bad apples but a lot of them generally are fairly good to deal with.

If I were you, I might change the title of your post to be more specific on the type of recruiters you are referring to.

<comment><USER45> If you actually read the article, that distinction is completely irrelevant, since the proper title is more like "Recruiters are a waste of time _for start-ups._ "

</comment></comment><comment><USER46> I've had very mixed experiences with recruiters. At best they ask reasonable questions and provide informative information about companies. At worst they're rude, condescending, unprofessional and openly confess that "I know nothing about software". I think that the probability distribution is skewed more towards the latter than the former.

</comment><comment><USER47> There is a bigger problem with recruiters and that is there is a whole group of candidates who don't need them and won't use them. The best candidates are falling over opportunities, they find positions through their own network, not agencies. If you only use recruiters you will never find these people.

</comment><comment><USER48> <http://www.hackruiter.com/> is maybe the only exception to the rule. But they're sort of an invite-only thing... so not exactly useful for most startups.

</comment><comment><USER49> As far as I can tell, it's like choosing between paying for display advertising or affiliate marketing. Two different mindsets and neither are a bad thing.

You either do the recruitment in-house and absorb all the risk to your business, which means lots of extra interviews, staff looking through high volumes of paperwork, etc. Or you hand it over to a third party, pay them nothing for the process (i.e. they take on the risk) but then pay highly for results.

<comment><USER27> Except that, most 3rd-party recruiters being clueless, you still have to do all the work. The only sensible way I've seen to use them is as a pure resume source. The folks at Wealthfront explain how: [http://eng.wealthfront.com/2011/07/hire-best-forget- rest.htm...](http://eng.wealthfront.com/2011/07/hire-best-forget-rest.html)

</comment></comment><comment><USER50> I think if you use a recruiter to help find new people to put through your interview process then they can be highly effective. It doesn't make sense to trust in a recruiter to bring you the perfect fit. They bring you people and then you must determine if they are the proper fit. I see no harm in recruiters. You don't have to hire anyone they bring you.

</comment><comment><USER51> Try CodeEval.com. It's free to use and we've helped dozens of awesome companies hire awesome talent through out platform. We run the only free technical screening tool out there.

And with out sourcing product, unlike annoying recruiters - we send very little but high quality candidates that are pre-screened.

</comment><comment><USER52> Shameless plug:

<http://recruiter-review.com/>

<comment><USER38> Pretty awesome idea, thanks!

</comment></comment><comment><USER53> As an in-house Recruiter at a startup, I thought I'd throw my $0.02 in.

The bottom line: you're right. There are a LOT of "bad apples" out there spoiling the names of the decent Recruiters. One person here said they've never had a call from a good Recruiter; that's because we know better. I don't call people for jobs they're not suited for. In fact, I rarely call Techies at all; you don't answer your phones.

Anyway, it's NOT always the Recruiter that sucks, as I demonstrated here: <http://bit.ly/pokYV1>

Loved this discussion though. Thank you all for solidifying so many of my beliefs.

</comment><comment><USER54> Mabye the title is a little off, but from my experience we have an inhouse recruiter and she is worth her salary and more.

I'd be interested in wow do other companies hire without a recruiter.

</comment><comment><USER55> I'm not sure what it's like in the states but here in the UK, recruiters are nothing more than glorified salespeople.

The problem also lies in the fact that many recruitment agents get paid a woeful base salary, the rest being topped up by commission payments based on successful candidates they pass on. So it's in their interest to ring as many people as possible and put forward as many people as they can because there's a good chance they'll get a "bite" somewhere down the line

</comment><comment><USER56> Everyone seems to be hating on recruiters -- I actually just found a near dream job by me actually getting in touch with one. I'm using the language and tools I love, and I'm rolling over into a full time position with awesome benefits. Yes they take a cut of your hourly wage -- but I quoted it so up there that its not that big of a deal anyways (don't undersell yourself).

</comment><comment><USER57> The comments here about how bad contingency recruiters are point to a lack of three skills:

\- listening \- industry knowledge \- politeness

Now, a new recruiter might not have much industry knowledge, but you can pick up listening skills and politeness (for any given culture) in a myriad of jobs. Why don't they have these valuable skills?

Recruiters aren't very good at recognizing people who would be good recruiters...

</comment><comment><USER58> It seems that the author is confusing placement agencies with recruiters working directly for a company (also see pg's comment).

I've had great experiences with experienced recruiters at a startup, to help us find candidates, while at the same time getting pounded by unethical, moronic agencies.

The recruiter helped us wade through hundreds of CV's, and eventually found an awesome Stanford CS MS that we ended up hiring. And this was while the recruiter worked for us as a contractor, so no 30% salary fee.

So a great recruiter is worth their weight in gold, because they can potentially find that perfectly fitting candidate, giving a boost to a fledgling startup. Unfortunately, such recruiters a far and few between.

Opposed to that, I got daily calls from agencies, that weaseled their way past our operators to talk to me directly, being super aggressive and downright rude when I politely told them to stop calling. This all, of course, because of the 30% fee salary fee when they place someone.

<comment><USER59> The problem is that placement agencies also call themselves recruiters.

</comment></comment><comment><USER60> I had an idea of making recruiters put their money where their mouths are. Want to represent me? That'll cost you $2000/year. If you find me a great job in the next few years you you get many times your money back.

<comment><USER61> Recruiters would not consider such an arrangement unless you're a celebrity of the industry with a stellar resume. The candidate pool is too large to spend time and effort trying to place a single candidate in a great job.

<comment><USER62> At least this might force the recruiter to admit what his true assessment of your chances are - if you don't call their bluff, they will forever tell you that you are a perfect fit, and that the job is the best ever.

</comment></comment></comment><comment><USER63> If it's a designer you're looking for, skip recruiters and check out my project: <http://folyo.me>

It'll cost far less than a recruiter, and I'll refund you if you don't find someone.

<comment><USER64> Are you using the same model as dibbble, or why are you not accepting signups?

<comment><USER63> It's not the same model, every application has to be accepted by me personally.

I've got enough designers (about 250) compared to the number of job offers currently on the site, so I've decided to close sign-ups for now.

</comment></comment></comment><comment><USER65> Perhaps we need a startup to disrupt the job recruiting industry?

</comment><comment><USER66> LinkedIn both relies on recruiters for revenue and reduces their necessity in the market. I would like to know how they think that will play out.

</comment><comment><USER67> cue Peroni...

<comment><USER68> Typical. The one day I chose to ignore HN in order to make a dent in my mountain of work and this article gets posted.

For what it's worth, I agree wholeheartedly with the majority of the article. Recruiters who focus on start-ups are wasting their own time as well as the start-up's time. Yes there are a huge number of start-ups hiring and yes some will engage recruiters but the vast majority don't have the budget.

I would happily wager a months salary on the fact that 99.9% of recruiters don't understand how a tech start-up works and even if they make the effort to learn, every start-up is different with their own individual challenges and quirks and it requires an immense amount of time and effort to truly understand what these companies need. No recruiter will invest that level of time. Instead they will throw a few CV's at the role and hope something sticks.

I'm giving a talk at Hacker News London tomorrow night and the topic is 'Recruiting for start-ups'. It's a brief talk on how to improve your chances of finding the right people for your start-up and how to keep them. At no point do I even attempt to suggest they engage a recruiter, in fact I strongly encourage them not to.

</comment></comment><comment><USER69> So where do you search for engineers if you're a startup?

<comment><USER70> Go to various tech meetups/meetings that seem relevant and then try to identify smart people by what they say in meetings. Then approach them.

Maybe sponsor a relevant meeting by buying the pizza/beer for it. This buys you the opportunity to give a company introduction and let people know you're hiring. Very cheap advertising.

meetup.com is probably the generic place to look for these types of meetings, but every city probably has its own, better site. In Seattle, among others, we have <http://www.seattletechcalendar.com/> and <http://www.seattle20.com/> among probably others.

Sitting in your office waiting for resumes/cover letters is the absolute worst way to try to get people. It also leaves you with only an interview to get to know the potential hire, while at the meeting you may find out all sorts of desirable or undesirable characteristics about them. Additionally you may get input from that person's peers.

<comment><USER34> Yes.

An onsite interview is 4-hour sell meeting: sell your coworkers on the candidate, and sell the candidate on the coworkers and office and etc.

If you aren't pretty confident in a hire before the onsite, the onsite won't help.</comment></comment></comment>


Artificial Intelligence Will Kill Capitalism - <AUTHOR> http://www.worldcrunch.com/opinion-analysis/artificial-intelligence-will-kill-capitalism/google-transhumanism-ai-facebook-ibm-apple-cyborg-robot/c7s17977/#.VMY744FHarU <comment><USER3> I don't think a future with AI is as easy as "robots will produce everything". There are two important factors usually being ignored in these discussions:

\- Human labour is not only about producing _stuff_ or advancing sciences, that's a narrow-minded, or even consumerist view of life. We are also creative, we like to appreciate beauty and like to be appreciated for things we create. Even when every thinkable and unthinkable _thing_ will be produced by the machines, we will be able to focus on arts. We will be creating unique things, re-distribute it among ourselves and inevitably some of us will prove to be better at that than the others. Appreciation on one hand and the show- off factor on the other are in our nature, I don't think they will ever become an atavism.

\- Scarcity of natural resources: something suggests our fight for (possession of) resources will continue until at least we have a technology that transforms any matter to anything, which has nothing or very little to do with the AI per se. But then there will be a problem of habitable space. We will fight for possessing more than the others, that's also part of our nature. More habitable planets, more powerful machines, more possibilities. Because those who possess less have smaller chances to survive.

So no, inequality and survivalism (in a broader sense) are not going away. And to satisfy our need to be better and to have more we'll have to maintain a socio-economic system that ensures inequality is always in place. It's going to be Capitalism 2.0 rather than Communism.

<comment><USER4> I think those factors are being ignored partly because not relevant to the "capitalism vs. technology discussion" and partly because they're not on capitalism's side.

\- RE creation. It's true that people like to create and be created, but as sad as it is, being "human-made" is hardly a quality you can run entire economics on. Say, I create a new work of art. How can I exchange it for food? I could sell it for a bunch of credits to someone who appreciates my work, and then use those credits to buy food made by machines. But machines don't need that money, they just make food and run on machine-controlled power generation, etc. They're not participating in the economy; paying for that food becomes just a weird kind of tax, at which point we'd be better off just giving the food away - eliminating the unneccessary theatre.

Being able to focus on creating beautiful and interesting things is the promise of automated future. I don't think anyone in their right mind could think that there will be no economy of anything at all - as long as you have people exchanging things, there's always one. It doesn't really matter much what kind of economy it is (though you could get a good guess by observing content creation and sharing dynamics on the Internet when no one is trying to monetize it) - what matters is that it won't be the same system you use for getting your basic needs (food, shelter) fulfilled.

\- RE scarcity. I see two problems with it. One that in western societies there's almost no visible scarcity at all. You look around and see shops overflowing with all sorts of dirt-cheap products. We're isolated from the actual scarcity by many layers of abstraction, and it isn't even reflected in the prices - otherwise gas or meat wouldn't be so cheap as they are right now.

Humans do have a drive for achieving more (or more than others), but our economic system is built completely around the concept of growth - to the point that we use up resources in an exponentially increasing rate and have to invent artificial scarcities just to keep the whole machine from falling apart on the edges (see the thread about jewlery currently on HN). Not to mention increasing numbers of bullshit jobs. That system needs to go, it's unsustainable.

Inequality and survivalism are not going away. Nor should they, really - <USER2> history showed us is that people don't react well when forced to be equal. The drive to get more will always be there, but it doesn't preclude us from raising the low point from "starving to death" to "comfortable, if minimalist, life". Basics can totally be Communism 2.0 and let sulprus be whatever develops, only moderated so that it doesn't go exponential.

<comment><USER3> But there will always be scarcity of something: materials, energy, land. Things will always have a price, because they at least require energy and space for production. It will be unbelievably cheap, but never absolute zero.

And then someone will own the land (along with what's underneath) and the machines themselves, unless of course we dump private property altogether. I don't see how this is substantially different from what we have now. It's just the machines will get better, living standards will be higher, there will be more bullshit jobs, and more time for creativity.

So essentially the discussion comes down to whether there will be private property rights in the future. As someone who grew up in the USSR, where nominally there was no private property per se, I can only say the humanity will need a very, very good reason to abandon it for good. It's almost like saying prohibit sex in all of its forms and purposes, i.e. going against the nature.

</comment></comment></comment><comment><USER5> "In our meritocratic societies, the difference in intellectual abilities are, rightly or wrongly, the primary reason for the wage and capital gap"

Which country do you live in? I want to live there...

<comment><USER6> The fact that people use the word meritocracy in complete seriousness always jars with me, as the word was coined in satire as a warning.

[http://www.theguardian.com/politics/2001/jun/29/comment](http://www.theguardian.com/politics/2001/jun/29/comment)

<comment><USER7> The word is separate from the political beliefs of the person who happened to come up with it (or popularize it) first.

<comment><USER8> Exactly. If you're going to complain about "meritocracy", just wait until you find out where "capitalism" originates!

<comment><USER6> I'm not likely to be that surprised. My dad used to be a Marxist.

</comment></comment></comment></comment></comment><comment><USER9> This probably will not be a popular opinion, but here it is anyway. I'm sure all the famous names mentioned in this article as prophets of the AI impending doom are exceptionally gifted entrepreneurs, technologists and business men (and I should add I hold some in the highest respect in terms of their vision and work), but that does not convince me they have any qualifications to predict where artificial intelligence will take us. To speak about if and when AI will become self-aware is nothing more than a guess. To speak about whether capitalism will end as a result is a guess upon a guess. As far as I'm concerned there's one sure way to tell if someone has the ability to predict the future at these levels: if they are human - they can't.

</comment><comment><USER10> _It will kill capitalism_

Who controls/owns the hardware that the AI runs on, and the AI itself? Who are these people giving away augments for free? Why would they not limit the capabilities of low-tier augments to ensure the very few at the top can still remain there?

Clearly you haven't played deus ex.

</comment><comment><USER11> There's a short story, Manna [1] by Marshall Brain, which illustrates two potential outcomes of an AI revolution.

In one scenario, the people who are out of work (i.e., most of them) are housed in prison-like welfare dorms. Most jobs are done by robots, so ending up in the dorms is basically a life sentence. They are "strongly discouraged" from even going too far from the dorm buildings. This scenario plays out in the United States.

In the other scenario, taking place in Australia, the AI/machines handle all of the infrastructure of the society, and due to the efficiency with which they do so, the human members live a leisurely existence. It's essentially a post-scarcity society in which energy is the only limiting factor, so everyone gets a certain amount of disposable credits per month with which they can do what they choose--be it create art, buy custom clothing, participate in entertainment, perform research, or donate it to someone whose work they enjoy or find to be worthwhile. It reminds me of the outcome hoped for by proponents of Basic Income, which is that letting everyone do what they are most interested in doing will result in a richer existence for all of humankind.

The author also explores interesting interpersonal and technological aspects whose degree of plausibility will depend on how much you buy into the Singularity, but my main point in posting this, besides hopefully introducing someone to a good read, is to say that AI does not represent some inevitable force that is bound to change the course of human existence in one particular way. AI itself and the environment in which it exists are things that are continually shaped by human actions both individual and collective. If an AI revolution is brewing, it is in its very early stages. I have no doubt that powerful interests are interested in remaining powerful, so those of us who pay attention to developments in the field should also pay attention to the uses they are put to, and to do all we can to guide them towards humanitarian ends instead of pure profitability.

[1] [http://marshallbrain.com/manna1.htm](http://marshallbrain.com/manna1.htm)

</comment><comment><USER8> The trend has been just the opposite: Computers display far more "intelligence" than ever before, if you accept the (admittedly somewhat glib) definition of AI as anything computers still can't do. Logistics, navigation, trading... Yet globally, despite a few shocks, poverty has been falling for decades. Essentially everywhere capitalism is practiced and computers with their existing labor-replacing-intelligence are increasingly integral, the standard of living has been rising steadily. What about more capable computers will reverse this trend?

</comment><comment><USER12> Good riddance! Better start working on it.. :-)

Seriously, we have problems with distribution of capital already. There are two mainstream theories of value:

First pretty much argues that value is subjective, so the distribution of capital, whatever it is, is never a problem. It's more of a cop-out than an attempt to describe the world.

The second is the (Marxist) labor theory of value, which argues that all the value comes from human work. However, it is unfortunately logically inconsistent.

In the reality probably lots of value already comes from machines (and biological systems), and we have lots of social conventions and pretending (calling it "meritocracy" etc.) around it concerning how to distribute this added value to humans. In reality I don't believe there are many people (maybe some homeless ones) in the western society who deserve their current living standard, for the labor they do.

I don't think that this sort of social construction is going to be eliminated by artificial intelligence. We will just see more of it, if anything.

</comment><comment><USER13> ..and cancer drugs will cure cancer. Unless they start charging $1000 a pill. Which they did.

</comment><comment><USER14> I think capitalism will kill artificial intelligence. Just look in boardrooms and classrooms and senates across the world and see what it's done to the real thing.

<comment><USER15> You mean politics.

<comment><USER16> No. They likely mean capitalism, as capitalism comes with politics and ideology.

Hard to generate an AI when bright young minds have their schooling, medical, public libraries, social welfare systems, to name a few, degraded beyond all repair to shave a few billion off of the balance books.

It is capitalism's endgame, much like a UFC match without a referee.

<comment><USER15> Ok, then you mean crony-capitalism: [https://en.wikipedia.org/wiki/Crony_capitalism](https://en.wikipedia.org/wiki/Crony_capitalism)

</comment></comment></comment></comment><comment><USER17> But [http://en.wikipedia.org/wiki/I,_Robot_%28film%29](http://en.wikipedia.org/wiki/I,_Robot_%28film%29) differs

</comment><comment><USER18> edge.org "2015 : WHAT DO YOU THINK ABOUT MACHINES THAT THINK?"

[http://edge.org/responses/what-do-you-think-about- machines-t...](http://edge.org/responses/what-do-you-think-about-machines- that-think)

</comment><comment><USER19> Oh good, kurzweil is on board. We should be safe for another three centuries or so :-)

</comment><comment><USER20> Should an AI be able to own its own hardware and have a company by itself?

How will this affect us?

<comment><USER6> Should stops being part of the discussion when you are no longer the adult. If it is considerably brighter than us and it wants these things, then it will get them.

</comment></comment><comment><AUTHOR> I submitted this story. This is a conversation I have with non-tech friends who don't yet grok just how much society is going to change in the next few decades because of technology.

<comment><USER21> And you have a single paragraph explaining anything at all as to why you think that will happen, and it's basically, "too much efficiency will ruin capitalism." I'm going to need more convincing than that.

<comment><USER4> Try this: [http://slatestarcodex.com/2014/07/30/meditations-on- moloch/](http://slatestarcodex.com/2014/07/30/meditations-on-moloch/). Long, but very much worth the time.

<comment><AUTHOR> Wow, quite a good rant on the state of the world! I only had time to read the first 1/3 but will enjoy the rest tonight. Thanks.

<comment><AUTHOR> edit: I finished it. Good read, and recommended.

</comment></comment></comment></comment></comment><comment><USER22> When the right control most of the media "We will have 20 hour work weeks!". When the left control most of the media "capitalism will destroy us all!".</comment>


You Can Do Any Kind of Atomic Read-Modify-Write Operation - <AUTHOR> http://preshing.com/20150402/you-can-do-any-kind-of-atomic-read-modify-write-operation/ <comment><USER3> When spinning on an atomic operation, you should add some kind of yield operation. The "pause" instruction in x86 at least. On newer CPUs a "monitor" and "mwait" can work a little better (but may not be available in user space). Do not use thread_yield or other kernel calls, because at that point you're better off using a mutex.

When spinning on atomics, the interconnect between cores needs to do a lot of work synchronizing the memory. When adding a pause or monitor instruction, the core will give an opportunity for the other hyperthread in the same core to execute, reducing the amount of churn on the interconnect.

Counterintuitively, adding a "pause" in the spin loop will make the program faster by yielding the CPU core for a few nanoseconds.

<comment><USER4> That's interesting. Any references about this? What about ARM CPUs?

<comment><USER3> See Intel Programmer's manual (vol 3a iirc) about pause, monitor and mwait. Not sure about ARM, that architecture has different rules of cache coherence and no hyperthreading and you need to add explicit memory barriers.

</comment></comment></comment><comment><USER5> It looks like the examples are broken.

      uint32_t fetch_multiply(std::atomic<uint32_t>& shared, uint32_t multiplier)
      {
          uint32_t oldValue = shared.load();
          while (!shared.compare_exchange_weak(oldValue, oldValue * multiplier))
          {
          }
          return oldValue;
      }

If shared.compare_exchange_weak() fails because of a concurrent writer, then the while loop will never exit (unless another writer sets the value of shared to oldValue).

What the author wants is something like this:

      uint32_t fetch_multiply(std::atomic<uint32_t>& shared, uint32_t multiplier)
      {
          while(1) {
              uint32_t oldValue = shared.load();
              if(shared.compare_exchange_weak(oldValue, oldValue * multiplier)) {
                  return oldValue;
              }
          }
      }

The other examples have the same problem.

<comment><USER6> I'm pretty sure that example is actually fine - if there's a concurrent writer, the modified value will be loaded into oldValue. Repeating shared.load is unnecessary since that operation occurs as part of compare_exchange_weak.

<comment><USER5> Thanks, my mistake.

I've been bit by this before. I sometimes wish you had to mark "reference-of" just like you have to mark "address-of" with &.

It's obvious that blah(&foo) might modify foo, without needing to examine the signature of blah().

The tradeoff is a bit of code clutter when you add a symbol (something like shared.compare_exchange_weak(%oldValue, oldValue * multiplier)), and a violation of DRY, since the information provided by this symbol would be redundant.

</comment></comment></comment><comment><USER7> I remember how I was surprised to find that you can implement compare-and-swap purely in software (with some caveats :) but still, IMO, preserving the idea). See, e.g. Peterson's algorithm. It does have a lock, but only for finite (and brief, if you will) moment, and, wrapped as procedure, is externally lock- free.

You still rely on ability to locally order operations in time (i.e., you have to be sure that you - the CPU - reads from that memory cell strictly after it completed writing into this one) - so there are restrictions for modern CPUs. You might have issues if you want to synchronize many threads/processes - <USER2> the locking delay may become too long. But still.</comment>


Crytek: Graphics Are "60% Of The Game" - <AUTHOR> http://www.x360magazine.com/general/crytek-graphics-are-60-of-the-game/

<comment><USER2> To claim that graphics don't matter to games would be like claiming cinematography doesn't matter to movies. On the other hand, making a generalization that graphics are "60%" of all games would be just as silly as making a percentage claim for movies.

That 60% of Crytek's development resources are spent on graphics seems plausible, and it's working for them. But that's just their games. FTL, Binding of Isaac, Minecraft, or lots of other smaller games are almost certainly not spending 60% of their time on graphics, and most of what they do spend is more likely art direction than technology (The quote in the article did include art direction in "graphics"). That works for them too. "The Game" is a very broad term if you take that statement at face value. We should take it as a statement only about Crytek's games.

<comment><USER3> > _To claim that graphics don't matter to games would be like claiming > cinematography doesn't matter to movies._

Cinematography (camera work) is not graphics though. It's as essential to cinema as the plot and acting is.

Graphics in relation to games not so much. You can make a great game with very simple graphics, even today (from Tetris and Pacman to Leterpress). You can even make a great game with no graphics at all (all the much loved text adventure games, MUDs, etc) -- whereas you cannot make a movie at all without any cinematography.

If you want to make a movie comparison, visual effects and/or fancy sets would be a better one.

<comment><USER4> Tetris requires a properly sized view and clear, effortlessly distinguishable artwork.

MUDs require a quality typeface, proper layout, and appropriate coloring.

Pacman would be nothing without its remarkable low-fi charm.

Letterpress wouldn't have gathered a moment's attention if the tiles weren't immaculate and well rendered.

Graphical quality matters in everything, everything, everything visual. And it's a reasonably reliable proxy for how much effort has been put into the rest of the game.

<comment><USER3> > _Graphical quality matters in everything, everything, everything visual. And > it's a reasonably reliable proxy for how much effort has been put into the > rest of the game._

You sidestepped the whole "text only adventure games" thing with the "visual" word. We were talking about computer games in general.

As for all the above examples, I don't think they show at all how graphics matter. They show that great addictive gameplay trumps flashy graphics any day of the week.

I mean "properly sized view and clear, effortlessly distinguishable artwork"?

Those goes without saying. Of course graphics should not obscure the game's goals.

That doesn't mean that the graphics are "60% of Tetris" -- in the same way the fact that Tetris would be totally unplayable if the background was all black and the tiles dark gray --doesn't mean that graphics are "100% of Tetris".

As for "MUDs require a quality typeface, proper layout, and appropriate coloring", yet people have played them in bad typefaces --heck, even not typeface at all, just bitmap graphic card fonts-- and black and white. Matter of fact, in the eighties lots of people on DOS PCs played color games in black and white monitors, and enjoyed them too.

<comment><USER5> > Of course graphics should not obscure the game's goals.

Indeed, you might say in that case the graphics are of high quality! Low quality graphics can easily detract from gameplay.

I think there are two definitions for "graphics quality" commonly used.

\- Realism, as used in this article.

\- Aesthetics, an example of which would be Team Fortress 2. By any definition it is not a very realistic game, but it still has very well-made/high quality graphics (before hats, anyway.)

</comment></comment></comment></comment></comment><comment><USER6> In my opinion, far too much time is wasted these days discussing which elements of game design are most "important".

For Crysis, the graphics are absolutely the focus. For a game like Minecraft, graphics take a backseat to the innovative game mechanics. For Team Fortress 2, game mechanics and graphics both take a backseat to unique team-based multiplayer.

Games are a type of art, and art can take many forms. Are Michelangelo's sculptures "worse" than Picaso's masterpieces simply because they are far more life-like? Are modern movies worse than early 1900s films simply because they use high end special effects?

Take a step back and judge a game by its own merits - any individual element of a game's design is dependant on it's genre, art direction, platform, and specific goals. The question should not be "are graphics the most important part of video games?", but rather "are graphics the most important part of THIS game?".

</comment><comment><USER7> First thought: "Necessary but not sufficient"

Second thought: "Minecraft rather undermines the 'necessary' part."

Flippancy aside - there is a place for incredible graphics and I'd be the last person to want progress to halt in that direction but many games wade through the uncanny valley and the immersiveness(?) suffers. Others refuse to even play that game and seem no poorer because of it.

I rate Doom (original), Journey on the PS3 and of course Minecraft as amongst the most captivating gaming experiences I've had. Food for thought.

<comment><USER8> Minecraft doesn't really undermine it. Minecraft's inspiration was Dwarf Fortress, which is an ANSI game. Minecraft's most significant contribution which made the game feasible was the addition of graphics. All the non-graphic 'cool elements' of Minecraft are found in abundance in Dwarf Fortress, (and then some (and then some)). Graphics made that style of game accessible to the masses.

<comment><USER9> I thought Minecraft's inspiration was Infiniminer?

<comment><USER10> More or less.

When creative mode was made, the intention was emulate infiniminer and grab the fans of the abandoned game, but the survival mode, that is mostly a copy of dwarf fortress, was planned from start.

Minecraft is dwarf fortress with infiniminer interface.

</comment></comment></comment></comment><comment><USER11> Good thing there are still developers seeing things differently. I'll take story and mechanics over graphics every day.

<comment><USER12> Good thing it's not a zero sum game. I'll take graphics/story/mechanics anyday. And graphics don't necessarily need to be intensive, it could be chalked up to creative art direction, like Heroes of Might and Magic 3 or Team Fortress 2.

<comment><USER13> "Good thing it's not a zero sum game."

Unfortunately, costs for creating content of high graphical fidelity and for reworking it and whatnot very much mean that it is not a zero sum game.

You cannot take story and mechanical risks if you spend millions and millions on the engine licensing, art talent, and man hours required to perfectly render every shade of grass.

This is, incidentally, what is driving the AAA bubble and the slow implosion of studios we see today.

</comment></comment></comment><comment><USER14> "And immersion is effectively the number one thing we can use to help you buy into the world."

I agree wholeheartedly with this statement. However, the devil is in the details. There are a plethora of ways to ruin immersion in video games, and the list is different for each player.

One of my personal immersion breakers is the appearance of the characters. As someone with brown skin, it gets hard to immerse myself in a game where the main acting character looks and sounds so different than I would imagine them myself. And in the rare event that a game has a minority main character, it's common for them to be laden with negative stereotypes.

The reason I can become immersed in a _book_ has a great deal to do with the freedom to envision characters a certain way. For _games_, this counts even more because we are often asked assume control of a character's actions, but have little control over much else.

For the majority of male players, this might not be an issue. But for females and minorities, this is so common that we really have to do some mental jujitsu just to get past it to the point of true immersion.

I think this phenomenon also explains why basketball and football games have a much higher amount of minority gamers than, say, adventure games and RPGs.

This isn't just an issue in video games-- Hollywood mirrors this to a degree. It's why a movie with all black characters (or a black lead) routinely draws large black audiences, often regardless of the quality of the movie.

One of the things I liked most about playing Crysis 3 was being able to identify with the main character more-so than in most FPS games.

Sometimes people want to be able to see themselves in the heroes, the side- kicks, the minions, and even the villains.

</comment><comment><USER15> Pretty sensationalist title here. The quote literally says graphics + physics engine + sound design + assets + production values + art direction = 60%.

Given the reality of the statement, it seems very acceptable. Graphics are only a small piece of the puzzle. Adding in realistic physics and immersive sound scores add a ton to a game. Leaving that out in the title is pretty weak journalism imho.

</comment><comment><USER16> In any recent Crytek game, they're 100% of the game.

<comment><USER13> The AI and physics in Far Cry (the first one) were actually pretty legit.

</comment></comment><comment><USER17> Depends on how you define "graphics".

To me, graphics constitutes aesthetics, and usability. Good art direction combined with great user interface = 60% of a good game.

If in this article, they're talking about as many pixels, particles and polygons as possible, it's a bit of a rough statement; it might be true for _their_ games, but not for _all_ games.

</comment><comment><USER18> The "percent" of X that makes up a game is purely in the eye of the beholder. For some people graphics has a huge part in their like or dislike of a game. Conversely there are plenty of games that go out of their way to have retro graphics specifically to evoke a certain nostalgia or just to shift the focus to the story and the gameplay.

</comment><comment><USER19> In my opinion _animation_ is more important than just "graphics". In my experience a well motion-capped game is easily more enjoyable and immersive than a traditionally animated one. Of course this doesn't apply everywhere, but I think fluid life-like animation goes a long way regardless of the graphics that package it.

</comment><comment><USER20> As it happens I wrote something that addresses this type of statement, just today.

[http://spacecreeps.com/essays/when_a_game_means_something/in...](http://spacecreeps.com/essays/when_a_game_means_something/index.html)

</comment><comment><USER21> _Graphics Are "60% Of The Game"_

It pretty much depends on the genre. For some hack-n-slash or RTS that may be true, but let's say for a good RPG - the depth of the story outweighs the graphics factor.

</comment><comment><USER22> Seems like sound design is nearly half of the game. Art direction is another big chunk. Playability is another big chunk. How technically advanced the graphics are fits into the remainder.

</comment><comment><USER23> Gameplay is 100%

A game without gameplay is a 3d demo.

<comment><USER24> What's a game with 100% gameplay? A design doc? These percentages are silly.

<comment><USER23> dodgeball

</comment></comment></comment><comment><USER25> They've obviously never played Zork.

</comment><comment><USER26> Depends on what you mean by graphics and what the game in question is.

If you mean the graphics appropriate to the game in question, then sure (whether the 60% was just a silly rounding above half because Yerli is graphics biased, who knows).

Minecraft has graphics, and they added substantially to the aesthetic style of the game, helping to make it stand out. It obviously doesn't have Crysis level graphics however, and would have failed if that had been attempted.

Tetris, as a classic example, doesn't need high powered graphics, but its style of graphics are still a substantial part the game, as necessary for the basic gameplay.

</comment><comment><USER27> Yeah, Crytek makes it a point of pimping their engines quite a bit. The problem is that they're not comparing their engine to Minecraft, but to Unreal 4, Frostbite 3, Id Tech whatever, etc..

Is someone going to choose to play Crysis 3 because the graphics are a bit better than say the Frostbite 2 engine in Battlefield 3, or whatever Call of Duty is using? I don't think so.

I do applaud Crytek for being pretty developer friendly with their Crytek SDK, but maybe they should look at how Dayz really helped out Bohemia and release a game SDK too.

I'm not sure what engine Far Cry 3 is using, but I'm sure some developers could create some real interesting mods with an SDK.

<comment><USER28> People _do_ chose to play Crysis 3 over other AAA titles because of its graphics. That's literally the most-used reason for buying it.</comment></comment>