Launch HN: Searchlight (YC W19) – Hiring based on past performance, not resumes - <AUTHOR> Hi HN community! We're Anna and Kerry, co-founders of Searchlight (https://www.searchlight.ai). Our software helps candidates be judged by their past performance rather than their resume or where they went to school.
We built this product to help job candidates and hiring managers. With platforms like Linkedin and Indeed, hundreds of applicants with indistinguishable resumes apply for the same job with just one click. Kerry and I both have backgrounds in software engineering, and we were frustrated by how time-strapped hiring managers increasingly over-index on the "snob test" (a.k.a. where the candidate went to school) or contrived technical screens [1][2]. We're also twin sisters who went to the same school and worked at the same companies. We look indistinguishable on paper, so we are especially keen to bring a new product to the hiring space that will allow candidates to express their individuality beyond their resumes. When we looked at the landscape of current hiring tools, we realized that the majority of them are self-promotional (resumes, personal websites, Linkedin, etc) and difficult to substantiate at first glance. This disadvantages people who aren't good at promoting themselves, or don't like to, and these are often the best candidates! We saw that a poorly conducted technical screen can penalize the most talented engineers. Worse yet, we learned that take-home coding challenges are a real pain point for certain demographics, like parents who don't have the time to thoroughly attack a 24 hour coding challenge because they have to take care of their kids.
This made us think - why are we ignoring the the perspectives of people who actually know what it's like to work with a candidate? This data is the most indicative of success on the job [3][4], but isn't currently being leveraged until the end of the process, if the employer conducts reference checks. This is why we built Searchlight to better assess candidates early in the hiring process. Currently, we work directly with employers to invite their applicants to the platform. Job seekers can invite as many advocates as they want to speak to their accomplishments and capabilities (some invite as many as 10). The references share feedback like specific examples of how the candidate demonstrated desired competencies and how future managers can set the candidate up for success. Then, we analyze this feedback to assess candidate-position compatibility by matching the requirements of the role to the candidate's strengths. Our recommendations for strong candidates are based on a mix of quantitative factors like average ratings of core competencies, and qualitative factors like work style and environmental fit (which we currently human QA). One of our core beliefs is that every candidate is exceptional in their ideal environment, so all the feedback gathered on Searchlight - regardless of whether the candidate gets an offer - is saved and available for the candidate to use and share.
We aim to make the hiring process more fair. We are building trust and legitimacy into our platform by tying each reference to a specific job experience, verifying references through work emails or Linkedin profiles, and keeping the feedback hidden from candidates. While no tool is perfect, we know that the insights surfaced by Searchlight allow for better decision-making than traditional resume scans, with no extra time commitment for employers. We are especially excited to see that Searchlight is already helping diverse applicants get to the on-site interview stage after being initially screened out.
We'd love to hear about your experiences in today's hiring process and if Searchlight would be helpful to you! Thanks for reading.
[1] https://news.ycombinator.com/item?id=15688972
[2] https://news.ycombinator.com/item?id=2175147
[3] https://dornsife.usc.edu/assets/sites/208/docs/Ouellette.Woo...
[4] https://www.linkedin.com/pulse/how-predict-on-the-job-perfor... <comment><USER3> > Gather true, unbiased insights from those who already know what it's like to > work with your candidate

How can you be so sure they're unbiased? I would worry a lot that this will just favor candidates who are more amiable and have less honest friends.

<comment><AUTHOR> Thanks for the question (and visiting our landing page)! You're right that opinions always carry the perspective of the person giving them. We try to minimize the bias by:

1) Gathering a larger number of perspectives

2) Asking gold-standard questions that get to the specifics and contextualized examples

3) Every referencer is tied to a specific work experience and provides their contact information, which helps hold referencers accountable to their feedback.

<comment><USER4> > gold-standard questions

Does this not just mean trendy? Can you provide an example?

<comment><AUTHOR> One question that we ask is: "We are all learning every day. What was an area of development for this candidate back when you worked together?" An example of a less open-ended question is asking the reference to stack rank strengths that the employer wants to learn more about.

</comment></comment></comment></comment><comment><USER5> Unsure if this actually makes the process more fair. A lot of job seekers probably don't want all of their history carried with them. A previous employer that could've been a bad fit could ruin your future if you don't have a clean slate.

<comment><AUTHOR> Being a candidate-centric platform, we give candidates the ability to choose who advocates on their behalf. We also do not require that candidates share references from every experience, knowing that sometimes candidates are uncomfortable with their current employer knowing that they are job-searching.

We also understand that everyone's ideal work environment is different. We ask a specific question related to which environments and work styles can set a particular candidate up for success.

</comment></comment><comment><USER6> Recently went through a process where references were needed. One of my references shared what was asked, and one of those was "what is person X's biggest weakness". There was an indication that submissions _without_ answering that question would be discounted entirely.

I looked at your sample report, and it seems very much like an expanded "linkedin", where it's just about the positives. Knowing - honestly - <USER2> someone is weak will help avoid bad fits.

I've been a reference for a couple of folks recently, and I always ask the person calling what the style of the role in question is. Person X might be a great person and _capable_ of XYZ, but only with support/oversight from a more senior person. If the role is expecting them to be self sufficient, and I know they need oversight, I'd try to push that. Me just saying "oh yeah, they're great", or only talking about the positives doesn't really help as much as a full set of questions which also include honest downsides/weaknesses.

<comment><USER7> Here in the UK, while it's not actually illegal to give someone a bad reference, no employer does because an inaccurate reference is grounds for legal action. You have to be able to provide evidence for everything you say. Consequently lots of employers only provide the dates you worked from and to and what your final salary was.

The fallout from someone not getting hired because the "honest downsides" were a bit _too_ honest (or not honest enough) would not be pretty.

<comment><USER8> This is true everywhere in the sense that there's absolutely no upside to giving someone a bad references. About all you can do for someone terrible is confirm factual data and refuse to comment on any opinion based questions. The requester can't even interpret that as a negative endorsement because lots of companies only do the same thing for good candidates.

</comment></comment></comment><comment><USER9> Slippery slope here. Why precisely is past performance indicative of future performance beyond some very weak correlations? Analogously, is a person's last relationship truly indicative of their future mating potential?

I understand that /some/ people believe it to be so, but I believe it's wrong to move society towards this. We should be going in the opposite direction- that you are as good as what you are aspiring to and putting in the work to become tomorrow.

<comment><USER10> I think past performance of a human is generally pretty strong as an indicator.

It's rare (though possible) for someone to turn from a lazy, self-entitled dilettante into a hard-working ace coder. It's rare for someone to turn from an excellent coach and mentor into an insufferable bastard.

<comment><USER11> > It's rare (though possible) for someone to turn from a lazy, self-entitled > dilettante into a hard-working ace coder.

Is it though? In my experience people are good at building things that they care about and bad at building things they don't care about.

If someone has been trapped working on sisyphean bullshit projects I would expect them to do much better on something they cared about, and the initial sorting of people into what they work on is often largely indiscriminate when it comes to what they care about, so I would expect there to be a significant percentage of people underperforming at their current job who would excel elsewhere.

I know at least personally, my job performance has fluctuated wildly depending on whether or not I cared about what I was supposed to do.

If you managed to incentivize me enough to get me to leave what I'm doing well now to go do something that I thought was rent seeking and deeply meaningless, I would most likely perform very poorly, and I've already had the opposite happen where I performed mediocrely in a boring area and then performed incomparably better when I got into doing things I cared about.

</comment></comment></comment><comment><USER12> I said this as a reply, but I thought it warranted repeating.

I would never give negative feedback about a former employee to anyone ever. I have no guarantee that that information is private, and they could sue me for defamation or a whole host of other things if they found out, especially if it ended up causing them not to get a job. That's real damages.

I know I'm not the only one who feels this way. In fact every hiring manager I've ever talked to about this has said the same thing.

I think they only way I'd feel comfortable giving negative feedback is if you insured me against lawsuits due to my feedback, to the tune of many millions of dollars. And even though I'm not so sure I'd do it.

<comment><AUTHOR> I hear your points. The feedback on Searchlight is kept private to the candidate, and only shared with employers the candidate has chosen.

<comment><USER12> Ah ok maybe I didn't understand the process. So all my feedback goes to the candidate first? And they only choose to forward that on after they've reviewed it? In that case I'd probably be comfortable giving them the negative feedback.

As a consumer of your service, I'd be a little worried about the client only showing me the good stuff. But I guess if the questions are complex enough it would be hard to filter for only good stuff.

<comment><USER13> I don't think that's what they mean. Its hidden from the candidate and the candidate doesn't know whats in it - except from the result of that application. but only the candidate can decide which employers to show it to. Weird why I might want to show or not show references when I don't know what's in it.

</comment></comment></comment></comment><comment><USER14> Since we are talking about hiring and since this is a YC startup, having seeing other YC startups in this general area, I would caution people regarding using these services.

1)If employer 'X' invites me to platform 'Y', I should not have to accept 'Y' privacy policy. It is between 'X' and 'Y' and I do not have much choice except rejecting it outright.

2) Just because 'Y' has my email address does not mean, 'Y' can start sending me marketing mail (Hello Hacker rank?). I don't need your tips, suggestions and all that crap.

3) Don't use forums like triplebyte. You are better off talking to individual companies yourself. Triplebyte is basically a funnel to make YC companies' hiring more efficient. Lets say you failed triplebyte's interview/test and they share this info with company 'Y'. What will you do if 'Y' chooses to never call you for an interview again? Your potential companies list is now smaller, and worse you also don't know why!

4) Too many recruiting emails now track you through open rates. It is ok if Twilio sends me an email asking me for an exploratory call, but if they send my name, email, actively searching status to their "vendor" named eightfold.ai or searchlight.ai or to some random website called zen.sr, Its unacceptable. What if eightfold.ai or searchlight.ai also sells a product called "predict attrition". I don't want to be in trouble.

<comment><USER15> That's not an accurate description of Triplebyte, which isn't affiliated with YC beyond being funded by YC, just like numerous other startups trying to fix hiring have been. Presumably they work with tons of non-YC companies since they'd be dumb not to. (Also, I'd be shocked if they did the kind of data sharing you're talking about, because it sounds like a bad thing to do and also a dumb one). You might be thinking of YC's Work at a Startup program? That is a bit closer to what you describe, though hopefully not bad (or dumb) either!

<comment><USER16> Triplebyte is clean. I work at a non-YC and they don't share candidates who failed tests. We literally don't see them. And I prefer it this way. The other one would be a waste of time.

</comment></comment></comment><comment><USER17> I'd love to see something like this for abusive bosses. I had a short stint with a manager who yelled at the team, said inappropriate things about my girlfriend, called at all hours, embarrassed teammates to the point of tears etc. I left early into the job, but 2 years later I still get emails from the team about fruitless stories trying to get HR to do something.

There should be a yelp for bad bosses so you can avoid these situations.

</comment><comment><USER18> I see your wall of text, and raise you a wall of text:

I've started looking for a new job recently. I clicked the button on LinkedIn saying recruiters could contact me and have gotten over 100 incoming messages since Christmas. I'm not sure which buzzwords I have that are causing it. Many are junk, but not all. They all want a 'quick chat' to proceed. There are probably 20 I would talk to but getting started scheduling phone calls is going to be a huge pain given I have a software job that I do all day. Just responding to the 100 messages as "no thanks" or "let's schedule something" is going to take a day. I outsourced updating my resume and website, and am trying to get Hubspot working to set up a sales funnel (but that works horribly with linkedin email redirects, so not sure that's going to stick). I'm trying to get organized and would even pay for someone to actually represent me and help with logistics, but the 3rd party recruiting industry is built around pleasing companies for money, never the candidate.

So I'm not sure recruiters and hiring managers need more coddling and new tools to target candidates. They already have the candidates out-gunned by a few orders of magnitude. And they have resources (and their own time) to deal with their pipeline. I don't. Maybe they could spend a little time and effort differentiating themselves? Cuz these job ads all look the same, and they all think they're doing me a favor for allowing me to be considered for an interview. If they are getting a lot of resumes that look the same, maybe that's a signal about themselves and not about the overall population of developers looking for jobs?

And as for getting my past employers to provide recommendations... They are all hiring. It seems crass to ask them to write me a recommendation while making it clear I don't want my old job back. The time to ask for the recommendation would have been when I was working there.

<comment><USER19> I think if you go to peers who you trust from old roles there should be no awkwardness in asking for a rec. However, I totally empathize with every company seeming similar when you're an applicant. It's hard to get a sense of the intangibles of a workplace until you actually work there. That's something I'd like to see a startup tackle

<comment><USER20> Glassdoor?

</comment></comment></comment><comment><USER21> > This made us think - why are we ignoring the the perspectives of people who > actually know what it's like to work with a candidate?

Because saying negative things about a past employee can cause you to become liable for libel or slander.

<comment><AUTHOR> The legal aspect is another interesting part of this problem. We have reviewed the legal statutes by state and our terms and conditions make sure our references comply with the law. We require that our references give their feedback without malice and to the best of their knowledge.

<comment><USER12> That won't stop a former employee from suing me though. I might win the case, but I'd still have to pay a fortune to defend myself.

<comment><USER21> Yeah, my business law professor told the entire class to just never give references period. There is no gain, and everything to lose.

Even saying positive things like 'Employee X was fantastic; she was really good at Y' can be grounds for libel. The employee will claim standing based on the fact that, by saying she was good at 'Y' and not 'Z' (which is what the employer really wanted), it was libel by omission.

</comment></comment></comment></comment><comment><USER22> In my opinion, the only two entities that should be involved in the process is the employer and the candidate. Adding more human elements will I think increase potential for abuse (favoritism) and in my opinion single out more extroverted people who have more references and whom they know will say positive things about them. Contrived example; extremely introverted developer who cant make friends and would never ask for references due to various reasons (shyness, anxiety, apathy). He will not benefit from this because he has no references to begin with even if he is a brilliant engineer.

<comment><AUTHOR> Thank you for sharing your thoughts. It's hard to build a product that suits everyone, but we try to be as inclusive as possible. In your example, the introverted engineer probably has standard professional relationships with a manager and/or a direct report that he can invite to speak to his capabilities.

I hear your opinion that a hiring process should stay between the employer and the candidate. Unfortunately, the reality is that most hiring processes today depend on a more human element, like a referral. We hope to make it possible for people without this access can also gather support around their candidacy.

<comment><USER23> Honestly this product discriminates against tons of people and would make the hiring process much worse. I hope it doesn't succeed as I believe this product would make the world a worse place.

I have an anxiety condition, this would be awful for me, for others with anxiety conditions, for introverts, for people with families, for people who care more about work than about networking.

I disagree that most hiring depends on referrals. I recently did a job search and didn't use any referrals. I have anxiety about posting an anonymous comment on a board to a stranger I have never met and don't care anything about. I can't just invite my manager to speak about my capabilities. Also what if my manager was toxic?

</comment></comment></comment><comment><USER24> Replying again because this question occurred to me: At what point in the hiring process do you expect these references to be used? If I'm a hiring manager getting ~200 applicants per opening, there's no way I'm reading ~2,000 of these references individually. But if I'm only requesting references for candidates who've passed the resume screening process, the bias in that part of the hiring process will remain. Will you provide tools for hiring managers to use NLP, etc., to allow for automated/aggregated reference-based screening?

<comment><AUTHOR> Thanks for the question. Our process can plug in during the resume screen or after an initial screen. Once the references are completed, we provide high level summaries and recommendations for which candidates have the strongest fit for the position. This gives the hiring manager better data while still saving time. The full references are available if the hiring manager wants to dive deeper.

These recommendations are based on on both quantitative ratings and open-ended questions like working style and ideal environment. We do use tools to help automate the recommendation calculation, but also currently human QA because we want to get these recommendations right when we're early-stage.

</comment></comment><comment><USER25> I can find 10 people to say nearly anything I want them to on my behalf. Whatever the job requires, they will say I have it. Whatever competency is required, I nail it. Whatever cultural fit is required, I blend into it.

Taking my own example, the signal to noise would be difficult for an employer to separate from my own (equally glowing) LinkedIn profile.

<comment><AUTHOR> The references submitted to Searchlight are saved and available for the candidate to use and share, and referencers are not asked to speak to a particular culture. This means that the references don't change per position, which makes sense because a candidate's top strengths/personality shouldn't change for every potential employer.

</comment></comment><comment><USER26> Best of luck! Just went through a lot of interviews with a number of engineering companies before finally settling on an offer, and can strongly agree a more human approach is sorely needed. I get that there is a strong fear of hiring under-qualified candidates, but the true cost is a loss of perspective on long-term hiring.

<comment><AUTHOR> Thanks for sharing your experience. We've found the fear of hiring under- qualified candidates is one of the primary reasons why hiring managers screen out candidates without recognizable line items on their resume.

By re-humanizing this process and getting better data around a candidate, we hope that hiring managers will qualify more diverse candidates and think more long-term.

</comment></comment><comment><USER27> I have nothing useful to add, though I do have two things to praise.

First, your write up is excellent and you have a good story. I genuinely wish you the best.

And second, the way you have been responding to feedback here is absolutely incredible. I'm both humbled and inspired by it. Great work!

Keep kicking ass and best of luck!!

<comment><AUTHOR> Thank you!!!

</comment></comment><comment><USER28> Isn't this like trying to solve the fake review problem, but for the hiring community? What differentiators does Searchlight bring to the table that would help it solve this problem, where countless bigger companies have failed?

I don't see anything in your literature that gives me confidence that your offering will be a game-changer.

Language like the following doesn't help:

> Our recommendations for strong candidates are based on a mix of quantitative > factors like average ratings of core competencies, and qualitative factors > like work style and environmental fit

<comment><AUTHOR> Validating the references are a top priority for us. By anchoring the reference to a job context and verifying the referencer through work emails or Linkedin profiles, we hope to address most of the fake review problems. This is a feature that we will develop further as we grow.

</comment></comment><comment><USER29> I think it's great you are doing something different. What I will say is that for many jobs or contracts I have had, the performance report given in the middle would be extremely positive, but then at the end when we had a disagreement or I had decided there was a better opportunity, it would be negative because I was giving them extra work to replace me. I am sure I am not the only one.

So I think that you may have trouble getting really objective evaluations of performance.

<comment><AUTHOR> Thanks for sharing your experience. That will help us with product development as we think about how to support other use cases when people request references, even when they're not actively job-seeking.

</comment></comment><comment><USER30> "We'd love to hear about your experiences in today's hiring process and if Searchlight would be helpful to you! Thanks for reading."

This isn't a "hair on fire" problem for me. Perhaps your report would nudge me towards interviewing 1-in-5 candidates who I would have otherwise rejected at CV screen. If I want to judge past performance, I'll interview them.

Good luck!

<comment><AUTHOR> Thanks for the feedback! We find that Searchlight is most helpful for hiring managers who are time-strapped and have high inbound applicants. We realize that this may lean more towards non-technical, early to mid-level roles more so than engineers.

</comment></comment><comment><USER31> I'm the type of candidate you describe, I think. I'm concerned. Does the average hiring manager wants a good candidate or a candidate who makes him look good? The way I see it, it's mostly the former, even with people I respect for their honesty. How can you make a hiring manager want a good candidate -- assuming he unconsciously think this is not the best option? I have a partial answer: every single one of my managers got promoted, mostly because of my results. In the long term, good employees are obviously better for everybody but the fool doesn't know it. Which makes me think that, when they got promoted, that's when I want them to talk about me and capture that in your thing.

A manager promotion, maybe, is a key moment. Because at that moment, some will think twice before forgetting the guys who got them promoted. At that moment, I wish I could have got to them with your form.

<comment><AUTHOR> Thanks for sharing! We think more about capturing reference feedback outside of the active job-seeking context.

</comment></comment><comment><USER32> This is very cool, shared with my team. I am worried about bias in the results questionnaire - both biased against and biased towards the candidate. How do you plan to avoid extremely damaging and false information from getting in? One bad review is now a black spot on a candidate. Dipping into black-mirror rating territory...

<comment><USER33> From what I read you invite people to comment, so you would only invite people who would speak to your strengths. This still leaves positive bias but that's inherent in reference checks because you choose the reference.

<comment><USER10> You would imagine this to be the case, but I've definitely been asked to serve as a reference by people who should have no rational reason to think I'd be able to speak positively about their employment. In those situations, I've always declined, indicating I was not comfortable and suggested that they use HR instead to validate their dates of employment here.

<comment><AUTHOR> Thanks for sharing your approach. We give references on Searchlight the option to decline for this reason.

</comment></comment></comment></comment><comment><USER34> Last week I had an epiphany about why I do so bad in interviews. Lately I have been getting into playing holdem poker online, and I decided to go to a live casino and see how that would feel.

While there I was as stressed as I am during an interview. When I got into a hand and one of the players raised my bet, I found myself unable to do the basic arithmetic operation of subtracting my initial bet from his raise to figure out how much more I should add to the pot. Even when I was told the amount from the dealer, I was still unable to calculate how many chips that would be.

The feeling was exactly like the one I have when interviewing, my brain is so stunned that is unable to do any thinking at all.

</comment><comment><USER35> How do you plan on handling "voting rings" where people all vouch for each other?

<comment><AUTHOR> We're aware of this possibility, and we flag it to the candidates and the employers.

</comment></comment><comment><USER36> The problem with systems like this is that your success rate is based on quantity of data, and not quality. The more data you have on a candidate, the more likely that candidate will move forward. This shifts the focus from knowing the job, to polishing your profile.

Simply put, your "performance gauge" as a programmer is directly correlated to your knowledge and expertise, NOT to how well you are at updating your profile and recommendations.

To me this is primary problem of hiring websites at the moment. It creates a sort of game that you can then get better at (but which I don't care for, and unfortunately creating a disadvantage for myself).

<comment><AUTHOR> From the references that have come through our system, we've seen that quantity doesn't outweigh quality. We'll make sure that this stays true as we grow. Our goal is to elevate candidates who aren't self-promotional through the feedback collected on Searchlight.

</comment></comment><comment><USER37> > _This disadvantages people who aren 't good at promoting themselves, or > don't like to, and these are often the best candidates!_

...

> _Job-seekers can invite as many advocates as they want to speak to their > accomplishments and capabilities (some invite as many as 10!). The > references share feedback like specific examples of how the candidate > demonstrated desired competencies and how future managers can set the > candidate up for success._

Why do you think folks who aren't good at self-promotion would be adept at inviting advocates to speak to their achievements? Moreover, LinkedIn has a 'recommendations' feature to cover this.

<comment><USER38> Giving references is an established part of the hiring process, so we believe that folks who don't want to promote themselves in other ways will be comfortable giving their references to their prospective employers earlier. Reference feedback collected on Searchlight is not public.

<comment><USER37> Isn't that biased too? You always have some people who think you're good at your job and some people who think you're not. A job-seeker would only seek recommendations from people s/he believes think highly of them.

</comment></comment></comment><comment><USER24> I don't use LinkedIn, but I know it has a feature called "endorsements" that does materially the same thing as this, and my understanding is that those endorsements aren't taken all that seriously. What makes these more credible?

Also, looking at your landing page, it looks like this is company-driven, not candidate driven (that is, I send my resume to the company, which sends it to Searchlight). If I'm in the early stages of the interview process for a company, and they request that I invite a bunch of former coworkers to endorse me on a platform I've barely heard of, there's basically 0 chance that I'll be continuing the application process.

<comment><USER10> > those endorsements aren't taken all that seriously

I've only ever heard them mentioned in a work situation in the context of endorsing people for bogus things (like a hard-core C/C++ programmer for PHP and Visual Basic).

<comment><USER12> You can get endorsed for murder. It's supposed to be for lawyers who specialize in murder cases, but the tag just says "murder". I currently have three endorsements for murder.

<comment><USER39> Now I have to get this.

</comment></comment></comment></comment><comment><USER40> Many employers, including mine, have "no reference" policies in place (its a legal thing). It's this an issue you've encountered?

<comment><AUTHOR> We haven't run into many causes where a referencer's employer has a "no reference" policy. Our terms and conditions has language that the reference's opinion does not reflect their employer, but we will continue thinking about this as we develop our product.

</comment></comment><comment><USER41> In 1016 I worked for [https://workgrades.com](https://workgrades.com). This sounds pretty similar.

<comment><USER42> Did you go to work with your horse in 1016?

PS: I know that I'm probably getting downvoted for this, but sometimes we also need to have fun on here, no?

</comment></comment><comment><USER43> Can't go back from: [https://www.searchlight.ai/candidate/login](https://www.searchlight.ai/candidate/login) by pressing the back button, redirects me back to the same page.

<comment><AUTHOR> Sorry to hear about the trouble! We'll address that soon.

</comment></comment><comment><USER44> Who "owns" my recommendations? In other words, once someone gives me a recommendation can I send it to whomever whenever I want or does a company have to pay to see it?

<comment><AUTHOR> Candidates can invite other employers to view their Searchlight recommendations. We haven't yet decided on the revenue model for this use case.

<comment><USER44> There's probably mutually exclusive options here. Either I can send my recommendations to anyone or recommendations stay private. If I can send them to anyone, I can send them to myself and read the report.

</comment></comment></comment><comment><USER45> Is it just me or is the site not loading for anyone else?

</comment><comment><USER46> This sounds like a recipe for being sued. You're maintaining a record based on potential libel and slander.

</comment><comment><USER47> The "sample" button at the bottom doesn't work.

<comment><AUTHOR> Sorry to hear you're having trouble - it should work now

</comment></comment><comment><USER48> Wait aren't Resumes just statements of past performances? Also where does A.I. come into this? By having a .ai domain you're clearly advertising to have a Machine Learning product of some sort.

<comment><AUTHOR> Resumes are statements of past performances, and our mission at Searchlight is to help candidates gather support around these statements.

AI helps us analyze the references, and will be developed more as we grow.

</comment></comment><comment><USER49> Love it! We'll try it out :)

</comment><comment><USER50> Clickable link: [https://www.searchlight.ai](https://www.searchlight.ai)

<comment><USER15> Fixed above. Thanks!

</comment></comment><comment><USER51> This is what references are for.</comment>


Adding a SHA1 collision vulnerability test hoses WebKit's source repository - <AUTHOR> https://bugs.webkit.org/show_bug.cgi?id=168774&comment=c27#c27 <comment><USER2> OK, this is quite a serious vulnerability in Subversion. SVN depends more on raw file SHA1 hashes than git because git prepends a header which prevents raw SHA1 collisions from translating directly into easy svn-style repository corruption.

The reason svn is broken is its "rep-sharing" feature, i.e. file content deduplication. It uses a SQLite database to share the representation of files based on their raw SHA1 checksum - for details see [http://svn.apache.org/repos/asf/subversion/trunk/subversion/...](http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/structure)

You can mitigate this vulnerability by setting enable-rep-sharing = false in fsfs.conf - see documentation in that file or in the source at [http://svn.apache.org/viewvc/subversion/trunk/subversion/lib...](http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_fs_fs/fs_fs.c?revision=1737356&view=markup#l862)

This feature was introduced in svn 1.6 released 2009, and made more aggressive in svn 1.8 released 2013 [https://subversion.apache.org/docs/release- notes/](https://subversion.apache.org/docs/release-notes/)

SVN exposes the SHA1 checksum as part of its external API, but its deduplication could easily have been built on a more secure foundation. Their decision to double down on SHA1 in 2013 was foolish.

<comment><USER3> > this is quite a serious vulnerability in Subversion

I rather believe it's a minor bug, and that once it is fixed, they can actually keep using SHA1 as before, without having the denial of service when somebody tries. Then, for example, if somebody actually tries to put two files with the same SHA1 but different MD5 they can reject the second one before accepting it. Or they if there are two different files with same SHA1 and they accepted both and they store only one content, SVN can still continue to work. So you can't get the second unless you, for example, put it in some archive format first and then put in the SVN, OK, your problem, the SVN would still work for anything else.

In short, it sounds like a denial of service at the moment, but I think that DOS can be avoided without changing the hash algorithm.

However, I'm sure that SVN is not the only source base that was never up to now tested with two different files that have the same SHA1.

<comment><USER4> Apache Subversion developer here.

Andreas Stieger (SUSE, SVN) has written a pre-commit hook script which rejects commits of shattered.io style PDFs

[https://svn.apache.org/viewvc/subversion/trunk/tools/hook- sc...](https://svn.apache.org/viewvc/subversion/trunk/tools/hook- scripts/reject-known-sha1-collisions.sh?view=markup&pathrev=1784336)

This is the first mitigation available. If you are responsible for an SVN server at risk, please make use of this hook.

If somebody could make a similar hook for Windows and post it here or to firstname.lastname@example.com that would be highly appreciated.

(edit: switched script link to HTTPS)

<comment><USER4> There have been follow-up fixes to the script and the link above is now stale. See [https://svn.apache.org/viewvc/subversion/trunk/tools/hook- sc...](https://svn.apache.org/viewvc/subversion/trunk/tools/hook- scripts/reject-known-sha1-collisions.sh?view=log)

</comment></comment></comment></comment><comment><USER5> As mentioned in a previous comment ( [https://news.ycombinator.com/item?id=13722469](https://news.ycombinator.com/item?id=13722469) ) git doesn't see these the same as it hashes the header+content which breaks the identical SHA trick.

Of course, I first tested this on our main production repository at work because...oh, wait, I didn't because _what were you thinking_?!

<comment><USER6> It could be made to work on Git, but you'd need to make a collision that included the git blob header. The resulting files would not have the same SHA-1 hash until the header was added though, so they wouldn't be useful except for testing Git itself.

My guess is that Git wouldn't be 'hosed' like SVN, since it currently doesn't have a secondary hash to detect the corruption. It would simply restore the wrong file without noticing anything was amiss.

<comment><USER7> > It would simply restore the wrong file without noticing anything was amiss.

Why hasn't Git switched to SHA-2? People have been warning that SHA-1 is vulnerable for over a decade, but that vulnerability was dismissed with a lot of hand-waving over at Git. Is it a very difficult technical problem to switch, or just a problem of backward compatibility for existing repos (i.e., it would be expensive to change everything over)?

<comment><USER8> > Why hasn't Git switched to SHA-2? People have been warning that SHA-1 is > vulnerable for over a decade

Back then Linus shot this down in his typical abrasive fashion:

[http://www.gelato.unsw.edu.au/archives/git/0504/0885.html](http://www.gelato.unsw.edu.au/archives/git/0504/0885.html)

<comment><USER9> His style is a super abrasive unnecessary power trip but this key point is relevant:

 _> It is simply NOT TRUE that you can generate an object that looks halfway sane and still gets you the sha1 you want._

The key phrase being "looks halfway sane". Git doesn't just look at the hash. It looks at the object structure too (headers) and that makes it highly resistant to weaknesses in the crypto alone. His point essentially is you should design to expect crypto/hash vulnerabilities, and that's a smart stance, as they are discovered every few years.

<comment><USER10> > It looks at the object structure too (headers)

Linus was not talking about the object headers, but about the object contents. It's harder to make the colliding objects look like sane C code, without some strange noise in the middle (which wouldn't be accepted by the project maintainers).

Yes, it's a "C project"-centric view, but consider the date: it was the early days of git. The main way of receiving changes was emailed patches, not pull requests. Binary junk would have a hard time getting in. And even if it did get in, the earliest copy of the object wins, as long as the maintainers added "\--ignore-existing" to the rsync command in their pull scripts (yeah, this thread seems to be from before the git fetch protocol), as mentioned earlier in the thread.

</comment></comment></comment></comment></comment></comment><comment><USER11> (from the link) "For the record: the commits have been deleted, but the SVN is still hosed." That is pretty much my memory of working with SVN. I remember SVN fouling its database a few times. Sure I've broken git a few times, but I am always able to (as Jenny Bryan says) "burn the whole thing down" and take state from another copy of the repository.

I really tried with SVN (wanted something better than CVS) for quite a long time.

<comment><USER12> I've done surgery on svn repos to unhose things a few times over the years, _usually_ due to PEBCAK rather than svn shitting itself. It's actually pretty doable, up to and including the equivalent of interactive rebase.

I much prefer that git's _designed_ to let me do such things and provides tools for doing so, but you can totally rewire svn repos with vi and a bunch of swearing if necessary.

(and I was using svk for a merge tool at the time so I _did_ have the option to burn it down and rebuild from scratch; unhosing svn repos wasn't quite unpleasant enough for me to want to do so)

Then again, I started off doing more ops than dev and have also happily hand- edited mysql replication logs to unfuck things after a partial failover, so I may have more of a masochistic streak than you do :)

<comment><USER13> I fondly remember editing the raw metadata in a Gluster cluster to recover it after a three-node split brain :)

</comment></comment></comment><comment><USER14> Isn't it the SVN repo that's "hosed", not the Git repo as suggested by the title?

<comment><USER15> Yes, the mailing list post backs this up:

[https://lists.webkit.org/pipermail/webkit- dev/2017-February/...](https://lists.webkit.org/pipermail/webkit- dev/2017-February/028792.html)

</comment></comment><comment><USER16> Reminds me of when I worked at an antivirus company. We had be careful with the EICAR file in test code because it would set off AV alarms. [http://www.eicar.org/86-0-Intended- use.html](http://www.eicar.org/86-0-Intended-use.html)

</comment><comment><USER17> New SVN attack category: denial-of-service by SHA-1 collision.

<comment><USER18> New SaaS service: Repository SHA-1 collision detection

</comment></comment><comment><USER19> A bit hard for me to tell what happened here, maybe because I don't know anything about SVN. The two PDFs with equal SHA1 hashes were git commited to the repository, but converting that to an SVN commit failed because... SVN can't handle two separate files with the same SHA1 hash?

<comment><USER20> It's likely some part of the svn implementation that assumes that the SHA1 signatures guarantee uniqueness within a repo. And they might use that hash as an identifier.

I'm guessing shattered-1.pdf and shattered-2.pdf have identical hashes but distinct contents. It's not clear for me to know why this results in a "checksum mismatch."

        Checksum mismatch: LayoutTests/http/tests/cache/disk-cache/resources/shattered-2.pdf
        expected: 5bd9d8cabc46041579a311230539b8d1
            got: ee4aa52b139d925f8d8884402b0a750c

EDIT: see [https://news.ycombinator.com/item?id=13725312](https://news.ycombinator.com/item?id=13725312) for the answer

<comment><USER5> Heh, because those are the md5 checksums which _don 't_ match.

      $ sha1sum shattered*
      38762cf7f55934b34d179ae6a4c80cadccbb7f0a  shattered-1.pdf
      38762cf7f55934b34d179ae6a4c80cadccbb7f0a  shattered-2.pdf

      $ md5sum shattered*
      ee4aa52b139d925f8d8884402b0a750c  shattered-1.pdf
      5bd9d8cabc46041579a311230539b8d1  shattered-2.pdf

As you can see.

<comment><USER21> Wouldn't using both sha1 and md5 solves our problem or the fact both have collision in some cases dooms that combination ?

<comment><USER22> I asked this question yesterday. Apparently using both is not much more of a barrier than the stronger of the two by itself.

[1] [https://news.ycombinator.com/item?id=13715146](https://news.ycombinator.com/item?id=13715146)

[2] [https://www.iacr.org/archive/crypto2004/31520306/multicollis...](https://www.iacr.org/archive/crypto2004/31520306/multicollisions.pdf)

</comment></comment></comment></comment></comment><comment><USER23> I have to just say here that WebKit is one of the most over-the-top software projects I've ever tried to dig into, in my twenty years of programming. Building it inside a vanilla container was impossible following their directions exactly and required _so much_ research on my part to get working. I'm used to a bit of back-and-forth with just about every project, but WebKit was ridiculous. After two workdays of trying, I'd been able to build a WebKit from the source, but at that point had to concede to the universe the futility of trying to build a golang-based Phantom, as my friend and former coworker originally wanted. And that also gave me _mad_ respect for Phantom's author and immediately taught me why they do not often incorporate new WebKit versions into the project instead of just pegging to the first one they can get to build.

</comment><comment><USER24> Site is down.

</comment><comment><USER25> This is why a git clone is not a real backup.</comment>


Ping Brigade - measure how quickly your website loads from around the world - <AUTHOR> http://www.pingbrigade.com/

<comment><USER2> If you are asking people to review then ask. If not, then don't state your service as around the world if it can only do 2 countries. Disappointing after getting excited by the title and spending 5 mins to learn about pinging from USA and France.

<comment><AUTHOR> > _If you are asking people to review then ask._

That was not why I posted it.

> _If not, then don't state your service as around the world if it can only do > 2 countries._

I have no affiliation with this service, I just found it useful and thought others might as well.

<comment><USER2> I assumed the OP was the owner of the site!!

<comment><USER3> Well... assuming "vanity" is often a safe bet! But not this time.

</comment></comment></comment></comment><comment><USER4> Interesting - hope it grows.

I ran netstat.net back in the day - took it over when MCI threatened to sue its initial creator, then updated it and expanded it.

We tracked latency to common ISP nameservers from ~15 locations in the US and Australia. It seemed a little more important when we had to deal with non- broadband latency (dialup was awesome).

Interweather was another 'competitor.' As was the internet traffic report.

And this makes me feel old: [http://www.merit.edu/mail.archives/nanog/1998-06/msg00401.ht...](http://www.merit.edu/mail.archives/nanog/1998-06/msg00401.html)

</comment><comment><USER5> There is more to "around the world" than Five locations in US and One in France

<comment><AUTHOR> Yeah I agree that they do have a little few servers, but I'm hoping they'll add more over time.

</comment></comment><comment><USER6> Very nice, and potentially very useful, but you need more servers. 5 in US and 1 in France doesn't give very much information.

<comment><AUTHOR> > _[...] you need more servers._

I have no affiliation with this service, but I agree with your point and I too hope that they will get more servers.

</comment></comment><comment><USER7> I was excited this might measure user experienced page load times (with all referenced resources) but it's just another Pingdom Clone.

This is a useful tool, but there are already many competitors who do this better (like Pingdom, or Yottaa).

<comment><USER3> I had no clue that Pingdom could do that, I use it to monitor availibility, not to compare speed.

I feel like PingParade is more a troubleshooting tool than a monitoring tool (this point), ie you use it one shot when you feel like something might be wrong, whereas you need Pingdom to keep trying, in order to detect failure ASAP.

</comment></comment><comment><USER3> This is nice. You may want to share the code in order to set up some more locations. Assuming you implement some rate limit, I am sure that there would be some nice people happy to share some CPU with you.

<comment><AUTHOR> > _You may want to share the code in order to set up some more locations._

Again, I have no affiliation with this service. I do realize now that the title of the post might make it easy to assume I had though. Sorry about that.

</comment></comment><comment><USER8> I don't really like the birds ("what does this have to do with twitter?"), but the site does look nice.

Why should I go with this over, say, pingdom?</comment>


Do pro web developer's use web authoring package? - <AUTHOR>

Hello, I am new to web developing world and I was just wondering if popular websites such as Google, Facebook etc use web authoring softwares such as Dreamweaver to design their websites? <comment><USER2> They use tools to help them, but WYSIWYG design tools like Dreamweaver tend to be strictly amateur-level. Once you know what you're doing, Dreamweaver doesn't make things that much faster, and it does produce a vastly inferior product (from a standpoint of code quality and maintainability).

</comment><comment><USER3> On some of my freelance work I've worked with people who used Dreamweaver and, how can I put this, I wouldn't consider their code to be very professional.

Dreamweaver might be alright for developing simple, static websites but for anything dynamic I can't see a professional web developer using it.

</comment><comment><USER4> No.

</comment><comment><USER5> I've worked as a webmaster for 10 years. I used notepad for years but finaly switched to Dreamweaver. The code isn't all ways clean but it gets the job done. If I have trouble I can edit the code as needed.

Since you're new, I think it's best for you to stick with a text editor for a while so you can get a strong hold on the code. It will come in handy later. But once you have a true hold, switch to something like Dreamweaver.</comment>


Opinion: Why students persist in studying English lit in a tech world - <AUTHOR> https://montrealgazette.com/opinion/opinion-why-students-persist-in-studying-english-lit-in-a-tech-world <comment><USER2> Ahh, McGill

TLDR: Students studying English seek opportunities for exploration and critical thinking</comment>


Facebook threatens to sue UK newspaper - <AUTHOR> http://www.guardian.co.uk/media/2010/mar/11/facebook-daily-mail <comment><USER2> They're threatening to sue the Daily Mail.

For those outside UK, the Daily Mail is generally thought of as a joke scaremongering paper, not to be taken seriously.

Unfortunately they do have a large readership who do take them seriously, so Facebook have good reason to be annoyed here.</comment>


I want to hack software like my uncle hacks machines - <AUTHOR> http://new.novelog.com/my-first-story/machine-hackers

<comment><USER2> My first thought was why not hack machines with your uncle. Learn the Arduino platform and bring the flexibility of embedded logic to his creations, then you could provide your expertise to a portion of his hacking machines.</comment>


SPACEWAR: Fanatic Life and Symbolic Death Among the Computer Bums (1972) - <AUTHOR> http://www.wheels.org/spacewar/stone/rolling_stone.html <comment><USER2> Appendix 2 is my favorite part of that article.

Otherwise... it sounds like we've been doing LAN parties for a long, long time.</comment>


The Worst Job in Technology: Staring at Human Depravity to Keep It Off Facebook - <AUTHOR> https://www.wsj.com/articles/the-worst-job-in-technology-staring-at-human-depravity-to-keep-it-off-facebook-1514398398 <comment><USER2> I found this juxtaposition unsettling:

"I was watching the content of deranged psychos in the woods somewhere who don't have a conscience for the texture or feel of human connection,"

"...If the managers noticed a few minutes of inactivity, they would ping him on workplace messaging tool Slack to ask why he wasn't working."

The texture of human connection is severely diminished when you are managing what is essentially a drip-fed trauma survivor remotely using a metric of trauma exposure per minute.

<comment><USER3> Likewise this part...

> Whisper no longer employs U.S.-based moderators. It uses a team in the > Philippines along with machine-learning technology.

Great that they moved the problem to a place with even fewer protections for workers.

The underlying issue is seeing content moderation purely as cost that should be minimized. Taking YouTube Kids for example I'm sure you could sell parents products based around moderated content that educates their children, supports their morals and beliefs etc. i.e. find ways to generate enough income with moderated content to treat people fairly.

<comment><USER4> > Great that they moved the problem to a place with even fewer protections for > workers.

Other big problem with this is the lack of cultural background for doing this.

Only if you belong to a given culture you can understand the subtleties and the cultural tolerance according to each culture.

This is one of the things that should make us think, about how bad is the situation of centralized social networks, when the values of one culture can be arbitrarily imposed into another.

People and cultures in general should have the right to organize according to their own cultural values. So the only right way to do this, at least if its a centrally managed network, is to have native people doing this for each country they have presence for.

If this is too expensive to do it in the beggining .. maybe hire not just cheaper labor, but from countries (cultures) that have good sensibility about foreign cultures, and select the people with talent to understand cultural nuances.

I feel particularlt outraged when according to a given foreign culture, one thing that locally is not a big deal, get censured, and then you think how close we are to the big brother (by the way theres a certain country already going through this path).

But in the western hemisphere, if nothing change, we are slowly consolidating a regime of digital feudalism were corporations are free to dictate rules for our lives.

</comment></comment></comment><comment><USER5> The real problem is not that such content exists, the problem is that Facebook makes it horrifically easy to distribute such content. Back when Facebook was more of interacting with people rather that liking and subscribing to whatever these content and joke pages shoved down these concerns where not present much. Clickbait, etc is just a natural progression of like and subscribing culture.

My Facebook feed about a year ago, when i last opened facebook was filled with the same memes and "inspirational messages" shoved down by a handful of pages being shared by dumb idiots. I had to search and go directly to a person's profile to see what they were up to.

The real problem is that Facebook started behaving like a TV channel that a social network.

If Facebook goes to only keeping profiles of real persons and removes any and all of these pages and blogs and news agencies and the lot, a lot of its woes with regard to content will be solved.

These pages give a sense of anonymity to the people behind them. Take that anonymity away. Once you know that your personal image will be directly tied to whatever you post and held responsible by everyone in your friend list, you will begin to curb your tendencies in public.

<comment><USER6> I believe that the "one account = one real person" policy actually makes things worse.

Regardless of what Zuckerberg claims to believe, in real life I have multiple "profiles". For example, I don't debate Java programming with my family members, I don't debate politics or religion (or generally any opinions, because no one knows what will become a hot political topic tomorrow) with my colleagues, etc.

Similarly, I have different rules for reading stuff. I hate memes, but if my cousins post some, I am not going to block them like I would most other people; I will just sigh and scroll down. More importantly, I do not want to read stuff from all my social circles at the same times; sometimes I am in a mood for family stuff, sometimes programming, sometimes other things.

Even better would be to have advanced tools (but with simple user interface) to specify what I want to read; e.g. "what my cousin wrote, but not what someone else wrote and my cousin shared"; to filter posts containing certain words, etc.

Merely removing anonymity will simply divide users into two groups: those who care about not getting fired tomorrow so they only post completely unobjectionable politically correct vanilla stuff, or those who are not very smart or young and inexperienced or too rich to care about what other people think, so they post anything that comes to their minds.

</comment></comment><comment><USER7> This sort of stuff at one point was handled by hosting providers. It now seems that the internet is "facebook/google" so it is no wonder they are getting the brunt of this sot of work.

I can tell you back in the day when I worked for Rackshack I never envied the abuse department. They always looked stressed out.

8k post in a day is just too many for one person. Its not about how much work they are doing, its about the content they are subjected to look at and review. To do that you have to actually think about it and make a decision -- and that takes a toll on people -- you don't get to forget.

I am not one for regulation but if there is on place in tech that should be considered for regulation, then I think this is a good place to start.

These workers need paid more, access to therapy and much more time off. I also think there are technical solutions to help ease the work needed, but that cost, and nobody seems to want to pay with human workers left holding the bag. This would also include stricter rules to make filtering easier.

Good job, you guys can put a dancing hotdog on the screen why not use that talent to work and make back-end systems to automate this sort of horrid work away. I know it will be hard, but if NN and deep learning is all cracked up to what keeps being preached then it should be within the realm of possible.

Also, while with the when it comes to the law, I am for the notion that it is okay if 10 bad people get away if it means not falsely convicting 1 good person. However on the internet with regards to what normally amounts to pointless shit that people post on the internet. I am okay with 100 good post being automatically removed it if means 1 bad post is also removed.

<comment><USER8> > Facebook will have 7,500 content reviewers by the end of December, up from > 4,500, and it plans to double the number of employees and contractors who > handle safety and security issues to 20,000 by the end of 2018.

Seems like the bulk of these roles are contract. "$24/hour" was mentioned in the article. Probably no health benefits. Probably no counseling.

251 working days * 8 hours * $24 = $48,192. Less taxes. Less sick days. Safe to assume these workers aren't all going to work out of the main office in SF.

Seems like Facebook would have to set up some sort of work from home system, like people who transcribe medical records, or some sort of off-shore system, to make this work for workers. But... spreading out the people who do this work is likely to increase security risks that sensitive data will be shared, and reduce workers ability to find comfort with peers.

No way to look at this where automation isn't going to be better.

<comment><USER9> This definitely seems like a good place for either regulation or unionization.

Unionization would likely be difficult though due to the low barrier of entry for the job - artificial barriers would have to be created by the union and those break down pretty easily due to human factors (see Walmart).

I'm all for regulation in this case. Require that folks who review content for illegal posts be limited in the amount of content they review in a weekday, must have certain benefits like mental health counseling, and get some vacation / sabbatical from the work.

Risks might be reduced innovation in the field, but the big boys like Facebook and Google have already found excellent ways to snuff out their competition within the existing market.

<comment><USER10> Unionization doesn't work very well when you can trivially outsource the job/task. It'd be necessary to entirely remake the US economy, because you'd have to enable union powers that could directly control all actions by the parent corporation, including setting up overseas / outsourcing.

</comment></comment></comment></comment><comment><USER11> The internet really is an incredible cesspool. It would be interesting to see how the public reacted if YouTube and Facebook turned off their content moderation for a week. It would make the goatse meme look like a Sunday school picnic.

<comment><USER12> Yes, but really, humanity is the incredible cesspool. Facebook is just mirror.

The interesting thing is that instead of trying to prosecute the criminals behind this content we would rather just censor it so we can pretend it doesn't exist.

<comment><USER13> I disagree. 99% of us are good people. The cesspool loves a stage/soapbox though.

</comment></comment></comment><comment><USER14> Why are we surprised or shocked? Hasn't society always used servants, police, soldiers, miners, loggers, garbage men, wardens, and such to do tasks that the rest of us are loath to do, and to keep certain stuff away from 'civilised' society?

Why would online be different suddenly? Analogues of all the above are needed online too. And somebody who has no other option will be unfortunate enough to fill these roles.

<comment><USER15> > Why would online be different?

Because you don't have to look at anything you don't want. Because it's not bounded by physical space limitations, so everyone can have as much "space" as they want. Because you can travel from any point to any other point almost instantaneously. Because you can be in as many different places simultaneously as you have the capacity to keep track of. Because you can come and go as you please without anyone else even knowing.

The internet is _nothing_ like the physical world.

</comment></comment><comment><USER16> What they don't tell you, you can get PTSD from being a witness to trauma. It happens to cops and lawyers investigating child abuse all the time.

</comment><comment><USER17> I would expect that social media websites whose content is largely decided democratically (via votes, shares, or the like) would relegate the majority of this content to a place where it is not seen by many. I would argue that the best way to handle this issue is to let the sites mechanisms deal with the content accordingly and then focus efforts on developing processes that will be able to detect and remove it automatically.

The article implies that they are forcing moderators to view the content at a high clip. Why, so as to get false positives back online as quickly as possible? Maybe moderators should only review content that reaches a certain threshold of complaint, and other content is left as is?

<comment><USER18> Reddit is such a website and it's generally consensus there that voting alone does not work.

<comment><USER19> Subreddits are pretty heavily moderated via the "report" buttons. Moderators are basically whatever community member created the community plus the other volunteer users that they picked, and if you have a problem with a subreddit's moderation the usual response is "It's my sub, you can leave."

I'm not sure how that would work for facebook.

<comment><USER20> Reddit recently deleted a whole bunch of subreddits because they decided that wasn't good enough.

Anecdotally, I know someone who was interested in one of those topics (zoophilia), who thought the subreddit in question was "a dumpster fire" yet was still upset about it being banned.

</comment></comment></comment></comment><comment><USER21> Can't help but being reminded of this Silicon Valley episode: [https://youtu.be/dvn-hpZdElo](https://youtu.be/dvn-hpZdElo)

</comment><comment><USER22> In the very early days of the internet being used at a very large corporation, I had the task of reviewing proxy logs to monitor what was euphemistically called "non-business use of the internet". I started by scanning for URLs with "XXX" in them, then pivoted to make a more extensive list.

I never looked at the content itself. Just the seeing the URLs was corrosive enough.

</comment><comment><USER23> A Kaggle competition just started a week ago, hosted by Jigsaw (part of Alphabet) for classifying toxic content (insults, threats, vulgarity, etc) in online comments. $35,000 prize pool. [https://www.kaggle.com/c/jigsaw-toxic- comment-classification...](https://www.kaggle.com/c/jigsaw-toxic-comment- classification-challenge)

Also interesting, they already have an API for doing this sort of classification: [https://perspectiveapi.com/](https://perspectiveapi.com/)

</comment><comment><AUTHOR> non paywall link [http://archive.is/tmFps](http://archive.is/tmFps)

</comment><comment><USER24> It's a good space for regulation. The treatment these workers are subjected to is ignominious.

</comment><comment><USER25> One of our team members used to do this job; luckily, she managed to do it with deep learning, so didn't have to spend too much time looking at unpleasant images.

This experience is one of the main drivers that pushes our team to develop an open, nonprofit conversation platform on which harrassment is difficult by design.

www.hellolyra.com/introduction

</comment><comment><USER26> I wonder what AI thinks and then does to us when we have one.

</comment><comment><USER27> The problem is more the job conditions than the job itself.

</comment><comment><USER28> It's not a popular opinion but remove the anonymity/fake accounts and you eliminate a very significant portion of these issues.

</comment><comment><USER29> I want to do this job.

</comment><comment><USER30> South Park did an episode on this. Where everyone was so afraid of "reality" they elected someone to censor it all and keep their tweets only positive.

Butters had to see every depraved thing. And he ended up trying to kill himself.

And in the end, when he almost died, they blamed _him_ for failing to be the perfect filter.

<comment><USER31> Great episode

[http://www.esquire.com/entertainment/tv/videos/a39070/south-...](http://www.esquire.com/entertainment/tv/videos/a39070/south- park-season-19-episode-5-shaming/)

</comment></comment><comment><USER32> If there ever was a ISIS recruiting pool, this is it.

</comment><comment><USER33> It starts censoring topics that are illegal, then topics that aren't family friendly and finally, it will end censoring point of view and free speech.

Its 1984 all over.

Im not an anarchist however, if something is illegal then go ahead and cut it but, censoring without a legal cause is a crime.

<comment><USER34> Social media platforms are owned by privately held companies. Their moderation is up to their discretion. Sure there's a larger conversation about their ubiquity and how social media posts relate to free speech, but as things work now in the US, Facebook isn't breaking any rules by censoring anything for the most part.

If you want to collect pictures of horrible crap and start a website, as long as it's not illegal you can host it on your own.</comment></comment>


The strange reason why big, innovative companies are hit by 'brain drain' - <AUTHOR> https://www.marketwatch.com/story/the-weird-reason-why-big-innovative-companies-hit-by-brain-drain-2019-01-14 <comment><USER2> > A 100% within-firm increase in R&D leads to an 8.4% increase in the employee > departure rate to entrepreneurship

And how much does the company gains, more or less the increase in funding for R&D? If more, it's OK. People come, contribute new ideas and leave.</comment>


4-bit calculator made from cardboard and marbles - <AUTHOR> https://lapinozz.github.io/learning/2016/11/19/calculator-with-caordboard-and-marbles.html <comment><USER2> Thought experiment: if sentient AI is possible with nothing more than software, does that mean if you "load" the sentient AI program into a "computer" made of cardboard and marbles, that the cardboard and marbles will be self-aware?

<comment><USER3> Yea, sure. What's so hard about that? XKCD made a comic about something similar: [https://xkcd.com/505/](https://xkcd.com/505/).

It's unintuitive, sure, but that's just because of the enormous size of the cardboard you'd need to do it. Human intuition is pretty bad outside of its human-sized comfort zone.

<comment><USER2> > Yea, sure. What's so hard about that?

The hard part is coming to grips with consciousness merely being an abstract or concrete system changing states and nothing more. If rocks on the ground can be (slowly) self aware, then doesn't that mean everything in the universe is self-aware?

For example, say I throw a bucket full of sand on the ground. Then, later, I define an incredibly complex abstract computer, such that the sand falling out of the bucket and shifting along the ground exactly represents a self-aware AI within my defined computer going through its states. Does that mean as I was pouring the the sand on the ground, the sand was momentarily self-aware?

<comment><USER4> > If rocks on the ground can be (slowly) self aware

If weird assemblages of water, carbon, calcium and trace minerals can be self- aware, why not?

<comment><USER5> > If weird assemblages of water, carbon, calcium and trace minerals can be > self-aware, why not?

Because humans (and indeed all life) have "spirits" which can think independently of the body, and that's the true reason we are conscious/self- aware. The body is merely a glove for the spirit. Before birth and after death your spirit still exists and is self-aware (don't take my word for it; try it out - when you die, think to yourself "wow, I'm still conscious without a body!"). Spirits are created by God beings from intelligences (which is a spiritual resource that can be neither created nor destroyed). Gods are created by granting humans more knowledge and powers. The progression is this: intelligence -> spirit -> human (body + spirit) -> God. However, not everyone makes it to God status since Gods choose which humans are to be granted further knowledge and power. And people that have been really bad can be stripped of their body and spirit to have their intelligence recycled back into the immutable pool.

Hence Strong AI is impossible since we aren't/don't know how to mine/utilize intelligences (the immutable spiritual resource) like God does.

<comment><USER6> I understand the downvotes for retina_sam's comment for the number of unproven ideas it asserts, but as unlikely as it seems in our materialistic view of the universe, I think it's important to not lose sight of the possibility that maybe there really is something special, something non-physical, about awareness. It is really poorly understood, and I still have not seen an adequate explanation for why I can perceive things and be aware of them.

"Awareness is an illusion", as proposed by some thinkers, certainly doesn't do it for me, because there's still something that has to perceive that illusion. It's got to be an emergent property of sufficiently complex logical processing, but how does it emerge? What is it really? I feel like I'm still missing a vital step.

<comment><USER7> The lack of an adequate explanation does not demand the need for a non- physical explanation.

I think that because consciousness is part of the core understanding of ourselves, we're drawn to a non-physical and spiritual explanation.

But as a phenomenon the conscious mind is not different than other observed phenomenons.

The rational approach would therefore be to expect an explanation that can be provided by our physical environment just as we do with any other unexplained phenomenon we observe.

<comment><USER6> > But as a phenomenon the conscious mind is not different than other observed > phenomenons.

Yes it is, because it's the phenomenon itself that does the observing. If not, then what is it that does the observing?

<comment><USER8> I have never understood the argument that "observation" is something special and that it must be taken into consideration. In fact, is it really the brain that is observing a phenomenon or is it something else?

In the double slit experiment, it really isn't the brain, is it? It's the screen you place some distance away from the slits, or the electron detector if you place it in one slit only. The fact that "an outside observer" only sees the pattern made by these detectors later and may reason about them is entirely unnecessary to the experiment itself and to the collapse of the wave function. If you leave the results of a double slit experiment lying overnight, has the wave function not collapsed for longer? As such, the "observer" in these experiments are devices, just like in most other experiments.

Just as the infamous cat is an observer of a quantum system, for whom the wave function has collapsed potentially much earlier than for an outside observer. A wave function collapse is in that case much more of a statement of information state than about the state of reality, which is the exact point of quantum mechanics, in that there must be a distinction now between "observers" (which my be single atoms, mind!) inside of the sphere of influence of an event (whether that event may be bounded by physical barriers or a light cone is irrelevant by the way). [Note that Schrödinger brought up the cat example exactly to point out that the world does not exist in a blurry double-state way even if unobserved.]

The same is true for most psychological observations, which get recorded by a computer, by a pen or by an undergrad. The fact that someone else observes these observations doesn't change the phenomenon in any reasonable way.

One more analogy: we now have computers that are entirely capable of recording/observing phenomena by themselves, some of them internal to themselves even, where we have no other way of recording such phenomenon. If you run an A/B test on a web site, let it run for long enough, then look at your analytics page and it says to accept one hypothesis with 99.99% probability, then you have exactly the same situation as if you had done the experiment through other people and publishing it as a paper. Except, of course, no human mind has "observed" the phenomenon, neither directly nor indirectly, until you opened the results page. So what it comes down to is this: did the result exist before you read the results page? I would answer yes.

All of this is a long way of saying that IMHO the observation of a phenomenon does not change the phenomenon itself.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> From the woodworking site posted a couple of days ago, Matthias Wandel made one out of wood:

[http://woodgears.ca/marbleadd/index.html](http://woodgears.ca/marbleadd/index.html)

</comment><comment><USER10> People build these, and they are fun, and I'm surprised that they don't still make and sell the Digicomp[1]. (yes I know some people have been doing limited runs but it seems like there should be a persistent market for it)

[1] [https://en.wikipedia.org/wiki/Digi- Comp_I](https://en.wikipedia.org/wiki/Digi-Comp_I)

<comment><USER11> Evil Mad Scientist makes and sells a Digi Comp II replica: [http://shop.evilmadscientist.com/tinykitlist/375-dcii](http://shop.evilmadscientist.com/tinykitlist/375-dcii)

</comment></comment><comment><USER12> A literal example of race conditions ;)

</comment><comment><USER13> As one of the comments on the bottom of the page points out, Canadian woodworker Mathias Wandel built a similar machine some time back: [https://www.youtube.com/watch?v=GcDshWmhF4A](https://www.youtube.com/watch?v=GcDshWmhF4A)

</comment><comment><USER14> These remind me of Dr. Nim, the unbeatable single-player board game from the 60s, which also used marbles to do binary math:

[https://www.youtube.com/watch?v=9KABcmczPdg](https://www.youtube.com/watch?v=9KABcmczPdg)

</comment><comment><USER15> this calculator nicely illustrates an insight that I feel gets lost the way most people learn and reteach some simple CS concepts:

"two's complement" is not a different system for arithmetic that includes a "sign bit", it's just a different encoding or labelling of states which happens to have a bit that reflects the sign. So, inputs to this calculator can be said to go from 0-15, but more interestingly it can also add numbers in the range -8 to +7 (and therefore, it can also subtract, though it can't negate so you'd have to manually do that to your input by performing a different encoding table lookup).

(edit: now I'm realizing you could negate by performing a two's complement multiplication by -1, performed using this calculator via a sequence of 3 (shift+adds) of your input number to itself... that's correct at least up to some fencepost)

And then by extension, you could test "what about treating the range as -10 to +5", would that encoding succeed or break down? for starters, you would no longer have a sign bit...

</comment><comment><USER16> This is a cool project. But when I was in school, if a classmate of mine came in with this, it would have really rustled my jimmies. Some kids have much better resources than other kids.

<comment><USER17> > _Some kids have much better resources than other kids_

Do you mean in the sense of "engineer parents who encourage this kind of thing", or in the sense of "their own computer to research this on and time to build it", or in some other sense?

I'm assuming you wouldn't have been envious of some kids' access to cardboard and glue.

<comment><USER18> He didn't make the context he was referring to very clear, which is this: "Then my sisters had a science activity where they had to present a science project and I was helping them to choose a subject."

</comment></comment></comment><comment><USER19> I won't be impressed till they run DOOM or the Linux kernel on it

<comment><USER20> Also, it should be self replicating.

And if it ever gains self-awareness and turns into Skynet, humanity could take it down with a Zippo lighter.

</comment></comment><comment><USER21> Reminds me of time that I was working as a clerk and doing CS part-time. It was the early days of computers in the office and was asked how do these magic machines work?

I replied that there was nothing special about electronics, computers could be constructed out of many things. Water/pipes/valves. Ropes and pulleys [1][2]

They looked at me like I was an alien :-)

1\. [http://www.geek.com/geek-cetera/digital-computing- demonstrat...](http://www.geek.com/geek-cetera/digital-computing-demonstrated- with-ropes-pulleys-and-weights-1595808/) 2\. Nice April fool's joke (though the principle is fine) [https://www.scientificamerican.com/article/computer- recreati...](https://www.scientificamerican.com/article/computer- recreations-1988-04/)

</comment><comment><USER22> This is probably the coolest thing I have seen today.

</comment><comment><USER23> Is the AND gate really an AND gate? It seems to be a counter. More like a .skip(1) gate.

<comment><USER24> It really is, with the caveat that a delay line is needed upstream to provide the inputs in sequential time. But it satisfies the AND truth table:

        A  B  Y
        -  -  -
        0  0  0
        0  1  0
        1  0  0
        1  1  1

<comment><USER25> What does the delay line do?

<comment><USER24> I think I may have used the wrong term. What I mean is that if you drop two marbles into the contraption at the same time, they need to be serialized before reaching this AND gate. A typical logic gate allows the signals to arrive simultaneously, but this one doesn't, so one of the signals (marbles) needs to be held back while the other one goes first.

</comment></comment></comment></comment><comment><USER26> Wood version: [https://www.youtube.com/watch?v=GcDshWmhF4A](https://www.youtube.com/watch?v=GcDshWmhF4A)

</comment><comment><USER27> Here's a guy who made a 4-bit adder through water[0]. Creating a computer through water seems to be really doable. I'd be interested to watch more stuffs like this.

[0] - [http://www.blikstein.com/paulo/projects/project_water.html](http://www.blikstein.com/paulo/projects/project_water.html)

<comment><USER28> Back in the late 80s or early 90s there was a group working on this a a backup for military planes. The thought was the "water" (they used some oil like fluid and machined metal blocks for the gates) wouldn't be susceptible to EM pulses and could keep a plane in the air if hit with a pulse weapon.

I believe it was abandoned because solid state ICs grew way too quickly and outpaced the ability of the liquid ICs to keep up.

<comment><USER29> "Liquid IC" as a pun on "solid state" is quite fun.

</comment></comment></comment><comment><USER30> This is awesome! Would be really cool if something like this would exist as a 2d animation so we can see what would happen with different kinds of input.

</comment><comment><USER31> I suggest adding a binary-to-decimal decoder at the bottom to make demos more exciting for people who can't read binary

I did that for a 4bit knex adder/subtractor

</comment><comment><USER32> Given enough free time awesome stuff will happen.

</comment><comment><USER33> Wow, that is pretty cool. It made me think of the book The Difference Engine.

</comment><comment><USER34> Now implement carry look ahead

<comment><USER35> Calculators are a lot easier in unary, no carry-look-ahead needed either.</comment></comment>


Mozilla suspends advertising on Facebook after Cambridge Analytica scandal - <AUTHOR> https://thenextweb.com/facebook/2018/03/22/mozilla-suspends-advertising-facebook-cambridge-analytica-scandal/ <comment><USER2> Please!! They knew of Facebook's bad privacy policies all this time. This is just an attempt by them to gain some good P.R., like this article.</comment>


Wikileaks releases "The Kissinger Cables" with 2 million new documents - <AUTHOR> https://search.wikileaks.org/plusd/pressrelease/?nocache

<comment><AUTHOR> On a related note, Assange's new book "Cypherpunks" is also now available for purchase in Bitcoin:

<http://www.orbooks.com/catalog/cypherpunks/>

And "Hacking Politics," with essays from Aaron Swartz, Kim Dotcom, Lawrence Lessig, Cory Doctorow, Ron Paul, Alex Ohanian, and U.S. Representative Zoe Lofgren, is available as a pay-what-you-want ebook download from the same publisher:

<http://www.orbooks.com/catalog/hacking-politics-2/>

</comment><comment><USER2> _At around 700 million words, the Kissinger Cables collection is approximately five times the size of WikiLeaks' Cablegate. The raw PDF data is more than 380 Gigabytes in size and is the largest WikiLeaks publication to date._

That's a big file.</comment>


Columbus Lowered World Temperature - <AUTHOR> https://www.sciencedirect.com/science/article/pii/S0277379118307261 <comment><USER2> Original title: " _Earth system impacts of the European arrival and Great Dying in the Americas after 1492_ "

Title in HN: " _Columbus Lowered World Temperature_ "

Columbus was not a saint and committed a few of the earlier atrocities in that period, but it's unjust to blame him for all of them (and also the involuntary spread of illnesses) that caused the drop of population in America.

<comment><USER3> I believe this is from the post on Marginal Revolution [0]

[0][https://marginalrevolution.com/marginalrevolution/2019/02/co...](https://marginalrevolution.com/marginalrevolution/2019/02/columbus- lowered-world-temperature.html)</comment></comment>


Ask HN:  How did Google build their July 4th doodle? - <AUTHOR>

They seem to be using one large sprite graphic and some JS for the animation. <comment><USER2> Out of interest I went to check. I don't get a doodle, neither from the URL that automatically comes up - <http://www.google.co.uk> \- nor from the more likely/probable one of <http://www.google.com>

Sometime it would be nice to know what Google actually does. I'm sure it's usually right, but sometimes it's wrong for opaque and mysterious reasons.</comment>


EPUB, open PDF alternative supported by the largest number of hardware readers - <AUTHOR> https://en.wikipedia.org/wiki/EPUB <comment><USER2> It's not an alternative to PDF. It's an alternative to Kindle's MOBI. PDF is intended for printing, and ePUB with its XHTML and CSS has nothing that makes printing easy.

</comment><comment><USER3> By the largest number of brands, sure (everything except Kindle?)

But certainly not by the largest number of devices. Kindle dwarfs everything.</comment>