Internet Cafe Refugees – Japan's Disposable Workers - <AUTHOR> http://disposableworkers.com/?page_id=37 <comment><USER2> c.f. this discussion of the same phenomenon on HN from 6 years ago

[https://news.ycombinator.com/item?id=551794](https://news.ycombinator.com/item?id=551794)

In particular, if you want to understand why someone would opt for paying more daily rather than less monthly, the big non-obvious factors are a) landlords have a very particular set of requirements regarding social status to be allowed to rent in a building ( _gradually_ loosening) and b) a few institutions like key-money balloon the upfront payment required to move into an apartment.

[https://news.ycombinator.com/item?id=552175](https://news.ycombinator.com/item?id=552175)

When we moved to Tokyo, my wire to the new apartment's management company (introduction fee, key money, deposit, first month's rent, insurance, guarantor fee) was, well, call it 50% of a young salaryman's take home pay for a year.

<comment><USER3> Property market in Japan is strange but I still find it much better than where I live now (Singapore).

When I stayed in Tokyo all buildings I've looked at were owned by big corporations. The upside was that rent was quite reasonable. Also one can always make an offer. Those companies don't go for a bidding war. The property will go to the first person that makes an offer provided that both the person and the offer are 'acceptable', rather than the highest bidder.

The rental process can be quite complicated though. I can't even begin to imagine how to rent a flat directly without getting everything arranged through the company that hired you. If I tried to rent by myself I have a feeling that I would be refused even if I offered to pay the whole lease (1 year or 2 years) upfront...

And while rent itself is not bad there are some crazy cost to deal with. Moving from company A to company B and changing the name on the lease? That's monthly rent gone! And there's the "key money" (1 month rent as a fixed fee when signing the lease). Fortunately one can often find "no key money deals".

I will be moving back to Japan soon so fingers crossed. Hopefully I'll manage to work something out.

<comment><USER4> What's it like in Singapore?

<comment><USER3> The rental prices in Singapore are very high. If I wanted to rent a studio flat close to where I work I would have to pay S$3500+ and it wouldn't be easy to find. To live alone it is very expensive; sharing is a bit better and that's what I opted for.

A lot of Tokyo apartments are small and well designed and the quality of materials is fantastic. They are also very well maintained. Apartments in older buildings still look great. And the prices are reasonable. Varying the size / distance to the centre / condition of the building gives you a wide spectrum. Something should fit your pocket.

In Singapore the options are much more limited. I'd rather take a well designed 20-30m^2 studio than overpay for 50m^2; for me that's a waste of space. I don't need a small and mostly useless condo gym; I'd rather pay for a proper gym membership. Swimming pool might seem nice initially but think about how much you pay for it in rent (whether you use it or not). Unfortunately all condos have "amenities". And that "added value" isn't free.

There's also a crazy "double agent" system in place in Singapore. When you see an ad online it is not uncommon that it has been placed by "agent" that has nothing to do with the landlord. They insert themselves between you and the landlord's agent. So you call them up and it turns out they might not even know on which floor the apartment is on! (happened to me more than once). Because the other agent didn't tell them... And as you might expect both agents expect to be paid. This is usually 1 month per year of lease, even on your re-contract. You're supposed to pay your agent off and the landlord is supposed to pay their agent off.

Another weird thing is that you'll never see the photos of the apartments online. Most of the time the photos are just generic filler. The agents post their own mugshots though (crazy, right?).

Finding a flat in Singapore wasn't a pleasant experience.

</comment></comment></comment></comment><comment><USER5> As far as I remember, the term "Net-cafe Refugee" was already common in early 2000s, but I feel that Japan hasn't done much to reduce the number of them (in a good way). And we were too busy paying attention to more sensational things.

As a Japanese citizen, I welcome these reports. It's repeatedly shown that we have no self-correction ability, and I think being exposed and embarrassed in front of the world is one of few effective ways to change this society. We need more of this.

<comment><USER6> Forgive me if I'm simply being ignorant in my questioning here.

As a foreigner with interest in your country but an obviously limited perspective, I have noticed what seems to be a growing trend there in willingness to publicly challenge traditions and institutions and to draw attention to societal problems. This strikes me as being very "unjapanese". While I can see evidence of it going back to the '50s, it seems to be an attitude that has become much more common in the last 10 years or so. Am I totally misunderstanding this, and if not, do you have a positive outlook on some large cultural shift fixing many of these problems in the near-future? "Revolution happens in the mind" and all...

As a side note, from the little bit that I've seen, the amount of attention that the hikikomori phenomenon has received seems to have resulted in some positive steps to dealing with mental health issues like that and it looks like Japan is ahead of other countries that have and don't admit to the same problem.

<comment><USER7> There were arguably more challenges to traditions and institutions in decades past than in today's Japan. Student anti-war riots in the late 60s, etc. Today there are limited protests, especially strong after Fukushima, but none as strong as what happened in 1969.

<comment><USER6> Hmm, a very good point and counterargument. I've come across this once or twice before but I realize I know frustratingly little about the student protests there in '69\. Thanks @gen!

(Been to your blog before for motorcycle reasons. Big fan!)

</comment></comment></comment></comment><comment><USER8> I've stayed in one of these overnight Japanese net cafes. They're absolutely amazing, with TV's libraries, a collection of DVDs, games and magazines. You can pay to shower, theres even a kitchen with food.

That being said, It's a novelty and definitely something I wouldn't want to experience night after night.

<comment><USER9> Off Topic: When I was in Sapporo, I spent a lot of time in Internet Cafes. I LOVED them. Huge up-to-date library, fast internet connection, free lunch, shower, sofa... This place was in the center of the city. Hotels in the same area would have costed 3/4 times that.

The only downside is the smoke. Too many people were smoking and polluting the cafe. As a minor, I could not go to the non-smoking area.

<comment><USER10> > As a minor, I could not go to the non-smoking area.

Is that backwards?

</comment></comment></comment><comment><USER11> I just spent the last couple of minutes watching those videos. That "hostess girls" really got me.

The US is by no means perfect, but to think that human beings could be stuck in such a situation (to literally be worked to death) seems obscene. Here comes my Ignorant American side talking, but to even imagine such lack of opportunity and freedom in a first world country makes no sense to me.

<comment><USER12> > to even imagine such lack of opportunity and freedom in a first world > country makes no sense to me.

What about the lack of opportunity for black people in the US and the fact that most of the prison population is black?, both opportunity and freedom lacking here too. Just different contexts and cultures.

<comment><USER13> It is not a lack of opportunity, it is a lack of dual parent opportunities that can place the proper amount of time into their children to develop them for school and career opportunities.

Taking a step back from race, let us focus it on one parent families. While it is possible to successfully raise a child in a one-parent family it is a difficult variable; there is a reason why the news includes single parent as a hardship in their tales. Locating child-care, staying home with a kid who is sick and cannot attend day care, one income wages, and a host of other variables create issues for allocating the one-on-one time necessary to develop the ABC's, numbers, and mannerisms that a two-parent family is capable of; be they gay, straight, black, white, asiatic, hispanic.

The question is how can we, as a society, encourage two-parent families, so we can get people a good start to be successful?

On the website, there are societal factors within Japan that are creating suicidal individuals without any hope for the future:

* Salary careers are no longer as optimum as before, Mr. Tadayuki is taking the same action as those of second born individuals from centuries past, seeking opportunities in other lands. Staying in Japan only creates suicidal thoughts.

* Hostess girls fall into the same trap as strippers, there is so much up front monies to be made that pursuing a career seems dubious for those who are not able to see long term benefits or have requirements that necessitate large up-front monies (drug habits). Getting them off of these careers would require a government or privately run stripper wean off program.

<comment><USER14> I think maybe you're viewing hostesses from a decidedly western pov. I have lots of hostess friends or x-hostess friends. All but one of them saw it as a pleasant experience. They talked a lot and got paid very very well. They didn't have to put out and some of them gained amazing social skills that have helped them in their current careers. All of them (except 1) go back from time to time for extra cash.

My point being its not seedy to Japanese culture where as to westerners it's often associated with strippers. It's not or at least a large percentage of it is not

<comment><USER13> Thanks for your insight Mr. GreggMan, I figured the hostess title meant that they were not doing seedy thing. I wanted to point out the addictive property of going to work for a high pay job that was only good for a period of time and having the ability to get away from that job.

I wondered about the seediness level and your comment helped explain it better.

</comment></comment></comment></comment></comment><comment><USER15> Interesting. 230,000 seems a surprisingly average wage to be stuck in a net cafe. I think there may be other factors at play - perhaps the part time nature of his job prevents him from being accepted into apartments.

edit: Just my opinion, but I'd say ¥1,000,000 is ... overestimating it a fair bit. Assuming 70,000 rent - a little less than what I pay in a 1DK quite near Shinjuku/Shibuya - 2 months cleaning deposit, 1 month rent in advance, 1 month to the agent, 1 month guarantor, ~20,000 disaster insurance fees, ~5万 to get very basic furnishings (Nittori delivers!) - comes in under half that.

Key money seems to be less and less common. More often than not, it's only one month, or zero with a 2 month cleaning fee.

<comment><USER16> As the article says, when you're temping and have an irregular income, you need to pay _more_ in guarantor fees, advance rent etc to make up for your perceived/actual lack of stability.

<comment><USER17> Also a close reading of the article shows the intro dude was getting over twice the income of his fellow cafe residents.

So if overtime goes away and they only need him 3 days/week instead of current 6 days/week, and maybe he loses his temp job entirely for a few months, or gets stuck at 1/2 average income instead of twice for awhile... he could be homeless and penniless on the street in just a couple months. So saving up a very large cushion sounds wise to me.

A long time ago I went into my first bachelor pad expecting to spend X in the first month and ended up spending 2X. Things add up!

</comment></comment></comment><comment><USER18> I've read several articles about some Silicon Valley employees living in their cars. Especially if the company provides most everything else- food, showers, entertainment. Why spend huge amounts on apartment you just sleep in? This happened at MIT and Stanford too when I was there.

[http://www.telegraph.co.uk/technology/google/11089188/Google...](http://www.telegraph.co.uk/technology/google/11089188/Google- staff-shun-homes-and-live-in-car-park.html)

<comment><USER19> Is there anything in your life other than work at that point? You are either at work or a bunk bed on wheels.

How is that healthy?

</comment></comment><comment><USER20> The "key money" is an archaic thing that really needs to be addressed. There's been a little bit of loosening up on it, but it is still quite obnoxious the amount you have to pay for the "privilege" of a lease.

<comment><USER21> Yeah, totally agree. This is so ridiculous. Japan is one of the only developed countries where owners have about everything to their advantage.

<comment><USER17> Stereotypically this is a symptom of strong renters rights laws. Its a tradeoff.

I've rented in places where renters are treated little better than cattle, at least legally, and you show up with first and last in a cashiers check, they run a credit check while you wait (which I never had a problem with although I assume people must occasionally fail?), sign a one year lease, they hand you the keys, whole process is done in like 10 minutes. Then again if you don't pay up the judge will issue an eviction before the month is out, seen those papers posted on doors. And that's precisely why they'll rent on the spot to almost anyone with a pulse no hassle.

Then again I've heard of places where women with children can't be legally evicted because the judges just won't approve, any code violation or maint issue means rent goes into escrow not to the owner, endless .gov permits and hassle. Thats nice that .gov has your back when there's a problem, but when there isn't a problem, you end up with key fees and guarantors and endless paperwork and social standing stuff.

Given that experience, I would assume Japan is some kind of renters paradise when/if a problem occurs, whereas at a net cafe in Japan if a problem occurs either you walk out or are physically tossed out, although I didn't read anything in the article on that topic (Or I missed it if it was there)

</comment></comment></comment><comment><USER22> These sound like aPodments in Seattle, though even smaller: accommodations for people who either can't afford or don't want to live in "traditional" units.

<comment><USER23> They seem great to me, but sheesh, $775/month? Is the city that expensive?

<comment><USER24> [http://seattle.craigslist.org/search/apa?maxAsk=775](http://seattle.craigslist.org/search/apa?maxAsk=775) (+ click Map)

Not a whole lot of options at that price -- at least, not anywhere near the city center. Things are a little better in, like, Tacoma :P

</comment></comment></comment><comment><USER25> Is this just a market inefficiency? A cubicle with electricity and internet should cost more than a cubicle with bed and nothing else.

<comment><USER26> If you're providing a cubicle with bed and available showers, laundry service and suchlike, the added cost of providing electricity and Internet is small whereas the added value is large, so it strikes me as efficient.

</comment></comment><comment><USER27> It is dishonarable in Japan to live in your parents' home until you find something stable ? Or to live in a flatshare ?

The average salary for those Coffee Shop dwellers is higher than a starting salary for many jobs in Southern Europe.

Even if the cost of living is inferior to Tokyo, it's still not enough to rent an apartment by yourself for most people, so many 20something stay at their parents' home until they saved enough to move out, or found a better job, of just find a flatshare.

I know it's cultural, and not very flattering for those still living off their parents in their mid-late twenties, but given the alternative...

</comment><comment><USER28> The worst part I think is that, even though they don't have family in the city, work is hard to find, and the rent is very expensive, they still won't leave Tokio.

Does that mean the other cities are worse?</comment>


Data scientists shouldn't use Upwork - <AUTHOR> https://www.racketracer.com/2019/07/11/why-data-scientists-should-never-use-upwork/ <comment><USER2> > the dataset contained ... demographic attributes like _race_ ... to predict > actual work performance

Regardless of the original intent, that's a request to launder a protected class into an opaque "work performance" metric. Also, while they are not on the unfortunately short list of protected classes, using demographics like age is probably going to create similar discrimination problems.

If you want to know about job performance, use data that is at least theoretically related to someone's performance. While machine learning is going to find some sort opf signal in whatever pile of data you feed it, that doesn't mean it actually predicts something complicated like job performance. When you feed it a bunch of general demographics, it's going to find one of the signals that we know are present throughout existing demographic: the institutional discrimination and similar problems of the past.

<comment><USER3> If you think it is not possible for race be related to job performance you must not have watched much NBA basketball.

<comment><USER4> The best athletes are extremely intelligent and driven people. The over- representation of black people in athletics is most definitely related to their under-representation in other fields.

<comment><USER5> Nah, intelligence is orthogonal to sports skill, that is, it is not correlated. Plenty of dumb and smart sportsmen. Now, mental fortitude, self- belief, ability to perform under pressure, now those things I believe. But none of them are strongly correlated with intelligence.

</comment></comment></comment></comment><comment><USER6> I freelanced on Upwork for a bit. It was all the worst aspects of having an abusive boss (the platform itself), and all of the worst aspects of freelancing (unpredictability, getting ripped off by shifty clients).

"Creepiest shit ever" is a pretty accurate description of Upwork. If Upwork's attitudes and policies were described to me in the 90s, I would have said it was lame, unrealistic science fiction with an over-the-top villain.

If anyone wants to do freelance work, I'd recommend marketing yourself directly to local businesses, print some business cards, schmooze. It's about as much work and stress as using Upwork, and you don't have a misanthropic middleman taking their cut.

<comment><USER7> Any tips for freelancers who are recent engineering grads? I have some portfolio items, but I still can't claim more than 1 year experience

<comment><USER8> Spend a year working for a consulting company, to learn the ropes. If I were a business looking to have a freelancer build something for me, 'a fresh grad, with less than a year work experience, who has never done this sort of thing before' is not who I'd hire.

I have 9 years experience, and if I were to to go into freelancing (The kind where you solve problems for businesses, as opposed to the kind where you're a 9-5 1099 contractor), the first thing I would do is to take a job at a consulting firm, and spend my time there _learning_. Figure out what works, and what really, really doesn't.

That job is probably going to suck, but you'll have a much better idea of what to do, what your customers really hate about working with big firms, and your resume will look much better, when you can put down "I did the thing I want you to hire me for, in a consulting firm, for 18 months, I know how this works, and I can do it better."

<comment><USER9> > I have 9 years experience, and if I were to to go into freelancing (The kind > where you solve problems for businesses, as opposed to the kind where you're > a 9-5 1099 contractor), the first thing I would do is to take a job at a > consulting firm, and spend my time there learning. Figure out what works, > and what really, really doesn't.

I have to disagree.

I've worked for several tech consulting firms(8 years of experience) and then started my own consulting company 2 years ago. And in my running a successful consulting firm involves 3 key skills(in order of importance).

Sales - This is definitely the hardest and most important skill. And unfortunately is also the skill developers usually have the least experience in. This involves finding leads and closing deals. This will take a long time to do and is emotionally very rough for most developers.

Customer/Relationship Skills - If you're already a nice a polite person who doesn't mind eating the occasional shit sandwich this mostly involves repeatedly learning the lesson that however important you think communication it's more important you think.

Execution - This is mostly involves a combination of being able to correctly identify requirements and technical execution skills. Most developers usually have plenty of experience hear.

The experience most developers lack is sales. And most consulting shops won't put you anywhere near a sales or account management role unless you have previous sales experience or have spent significant time at the company. A much faster route to learn these skills is to just start a company and start doing them. (while reading a bunch of books and talking to anyone you can who has relevant experience)

<comment><USER8> > Execution - This is mostly involves a combination of being able to correctly > identify requirements and technical execution skills. Most developers > usually have plenty of experience hear.

How do you think a fresh-out-of-college junior with less than a year of experience going to be at execution?

Knowing nothing but that, I'm going to take a statistical guess, and say: "Probably not great." I'm going to make another statistical guess, and say: "They probably don't even know what they don't know."

Also, joining someplace that's not consultancy-focused (like Google or Facebook) isn't necessarilly going to help with building relevant experience.

Yes, you're going to learn a lot about how to move protobufs around from one distributed system to another, and how to do on-call, and how to work in the cloud.

No, this is not the most useful set of skills to have, when the insurance broker down the street wants to pay you $XYZW to build them a custom SalesForce widget.

<comment><USER9> Sorry I wasn't disagreeing that the fresh out should get some experience. Just I don't think that working at a consultancy is going to be much more helpful than working any other development job.

> Also, joining someplace that's not consultancy-focused (like Google or > Facebook) isn't necessarilly going to help with building relevant > experience.

It's soo much more important to have worked for one of these companies. If you have Google on your resume it will work magic. Clients love that. It'll help with leads, it will help with negotiation, and it will help close deals.

Honestly the two things I wish I'd done before I started my company was work for amagoobooksoft and put in some time at a sales job.(cold calling or some type of lead gen)

</comment></comment></comment></comment></comment></comment><comment><USER10> Long term upwork Data Scientist here. It seems he had a bad experience, that could have happened on any other place / platform. There are always horror stories about how platforms deal with conflicts (think PayPal, Stripe, Uber...)

Out of >30 contracts I had, only 1 resulted in a dispute, and it was decided in my favor. I do agree the whole monitoring thing is absurd and I generally don't take jobs from clients that demand hours to be logged through the monitoring app. If you want to see how the jungle looks like, please have a go at freelancer.com.

<comment><USER11> > I generally don't take jobs from clients that demand hours to be logged > through the monitoring app

Isn't like 9/10 jobs there require you to use the monitoring?

<comment><USER10> Not at all. Maybe for low paying jobs.

</comment></comment></comment><comment><USER12> My company uses Upwork (as a services buyers) for hundreds of contracted hours per week, and have been for many years. We've never required any contractor to use screen recording for hourly work — that is optional — and have never heard of anyone using a webcam. We've also never disputed a single hour of work that a contractor has billed, nor have we heard any of the dozens of contractors we've hired mention this as an issue.

I think you might be extrapolating from a single bad data point (never a good idea, especially for a data scientist) and reaching an incorrect conclusion about the Upwork ecosystem.

<comment><USER13> While I appreciate your perspective, I can imagine there's a lot of people hiring that have zero idea how developers & data scientists work and think screen recording is a brilliant idea and thus insist on it.

I've seen a lot of really shit job postings on Upwork & similar freelancing platforms that confirm the person posting the job has no idea of what they're doing.

<comment><USER12> I imagine there could be employers who insist on this sort of thing screen monitoring (which sounds crazy to me) -- but the good thing about this marketplace is that contractors aren't forced into any particular contracts. If they don't like the terms with one buyer, they could "vote with their dollars" and work for another.

</comment></comment></comment><comment><USER14> I've been using UpWork for over a year now and never had a dispute so far.

The author is right in saying that UpWork only guarantees payment when time tracking using their app. Where he is mistaken though is where he says that the app requires camera to be on at all times. There is such an option but personally I'd never go for a project like this. And yes, the app will take the screenshot of the stuff you are working on.

I don't get why author's customer would launch a dispute over not completing a 10 minute survey. Given that it was an hourly project I bet there were issues with communication and perhaps the author should have double checked with the client before closing the job.

UpWork is far from being flawless but for me it has been working quite ok.

<comment><USER15> Yeah, I've been working on Upwork for about 1.5 years now and it has been such a vast improvement over other freelancing sites - both in terms of the size and scope of the projects and in terms of the types of clients available. Not one of my hourly clients has ever insisted that I use the monitoring app; yet I do it anyway because then I don't underbill and the payments are guaranteed.

In my mind, the screenshot every 10 minutes is no more invasive than sitting next to a colleague in an office and if you want to do something private, simply pause the timer and do it on your own time. Upwork is a pretty great site if you're a freelancer.

</comment></comment><comment><USER16> I'm surprised that the complaint is about getting paid, rather than pointing out that Upwork facilitated an illegal and immoral system to be built. You don't get to use "demographic information" like "race" or "age" or probably many of the columns the post mentions to make hiring decisions. For the sake of the jobs industry, the author should change their post to say this.

<comment><USER17> I feel there's such a huge potential for serious sampling bias in data like that, I wouldn't be comfortable to train classifiers on it.

Any company with dubious ethics could easily manipulate the data (collection) to match their intentions.

<comment><USER18> Things like job performance usually have a subjective component, so the training data will already have human bias built in without any intentional manipulation.

</comment></comment></comment><comment><USER19> > The central computer notices just about everything. Keeps track of every key > you hit on the keyboard, all day long, what time you hit it, down to the > microsecond, whether it was the right key or the wrong key, how many > mistakes you make and when you make them. You're only required to be at your > workstation from eight to five, with a half-hour lunch break and two ten- > minute coffee breaks, but if you stuck to that schedule it would definitely > be noticed, which is why Y.T.'s mom is sliding into the first unoccupied > workstation and signing on to her machine at quarter to seven. Half a dozen > other people are already here, signed on to workstations closer to the > entrance, but this isn't bad. She can look forward to a reasonably stable > career if she can keep up this sort of performance.

Snow Crash, Neal Stephenson

And now, from the convenience of your home...

</comment><comment><USER20> As a buyer I had a few good experiences finding contractors on Upwork. If "no one should ever use it," does anyone have a good suggestion for a better place to find good freelance coders for small projects?

<comment><USER21> I have used Turtle for 2 projects and had an exceptional experience with both.

Very well vetted devs, excellent platform and framework for managing them, and human customer service at (currently underpriced IMO) $55/hr.

The two apps I've used Turtle for:

\- React site for my business \- Ongoing Node integration work at about 10/hr per week.

[https://www.turtle.ai](https://www.turtle.ai)

<comment><USER22> Thanks for mentioning Turtle! Really appreciate it.

(1) Great talent, (2) great customers, and (3) an easy to follow process (platform) are critical for making freelancing work. UpWork typically misses at least 2/3.

</comment></comment></comment><comment><USER22> Upwork is ruining it for gig economy / freelancing / outsourcing. They create a race to the bottom and prioritize the client over fairness. Fiverr isn't much better.

It's hard for any "we focus on everything" platform to get the specific needs of each vertical (customer + talent side) right. It's why TaskRabbit (focus on everything) never reached the scale of Uber/Lyft (press a button, get a ride).

I'm not deep in the Data Science world, but hopefully there's a platform focused on vetting customers + freelancers and providing a platform for working together easily, fairly, transparently (without the creepiness). Platforms should also decide what kind of work is fine to outsource. Long- term/strategic work the company has to live with for many years (i.e. a core prediction model or fundamental backend arch) shouldn't be outsourced. People who will see the benefits of their work in 5+ years should be the ones working on stuff that'll benefit the company in 5+ years.

I appreciate the mention of Turtle ([https://www.turtle.dev/](https://www.turtle.dev/)) in this thread already (I'm one of the founders).

For freelancers, Turtle: \- Makes sure customers are vetted (just as freelancers are). \- Pays for all hours billed (even if customers don't pay). Freelancers are trusted to enter all hours manually. No creepy screenshots. \- Provides an easy-to-use platform for task management, chat, video, payments.

We're specifically focused on React, React Native, iOS, and Android dev now. Front-end/mobile is easier to compartmentalize. I'm not sure if that's possible for many forms of data science.

Part-time work is an important part of our economy and will become more important as the world further globalizes (ex: "Remote" now being a hot topic) -- but the infrastructure is seriously lacking. UpWork is a sloppy solution attempt.

I encourage freelancers, customers, and platforms to be honest about what should and shouldn't be outsourced, to make sure there's quality on both sides (great freelancers and shit customers fail), and to make sure there's an easy process to follow (great freelancers, customers, with shit process still fail).

<comment><USER23> I'm happy to see an alternative to Upwork.

I hope you will consider branching out beyond those particular technologies. For example, I'm a Node dev who has done Angular and React before but would probably rather use Vue on the front end (if it's appropriate) these days. Also have a fair bit of Linux experience which might be an important skill (as well as C++ and many other technologies from 20+ years). And I am thinking I might focus more on AI by learning to use some of the high level APIs out there. I assume there is demand for that type of thing.

I guess one of the main difficult things about branching out beyond React etc. is, how can you necessarily do a weekly demo for something that really isn't visual or interactive or may not have a short term deliverable? I think the simplest answer is just some type of screen share or short video that shows the raw progress, for example just a run-through of data cleanup scripts highlighting some of the data issues being resolved.

<comment><USER22> Thank you.

We will absolutely branch out, but not sure on timing or specific skillsets.

Fixing what companies (Startups) and freelancers (React, React native, iOS, Android) are on Turtle lets us focus on our platform.

We believe the infrastructure is the biggest missing element in making remote, part-time work successful. Something as simple as how fast or reliably a chat message gets delivered becomes important in asynchronous work. Chat, tasks, meetings, payments, and vetting processes are the core elements of our platform today and these feel critical to any vertical. The vetting processes and meeting templates are the only unique pieces to each vertical.

Since most of our customers are technical, the non-visual demo is not a concern. We are more concerned with distracting ourselves away form focusing on the platform or outsourcing things that shouldn't be outsourced (ex: strategic, long-term impact decisions should not be outsourced. Co-founders should never be outsourced).

</comment></comment></comment><comment><USER24> Archive link: [http://archive.is/Is2Uk](http://archive.is/Is2Uk)

The title is "Why data scientists should never use Upwork"

</comment><comment><USER25> Upwork rules require you to have their desktop app installed and let yourself be filmed for the duration of the task? Holy s __ _

<comment><USER14> UpWork doesn't do filming using your camera, only takes a picture every ~10 minutes. Also the picture taking is optional and can be disabled by client (Throughout 1 year working on UpWork I never had a project which required of me the camera bit). The screenshot taking is required for payment protection.

<comment><USER26> > UpWork doesn't do filming using your camera, only takes a picture every ~10 > minutes.

And how can I, the person being filmed, verify that?

<comment><USER27> Probably notice that your webcam light comes on only once every ten minutes, or that your network traffic is consistent with one picture every ten minutes.

Isn't this site for nerds?

</comment></comment></comment></comment><comment><USER28> You can't be precious on Upwork. And there's a strong GIGO factor.

What you have to do is churn a bunch of jobs/contractors until you get the one that really pays off. Don't let the hassles get to you, just keep turning the crank.

</comment><comment><USER29> A lot of people from India on there posting dumb jobs. Though, dumb jobs can come from anywhere. It takes hours of sifting to find real offers. Most people want you to solve climate change for $1k or build their amazing idea they haven't thought through. It's a dumpster fire but there are real offers on there. With data science, you have to stress that it is research. You can only give probabilities, there's a chance it never works as you suspected or you don't come to a conclusion.

</comment><comment><USER30> Archive link: [https://archive.is/Is2Uk](https://archive.is/Is2Uk)

</comment><comment><USER31> _Nobody_ should use Upwork. You won't get paid what you're worth, and it's the worst of both worlds when you compare it with normal freelancing or working in a regular job.

</comment><comment><USER32> Site down, we 'll never know why ...

<comment><USER33> Maybe the site was built by Upwork? ;)

—-just a joke, I have nothing against them

<comment><USER34> After reading the article, you will.

</comment></comment></comment><comment><USER35> I've probably hired about $150,000 of contracts through up work and never once have we required the monitoring app to be used. However, we have a bit of a different approach, we will "pay" you through upwork for the first month, but if we are satisfied with your work after a month we pay you directly through PayPal or wire transfer and then we cancel the contract on upwork. Out of all of the contractors we have hired (20-30) I can only think of one who wanted complete payment through upwork.

I know how bad upwork is and when we hired we genuinely always set out to get the contractors we worked with off using it so they can keep more money.

<comment><USER36> Might not want to post that here if you intend on keeping that account - [https://www.upwork.com/legal#noncircumvention](https://www.upwork.com/legal#noncircumvention)

<comment><USER35> We have such a solid team now and we have lots of other channels for hiring people now but thank you!

<comment><USER36> Care to share a channel or exchange some knowledge? We buy about 50k hours a year off of upwork, would love to learn how others are doing it. John @ curtisdigital . com

<comment><USER37> The e-mail address isn't working, neither is info@<website>. You might want to take a look into that.

</comment></comment></comment></comment></comment><comment><USER38> > This was the first task I ever received and it flickered my curiosity. In > the problem set, the dataset contained survey and demographic attributes > like race, age, computer proficiency, immigration status….etc.. to predict > actual work performance.

> I completed the survey which was asking me workplace situational questions > testing my unconscious biases and then quickly rejected the dispute.

This sounds like the "consulting company" was someone running a scientific experiment to see who is willing to create racist/ageist/otherwise unethical models.

</comment><comment><USER39> what would be a way to start contracting if you have no existing network, no existing clients to get references?

<comment><USER40> I actually do think Upwork is a good way to start but you could

1) Build a network - go to product meetups (assuming you're a dev) 2) Contact agencies near you and see if they have contract projects you can hop on 3) Use other sites - Toptal, Freelancer, Remote, Gigster, etc.

</comment></comment><comment><USER41> With Upwork, since they're an intermediary and marketplace and hirers are the ones who are paying and providing supply, it's in their interest to offer services that cater to them like time logging and charging connection fees, even if it's unfriendly for freelancers.

Your best best is to take matters in your own hands, and proactively email people you know you can help with your services. This way, there's no intermediary controlling your relationship with your client and you don't have to play the waiting game either (waiting for referrals or networking to pay off, waiting for people to find your website, etc.)

This may seem hard at first glance but let's the take the example of data science work. You can reach out to startups who recently raised a sizable investment round (sign they have the budget and bigger growth goals) and share a few interesting ways you can help them uncover user behavior patterns that would them hit their next milestones.

Further reading:

[1] [https://artofemails.com/get-new-clients](https://artofemails.com/get-new- clients)

[2] [https://blog.hubspot.com/marketing/best-guest-post-pitch- exa...](https://blog.hubspot.com/marketing/best-guest-post-pitch-example)

</comment><comment><USER42> this article has nothing to do with "data scientists". It is just a runt by some guy who used Upwork for the first time, did not follow their (admittedly stupid) rules and got into payment dispute.

</comment><comment><USER43> Anyone using upwork knows you need to have the software running when you have an hourly contract. Fix contracts are different and upworks judgements can be awful but for hourly as long as you have the software running you will be paid.

Data scientist doesn't read agreement.

</comment><comment><AUTHOR> Here's the post on medium. [https://medium.com/@jayfeng/why-no-one-should- never-use-upwo...](https://medium.com/@jayfeng/why-no-one-should-never-use- upwork-a2da9c42769b)

</comment><comment><USER44> The gig economy sucks.... surprise!

</comment><comment><USER45> What does this have to do with Data Scientists? So you didn't read up on how to log time, made unnecessary mistakes, and an inflexible collective of likely underpaid service reps didn't care enough to go beyond the letter of the law. That's hardly a scam. That's you not knowing what you're signing up for and then being upset when you didn't get the outcome you wanted. Hardly deserves front page news.

</comment><comment><USER7> > Their rules state that you must have the Upwork Desktop App on you computer > which installs some sort of video malware onto your laptop AND have your > webcam visible and on at all times so that the customer success agent can > audit that you looked like you were working through the webcam while staring > at a screen. How is that not the creepiest shit ever?

Gross

</comment><comment><USER46> >Their rules state that you must have the Upwork Desktop App on you computer which installs some sort of video malware onto your laptop AND have your webcam visible and on at all times so that the customer success agent can audit that you looked like you were working through the webcam while staring at a screen. How is that not the creepiest shit ever?

Compare this to the Upwork App's description [https://support.upwork.com/hc/en- us/articles/211064098-Log-T...](https://support.upwork.com/hc/en- us/articles/211064098-Log-Time-with-the-Desktop-App) .

It seems pretty reasonable. By default, it snaps a picture every 10 minutes. If you do not want to upload a snapshot, it removes the 10 minutes from your recorded hours. It captures info about how often you used keystrokes and clicked, but not what you typed or clicked.

The benefit of using the app is instant approval and payment.

On my reading, it looks like the author failed to take a 10 minute survey in 3 days, losing his work hours. That seems like a pretty reasonable ask that he failed on.

<comment><USER47> Yeah no. No kind of spyware is reasonable.

</comment></comment><comment><USER48> This is standard industry practice. I remember hearing about one of Upwork's constituent companies, oDesk, doing the same thing 13 years ago. You had to run their time logger program, which would periodically screenshot your desktop and take a webcam photo to ensure you were working.

</comment><comment><USER49> There's also an older post about the same topic: [https://hackernoon.com/why- you-should-never-use-upwork-ever-...](https://hackernoon.com/why-you-should- never-use-upwork-ever-5c62848bdf46)

</comment><comment><USER50> This is very typical for Upwork. I have entirely stopped using their platform.

</comment><comment><USER51> Elance was so much better

<comment><USER52> Looks like upwork ate them AND oDesk

</comment></comment><comment><USER53> Well, maybe a data scientist should do some research into a platform's terms before taking on a job there. And read up on a topic properly when writing a blog post.

</comment><comment><USER54> Nobody should use Upwork, it's filled with scam artists and Upwork doesn't actually adequately block the scammers.

</comment><comment><USER55> So hidden in the article is the real reason that data scientists might not want to use Upwork:

* They record your screen * Your screen contains data * GDPR.

</comment><comment><USER56> s/Data scientists shouldn't/No one should/

No?

</comment><comment><USER57> s/Data scientists/people/

</comment><comment><USER58> tl;dr OP accidentally left a survey unfinished for the client, this a dispute was made. OP completed survey, client was okay to pay, but OP hadn't logged hours with always-on webcam thus money was refunded to client.

<comment><USER59> Seems to me the client got the work for nearly free (gaming the system time only) - I see nothing in the article about UPWork requiring the client return work product as consequence of payment refusal.</comment></comment>


Breaking Through JVM Memory Limits - <AUTHOR> http://www.artima.com/lejava/articles/javaone_2009_gil_tene.html

<comment><USER2> That's a lot of words spent explaining the problem five times over, and none whatsoever describing the solution - just two paragraphs about a mysterious "device" that has to be contacted over the network to run this wondrous 100GB- JVM.

Having a JVM that transparently executes your app on a remote server is an interesting idea that might be useful for some kinds of apps and very bad for others, but I can't see what it has to do with the problem of JVMs not making use of lots of RAM.

All in all, it sounds like a rather contrived setup to justify selling a very expensive hardware dongle instead of a straightforward server.

<comment><USER3> The presentation by Cliff Click at the JVM language conference made that all a lot more clear; it's a shame the article on Artima doesn't go into much in the way of details.

Basically, the limitations on conventional VMs are at least somewhat imposed by the underlying x86 architecture. The Azul box is a custom processor specifically designed for the JVM, so it can do things like write barriers around gc in a single hardware instruction. That, in turn, allows them to produce a fairly pauseless garbage collector: I believe he claimed something like 10-20ms at most for a 100GB+ heap. Contrast that to the several second GC pauses you'll often get on a conventional VM with just a few GB of heap.

Their boxes are also designed to massively scale up; up to 54 cores per chip and 16 chips in a single box. I believe they use localized L1-caches, shared L2-caches (between each 9 cores, I think), and uniform memory accesses across all the chips in the box. The end result is incredibly high memory bandwidth. The other bit Click talked about was how they can profile under load with basically no overhead, unlike the standard JVMTI interface.

They used to sell a lot of gear to financial services companies, as I understand it: I get the impression that for high-load transaction processing their boxes would pretty much smoke anything remotely resembling a convention server or array of servers.

Again, it's a shame the Artima article didn't really talk about any of that. The fundamental limitations are more primarily in the JVM interaction with conventional x86 chips and the really cool work Azul has done is to create a chip (not unlike the Lisp machine of yore) specifically designed to make the JVM run fast and scale up.

<comment><USER2> Thanks - your explanation was far more interesting than the article :)</comment></comment></comment>


You don't get shit you don't ask for - <AUTHOR> http://www.humbledmba.com/you-dont-get-shit-you-dont-ask-for

<comment><USER2> Great advice! Some people can be a bit shy about asking for things, so I suggest you practice.

For example, I was recently at a reception which included a fixed menu dinner. For the non-native speakers, a fixed menu means that you don't order your meal, they just bring it to you. A waiter was walking around to each of the many tables, quietly filling glasses of red wine. Red wine makes me sleepy, so I asked for a beer.

A few minutes later, someone at my table asked me "How did you get a beer?"

I said "Um, I ordered it..."

"Oh? We can do that?"

"We're at a restaurant, he's a waiter. I asked and he didn't say no. So, ugh, yeah. We can do that."

That tiny unfamiliarity of a fixed menu and an automatically filled wine glass undid a lifetime of training on how to order things at restaurants.

You've got to ask for things, even if it feels unnatural or uncomfortable!

<comment><USER3> In the US, there are enough of us vegetarians around (and other dietary restrictions) that they're going to expect some people to request alternatives.

I've waited a few tables before (in the US) too. People with food principles and allergies can be downright rude. Waiters are usually going to be so pleasantly surprised if you ask for something nicely and politely, that they will usually try to make you happy if they can.

Bartending is a completely different business however. :-)

<comment><USER2> Just after we sat down, they asked if anyone was a vegetarian, had any allergies, or other dietary restrictions. I think that's pretty standard practice for fixed menus.

</comment></comment></comment><comment><USER4> This reminds me of the "Asker vs Guesser" culture comparison: [http://www.guardian.co.uk/lifeandstyle/2010/may/08/change- li...](http://www.guardian.co.uk/lifeandstyle/2010/may/08/change-life-asker- guesser)

<comment><USER5> >> This is a classic case of Ask Culture meets Guess Culture.

In some families, you grow up with the expectation that it's OK to ask for anything at all, but you gotta realize you might get no for an answer. This is Ask Culture.

In Guess Culture, you avoid putting a request into words unless you're pretty sure the answer will be yes. Guess Culture depends on a tight net of shared expectations. A key skill is putting out delicate feelers. If you do this with enough subtlety, you won't even have to make the request directly; you'll get an offer. Even then, the offer may be genuine or pro forma; it takes yet more skill and delicacy to discern whether you should accept.

All kinds of problems spring up around the edges. If you're a Guess Culture person -- and you obviously are -- then unwelcome requests from Ask Culture people seem presumptuous and out of line, and you're likely to feel angry, uncomfortable, and manipulated.

If you're an Ask Culture person, Guess Culture behavior can seem incomprehensible, inconsistent, and rife with passive aggression.

Obviously she's an Ask and you're a Guess. (I'm a Guess too. Let me tell you, it's great for, say, reading nuanced and subtle novels; not so great for, say, dating and getting raises.)

Thing is, Guess behaviors only work among a subset of other Guess people -- ones who share a fairly specific set of expectations and signalling techniques. The farther you get from your own family and friends and subculture, the more you'll have to embrace Ask behavior. Otherwise you'll spend your life in a cloud of mild outrage at (pace Moomin fans) the Cluelessness of Everyone. >> from [http://ask.metafilter.com/55153/Whats-the- middle-ground-betw...](http://ask.metafilter.com/55153/Whats-the-middle- ground-between-FU-and-Welcome#830421)

</comment></comment><comment><USER6> The article's bullet points are great, but I would emphasize one thing:

Don't be afraid to ask, but only AFTER YOU'VE DONE YOUR HOMEWORK. People can be surprisingly helpful, but especially if it's clear you're working hard to make each request count. They're much less likely to think you're wasting their time if it's clear you've been hustling and see their help as a catalyst for you working EVEN HARDER.

<comment><USER7> That reminds me a lot of ESR's "How to Ask Questions the Smart Way". Two variations of a general rule, I think: if you want someone's help, make it as easy and hassle-free as possible for them to provide it.

<http://www.catb.org/~esr/faqs/smart-questions.html>

</comment></comment><comment><USER8> The comments on the article directly quickly go into specifically not abusing VCs, they are very busy etc but I think they are missing out on the larger picture here that I am taking away from this story. No one is looking out for my best interests and if I need something or want something the best opportunity I have to get that is to ask.

</comment><comment><USER9> Excellent post, but I wouldn't have distilled quite the same catch phrase -- "You don't get shit you don't ask for" -- from that anecdote. It's more like, "When someone offers you help, take them up on it."

<comment><USER10> Except the general principle of "you don't get what you don't ask for" is true and encompasses your thoughts. Whether it is an introduction to a highly visible CEO or a conference you want to attend for work, if you don't ask, it isn't going to happen. I would argue that even if they didn't offer specifically to help (but didn't make it clear that they don't want to be annoyed by you :), if you think they can help, ask them.

This is a philosophy I've lived by for many years. The only time you need to worry is when the person you are asking is passive-aggressive. They may never give you an indication you are asking too much, while you are unknowingly destroying your relationship with them.

<comment><USER11> That risk certainly isn't just restricted to passive-aggressive people.

It's an issue with a lot of people from more polite cultures or higher context cultures than your own. They likely _are_ giving indications of their discomfort. Unfortunately askers are usually oblivious to those indications and barrel on ahead anyway.

<comment><USER10> You bring up a good point. If you are crossing cultures, tread carefully. Cultural missteps can kill relationships.

</comment></comment></comment></comment><comment><USER12> Good read. I had been sitting on an email for a few weeks to a possible business partner because I didn't want to look like a moron that doesn't know how things work (I am said moron). I just pressed send.

<comment><USER13> Well done! I too sit on emails for far too long - making up plenty of reasons not to send.

You've inspired me to scour my inbox now

</comment></comment><comment><USER14> Yes and no. There are some people who make a living asking 100 people a day, and banking on the five people who say yes. Then others are fortunate enough to have customers, investors and interested parties knocking on their door -- simply because they've made something really interesting. These people often get things _without_ asking. And if you are bad at asking -- it's better to be one of those people.

<comment><USER4> > There are some people who make a living asking 100 people a day, and banking > on the five people who say yes.

Yeah, I never give those guys any change.

<comment><USER15> Persistence is one of the major things PG looks for in founders, it is foolish to mock it as a things only beggars do.

</comment></comment></comment><comment><USER5> When leading my sales team, I always asked them to do two things clearly.

1\. State clearly why they were there

2\. State clearly what they needed.

Jason's note had two statements that did exactly that.

ONE >>Right now, we're preparing for the PhoCusRight conference next week, the travel industry's largest get together. We're trying to make some connections with people at the conference.

TWO >>Can you help us connect with leaders in the Business Travel divisions of Expedia or Orbitz? Or any other introduction that you feel would help us?

Remember without those two simple points, you are wasting yours and other peoples time. And then they wonder (very LA indeed) "And you are wasting my time because?" - which is the last thing anyone should be thinking.

</comment><comment><USER16> Good article, but it's "root" and "rooting" not "rout" and "routing." Somewhat hilarious reversal of meaning via that typo.

<comment><USER17> Australians would think it hilarious with the correct spelling.

"rooting" for me? Uh, thanks, mate. But keep that sort of information to yourself next time, eh?

<comment><USER18> yeah, I just learned about the meaning of that in Australia.

<comment><USER19> There's a clothing company in Canada called Roots. Their logo is a beaver. They sell shirts that say "Roots" next to their logo.

Makes for a hilarious gift for your beer drinking Australian chums.

[http://usa.roots.com/PendrellSlimFitT- shirt/MensTeeShirtTop/...](http://usa.roots.com/PendrellSlimFitT- shirt/MensTeeShirtTop//03022984,default,pd.html?cgid=mensGraphicTShirts&selectedColor=0100)

</comment></comment></comment></comment><comment><USER20> So nobody else is offended by the title? I don't take advice from people who talk dumb shit.

</comment><comment><USER21> "You should never ask anyone for anything. Never- and especially from those who are more powerful than yourself."

— Mikhail Bulgakov (The Master and Margarita)

<comment><USER22> This quote needs context.

Wolland goes on to say 'they will make the offer and they will give of their own accord,' and then, once he's made the offer, he encourages Margarita to ask for whatever her heart desires.

In other words, when someone offers to help, take them up on it. Which is what FlightCaster did.

</comment></comment><comment><USER23> Reminds me of "All you have to do is ask" by James Altucher: [http://www.jamesaltucher.com/2011/05/all-you-have-to-do- is-a...](http://www.jamesaltucher.com/2011/05/all-you-have-to-do-is-ask/)

</comment><comment><USER24> It's true. Ask for what you want in this world, it will usually not be offered.

I've had to get over my own personal shyness to help promote and market our startup (<http://infostripe.com>) and at times it's been difficult to just ask for help, opinion and testers. But when I do it's always quite rewarding.

Ask, speak your mind, be reasonable and the things you want are obtainable.

</comment><comment><USER25> Wow. Perfect timing on this. I've been struggling to develop a mindset like this when looking for a job. It's difficult, but you must not underestimate the value of a connection, regardless of the initial interaction (i.e. it's better to receive a "no" and be recognized than not known at all).

</comment><comment><USER26> Inaccurate. I frequently get things that I don't ask for. Also, asking for things inappropriately is a great way to annoy people.

</comment><comment><USER27> This is good advice. On a first date, after introductions, I present a brief presentation slideshow to the prospective mate where I lay out a dating schedule in Gantt format denoting dates, first kisses, sex and falling in love, including sexual position diagrams. For a personal touch, copy-and-paste both your Facebook profile photos onto the people in the diagrams.

</comment><comment><USER28> Wrong. If you have something people want, they'll force money on you. Make something people want. People are so naive, like they've never seen "Soup Nazi" episode of Seinfeld.</comment>


90% Coverage of Kanji in the wild with 777 glyphs - <AUTHOR> https://japanesecomplete.com/triple-seven <comment><USER2> Discussed 35 days ago [https://news.ycombinator.com/item?id=20721736](https://news.ycombinator.com/item?id=20721736)

<comment><USER3> And it was posted by the same person too.

<comment><USER2> Also posted 21 hours ago by the same person [https://news.ycombinator.com/item?id=21030168](https://news.ycombinator.com/item?id=21030168)</comment></comment></comment>


Byword: A Simple Text Editor for Mac - <AUTHOR> http://bywordapp.com/

<comment><USER2> Thanks, but BBEdit 6 worked just fine.</comment>


Calling All Moms: Stop Saying That - <AUTHOR> https://medium.com/code-like-a-girl/calling-all-moms-stop-saying-that-8505a7ce6eb8#.sycx5pvc1 <comment><USER2> The author claims that "go ask your dad" about math questions happened in her family as well, and yet she herself is a counterexample showing that a young girl in that situation will still pursue math if she enjoys it.

When I grew up (also in Texas, like the author), "go ask your dad" didn't happen at all. My sister (who was good at math, btw) became a nurse because she likes taking care of people. I'm pursuing a PhD in CS because I like programming and teaching, and my brother (who may have had the least amount of mathematical preparation of us 3 children) works with numbers all day long as a insurance/investments salesman because he likes sales. We are all doing what we like to do.

I suppose that my frustration is that many times these articles seem to ignore the fact that people like to do what they like to do (a tautology, for sure), and seem to fixate on, rather, pushing people into roles that the observer wants them to fill. In essence, they are no better than the mindset that they are fighting against.

Why not just let people do what they enjoy?

</comment><comment><USER3> And say what instead?

Is the author saying that moms are lazy, and say that they don't know the answer just to offload the work to the dad? Or are they genuinely bad at math, and in that case what are they supposed to say to their kids?

Not disputing the fact that the role model is important, but what is the alternative here?</comment>


Backbone.js Quickly - A 90+ Minute Live Coding Screencast - <AUTHOR> http://backbonescreencasts.com

<comment><USER2> Excellent screencast! Looking for more like this.

<comment><USER3> Indeed. Great screencast to get you started with Backbone.js.

<comment><AUTHOR> Thanks guys!</comment></comment></comment>


Wispay – Start Receiving Cryptocurrencies from Others in 30 Seconds, for Free - <AUTHOR> https://www.wispay.io/ <comment><USER2> Love the idea,but not sure how it performs in the current market</comment>


How to develop a Firefox extension - <AUTHOR> http://blog.mozilla.com/addons/2009/01/28/how-to-develop-a-firefox-extension/ <comment><USER2> This is why I love hacker news :) I get to learn something every day</comment>


EuroClojure 2012: Call for Presentation - <AUTHOR> http://euroclojure.com/2012/call-for-presentations/

<comment><USER2> Uhh America was way to expensive, England is much closer but still expensive. I will have to think about this, would be cool to talk with people that acctually care for clojure, my frinds are sick of me talking about it :)

<comment><USER3> the idea is to change European city every year to make it as easy as possible for as many people as possible to attend. We started with London because it's always a safe bet (and I'm based here, making it easier to organise it properly).</comment></comment>


The Web Setup: Runscope - <AUTHOR> http://blog.meldium.com/home/2013/12/18/the-web-setup-runscope

<comment><USER2> John from Runscope here...if I can answer any questions about our setup, let me know!

<comment><USER3> Hi John, Love the concept of RunScope. Want to talk to you about a possible partnership. Pls email me; my email address in my profile. Thanks.

</comment></comment><comment><USER4> > In this installment of the Web Setup, we spoke with John Sheehan, the CEO of > Runscope, a small startup based in San Francisco.

Who's we?

<comment><AUTHOR> We is Meldium (YC W13) - a startup helping companies manage passwords & accounts for the plethora of web apps we use to run companies today.

<comment><USER4> Okay. I'm not a fan of unattributed author blogs. It comes off as advertising.

<comment><AUTHOR> Very fair point. I wrote the post and I'll look at modding our blog template to make sure author is visible.

</comment></comment></comment></comment><comment><USER5> Sounds phishy to me.</comment>


What Is Happening with Web Components and Polymer - <AUTHOR> Recently it looks very slow in area of Web Components and Polymer. Do you think that there is still a future for it. Would you use it for your next big project that should last 5 or more years? <comment><USER2> Why would you say its looking slow? The announcements on google IO and other conferences were awesome.

There are lots of components released by users and big companies like General Electric. Google adopted polymer for youtube (including the "main" youtube.com - beta layout is polymer powered) - EA used it for battlefield 1 website, slack was looking for polymer experts. It looks very active to me on github. Just because it doesn't get the love it deserves here on HN it doesn't mean it is not used.

Web Components themselves as a standard are agreed on by vendors, it is a W3C standard, yes there is future for it.

[https://www.polymer-project.org/1.0/blog/](https://www.polymer- project.org/1.0/blog/) \- just recently App Toolbox was released.

<comment><USER2> Also chrome uses polymer components for parts of its UI.</comment></comment>


Google Adds Ads to Image Search... It Really Sucks. - <AUTHOR> http://ryanspoon.com/blog/2008/12/14/google-image-search-now-with-text-ads-ugh/ <comment><USER2> All right, this has to be the first time I've seen someone vocally complain that they visited a web page and didn't see _enough_ large, visually distracting, bandwidth-sucking graphical advertisements.

I think these ads on Image Search are great—they stay out of the way, they don't slow down my web browsing, and they're generally relevant to the search query. As for whether or not this will generate an extra $200 million in revenue, I trust Google's calculated and informed estimation on that account far more than I trust this random blogger's gut feeling to the contrary.

</comment><comment><USER3> what really sucks is when a blogger has "rich jerk" large image ads; much worse then adwords in image search

</comment><comment><USER4> How do advertisers feel, now that their clickthrough rates will likely decrease as a result of being on the seemingly-irrelevant images page?

<comment><USER5> The beauty of AdWords is you can track conversions and exclude domains/sites that don't convert well. Most AdWords advertisers will welcome an increase like this in inventory.

</comment></comment><comment><USER6> I think the author really nails it. He isn't object to the ads existing, he is objecting to them being irrelevant.

Ads that are relevant almost aren't Ads, they are almost content.

<comment><USER7> How are they irrelevant? He searched for Brett Farve and got ads about Brett Farve. He was just bummed that they were text ads, not images.

<comment><USER6> Isn't relevance defined by context?

If he was doing an image search he wanted images, not text.

What about Ads for products or Stock photography or Celeb sites with more photos?</comment></comment></comment>


You're Never Given Time to Build, You Have to Take It - <AUTHOR> http://www.victusspiritus.com/2010/09/29/youre-never-given-time-to-build-you-have-to-take-it/ <comment><USER2> I also have a day job (like many aspiring startup founders here I guess), so I expected to find some insight here :) (some way to cheat some sleep patterns, cramming code in some way or whatever).

So, in your case, you only do some coding on your day off and Saturday-Sunday afternoons/nights? Guess I'll be on the same boat.

The post then wanders into digressions on Focus, self improvement and other observations that read very disconnected to me, and took away from the main point of the post (they may be valuable, but I'll have to process and digest them myself, as they aren't explained much, just thrown together).

I don't want to sound harsh, just some constructive criticism which I hope better blog writers than myself can expand (or refute :) ).

<comment><USER3> Yeah, the title makes you think the article is going to be about ways to work on your projects while keeping your day job. I was looking for things like maximizing efficiency while coding on nights and weekends, or lunch, or downtime, or something.

<comment><AUTHOR> That's funny I was hoping by posting the article here I could get better tips from HN members about how to more efficiently cram in hacking time into my schedule.

I'll certainly continue posting along this topic line as I uncover more tips that work for me. I need to monitor hard metrics for application development (commits) while trying out different time saver tricks.

</comment></comment></comment><comment><USER4> I'd like to have the context of the "walk 2.5-3hrs" part of the schedule. While admirable, that's awfully extreme!

<comment><AUTHOR> You got it: <http://www.victusspiritus.com/2009/03/05/the-walker/>

Now my schedule is 7 days a week ~10 miles a day. I find it cathartic and calming. Helps my state of mind stay clear most of the day.

</comment></comment><comment><USER5> Try going to a University like MIT or CMU. Then you literally can't "take" time. 12-6am you say? Its already taken by your classes, homeworks, and projects...

<comment><AUTHOR> I've been out of school for a while (msee 97). Back when I was going it was pretty smooth sailing (state school).

Now it's harder than creating a startup? No wonder Mark Zuckerburg opted for the easy out. Maybe even Bill Gates too.

I kid I kid!

<comment><USER5> For some schools it may be on par with creating a startup. Of course, these schools are likely the ones famous for taking up all your time.

I'm referencing the famously time consuming universities like Caltech, Carnegie Mellon, and MIT. And even within these schools it depends on your major.

Harvard, I have no personal knowledge of. However, from what I've heard its not that time consuming and has a high level of grade inflation... Take that with a grain of salt though.</comment></comment></comment>


Tell HN: The truth about how startups are valued - <AUTHOR> Startups are valued in ever increasing numbers because people say so then make it real by giving each other money to prove its real.
Here's how they are actually valued:
Seed stage: Startup company to Angel "we are worth $1m" Angel: You ARE worth $1m. If I give you some money it will anchor the value and prove it true because I agreed.
Series A: Startup company to VC "we are worth $10m" VC: You ARE worth $10m. If I give you some money it will anchor the value and prove it true because I agreed.
Series B: Startup company to VC "we are worth $100m" VC: You ARE worth $100m. If I give you some money it will anchor the value and prove it true because I agreed.
Series C: Startup company to VC "we are worth $1b" VC: You ARE worth $1b. If I give you some money it will anchor the value and prove it true because I agreed.
IPO in a bubble Startup company to public markets "we are worth $10b" Public markets (when it's not a downturn): You ARE worth $10b. Take our money.
IPO in a correction Startup company to public markets "we are worth $10b" Public markets (when it's not a downturn): No you aren't. Here, take somewhat less of our money.
That's how money is conjured out of thin air.  I say I have some thin air worth $100. You agree with the value and give me $10 for 10% of it. The next guy both agrees and can see the increasing value of thin air, and can see there's another stage of people coming who will buy in at a higher valuation.  So he buys 10% of the thin air at 10 times the value.  Next folks come along and want in on the opportunity of thin air. etc etc <comment><USER2> Unless it's a mining startup, the kind I invest in. The ones who have almost finished their mine, and will start producing within the year.

I value them using a good rule of thumb: If they're priced less than cash at bank plus expected earnings within the first year, and their mine plan is solid, then it's a good buy.

If they make it through the year, making the expected profit, their share price will triple. A one year old gold mine is priced generally at three times earnings.

Occasionally they're wrong AND I'm wrong about the mine plan they had, and the costs of mining each ounce of gold is much greater than thought, the mine runs out of cash in a month or two and everything crash and burns and I lose all I invested in.

Or the gold price just falls hard.

I've made 30%+ per year for the past several years doing this. Requires a good eye on the reports to figure out if the directors of the company are credible.

</comment><comment><USER3> Exactly. It is a totally arbitrary process. Valuations have nothing to do with real economic value or real profits of a company. We see it all the time. In fact, valuations and the subsequent "exit" venues (IPOs, acquisitions, etc..) can be thought of as a form of money laundering in which money is collected from the public and funneled into the pockets of VCs. If you have the power to create the hype, you can "value" anything at any amount you like.

<comment><AUTHOR> >> We see it all the time.

Who is "we"?</comment></comment>


The Russian Government Runs a Troll Agency to Flood the Internet with Propaganda - <AUTHOR> https://www.yahoo.com/tech/the-russian-government-runs-a-troll-agency-to-115389567389.html

<comment><USER2> So do the Israelis, Chinese, British, Americans and probably many more countries. It's amusing to read propaganda like this article that is clearly designed to point the finger from a morale high ground when the moral high ground is an illusion. It's also a little tiring. We could try clean up our own backyards before pointing at our neighbors.</comment>


Get Back from SF after Caltrain Stops Running - <AUTHOR> http://www.indiegogo.com/projects/last-call-bus-club

<comment><USER2> Good luck, this benefits everyone who may need this service, unlike the screw- the-community-we'll-just-take-care-of-our-9-5 big company buses. Definitely need more ideas on how to improve commuting around this huge place stuck with the 60's transportation.

</comment><comment><USER3> whatever happened to the tradition of bouncing around after hours clubs and coffee shops before staking out the first caltrain??

<comment><AUTHOR> Haha, good point. 8am is a long wait though.

<comment><USER3> waiting is until fullness... some of the best things I've ever learned, I learned when I normally would have been asleep</comment></comment></comment>


GitHub was down - <AUTHOR> https://status.github.com/messages?ts=mon-jul-31-2017 <comment><USER2> I would love to see a chart of traffic to other sites when GitHub goes down. My bet is that HackerNews and Twitter both get significant spikes from all those bored developers.

<comment><USER3> > bored developers

Bored? Git's a distributed version control system, so no excuses. Get back to work!

But in all seriousness I kind of wish GitHub provided a way to mirror things like issues and PRs so you never have to be fully reliant on one service. Not being able to read these really does make it impossible to get work done offline.

<comment><USER4> But they want you to Rely on Github... if they could find away to make Git only work with Github they would in a heart bet.

Github is not a open source company, it is not really even a supporter of free software IMO they are a danger to free software for this very reason. They are following the old school Microsoft model of Embrace and Extend... I am waiting to see if they can extinguish,

<comment><USER5> What? Github employs members of the Rails core team including Aaron Patterson. They literally pay for open source software development.

I'm sure they employ direct maintainers/contributors to other open source projects as well.

<comment><USER6> Are they paying him to help make an open source alternative to github? If not, then it's not really relevant.

They are actively working against open source tooling for software development by developing features only for their closed platform.

Gitlab is a much better example of a company that fully embraces open source.

<comment><USER7> GitLab does not fully embrace Open Source.

GitLab Enterprise Edition does not have an open source license. Since GitLab.com runs GitLab EE, their two (presumably) primary sources of revenue (EE licenses and GitLab.com paid accounts) come from non-open source software.

But: it's still fair to call GitLab as a company way more Open Source than GitHub. All of their development happens out in the open, the vast majority of their codebase is Open Source, and the source code for GitLab EE is even made available.

This is a spectrum, not black and white.

</comment></comment></comment></comment></comment></comment><comment><USER8> When I break our GitHub webhooks, I joke it's time for people to practice our Disaster Recovery (DR) procedures. In all seriousness, this is a good opportunity to practice work without GitHub. Any service can go down; can you deploy a critical bug fix without it? If not, why not and what can you do to fix it?

</comment><comment><USER9> I had to change a username from capitalized to uncapitalized and use my updated remote afterwards, apologies if I broke it for everyone.

<comment><USER10> It's kmfrk's fault! Get them!

<comment><USER11> "Git 'em!" FTFY.

</comment></comment></comment><comment><USER12> If anyone is interested, I've been working with a git host that is actually distributed across a p2p network using SSB.

see:

[https://github.com/clehner/git-ssb](https://github.com/clehner/git-ssb)

[https://github.com/noffle/git-ssb-intro](https://github.com/noffle/git-ssb- intro)

It's been working fairly well so far. We are using git-ssb to manage a few projects instead of putting them into Github.

<comment> vertex-four Is there, at least theoretically, a way to prevent other people from pushing to my repo? That seems like it would suck re griefing for any project that might become even mildly politically sensitive for whatever reason.

<comment><USER12> Everything is key based, so only key holders push to your repo. It's all based on the SSB protocol.

I'd suggest reading [https://github.com/noffle/git-ssb- intro](https://github.com/noffle/git-ssb-intro) to get an idea.

<comment> vertex-four That states: "git-ssb's permissionless model has an interesting consequence: anybody can push to anybody else's git repository." The guide doesn't show any key-sharing in order to do that. Are you saying that's incorrect?

<comment><USER12> I don't want to give you the wrong answer so I've forward this question to the SSBC network for one of the core developers to answer better.

I'd be surprised if there was no security for pushes. The repos I've worked on did require an invite from the creator.

<comment><USER13> Marak It sounds like you are working with private repos. With git-ssb currently a repo is either public or private. Private repos are encrypted to a fixed set of recipients so only those keyholders can access it. Public repos are unencrypted.

</comment></comment></comment></comment></comment></comment><comment><USER14> Status now shows Major Service Outage:

12:32 EDTMajor service outage.

[https://status.github.com/](https://status.github.com/)

<comment><USER15> unicorns all around

</comment></comment><comment><USER16> Pages Builds Failure Rate spiked to over 2000%. I don't know how that's possible, but it seems pretty bad.

<comment><USER17> Maybe 20+ failing retries for any single page build?

</comment></comment><comment><USER18> Insert remark on why we use a centralized service for a distributed source control system, etc. No one seems to care, unfortunately

<comment><USER19> The reason is github did a fantastic job of implementing useful features. The visual design is unmatched and they have done a great job implementing developer oriented integrations and social features.

A more federated approach to this sort of thing might have been nice, but so far nothing I have seen comes close to the value-add offered by github.

<comment><USER20> You're sidestepping the main reason I believe it worked so well. It benefits from network effect. It is a collaborative tool and people like to have their work on there so others can collaborate with them.

<comment><USER19> That's important but I think people over-estimate it. It's both. I predict that if you analyzed the github network, you'd find many hubs are based around companies that chose to move their workflows to github based on features other than network effects. Or at least, the existing network was only one of many reasons.

<comment><USER21> As a business, we (and most other companies I know) chose github for features and performance. Nice that other open source stuff is there but doesnt matter for what we pay for.

</comment></comment></comment></comment></comment><comment><USER22> Looking at the status graphs, it seems like there was some clearly anomalous data starting around midnight, about 9 hours before the actual outage "began". Maybe a gradual botnet ramp-up, and 9:27 AM is when it got bad enough to overload some critical service? (Or really any other threshold-based failure scenario.)

<comment><USER23> or a bad commit being deployed on their fleet.

</comment></comment><comment><USER24> What was happening to Github for a week or so in late June - early July? I see "The status is still red at the beginning of the day" for a whole week.

[https://status.github.com/messages/2017-07-03](https://status.github.com/messages/2017-07-03)

<comment><USER25> They seemed to be suffering some external attacks / DDoS but I never saw a post-mortem from them on it, hopefully one is forthcoming (or maybe is out but I missed it)

<comment><USER26> They don't do post mortems.

<comment><USER7> They occasionally do, and they call them Incident Reports.

[https://github.com/blog/2106-january-28th-incident- report](https://github.com/blog/2106-january-28th-incident-report)

[https://github.com/blog/2273-incident-report-inadvertent- pri...](https://github.com/blog/2273-incident-report-inadvertent-private- repository-disclosure)

</comment></comment></comment></comment><comment><USER27> Do these general Github outages affect GH Pages as well, or is that service portion segmented to some degree?

<comment><USER28> Pages are static sites served from separate infra from the main github.com from what I've heard, so you should be fine in almost all cases. I've never had one go down unless I pushed a dud build.

</comment></comment><comment><USER29> I think it started as minor as I was receiving a unicorn once per 10 pages. It's currently happening on almost all.

Of course, I'm trying to dig into a WebKit issue and need the issues to load!

</comment><comment><USER30> Where is github hosted?

Do they use AWS or another commercial cloud provider, or do they have their own servers in data centers (hopefully scattered around the globe)?

If AWS, are their services spread among multiple availability groups? I'm just wondering how this could happen.

<comment><USER31> I like how this comment has 3 replies saying different things.

I am now feeling less informed than I was after the first reply.

<comment><USER32> An ex-github employee who used to give talks on Github scaling challenges revealed they shifted from Rackspace. See yourself - [https://github.com/holman/ama/issues/553](https://github.com/holman/ama/issues/553)

</comment></comment></comment><comment><USER33> Github is back online.

</comment><comment><USER34> It has leveled up to a major outage!

</comment><comment><USER35> Dang. It's too bad their customers' source control files aren't distributed and decentralized, or they could keep working and ignore this.

<comment><USER36> The problem is not the files, it's the project management stuff (issues and PR tracking). _That_ is a SPOF. If they could decentralize that, then nobody would care if they are down half the time.

<comment><USER37> I wonder why you can't export them as a read-only git repository. (Read-only to avoid people messing around with the history)

<comment><USER7> You can actually export them, but the format is undocumented, and is meant only for import into GitHub Enterprise:

[https://developer.github.com/v3/migration/migrations/](https://developer.github.com/v3/migration/migrations/)

</comment></comment></comment></comment><comment><USER38> Anyone have any knowledge of what specifically happened?

</comment><comment><USER39> I saw a comment earlier mentioning that GitHub allegedly doesn't release post mortems publicly? If this is true, that's upsetting.

<comment><USER16> They have published post-mortems before, but the previous bout of downtime went unacknowledged. Maybe it's a new policy.

Here's a postmortem they did several years ago: [https://github.com/blog/1261-github-availability-this- week](https://github.com/blog/1261-github-availability-this-week)

</comment></comment><comment><USER40> My apologies. I knew my Perl 6 wrapper for GLFW was bad, but never realized it'd be so bad that GitHub would choke to death on it.

</comment><comment><USER41> Are there any other major sites that are down?

<comment><USER42> All software depending on Github repo releases downloads are down, e.g: Rancher CLI.

<comment><USER43> Release downloads are affected?

<comment><USER42> yep, my Circle-CI deploy fails, the following URL is not available: [https://github.com/rancher/cli/releases/download/v0.6.0/ranc...](https://github.com/rancher/cli/releases/download/v0.6.0/rancher- linux-amd64-v0.6.0.tar.gz)

</comment></comment></comment></comment><comment><USER44> It just became a major service outage.

</comment><comment><USER45> This is happening too frequently now.

</comment><comment><USER46> It's starting to work again for me. I was able to approve a PR and merge it.

</comment><comment><USER47> Looks like I am still able to push to/pull from my repos without issue.

<comment><USER48> I've had a few CI jobs fail when attempting to pull, but I've also had some succeed. Seems that it isn't an all-or-nothing outage.

</comment></comment><comment><USER49> Whatever happened to gittorrent?

</comment><comment><USER50> Thoughts on the cause?

<comment><USER10> Definitely Bitcoin-related.

<comment><USER51> SegWit's first victim.

</comment></comment></comment><comment><USER52> How does this affects all your dependencies?

<comment><USER53> Why would it?

<comment><USER54> A lot of package manager tools download their packages from Github. If you happened to be refreshing your dependencies at the time, or doing a clean build, then you'd be SOL.

<comment><USER55> Really? I was under the impression that most language package managers downloaded from a CDN. I know that pip, npm, yarn, cargo, and hex do, at least.

<comment><USER54> Homebrew I believe goes through GitHub. Many of the Vim plugin managers also do, or at least have the option. I think CocoaPods and Carthage, both for iOS development, do as well.

I think it's somewhat common for a new package manager to use Github as a kind of CDN for a while, until they get big enough they can do their own.

</comment></comment></comment></comment></comment><comment><USER56> >GitHub is having a minor service outage

It's definitely not minor.

</comment><comment><USER57> I knew I shouldn't have released the new version of my project yesterday. :p

Sorry everyone

</comment><comment><USER58> Githubs uptime is pretty bad. Isn't it under 95% for the year now?

<comment><USER59> Not really.

It is just very public when they go out.

Steam, Blizzard, Activation are down at least 6 hours a week. While Bank of America is offline 24hours/month. Scheduled downtime is still downtime.

<comment><USER60> > Scheduled downtime is still downtime.

Except in the SLA.

I've argued more than once when watching a vendor announce "emergency scheduled maintenance"(!) as an outage is looming.

</comment></comment></comment><comment><USER61> In the face of a lack of information, HN comments begin to throw around unfounded speculation & tongue-in-cheek jokes run rampant. I suppose that in the absence of information, many stay silent, & the remaining see a thread lacking comments

& now we've got this meta one in the mix

</comment><comment><USER62> How many more can we expect before they develop appreciation for testing _before_ they push to prod?

<comment><USER10> Newly introduced errors in site code are only one of many sources of failure for a site like GitHub, and probably a fairly rare one.

<comment><USER62> It's a nearly weekly occurrence. Avoidable one, I might add. When was the last time a major Google service had a major outage?

<comment><USER10> You're still missing the point, by assuming that the outages are code related when you call them avoidable. Distributed systems at scale are terrifyingly hard to control.

When I said "probably a fairly rare one", I didn't mean that the outages are rare. I meant that new code is probably a rare cause of the outages that happen. They have other causes unrelated to new code.

(I'm also skeptical that GitHub major outages happen "nearly weekly", but I don't have data.)

<comment><USER62> I'm not "missing" anything. I worked at Google for 7 years much of which was spent working on, you guessed it, distributed systems infrastructure. You guard against this by carefully canarying things and putting robust testing, monitoring, and deployment procedures in place. A release might take a few days, but you can be reasonably certain your users won't be your guinea pigs, and if shit does hit the fan, rollback is easy, and you can reroute traffic elsewhere while you roll back. Most of the time no rollback is needed: you just flip a flag and do a rolling restart on the job in Borg. For some types of outages (most of which users never even see) Google has bots that calculate monetary loss. And the figures can be quite staggering and motivating, so people do postmortems and try their best to make sure the outages don't happen again.

<comment><USER63> So no Google service has ever experienced an outage? I distinctly remember Gmail being down on several occasions.

<comment><USER62> Gmail is several orders of magnitude larger than Github will ever be, and in recent memory I can only recall it being down once, and for a very small subset of users.</comment></comment></comment></comment></comment></comment></comment>