High Salaries Are Weapons in the AI Talent War - <AUTHOR> https://www.bloomberg.com/news/articles/2018-02-13/in-the-war-for-ai-talent-sky-high-salaries-are-the-weapons <comment><USER2> My personnal experience with data scientist and startups is that they're hired much much too early, when the product has a lot more fundamental issues to solve, and only because it looks cool to say that you're doing AI.

In practice, they're often frustrated for years by the lack of infrastructure to work on their ideas, but live with it because life is good.

I suspect AI today is like big data ten years ago : a lot of company think they need AI, but in fact what they need is a good product and a few algorithm requiring high-school level maths.

That could explain the shortage...

<comment><USER3> How would start ups compete well with those salaries?

<comment><USER4> They can't, and don't. The secret is that experienced PhDs (mostly) dominate the high end of "AI" hiring, but don't have much title or departmental differentiation from people who are "only" specialized software engineers in top tech companies. But this isn't very well known, large companies want to recruit as much talent as they can regardless of role, and startups want to compete _on paper_ \- therefore, you have the following effects:

1\. Startups give whatever sexy title they want to the people they can afford, which makes titles fairly useless, because they almost never can afford the talent commanding "sky-high" salaries.

2\. Within large tech companies like Google and Facebook, it's hard to immediately tell which of the many data sciencey, machine learning-y titles correspond to the truly stratospheric salaries versus the engineers that work with those roles. For some teams, like DeepMind, Google Brain or FAIR, it's easier to tell. But for others it's a mixed bag.

For comparison, see the fashionable term of art "quant" in the financial industry, which has similarly devolved into marketing and a bimodal distribution. As a rule of thumb, you generally can't trust AI titles or salaries at startups unless those startups are really known for their talent; further, you can safely assume that, at companies capable of paying for top talent, the very impressive salaries belong to titles which seem the most exotic and out of reach for general engineers in the job description.

Fortunately startups don't really need to compete for top talent as a genuine technological differentiator, they just need to engage in signaling, so this is mostly a non-problem. Startups almost never have problems usefully improved by the cutting edge of machine learning research, and can instead use off the shelf tools and existing software to accomplish the same things. Frankly, it's exceptionally rare for a startup to even have the massive data, pipeline and munging infrastructure requisite for actual research.

<comment><USER5> This actually makes a lot of sense. I have a lot of friends employed as Data Scientist or Data Engineers who don't seem to be able to explain exactly what they do, or describe what "research" they are doing. From what I understand, it seems like they are designing pipelines that ingest data, run an off the shelf AI algorithm and display nice graphs. There is a lot of pipelines you can design for different kinds of data so I imagine they always have something to do.

</comment></comment></comment></comment><comment><USER6> This is an interesting subject, because intuitively one wants to make the comparison "Is Conan O'Brein paid a multiple of how many times funnier he is than a local standup?". But economically what's important isn't how funny he is, but how many viewers he can draw. So then you might ask, does Conan draw 30x more viewers than some no-name comic? But that also isn't the comparison that matters. If no-name comic can draw 10M viewers, but Conan can draw 20M viewers then should he be paid 2x as much? It depends on the costs for the rest of the show. If a show costing $500K to produce and $10K for a no-name host earns 1M then if they were to replace the no-name host with Conan and could earn $2M per show, it would make sense to do that as long as you weren't paying more than $1,010,000 (101x the no-namer) for Conan to host the show.

The point I am trying to make here is that these figures vary from industry to industry and from job to job, I could have conversely changed these numbers around and shown that it doesn't make sense to pay Conan a huge multiple of the no-name comic's pay. For example, if the revenue does not increase substantially (like from 1M to 2M) or if the salary of the person in question makes up a much larger portion of the overall expense of the company.

<comment><USER7> I think it's more of a winner takes all phenomenon in the Conan case. Usain Bolt makes millions because he's a tenth of a second (or less!) faster than many other people.

Those other people - with the exception of maybe Justin Gatlan - make a far more modest salary. It's less than a 1% difference in performance that leads to orders of magnitudes differences in outcomes.

<comment><USER8> Reminds me my first year as a webdev. I was extremely lucky to ride the apex web 2.0 hype/insanity wave. Now I can't believe myself that I, as a 21 years old and just 3 years of for-profit programming experience, got CAD 85k on first real job Canada for just jquery animations.

Later down the career path in Canada, I was frequently asked "you worked for guy A and B, than must've been hell of a job?" or "how I got there to begin with?" All of them dismiss my explanation that "I just used to be on line 1 of google for thing A"

The back side of the coin? The moment web 2.0 became a more of an "in-house" production with mid-to-big sized companies that no longer needed a "hired gun" outsider, and hype wave moved to other things I really hit a wall. Actually, on my next 2 jobs I took salary in $70ks and was contemplating selling major life assets after my last employer in Canada was unable to extend my work permit.

Things went much better after I scaled down my appetites and stopped looking for employment with companies obsessed with "rockstar hiring"

</comment></comment></comment><comment><USER9> I suspect there may be some sort of hype in the AI area regarding salaries. Sure, there are a couple of rockstars, but I suspect that competition is really intense and compensation in ML area has an even more acused power law distribution than general vanilla software engineering that doesn't face as much pressure from maths, stats and other grads that look for a career to apply their quantitative skills.

I think that in the real world paradoxically math skills are easier to find than solid software design and development abilities. It may stem from the fact that the first one is taught quite well in school while the other one is more about individual learning and sometimes a contrarian stance to the system (can be reflected even a somewhat childish "I'll learn Haskell because the OOP and Java suck!") which is harder to find and therefore, more valuable.

<comment><USER10> In my experience with AI start-ups, if you have a rockstar CV in your deck you get funded. If you don't, you don't. It is very difficult for non-experts to evaluate the competence of budding AI teams. The best heuristic, track record, thus prevails, which in turn attaches a lot of value--from the company's perspective--to that single CV.

<comment><USER11> I'd argue this line as well. A lot of the earlier stage startups who don't need this talent are trying to secure it because it offers them that halo of prestige to grease the wheels of fundraising.

A $400k salary is nothing if it makes it easier to unlock $XXm in funding.

<comment><USER12> I'd laugh if I wasn't crying. I narrowly escaped high school where the popular kids and "school celebrities" won. All that studying in university, all those labs, grinding that entry level job, building my skills, grad school, more hard work... and at the end of the day the popular kids inevitably win again.

<comment><USER13> I don't understand the comparison. Aren't the highly salaried AI experts also people who undertook a lot of hard work to get there?

</comment></comment></comment></comment></comment><comment><USER14> A lot of the comments here are skeptical of the claims in the article. (This is how comments should, and do, function).

However, for all the reasons that the article may be wrong

\- the shortage is temporary

\- the shortage is overblown

\- the shortage is illusory

\- the shortage doesn't apply etc

Please compare with the situation for other high earners (CEOs, Entertainers and Bankers). Can the same arguments be made for them? (Conan O'Brien is funny, but he isn't 30x funnier than the person at my local comedy club?)

The work of an AI researcher is mechanically reproduced so a 1% benefit can be enormous. It could be the case that the competition isn't for more of them, but for the best of them.

<comment><USER15> Utility increase is not linear with cost increase. Conan or any other valued expert may not be 30 times better, maybe just something like 1.5 - 5 times better. But I do agree that demand will be satisfied in a long term.

<comment><USER16> Conan also gets paid what he does because his audience is millions or tens of millions of people. The guy at the comedy club who is _funnier_ than Conan but doesn't have the audience makes nothing.

</comment></comment></comment><comment><USER17> There's a pretty big difference between AI researcher and applied Data Scientist. Most companies don't need to develop novel algorithms, they just need to be able to apply what's already available.

Fortunately, the applied DS jobs don't really require a PhD in machine learning. A master's in CS, stats, etc is usually plenty.

<comment><USER18> What kind of salaries are MS applied Data Scientists commanding? I am very interested in enrolling in Georgia Tech's OMSA, but very little graduate statistics exist for these types of programs and job searches usually don't provide compensation. Further, the vagueness surrounding the name "data scientist" clutters up the information that is available.

<comment><USER16> It's not particularly popular to say here but the reputation of Georgia Tech has decreased more than little bit since the OMSCS program started.

<comment><USER19> Why so? Their CS program is now #8 in the world and CS research #1 (at least in one of those "proper British rankings").

</comment></comment></comment></comment><comment><USER20> This is the sort of thing which was starting to happen with software engineers way back in the 1990s as the Internet started growing explosively. It was stopped by the large tech companies engaging in illegal wage-fixing for years, setting the standard for software engineers being paid rates divorced from the value they create. Sure they got busted for it decades later, but by that point the danger was passed and astronomical profits assured.

<comment><USER21> I mean new grads still regularly make more than 180k at top tech companies...

<comment><USER22> Hmmm.... wonder if I could pass for a new grad. Would need to do something about this grey hair. And the wrinkles. And the attitude.

<comment><USER21> I mean, even lower-tier companies like Amazon pay that much for SDE2's these days. Its just a function of how much wages have gone up recently.

<comment><USER22> Ah well, I'm making ~$140k (salary) here in Florida. The Nerd Wallet COL calculator says that's equivalent to ~$260k in San Francisco (or $210k in Seattle), so maybe I'm doing alright :)

</comment></comment></comment></comment></comment><comment><USER23> As someone with a PhD in an engineering field (but currently working as a data scientist because all of my research was computational science), I wonder if independent machine learning publications would help with getting one of these jobs or if the CS PhD from a brand name school is an absolute requirement. I have 8 publications in my field, so I'm familiar with the peer review process, and it seems that with (a lot of) time permitting it might be possible to get a couple of NIPS presentations or JMLR papers. But I don't know if publications alone would be enough to get hired for these AI positions.

I had a hard time making the decision when I chose an engineering PhD over a CS one, but 6 years ago machine learning hadn't taken off like it has now, and engineering / hard science prospects seemed brighter at the time. If I had known it was going to become this big and this interesting, I definitely would have gone for CS instead.

<comment><USER24> _> but 6 years ago machine learning hadn't taken off... I definitely would have gone for CS instead._

I thnk you're doing it wrong because that's _exactly_ why there's so much demand for CS PhDs right now.

Rewind to mid-2000's when the CS postdocs/phds graduating over the past couple of years were choosing to major in CS. They were warned that everything was being outsourced to India and advised to choose a "real" engineering field, or perhaps finance/physics/math. It's hard to imagine today, but lot of smaller colleges/universities were killing CS majors back in the mid 00's!

So not only is there a shortage of CS PhDs in the pipeline, but the ones that made it through came in with a burning passion for the science (as opposed to the money/hotness). This combination of input bias and restricted supply is what makes the current labor market so damn hot.

In 3 years, that will invert, and some major struggling to justify its existence because it's hard but has "no future" will blow up. Rinse and repeat.

</comment></comment><comment><USER25> This is a PR piece put together by a company that offers AI services. Their pitch is that hiring for AI is incredibly expensive --- so you'd better outsource to us.

</comment><comment><USER26> > Designing AI systems requires a hard-to-come-by blend of high-level > mathematics and statistical understanding, a grounding in data science and > computer programming, _and a dose of intuition._

Doesn't intuition to resolve a domain specific problem require domain specific knowledge?

<comment><USER27> Not necessarily. Or more precisely, the intuition doesn't have to be in the domain itself.

I've done operational improvement work across supply chain/inventory management, marketing, digital analytics, ecommerce, healthcare revenue cycle management, and call center operations. In almost every case I started with little if any direct domain knowledge. The intuition that was valuable for my work was around systems-oriented thinking applied to business processes and being able to quickly suss out weak or suspiciously opaque areas of the system. The necessary domain knowledge to do so was always picked up from domain experts as I went along.

In fact, taking a naive approach on domain knowledge has always worked in my favor to uncover invalid assumptions that those with domain knowledge just accepted without question.

<comment><USER28> What kind of work has exposed you to so many areas; are you a data consultant (and if so do you work for yourself or a bigger shop?)

<comment><USER27> It was a bit of a winding process, but essentially early on in my career I learned that I fit this[1] description very well. And stumbled upon the same fact that patio11 did: it's incredibly hard for most companies to consistently fill that type of role. And now that I have a history of succeeding in those types of roles, it's a lot easier to talk my way into new ones.

That said, you're spot on that the core of what I do is data consulting, although most of it falls under a domain-specific name and has been W2, internal consulting roles. I'm actually in the process of switching my full time role to a less demanding one so I can focus on ramping up my actual consulting business.

<comment><USER27> Whoops, just realized I left off the reference and it's too late to edit. Here it is: [https://mobile.twitter.com/patio11/status/936616624378978304](https://mobile.twitter.com/patio11/status/936616624378978304)

</comment></comment></comment></comment></comment><comment><USER29> Any current/recently graduated CS students here who can confirm that students are required to take more mandatory Stats/Math classes in core CS curriculum instead of just electives?

"Designing AI systems requires a hard-to-come-by blend of high-level mathematics and statistical understanding, a grounding in data science and computer programming".

EDIT: Improved readability

<comment><USER30> At UChicago CS didn't have a mandatory core of statistics or math classes, but all the AI/ML classes had higher level statistics or math prerequisites.

<comment><USER4> _> At UChicago CS didn't have a mandatory core of statistics or math classes_

Not even Calculus or Linear Algebra? Do they take Discrete Math?

<comment><USER30> Two quarters (~1 semester) of Calculus is required, so a lot of integration is left out. Discrete Math is part of the CS curriculum, essentially as an introduction to proofs to prepare people for Algorithms [1]. Linear Algebra is a recommended prereq for some classes, but a lot of people don't take it because more fundamental algebra is covered in classes like Analysis.

[1] [http://collegecatalog.uchicago.edu/thecollege/computerscienc...](http://collegecatalog.uchicago.edu/thecollege/computerscience/)

</comment></comment></comment></comment><comment><USER31> > Even newly minted Ph.D.s in machine learning and data science can make more > than $300,000.

Wait, really? I'm just starting out as a research scientist and my pay is nowhere that high. Am I missing something?

<comment><USER32> They mean:

* PhD in deep learning / ML with papers at NIPS/ICML

* Working for BigCo (FAANG)

* Half of that is stock, annually.

* In the bay area or a big city

If you don't hit all those items then you pay is likely more modest. If you are academic, it is still likely laughable.

<comment><USER31> > Half of that is stock, annually

Ah ha. I read it as saying people getting 300k in cash. It makes more sense now. For me, I'll have to wait out several years until my RSU gets fully vested. Let's hope I don't get fired in the meantime :)

</comment></comment></comment><comment><USER33> You never get what you deserve.

You get what you negotiate.

</comment><comment><USER34> Is a Ph.D necessary for competing for those high salaries? It hasn't been for other CS jobs, and from my experience with ML and NLP - it seems wildly unnecessary. I'd pick experience over a degree every time.

<comment><USER35> Presumably the "fresh Phds" getting 300k right out of school have extremely relevant experience and are quite capable of actually putting their learning to use (I don't have data to prove this).

People who just know TF and couldn't implement it and know what it does and doesn't do well aren't the kind of people attracting 7 figures.

There's a huge amount of advanced statistics, math, and "intuition" that takes years working with data to build. Abstractions like TF (etc.etc.) make applying existing solutions to existing problems more tractable, but the real gold-rush is happening around the new/relatively-unsolved problems.

</comment></comment><comment><USER36> That's bad and good at the same time. If more PhDs work on it, the real capabilities and limitations will come to light leading to correction in salaries.

</comment><comment><USER22> Most of the Data "Scientists" I've met have been total BS artists. But I work in govt contracting, so... yeah.

</comment><comment><USER37> what is lecunn paid by FAIR? my roommate and I settled on 2-3 mil. thoughts?

</comment><comment><USER38> I am amazed that rather than see this as opportunity for racking up top $$$ for _at least_ the next 3-10 years by learning Calculus, Probability/Statistics, and Linear Algebra on the way to whatever degree one seeks, there is so much criticism of this.

This situation is a pure win for smart people who put their minds to the task at hand. Buzzwordy or not, AI/ML/Newfangled Regression has more than enough wins with speech recognition, game-playing, image recognition, and recommendations to have a strong future.

Or, if you insist on negativity and you prefer to expend your time kvetching about whether Famous CS Person X or Famous CS Person Y should make the most money (Fantasy Data Science?), then more for me getting $h!+ done whilst you bicker. I mean I scratch my head that Mark Wahlberg is the highest paid actor in Hollywood, but hey, good for him in my book. Too bad his burger shop is crap.

<comment><USER39> I suspect that calculus, statistics, and linear algebra are required for most 4-year CS degrees. Some schools even have an "AI" track for their undergrad degrees. This article specifically refers to PhDs in Machine Learning, which is much higher bar.

If you do research in the same area that Google's research department is throughout your graduate degree, net research internships at _insert fancy company with AI component here_ every summer, and get a PhD, $300k seems completely reasonable.

Personally, I think we're in a SaaS bubble and an AI bubble. "AI" isn't nearly as developed or useful as the amount of VC money being dumped into it.

<comment><USER40> You could also just become a plumber and probably make $300k a year[i] (~150$/hour). Engineers are incredibly undervalued for all the effort they put in their education (and _containing_ education). A plumber could pull in at least $150 per hour in the bay area. I just hate the headlines _Sky-High_ which in the rest of the professions is _normal_. Try finding an (below average english major) attorney at $150 per hour.

[i] [https://www.homeadvisor.com/cost/plumbing/](https://www.homeadvisor.com/cost/plumbing/)

<comment><USER41> $150/hr is the maximum of the range they give, and then you have to subtract payroll taxes, equipment, marketing, etc. On top of that, it's not like plumbers are working jobs back to back 40hrs a week.

<comment><USER40> Yes. They get to write off equipment, marketing, etc. "It's not like plumbers are working jobs back to back 40hrs a week." In California probably more like 60hrs.

<comment><USER42> Yup, the closer analogy is consulting but with the massive capital investment of equipment, vehicle + professional insurance.

I'd be surprised if the take-home on $150/hr is even close to $60/hr.

</comment></comment></comment></comment></comment></comment><comment><USER43> I wonder if the algorithms can determine a fair wage for the said talent. Like many have observed here, the salary is for an expectation of innovation and differentiation. However, an a-priori guarantee, or even a relatively high likelihood of such an outcome is hard to guarantees unless the said person is Geoffrey Hinton. So what exactly are the salaries for?

<comment><USER16> There is no such thing as a "fair" salary, especially when you're talking about the differences between $300k, $750k, and $2 million. There is what the market is paying similar people, what leverage you can use to negotiate higher pay, and how much [above|below] market your employer is willing to pay to keep you away from its competition, among other things.

The entirely idea that it's "fair" to pay someone $400k a year to do a job but "unfair" to pay them $350k to do the same job is silly.

</comment></comment><comment><USER44> What's the best way / best resources for someone with a full-time job to learn AI / machine learning on the side?

<comment><USER45> Check out online courses. Coursera, EDX, etc. Andrew Ng's intro to machine learning course is a nice way to get up to speed with some basic concepts without diving too far into math. It doesn't cover any state of the art machine learning concepts, though.

Some other resources I have bookmarked:

\- Convolutional Neural Networks for Visual Recognition Youtube playlist [1]

\- Deep Learning for Self-Driving Cars [2]

\- Natural Language Processing with Deep Learning [3]

[1]: [https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-z...](https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj- zLfQRF3EO8sYv)

[2]: [https://selfdrivingcars.mit.edu/](https://selfdrivingcars.mit.edu/)

[3]: [https://www.youtube.com/playlist?list=PL3FW7Lu3i5Jsnh1rnUwq_...](https://www.youtube.com/playlist?list=PL3FW7Lu3i5Jsnh1rnUwq_TcylNr7EkRe6)</comment></comment>


Pets, Cattle Politically Incorrect? - <AUTHOR> https://github.com/kubernetes/kubernetes/pull/18016#issuecomment-161932499 <comment><USER2> I actually think that term "pet" alone is politically incorrect as it implies that an owner is somehow superior. I prefer to call my non-human roommates "companions"!</comment>


FizzBuzz, minus the modulo operator, plus grit - <AUTHOR> http://www.globalnerdy.com/2020/06/17/programmer-interview-challenge-2-part-3-fizzbuzz-minus-the-modulo-operator-plus-grit/ <comment><AUTHOR> FWIW, I prefer the "grit" solutions. No modulo, no faking/implementing modulo. Why?

Because the original child's game isn't about "If a number is divisible by X," it's about "Every Xth number." And directly implementing "every Xth" can be interesting.

</comment><comment> java-man This is all good, until someone in Product changes the requirements. Now instead of 3 and 5 they need 7 and 11.

All these clever tricks will require a complete re-write and re-test. And since the company decided to pick theit employees based on how they answer these tricky interview questions, the code base is full of opaque idiosyncrasies with unexpected dependencies. Four sprints later, the code is put into production and some other part breaks, causing the company millions.

<comment><AUTHOR> The pleasure of solving eccentric problems is, of course, a completely separate thing from the question of, "What is the efficacy of asking job candidates to solve well-known programming puzzles?"

When nobody had heard of fizzbuzz, it had a certain value for screening out complete no-hopers when no other signal was available. There were other methods for achieving such a goal, but it had _some_ utility.

Today, using it for job screening it acts like a membership test. "Hey! Are you a member of the programming culture? Have you scoured the internet for examples of problems posed in job interviews? Prove it! Write fizzbuzz!"

<comment><AUTHOR> There's another thing about your comment. I suggest you reflect on this:

If you believe that someone who solves fizzbuzz in an idiosyncratic way would also do everything at work in an idiosyncratic way, you are essentially supporting the value of asking programmers to solve programming problems in job interviews.

I have a different belief. I have written a blog post on the subject, but the TL;DR is that the purpose of this kind of test is strictly to get rid of impostors whoc an't do any programming whatsoever.

The moment you are convinced that they are not a complete no-hoper, you can stop the test and get on with teh real interview.

As such, I believe it is harmful to take anything other than a single bit of information from it: Can/Can't.

If you actually study how they got it to work, and make extrapolations about them from it, you are incentivizing people to write millions of units tests, ask you whether there is a roadmap for future changes, set up a repo on github and submit their solution as a PR, &c. &c.

If that's what you want to know about a candidate, fizzbuzz is a ridiculously inadequate test.</comment></comment></comment>


Wayback Machine Hits 400,000,000,000 - <AUTHOR> http://blog.archive.org/2014/05/09/wayback-machine-hits-400000000000/ <comment><USER2> And not one of those hits is from Quora due to their robots.txt:

[https://web.archive.org/web/http://www.quora.com/](https://web.archive.org/web/http://www.quora.com/)

Good job Quora, preserving all that crowd sourced content away from the crowd, keeping it from everyone not logged in. Hats off to getting into YC so you can post your job openings on the HN home page and get some press. This doesn't add anything to your image though, it just takes a little away from YC.

On another better note, a great big thank you to the wayback machine for all of the public good it does. Now there's an organization that is amazing and wonderful and enriching our lives in an open and honest way with information.

<comment><USER3> Man. I hate on Quora. Didn't realize they were YC. Shame.

<comment><USER4> And why would you do that ? I can see why the must-register roadbloack is annoying in theory but It takes a few minutes to register, and then its an absolute gold-mine of knowledge, with an extra-ordinary community, which is very positive, constructive and helpful. I find it way more engaging than HN Actually to be honest.

<comment><USER5> I've found the Quora community to be great, but the register-walling and the robots.txt is very close minded, especially to the Internet as whole. Most of the time I find a link to a Quora I am turned down by the fact I can't read more than half an answer without having to sign in. Imagine if Stack Overflow did the same thing? They'd be dead.

<comment><USER6> Stack Overflow was created in response to a site that was doing something similar to what Quora is doing with search engines, except that site required a paid membership to see the answers. I recall one trick that people used before SO was around, which was to spoof the User-Agent to be GoogleBot. Doing so would show you the answer(s) to the question. :)

<comment><USER7> I thought Google specifically penalised sites which did that?

<comment><USER8> DISCLAIMER: I'm one of Experts Exchange's volunteer administrators.

Google penalized sites that redirected from SERPs or buried content below a lot of "sign up here" stuff very heavily; you can even make the case that it could have been called the "EE Penalty" instead of Panda, if only because Google's web search team (i.e. Matt Cutts) collaborated heavily with the Stack ownership in developing the algorithm changes and consequences.

That's not to excuse EE's management's behavior -- quite the contrary. From the perspective of longtime users, EE's string of decisions, the consequences of those decisions, and its reactions to those consequences starting not quite a decade ago nearly destroyed what had been a vibrant community.

If there's anything good to have come of it, it's that EE has finally moved -- about six or seven months ago -- to a business model that allows the non- member to see what members see: the entire question and solutions. Joining for free does have a few minor advantages; paying (either with a credit card or by answering questions) has more.

But it took a long time for EE to learn those lessons and begin to implement those fixes. Whether Quora will learn them is a whole 'nother story; like so many other sites, it built its systems and market-share without much thought given to how it was going to monetize.

EE made that mistake too, in 1997. Quora has a lot of money backing it up, so it can maintain its facade for a long time... but I wouldn't be placing any bets on it being around 15 years from now if it actually had to depend on income.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> PSA/ranty thing: Just because something is archived in the Wayback machine, do not trust that archive.org will keep it there for all time. If you need something, make a local copy! A few months ago TIA changed their stance on robots.txt. They now _retroactively_ honor robot blocks. Now any site can completely vanish from the archives.

Let's say I died tomorrow. My family lets my domain slip. A squatter buys it and throws up a stock landing page, with a robots.txt that forbids spidering. TIA would delete my entire site from their index.

I've already lost a few good sites to this sort of thing. If you depend on a resource, archive it yourself.

edit - Official policy: [https://archive.org/about/exclude.php](https://archive.org/about/exclude.php)

If I am reading it properly, once blocked they never check later in case of a change of heart? No procedure for getting re-indexed at all?

<comment><USER10> A fair rant, but to correct some misperceptions:

The retroactive application of robots.txt is not a new policy; it's been in place for at least 11 years, and I believe it arrived very soon after the Wayback Machine was first unveiled.

An updated robots.txt does not irreversibly delete prior captures, so if the robots.txt changes again, access to previously-collected material can be re- enabled.

This policy has served to minimize risk and automate the most common removal- scenario, when a webmaster wants a total opt-out of current crawling and past display. But, the collateral damage to unrelated content from prior domain- owners has grown as the web has aged, and more domains have changed hands. (The tradeoff that made sense in 2002 probably doesn't make sense in 2014.)

Figuring out a system that can automate most legitimate exclusions, while rejecting or reversing those that lack a firm basis in content ownership or personal privacy, is a thorny task, but it would be worth pursuing if/when the Wayback Machine has the necessary staff resources.

(My proposal since 2008 has been a DMCA-inspired 'put-back' procedure, where an original content owner can assert, formally, that they are the content owner and do _not_ want the current-day robots.txt applied to captures before a certain date. Then, the current domain-owner would have to counter-notify _that_ to maintain the block. This idea hasn't had legal review, but would reverse some current damage, and any bad-faith blockers would have to go on record with a false claim to maintain the block, potentially exposing them to a third-party legal challenge, with minimal risk to IA.)

<comment><USER9> Whoops, I probably should have checked the exclude.php page in the Wayback Machine before pinning a date on it. My bad.

> An updated robots.txt does not irreversibly delete prior captures, so if the > robots.txt changes again, access to previously-collected material can be re- > enabled.

Is there any official statement you can cite? Generally TIA does not mince words and is honest. If they say "removed" then I would assume they are not doing Facebook-style "deletion" shenanigans.

<comment><USER10> I don't know of a linkable official statement to that effect. Unfortunately much of the information available (for example in the official onsite FAQ) is incomplete or outdated. The Wayback Machine gets only a tiny fraction of the product-management, public communication, documentation, end-user support, and fundraising that it should.

</comment></comment></comment></comment><comment><USER3> I'd like to rave about an underappreciated but absolutely brilliant piece of the Internet Archive's infrastructure: its book reader (called, I gather, "BookReader").

TIA includes copious media archives including video, audio, and books. The latter are based on full-image scans and can be read online.

I generally dislike full-format reading tools: Adobe Acrobat, xpdf, evince, and other PDF readers all have various frustrations. Google's own online book reader is a mass of Web and UI frustrations.

I'm a guy who almost _always_ prefers local to Web-based apps.

TIA's book reader is the best I've seen anywhere, hands down.

It's fast, it's responsive. The UI gets out of the way. Find your text and hit "fullscreen". Hit 'F11' on your browser to maximize it, you can then dismiss the (subtle) UI controls off the page and you are now ... reading your book. Just the book. No additional crap.

Page turn is fast. Zoomed, the view seems to autocrop to the significant text on the page. _Unlike every last damned desktop client, the book remains positioned on the screen in the same position as you navigate forward or backward through the book. Evince, by contrast, will turn a page_ and then position it with the top left corner aligned. You've got to. Reposition. Every. Damned. Page. Drives me insane (but hey, it's a short trip).

You can seek rapidly through the text with the bottom slider navigation.

About the only additions I could think of would be some sort of temporary bookmark or ability to flip rapidly between sections of a book (I prefer reading and following up on footnotes and references, this often requires skipping between sections of a text).

Screenshot: [http://i.imgur.com/Reg8KLB.png](http://i.imgur.com/Reg8KLB.png)

Source: [http://archive.org/stream/industrialrevol00toyngoog#page/n6/...](http://archive.org/stream/industrialrevol00toyngoog#page/n6/mode/2up)

But, for whomever at TIA was responsible for this, thank you. From a grumpy old man who finds far too much online to be grumpy about, this is really a delight.

This appears to be an informational page with more links (including sources):

[https://openlibrary.org/dev/docs/bookreader](https://openlibrary.org/dev/docs/bookreader)

<comment><USER11> Wow, I had no idea. If you'd have asked me where to read older texts I'd have said "Project Gutenberg."

This is _miles_ better.

<comment><USER3> Gutenberg is also an absolute treasure, though its formats tend to be rather spartan. When you consider that it started on mainframes (and with founder Michael S. Hart hand-typing in the earliest works), and that for reasons of compatibility it has long standardized on flat ASCII text as a storage format (I'm not sure if this remains the case: ISO-8859-1 is now supported and many works are available in HTML, PDF, and ePub versions, though I understand ASCII remains the reference).

The wealth of material at Gutenberg is pretty staggering.

But yeah, TIA's BookReader is really nice.

If you go to the project page and view some of the sample works (from read.gov and the Internet Archive itself) you'll find some more colorful examples than my boring focus on 19th century economic heterdoxies ;-)

</comment></comment></comment><comment><USER12> Has there been any High Scalability articles on their infrastructure? We have a similar need: storing a large volume of text-based content over a period of time, with versioning as well. On top of it we have various metadata. We're currently storing everything in MySQL -- a lightweight metadata row and a separate table for the large (~400KB on average) BLOB fields in a compressed table.

We're looking at ways to improve our architecture: simply bigger+faster hardware? Riak with LevelDB as a backend? Filesystem storing with database for the metadata? We even considered using version control such as git or hg but that proved to be far too slow for reads compared to a PK database row lookup.

Any HN'ers have suggestions?

<comment><USER13> I am a former Archive employee. I can't speak to their current infrastructure (though more of it is open source now - [http://archive- access.sourceforge.net/projects/wayback/](http://archive- access.sourceforge.net/projects/wayback/) ), but as far as the wayback machine, there was no SQL database anywhere in it. For the purposes of making the wayback machine go:

\- Archived data was in ARC file format (predecessor to [http://en.wikipedia.org/wiki/Web_ARChive](http://en.wikipedia.org/wiki/Web_ARChive)) which is essentially a concatenation of seperately gzipped records. That is, you can seek to a particular offset and start decompressing a record. Thus you could get at any archived web page with a triple (server, filename, file- offset) Thus it was spread across a lot of commodity grade machines.

\- An sorted index of all the content was built that would let you lookup (url) and give you a list of times or (url, time) to (filename, file-offset). It was implemented by building a sorted text file (first sorted on the url, second on the time) and sharded across many machines by simply splitting it into N roughly equal sizes. Binary search across a sorted text file is surprisingly fast -- in part because the first few points you look at in the file remain cached in RAM, since you hit them frequently.

\- (Here's where I'm a little rusty) The web frontend would get a request, query the appropriate index machine. Then it would use a little mechanism (network broadcast maybe?) to find out what server that (unique) filename was on, then it would request the particular record from that server.

(Edit: FYI, my knowledge is 5 years old now. I know they've done some things to keep the index more current than they did back then.)

At the very least, I'd think about getting your blobs out of MySQL and putting them in the filesystem. Filesystems are good at this stuff. You can certainly do something as simple as a SHA-1 hash of the content as the filename, and then depending on your filesystem's performance characteristics, you can have a couple levels in the tree you store them in. e.g. da39a3ee5e6b4b0d3255bfef95601890afd80709 goes into the directory da/39/ Then you stick da39a3ee5e6b4b0d3255bfef95601890afd80709 into the 'pointer' field in your table that replaces the actual data. Obviously this design assumes the content of _that_ file doesn't change. If you want to change the data for that row in the table, you have to write a new file in the filesystem and update the 'pointer'.

<comment><USER14> Thanks! We were writing up a response at the same time:

The Wayback Machine data is stored in WARC or ARC files[0] which are written at web crawl time by the Heritrix crawler[1] (or other crawlers) and stored as regular files in the archive.org storage cluster.

Playback is accomplished by binary searching a 2-level index of pointers into the WARC data. The second level of this index is a 20TB compressed sorted list of (url, date, pointer) tuples called CDX records[2]. The first level fits in core, and is a 13GB sorted list of every 3000th entry in the CDX index, with a pointer to larger CDX block.

Index lookup works by binary searching the first level list stored in core, then HTTP range-request loading the appropriate second-level blocks from the CDX index. Finally, web page data is loaded by range-requesting WARC data pointed to by the CDX records. Before final output, link re-writing and other transforms are applied to make playback work correctly in the browser.

The server stack:

\- frontend: Tengine + HAProxy to a pool of Wayback tomcat app servers[3]

\- backend: The redis-backed archive.org metadata API[4] for object location and nginx on linux (via ext4) for data service

      [0] http://en.wikipedia.org/wiki/Web_ARChive
      [1] https://github.com/internetarchive/heritrix3
      [2] https://github.com/internetarchive/CDX-Writer
      [3] https://github.com/internetarchive/wayback
      [4] http://blog.archive.org/2013/07/04/metadata-api/

-sam and raj, Internet Archive

<comment><USER15> Why not use hashtable instead of binary search? I'm assuming your index is immutable and query for the data structure is essentially random. Another advantage of looking up item by URL hash may be that you can use hash-prefix to direct the query to appropriate machine (so basically, your infrastructure simply looks like giant distributed hashtable top to bottom with no binary searches required).

<comment><USER10> Former Archive employee (& still occasional contract contributor) here. This was one of my 1st questions when joining in 2003!

Some Wayback Machine queries require sorted key traversal: listing all dates for which captures of an URL are available, the discovery of the _nearest- date_ for an URL, and listing all available URLs beginning with a certain URL- prefix.

Maintaining the canonically-ordered master index of (URL, date, pointer) – that 20TB second-level index rajbot mentions – allows both kinds of queries to be satisfied. And once you've got that artifact, the individual capture lookups can be satisfied fairly efficiently, too. (A distributed-hashtable would then be something extra to maintain.)

Also, the queries _aren 't_ random: there are hot ranges, and even a single user's session begins with a range query (all dates for an URL), then visits one URL from that same range. Then loading nearest-date captures for the page's inline resources starts hitting similar ranges, as do followup clicks on outlinks or nearby dates. So even though the master index is still on spinning disk (unless there was a recent big SSD upgrade that escaped my notice), the ranges-being-browsed wind up in main-memory caches quite often.

There's no doubt many places that could be improved, but this basic sorted- index model has fit the application well for a long while, avoided too much domain-specific complexity, and been amenable to many generations of index/sharding/replication/internal-API tweaks.

BTW, the Archive is hiring for multiple technical roles, including a senior role developing a next-generation of the Wayback Machine: [https://archive.org/about/jobs.php](https://archive.org/about/jobs.php)

</comment></comment></comment></comment></comment><comment><USER16> If you're looking to donate, they take bitcoin too! [https://archive.org/donate/index.php](https://archive.org/donate/index.php)

</comment><comment><USER17> A little known fact is that there is a mirror of the Wayback Machine hosted by [The Bibliotheca Alexandrina:

[http://www.bibalex.org/isis/frontend/archive/archive_web.asp...](http://www.bibalex.org/isis/frontend/archive/archive_web.aspx)

I have sometimes had luck retrieving pages from this mirror that were unavailable (or returned errors) in the main site.

</comment><comment><USER18> awesome! it's a great tool to go back in time to check out our past websites full of blinking gifs and whatnot.

I didn't know that they also maintain the "HTTP Archive", showing website latency over time as well as some interesting live-statistics: [http://httparchive.org/](http://httparchive.org/)

<comment><USER19> As far as I know, Internet Archive does not maintain "HTTP Archive" ([http://httparchive.org/](http://httparchive.org/)). HTTP Archive was founded by and is being maintained by Steve Souders (Chief Performance Officer at Fastly - [http://www.fastly.com/](http://www.fastly.com/)). He's previously held titles at both Google ("Head Performance Engineer") and Yahoo!. He's also a co-founder of the popular web-development debug add-on Firebug.

Sources: [http://httparchive.org/about.php](http://httparchive.org/about.php) and [http://stevesouders.com/about.php](http://stevesouders.com/about.php)

<comment><USER18> from OPs linked article:

      June 15th, 2011 – The HTTP Archive becomes part of the Internet Archive [0], adding data about the performance of websites to our collection of web site content.

[0] [https://blog.archive.org/2011/06/15/http-archive-joins- with-...](https://blog.archive.org/2011/06/15/http-archive-joins-with- internet-archive/)

<comment><USER19> Oh schucks, I completely missed that! Thanks for linking to it! It doesn't hurt to be wrong occationally, then you get to learn something.. :-)

</comment></comment></comment></comment><comment><USER20> Can anyone explain to me how displaying those sites on demand is not copyright infringement? I'm seriously curious, I don't know much about copyright laws.

<comment><USER21> Not a lawyer, but I guess this probably falls under fair use, which includes (from the Wikipedia article) both search engine use, and library archiving of content. [http://en.wikipedia.org/wiki/Fair_use](http://en.wikipedia.org/wiki/Fair_use)

<comment><USER22> The Archive is also legally a library, which allows them to get away with a lot of things that companies cannot.

<comment><USER23> Also, doesn't the IA have specific privileges given it by Congressional legislation at some point? I seem to recall that coming up in abandonware discussions.

<comment><USER24> [https://archive.org/about/dmca.php](https://archive.org/about/dmca.php) \- it looks like the exemptions applied to everyone, for 3 years at least.

</comment></comment></comment></comment></comment><comment><USER25> I still cannot fathom how they are able to store huge amounts of data and not run out of space. Care anyone to explain?

<comment><USER26> From a conversation with Brewster a few years ago: The doubling of density of disk drives has allowed them to stay relatively neutral with respect to space for the Wayback machine. It _still_ occupies approximately the same size as it has for the last 10 years, which is essentially a set of racks about 15-20 feet long altogether I think?

However, the new TV news and search capability requires substantially more space than even the archive IIRC, or certainly is heading that way.

</comment></comment><comment><USER27> Funny story about the Wayback Machine and how it helped me. I had let my blog go into disrepair for a couple months, and eventually, when I went back to it, I found that since I hadn't kept up with security updates, I wasn't able to access any of my old posts.

When I went back to start writing again (this time using paid hosting so I didn't have to deal with that), I was disappointed that I wasn't going to have ~20-30 posts I had before. On a hunch, I checked the Wayback Machine and found that it had archived about 15 of my posts! Very excited that I could restore some of my previous writings.

</comment><comment><USER28> > Before there was Borat, there was Mahir Cagri. This site and the track it > inspired on mp3.com created quite a stir in the IDM world, with people > claiming that "Mahir Cagri" was Turkish for "Effects Twin" and that the > whole thing was an elaborate ruse by Richard D. James (Aphex Twin). > (Captured December 29, 2004 and December 7, 2000)

Okay this just blew my mind. Anyone else follow Aphex Twin's various shenanigans? Was this ever investigated further?

</comment><comment><USER29> Nice work on this over the years, gojomo et al.

</comment><comment><USER30> Cool, but on a lot of sites (including some of my own, from 10+ years ago to recently) it doesn't get hardly any of the images. Am I the only one experiencing this?

<comment><USER31> If you're using AWS S3 check your resource policy

</comment></comment><comment><USER32> Wow, that's one billion more pages than there are stars in our Milky Way galaxy. That's a lot!</comment>


Starry, Starry, Starry Night - <AUTHOR> http://www.nytimes.com/interactive/2013/02/03/magazine/look-stars.html

<comment><USER2> "WHEN I heard the learn'd astronomer; When the proofs, the figures, were ranged in columns before me; When I was shown the charts and the diagrams, to add, divide, and measure them; When I, sitting, heard the astronomer, where he lectured with much applause in the lecture-room, How soon, unaccountable, I became tired and sick; Till rising and gliding out, I wander'd off by myself, In the mystical moist night-air, and from time to time, Look'd up in perfect silence at the stars." - Walt Whitman

<comment><USER3> I never related to that poem until I read the Zen Pencils version: [http://zenpencils.com/comic/88-walt-whitman-when-i-heard- the...](http://zenpencils.com/comic/88-walt-whitman-when-i-heard-the-learnd- astronomer/)

</comment></comment><comment><USER4> this is a bit misleading. i've worked at major observatories - the best places to see stars in the world - and the sky doesn't look like that to the naked eye. the star images are over-exposed compared to what you would actually experience. yes, cities have light (and other) pollution, but this is over- stating the case.

</comment><comment><USER5> I think those photographs reveal how frail human creation really is.

<comment><USER6> Don't forget that these photographs _are_ a human creation.

</comment></comment><comment><USER7> It's about time we started to recognise light pollution as just that, pollution.

</comment><comment><USER8> Is the gallery specially designed in such a way to avoid allowing "Save as" link?

Well, I've managed to get direct links by looking at data-original property in Firebug, so it doesn't look hiding it was a real reason... But why then? Why such a non-accessible script?

</comment><comment><USER9> Absolutely gorgeous. Though it's still a major city, this is one of the reasons I'm grateful to have moved to the suburbs of Seattle from downtown New York City.

<comment><USER10> From what my friends that live in the Seattle area say, you can rarely see such stars. I'm always like "Hey the moon looks really great tonight" or "I can see the ISS really well" or "Venus/Jupiter are huge tonight." The usual response from them is "It's too cloudy here :("

I'm sure things are more visible there overall than say NYC though.

</comment></comment><comment><USER3> I was an avid reader of astronomy books as a child growing up in a major city. I thought the pictures in those books were always taken with a telescope or otherwise enhanced. Then we went to visit my grandmother, who lived back in a "holler" in the Blue Ridge mountains. I walked outside after dark one evening, looked up, and was shocked to see the Milky Way with my naked eyes. I hadn't realized that was possible until then.

</comment><comment><USER11> Idea for business.. or umm.. money:

\- A campaign by low pollution lighting using a similar set of images, except this time, with low emissions lights on. There'd be a before and after to show the difference between bright lights and less bright lights.

</comment><comment><USER12> There probably exists some real photographs from power outages, that shows the night sky over a big city without much light pollution.

<comment><USER13> Unfortunately, the regular particulate pollution and heat haze over megacities are going to wreak havoc with your seeing pretty badly even in the absence of light pollution.

</comment></comment><comment><USER14> I would love some high resolution images of these.

</comment><comment><USER15> Fake. And ridiculously so.

You can only see around 2000 stars in total with no light pollution across the entire sky

Perhaps the photographer is not saying they are realistic, it might be nytimes being misleading.

<comment><USER16> Get out into the wilderness sometime where there is no light pollution and take the time to look up. While 2000 may be the make without optical assistance, there is something humbling and awe inspiring about how vast things are.

Simply stating "Fake", at least for me, downplays the point the photographer/artist is trying to make.

<comment><USER15> YES "there is something humbling and awe inspiring about how vast things are."

So why fake it. Why show pretend scenes with orders of magnitude more stars then there should be.

Why not just show real life and how the stars would look if Paris etc had the lights turned off.

Why show a universe where somehow stars at hundreds of times more than they are. Isn't ours good enough?

Anyone scientific/technical knows how stupid those pictures look, and anyone artistic has been away from city lights and knows what stars really look like.

Why break from a great idea and create something so fake looking. Why not make them realistic and make a point on light pollution and the beauty we are missing out on. Why can't we appreciate the real universe and how great it is.

Have we become that sad we need to make everything look like a scifi movie? At least have a couple of moons if you're going to go down that road.

<comment><USER17> Some of those can be seen with naked eye. Others are taken with longer exposure times (and using tracers (or multiple shots) to avoid this <http://www.twanight.org/newTWAN/photos/3003666.jpg> ). While we can call them fake, they are not fake in a sense that they don't capture the real thing.

<comment><USER15> There's probably 10,000 stars in the first photo that's supposed to be part of the night sky?

It implies if the lights go out you'd see around 100,000 stars.

In real life you can see 2000 in the total night sky away from lights.

How will a long exposure allow you to see more stars? Even in the jpg you link you can see there's obviously one to two thousand stars tops in it.

These pictures are from telescopes, the latter ones even have fake colors applied.

<comment><USER17> > There's probably 10,000 stars in the first photo that's supposed to be part > of the night sky?

The first (or fourth) photo seems to me like naked eye would see.

> In real life you can see 2000 in the total night sky away from lights.

You can see much more than that. Just the fact that you can see Milky Way can give it away (although you can't see individual stars, they are still stars)

> How will a long exposure allow you to see more stars?

The less bright ones, that you can't see with naked eye, will appear. Just take any regular camera, go out and take a 20s photo of the night sky. You'll see how much more stars you'll see.

Yes, some of the pictures are zoomed in quite a lot, and I don't think they are colored though.

<comment><USER15> Interesting I assumed the colours from star pictures are mostly faked / enhanced / thickened / filtered.

I'll have to look into it more.</comment></comment></comment></comment></comment></comment></comment>


Ask HN: Are there many opportunities for part-time remote work in software? - <AUTHOR> I'm currently employed by a very flexible small company as a developer. I am also starting a vegetable farm which is requiring more and more of my attention, so I need to limit my hours behind a computer to 20-25 hrs/week, and it has to be remote. My employer is great because they will let me do that, however I don't enjoy the work and I feel stagnated. I want to look for another employer but I don't ever see part-time job listings. Are there many part-time remote software jobs, or is it hard to find? <comment><USER2> Basically you need to negotiate for these. It's much easier if you've already done it at your current job, because you have proof it can work. And it's easier to negotiate with remote companies, because they're not in a "seat in butt is all that matters" mindset. Ideally you get a job offer, and _then_ tell them you want to work fewer hours.

Lowest I've gone is 28 hours: started as contractor, got hired as employee. Later jobs have been 32-35 hours.

I interviewed someone who has been doing this for 15 years with 4-day week (also never advertised): [https://codewithoutrules.com/2018/01/08/part-time- programmer...](https://codewithoutrules.com/2018/01/08/part-time-programmer/) and I'm working on book on how to negotiate a 3-day weekend (i.e. 4-day workweek - 20-25 hours is harder): [https://codewithoutrules.com/3dayweekend/](https://codewithoutrules.com/3dayweekend/)

</comment><comment><USER3> Imagine putting yourself in the employers shoes. You have two devs who applied, and you're going to have to vet and train them, as well as risk they're a bad hire. And the reward for one is half the other. Which dev do you take a chance on?

The only time they might agree to it is if they already know you, you possess a unique skill set, or they're budget constrained. Both are rare and the second will probably be a stingier employer.

I would stay with the same employer and focus on how the is a means to achieving your dream of a vegetable farm.

<comment><USER2> Why is budget constrained a problem? E.g. if you can go to a bootstrapped company and credibly say "I am valuable enough you can't afford me, but—maybe we can work something out" that seems worth trying.

<comment><USER3> It's only a problem because budget constrained means they don't have very much money, so in addition to wanting to pay for fewer hours they probably want to pay for less per hour.

There probably aren't a lot of shops that can't afford a full time dev, but can afford a nice rate of 60-90/hr.

</comment></comment></comment><comment><USER4> It's entirely possible to get this going. There are the occasional posting for one of these kinds of jobs on the remote job boards. But also you could try working with them for a while and then say you want to go down to part time.

It's a lot easier to do this once they're already aware of your value, not to mention that you might be able to turn them into a more freelance-type client as well.

<comment><USER2> It's much easier to reduce your hours at an existing job, yeah.

</comment></comment><comment><USER5> This is a pretty non-standard spec and has to be negotiated for and networked for.

The only places that routinely hire for 20-25 hours a week are small restaurants with razor thin margins that need to skirt health care regulations that force an employer to offer health care to anyone working 30 or more hours per week (above a certain headcount).

</comment><comment><USER6> You can probably find this on a contract basis.

</comment><comment><USER7> What is your skill sets?

<comment><AUTHOR> I'm a full-stack web application developer. Strongest technologies are React, Redux, Ruby on Rails, Java/JEE, JBoss, Postgres, MySQL, Unix administration. Proficient with lots of other technologies and devops tools.

<comment><USER2> When you're selling yourself, don't just focus on technologies.

E.g. if you're experienced enough to not just solve problems, but also identify problems in the first place, that's hugely valuable, but you need to make sure to convey that to potential employers. (Specifically I'm thinking of the difference between "Solver" and "Finder" in [https://rkoutnik.com/2016/04/21/implementers-solvers-and- fin...](https://rkoutnik.com/2016/04/21/implementers-solvers-and- finders.html)).</comment></comment></comment>


RunKeeper's Boston Marathon social media stunt - running 26 miles in an iPhone costume - <AUTHOR> http://www.xconomy.com/national/2009/04/17/runkeepers-mad-dash-to-the-marathon-finish-of-foot-injuries-viral-video-and-dressing-up-as-an-iphone/

<comment><USER2> I'm training for a marathon or two. I actually came across the RunKeeper app just last night while searching. Does anyone here use it? Any feedback? How is the distance tracker? And speed tracker?

My main concern really is keeping the iPhone charged up for the long runs (5hrs+). That's when an app like this would really help but if it'll kill my battery, I might as well get a typical running stopwatch.

<comment><AUTHOR> If you put RunKeeper in lock mode and you don't run the iPod, your iPhone should last you a whole marathon (unless you're exceptionally slow). In the last couple of releases, the company has focused a lot on power management, and I've found that it is tremendously improved over the 1.0 release.

</comment></comment><comment><USER3> Wouldn't even get a second look at Bay to Breakers: <http://images.google.com/images?q=bay+to+breakers></comment>


Show HN: Linux Diagnostic Tool - <AUTHOR> https://www.acksin.com/strum/ <comment><USER2> As ibizaman wrote, that JSON output is not appropriate for human consumption.

I don't think the issue is having a dump of everything at a moment from /proc is that useful, since it gives you a "snapshot" of what happened when you collected the info.

What probably would be better is to take the advice and tools mentioned by Netflix in their Linux Performance Analysis in 60 seconds[0] and make some generic script to just kick it all off in a screen or tmux to give a quick overview for the admin.

I've been thinking about doing this myself for a while, but haven't gotten around to it.

[0] [http://techblog.netflix.com/2015/11/linux-performance- analys...](http://techblog.netflix.com/2015/11/linux-performance-analysis- in-60s.html)

</comment><comment><USER3> Although it's great to have everything gathered in one place (and what's everything, eh?), the json output is clearly not human readable. A 3am debugging will be difficult with that. A curse interface is a requirement IMHO.</comment>


Have Open Office Hours invites gone out yet? - <AUTHOR> They said "if you're selected we'll reach out by January 15"; I'm assuming this also means that people who are not selected will be notified, but I haven't heard anything.  Before emailing them to ask if they're behind schedule or if I lost an email, I figured I might as well check with HN to see if anyone here has heard back. <comment><USER2> We didn't get an email either. But perhaps they don't notify people who are not selected?</comment>


Ask HN: how do you promote a niche socail network? - <AUTHOR>

I recently launched a niche social networking site (lovelyco.de), but I have little insight into how to successfully promote it. What is the best way to do this? <comment><USER2> -Do you have a blog? Starting writing. Share the contents via RSS, Twitter, FB etc.

-Do you have a twitter account? Start following relevant people, get involved in discussions, share your link (don't spam).

-Write to bloggers/journalists who might be interested in covering social networking sites (mashable?)

-Since it's niche, promote your site on every blog, forum, site related to the niche. Again, don't spam but inform others why its to their benefit to join your network.

-Adwords?

EDIT: The site wasnt loading when I tried, so I wrote all of that without visiting the site. But now that I've seen it, I'm not really sure how this is a "niche" social network at all. Confused.

-Some sort of content where existing members can win prizes for referring others?

<comment><AUTHOR> It's a site where you can share pieces of code you find interesting and elegant. The social part comes from the fact vote/karma system and the ability to discuss code samples. A combination of HN-like interface with the <http://99-bottles-of-beer.net/> (recently covered on HN). Basically I was reading blogs like the Daily WTF and found that there are far more sites that talk about crappy code than good code.

Thanks for the advice. This is great help.</comment></comment>