Stack Overflow: The Architecture – 2016 Edition - Nick-Craver http://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture-2016-edition/ <comment><AUTHOR> "During peak, we have about 500,000 concurrent websocket connections open. That's a lot of browsers. Fun fact: some of those browsers have been open for over 18 months. We're not sure why. Someone should go check if those developers are still alive."

It's impressive that SO held the connection open for 18 months! That's some seriously good uptime for a process that manages a TCP connection.

<comment><USER2> Strange, no wonder why some Browsers stats are stuck with old versions.

</comment></comment><comment><USER3> They have 11 IIS servers but interestingly he says this:

 _What do we need to run Stack Overflow? That hasn't changed much since 2013, but due to the optimizations and new hardware mentioned above, we're down to needing only 1 web server. We have unintentionally tested this, successfully, a few times. To be clear: I'm saying it works. I'm not saying it's a good idea. It's fun though, every time._

<comment><USER4> Interesting to consider how some companies doing much less work on their machines than SO need clusters of hundreds of servers, meanwhile they can serve the 57th site by global traffic (according to Alexa) from _one physical machine_ .

Sure there is other stuff backing that one, but the next time you hear someone talk about big clusters and hundreds or thousands of nodes, just take a moment to appreciate how much can be done with one rack of gear these days.

<comment><USER5> Granted it's a machine with 768 GB RAM.

<comment><USER6> It's only got 64GB if I read data center upgrade post from 2015 correctly.

"The new web tier has dual Intel 2687W v3 processors and 64GB of DDR4 memory. We re-used the same dual Intel 320 300GB SSDs for the OS RAID 1."

</comment></comment></comment></comment><comment><USER7> Wish more sites/companies/employees published this sort of information - really interesting stuff.

Are there any other examples of big (largely) Windows stacks? Stack Exchange is the only one I've seen discussed.

<comment><USER8> Todd Hoff's High Scalability [1] is a very nice blog to follow if you like this kind of information about real-world stacks. He's been writing it for years. (Caveat: It's a while since I followed it closely.)

[1] [http://highscalability.com/](http://highscalability.com/)

<comment><USER9> Stuff the Internet Says on Scalability is an excellent weekly article he does with plenty of great resources. Lots of stuff that's just reposted from HN, but also many pieces on real-world architectures that I never see on HN or /r/programming.

</comment></comment></comment><comment><USER10> > Fun fact: some of those browsers have been open for over 18 months. We're > not sure why. Someone should go check if those developers are still alive.

How is that actually possible to keep a single TCP/IP connection open over 18 months?!

<comment><USER11> Someone probably opened up a browser on a server to look something up and then left it running. With a static IP and battery backup, only a reboot is likely to take it down. The connection should be able to survive short outages due to a firewall restart or momentary problem upstream, especially if there's no NAT in between.

I'm using SSH's ControlMaster in persistent mode on my desktop at work. The connections stay open for as long as the desktop is up, which can be months at a time.

</comment></comment><comment><USER12> So it's 125 million SQL queries or 6 million page loads per machine per day.

It also seems like average query is 1.2ms - am I missing something?

</comment><comment><USER13> Why the Windows stack though?

<comment><USER14> Jeff Atwood and Joel Spolsky have their roots in the Windows community, so they used what they were familiar with.</comment></comment>


Non-Euclidian Geometry Explained [video] - <AUTHOR> https://youtu.be/zQo_S3yNa2w <comment><USER2> This is amazing!

This video compares and contrasts three different types of Geometries:

o Spherical Geometry

o Euclidian Geometry (the one we all know)

o Hyperbolic Geometry

With the wonderful property that

 _Spherical Geometry is sort of the opposite of Hyperbolic Geometry_...

(Note to self, have to do more research on this!), i.e.,

Circumference of a circle:

Spherical: 2 x pi x sin(r)

Euclidian: 2 x pi x r

Hyperbolic: 2 x pi x sinh(r)</comment>


New 3D technology without glasses (what could possibly go wrong?) - <AUTHOR> http://www.youtube.com/watch?v=Uef17zOCDb8

<comment><USER2> Nice. Looks good.

But that's not how the facial nerve works. That wouldn't even be the right location to trigger the eye lids. This could possibly work if he put the stimulants on his eye lids. But even then other parts of his face would gringe.

No way to get that kind of specific control over one part of the face without surgery. Certainly not by just sticking something at your temple.

</comment><comment><AUTHOR> I cringe just watching him.</comment>


Twitter basically just admitted it doesn't understand its own policies - <AUTHOR> http://mashable.com/2017/12/01/twitter-doesnt-understand-its-own-policies/?utm_cid=mash-com-Tw-main-link <comment><USER2> Twitter still being unprofitable cannot afford to loose Trump from it's network so whatever he says on twitter is basically gonna get a pass from them. This is really sad. I think twitter should just scrap it's arbitrary posting guidelines and allow all the content. In few years Twitter is gonna degenerate into network of trash talkers and will loose many of it's sane users.

<comment><AUTHOR> But as an advertising-based media company, it's terrible business strategy to keep Trump, because major commercial advertisers - the kind that buys millions of dollars in ads, not the mom-and-pop social media outlets - actually have very specific and written standards on content that they can associate with. I've dealt with these organizations and their advertising standards.

Do you think Calvin Klein or Chanel wants their beautiful fashion ads next to an ISIS beheading video posted by Donald Trump?

Exactly.

Twitter really needs to learn how to control their content lik a professional media company would.

If I were Jack, I'd hire a strict "Editor-in-chief", and just set policy that states "don't piss off our editor-in-chief".</comment></comment>


Ask HN: What color is the dress? - <AUTHOR>

Does anyone have any science to explain the phenomena? <comment><USER2> You mean besides the 46 comments in [https://news.ycombinator.com/item?id=9117137](https://news.ycombinator.com/item?id=9117137) which was posted 12 hours ago concerning a Wired article titled "The Science of Why No One Agrees on the Color of This Dress"?</comment>


Heading to LA for next week, any open doors? - <AUTHOR>

Hey I'm flying out to LA tomorrow and I'll be there all next week visiting with my sister. She won't be able to take off work for the whole week I'm there so I'll have some time to myself to do whatever. I've seen these threads pop up now and again on HN and figured I'd take a shot at it.
My friend and I have been working on something here in Michigan for nearly a year now that we anticipate releasing in another couple months. We have a lot to teach ourselves coming up and I was hoping an established startup would open their doors for me to check out the culture and maybe learn a thing or two. So if this is something you want to make happen I can be reached at email@example.com
Thanks!
- Justen <comment><AUTHOR> Just wanted to specify I'll be in LA from 6/28 to 7/5</comment>


Five guilty pleasures only developers know - <AUTHOR> https://medium.com/@BenSassiAladin/five-guilty-pleasures-only-developers-know-395689b63f81 <comment><USER2> > I wrote my best lines of code high, drunk, and listening to Led Zeppelin.

I just don't find writing code a "write drunk, edit sober" task. It's just not an art in that way.

</comment><comment><USER3> Author forgot self-study time.

It's a great day when you get an hour or two to increase your knowledge of work-related technical stuff.

<comment><USER4> Oh this is soo true. I find that I can get home exhausted after a day full of dreary tasks and as soon as I hit some personal study material or a side project, I am fully charged up in an hour!

</comment></comment><comment><USER5> Emacs.

<comment><USER6> Vim

<comment><USER7> MS Word

<comment><USER8> grep

<comment><USER9> butterflies

<comment><USER10> M-x butterfly ?</comment></comment></comment></comment></comment></comment>


Ask HN:  If I died tomorrow - who gets my Google Adsense account? - <AUTHOR>

Any ideas? How does one put this in a 'Will' or how would one get this information from Google?  Just curious... <comment><USER2> Legally it probably goes along with any other assets.

Practically, since Google doesn't provide customer support they'll just lock the account, refuse to explain why, and respond to every inquiry with a FAQ about how to reset your password.

<comment><USER3> Even more practically, as long as your (estate's) bank account keeps accepting money, Google will keep putting money in to it.

</comment></comment><comment><USER4> Pete, all your adsense belongs to me.

</comment><comment><USER5> you could put your password in your will</comment>


Whitepixel v2 now brute forcing 33.1 billion password/sec - <AUTHOR> http://blog.zorinaq.com/?e=43

<comment><USER2> I suppose that means if someone managed to get their hands on a database with MD5 hashed password and we disregard the time it takes to query the harddrive for a match; it could (stupidly) brute-force it's way to any 5 character password consisting of a-Z, 0-9 and !"#€%&/()=?+ in about 0.06 seconds. Or a 6 character password just under a minute.

That's pretty impressive.

<comment><USER3> Then again, anyone who works on md5 password "cracking" for a living would likely already have the set of all 5 character printed passwords hashed and sitting in a database ready for recall with no need to speedily crank through them.

<comment><USER4> To understand why your comment does not make sense, please read this:

<http://en.wikipedia.org/wiki/Salt_%28cryptography%29>

<comment><USER5> There have been an embarrassing number of cases where an application has used the same default salt on all passwords.

</comment></comment></comment></comment><comment><USER6> This is very impressive.

Think of all the BitCoins they could create with a setup like that (I get an average of 50 coins every 26 minutes with <http://www.alloscomp.com/bitcoin/calculator.php>).

At a current market value of about USD$0.22 per coin, this equates to about USD$25/hour.

Of course, BitCoin uses SHA256, but how much different from MD5 can that be?

<comment><USER7> Hm, there is a market value for bitcoins? 25$/hour sounds like a very good deal - as long as power consumption is less, it is a license to print money?

Maybe this is the final incentive for me to look into EC2...

<comment><USER3> I'm not a bitcoins expert - but wouldn't any jump in the availability of bitcoins necessarily cause deflation, as with any currency?

If someone finds a way to generate,say, 5x more bitcoins that what the average expected amount is - they have a temporary advantage, but once those coins hit the market, the market is diluted and will quickly catch up.

Bitcoins has replaced mining gold with generating bitcoins - so it will be harnessing server infrastructure and good code / math -vs- digging big holes in the ground.

<comment><USER8> That's true. Also, as more computing power is added to its network, the difficulty of the hashing problem is increased to maintain the bitcoin creation rate at about 50 per ten minutes.

</comment></comment></comment></comment><comment><USER3> Let's stop calling it brute-forcing - it's really not - or at least not calling it decrypting.

<comment><USER9> Here are the usage instructions...sure sounds like brute force to me. Perhaps you could explain why it isn't?

        Usage: ./whitepixel [OPTION]... [<hash>]

        Main arguments:

          -c <charset>    Generate candidate passwords using the specified charset
         which can be:
                            lower  Lowercase
                            upper  Uppercase
                            digit  Digits
                            print  Printable ASCII characters and space [0x20-0x7e]
                            all    All bytes [0x00-0xff]
                          (default: lower)
          -l <length>     Attack passwords of this length (default 5)
          <hash>          MD5 hash to attack (default 00...00)

</comment></comment><comment><USER10> What is the minimum safest length when using MD5? I remember seeing 14 somewhere but I cant find the citation. Not forcing everyone to use 14 chars of course but adding salt.

<comment><USER11> Don't use MD5.

This only gets faster, not slower. No matter how much you salt your passwords, soon they will be able to crack your passwords fast enough. And soon there will be rainbow tables of salts.

<comment><USER12> That's just not true. If your salt is 1000 characters long, they will need more energy than the whole universe has to bruteforce it.

<comment><USER13> the point of adding a salt to the passwords is not to make decrypting the individual passwords harder - the salts are kept in plaintext - but to make it impossible to prevent the use of rainbow tables. that is, having longer salts will only make it harder for finding out all the people with the same password.

<comment><USER12> That's assuming hacker knows the salt, which is not always the case.

</comment></comment></comment></comment></comment><comment><USER7> Passwords and MD5?

<comment><AUTHOR> Microsoft is worse: passwords and MD4.

(Windows' newest password hashing algorithm, NTLM, is just a plain MD4 hash of the password. I estimate my 4x5970 machine can crack NTLM hashes at a rate of about 45 billion per second.)

<comment><USER14> Calling it Microsoft's newest algorithm is very misleading. Microsoft have themselves deprecated NTLM and recommend Kerberos.

Technically, NTLM will always be the "newest" unless they invent something newer, but it doesn't mean you need to use it.

<comment><AUTHOR> And Microsoft's Kerberos implementation (AD) hashes passwords on the domain controller using... NTLM!

How am I misleading?</comment></comment></comment></comment>


CVE-2018-10933 libssh authentication bypass in server code - <AUTHOR> https://www.libssh.org/security/advisories/CVE-2018-10933.txt <comment><AUTHOR> Bugfix release notes: [https://www.libssh.org/2018/10/16/libssh-0-8-4-and-0-7-6-sec...](https://www.libssh.org/2018/10/16/libssh-0-8-4-and-0-7-6-security- and-bugfix-release/)

</comment><comment><USER2> You're telling me that, basically, an attacker says "you've already said you trust me" and the server answers "sure, I did"? Wow.

</comment><comment><AUTHOR> "libssh versions 0.6 and above have an authentication bypass vulnerability in the server code. By presenting the server an SSH2_MSG_USERAUTH_SUCCESS message in place of the SSH2_MSG_USERAUTH_REQUEST message which the server would expect to initiate authentication, the attacker could successfully authentciate without any credentials.

The bug was discovered by Peter Winter-Smith of NCC Group."</comment>


Four years running Server Check.in - <AUTHOR> https://servercheck.in/blog/four-years-running-server-checkin <comment><USER2> Interesting read, since many of this type of articles I've read are from a perspective of failed startups and entrepreneurs, in your case it has been a side project that got successful enough to fund by itself while earning you some extra cash.

</comment><comment><USER3> This is awesome! Thanks for creating and sharing it at a reasonable cost.

<comment><AUTHOR> You're welcome! I wish more people would do retrospectives on their 'Show HN' projects a few years later. Especially if it went belly-up, because sometimes those stories are even more interesting and insightful!</comment></comment>


Pluto: a first concurrent web server in Gallina - <AUTHOR> http://coq-blog.clarus.me/pluto-a-first-concurrent-web-server-in-gallina.html

<comment><USER2> Reminds me of Lemmachine, which was written in Agda (since i'm a Haskeller i gravitated towards that). I wonder if Lemmachine's still alive... /me looks

edit: not so much.

</comment><comment><USER3> It's unclear what the grand scheme is here. Why using Coq instead of OCaml? How are the theorem proving of Coq used and to prove what? What do we expect from a web server anyway?

It is probably an interesting project but I don't see anything else on their website that witnesses more than the fact that they can write monadic code.

<comment><USER4> "It is a research project which aims to apply the pure and dependently typed Coq language to system programming, with inputs/outputs and fine grained concurrency in mind. … We try to develop new programming techniques with an extremist and purely functional approach, in the hope to lead to safer systems."

</comment></comment><comment><USER5> " _Note: Pluto is also the only planet discovered and undiscovered by the Americans._ "

As an aside, the way that's put is just...sad.

<comment><USER6> And not true. The IAU (an international organization) reclassified Pluto when they settled on a common definition of planet.

<comment><USER7> Yeah, I don't want to smear the work of good people. Neil deGrasse Tyson gets/got a surprising amount of hate mail from children due to his involvement in the definition of 'planet' and the reclassification, and man, he seems like a kind and thoughtful person. Obviously, there are people who wish to see Pluto be classified a planet that don't want good people to get hate mail from children, but this isn't the type of thing that should be "sad" or even controversial.

<comment><USER8> A world where classification of astronomical objects is cause for widespread controversy sounds kind of awesome to me.</comment></comment></comment></comment>


Top 10 Lies VCs Know You're Telling (Guy Kawasaki) - <AUTHOR> http://blog.guykawasaki.com/2006/01/the_top_ten_lie_1.html <comment><AUTHOR> I apologize if this is a repeat. I should have assumed Guy's whole site is probably somewhere on this news feed.

<comment><USER2> It's a good read, but yeah I believe it's been floating around here before.</comment></comment>


The unintended consequences of working from home - <AUTHOR> https://www.smh.com.au/business/workplace/the-unintended-consequences-of-working-from-home-20200519-p54ugp.html <comment><USER3> I've spent a lot of time working with people in London and Asia. I'm based in New York.

I can't speak to other professions, but for programming it's extremely difficult to work with people on the other side of the Earth. The 10-12 hour time difference makes any small issue or missing files/data that should take 20 minutes instead take a day to get resolved.

Working with people in Europe isn't as bad, you can sync up all morning. But the same applies to any issue that happens in the New York afternoon.

It pretty much guarantees your software will become architected around time zones due to Conway's Law: [https://en.wikipedia.org/wiki/Conway%27s_law](https://en.wikipedia.org/wiki/Conway%27s_law)

<comment><USER4> I normally work in the middle of four time zones my team is spread amongst.

Yes, it can be a pain. Sometimes you have a 6am meeting and sometimes an 11pm meeting and sometimes both in the same day. You have to have your shit together enough to deal with day-long lag for non-urgent stuff, and you have to care enough to jump on urgent stuff at all hours.

Even if you might rather have a chummy relationship with your colleagues, you have to be a bit more distanced and professional in order to accommodate the time difference.

But if you have a good team, you also should ask yourself: is the convenience of collocation worth losing the team?

At the risk of veering OT: there's a pretty good business in arbitrage around these cultural blockers. For instance there are thousands of people working for Indian companies in Central-Eastern Europe because (mostly) German companies are a lot more comfortable if they can meet the dev manager on any given day and fly home for supper.

<comment><USER5> > _Sometimes you have a 6am meeting and sometimes an 11pm meeting and > sometimes both in the same day._

I think we need to find a better way to deal with timezone-distributed teams. I personally would never find a 6am or 11pm meeting to be acceptable. If you require me anywhere from 9am to 5pm in my local time I'll be there; anything outside that must be strictly optional. And I mean actually optional, not "optional but we'll see you as not a team player if you don't show up".

<comment><USER4> Ok, that's fine, and there are lots of places you can work if you're good.

I'm not trying to virtue-signal about schedule flexibility, but if you have part of your team in SF and part in Bangalore somebody will have to work weird hours sometimes.

I fundamentally don't accept that it should always be the non-western side, and if you have a good team you will find a way to do this respectfully, without destroying anyone's private life.

Also, FWIW, during the 2/3 of my career I worked on site, a 9am to 5pm requirement would have been an absolute nonstarter for me. Biorhythms are diverse.

</comment></comment></comment></comment><comment><USER6> _Can_ be outsourced. Not _will_ or _should_ be outsourced.

If outsourcing were only an issue of money, then every all-remote company would exclusively hire in places like the Phillippines, where there seem to be an abundance of extremely low-cost, English-proficient workers.

Money isn't the only issue, of course. Language skills, culture, educational background, security clearance, tax laws, infosec, time zones, law enforcement, etc. are all also factors.

<comment><USER7> Thank you. Anyone who has worked extensively with offshore teams knows that there are always trade-offs involved. You can save money, but often at the cost of lower quality, lower maintainability, increased lag times due to time differences and miscommunications, added PM overhead, etc. Sometimes it makes sense, sometimes it doesn't.

<comment><USER8> it actually seems like the off-shoring craze has died down a bit in the U.S. I think companies realized its not as easy to get right as they thought. I've been working continuously with a team 10 hours away for the past 2-3 years, and it takes some accommodation on both sides....

<comment> blaser-waffle That has generally been my experience as well -- the concrete benefits in terms of lowered OpEx (aka paying less for a worker) is offset by the intangible inefficiencies. It's a lot harder and a lot more chaotic than people realize.

Plus automation is eating away at the lower level stuff.

</comment></comment></comment></comment><comment><USER9> I think if working-from-home becomes the new normal, a lot of companies will try off-shoring or outsourcing as a way to cut costs, but I'm not sure the results will match expectations.

I've worked at two previous companies that attempted to offshore coding production, and both times it led to unexpected delays, surprising bugs, badly written code, misunderstandings and cycling through multiple remote programmers.

I certainly don't think I'm a specially talented programmer or that coders in other countries are inferior to my skills (demonstrably untrue), it was just a natural consequence of geographic distance, timezone differences, personal commitment and cultural misunderstanding.

At both companies the experiment ended after just one project. If this is the 'normal' future then the infrastructure needed to really make offshoring work might be put in place, but if past experience is anything to go by, then simple geographic nearness - at least sharing timezone and the realistic ability to turn up in person at least occassionally, together with the personal commitment felt by 'real' employees - means I'm not sure whether distributed workforces are gonna be as productive as some people may hope.

On the other hand, I think truly productive individuals may gain from the situation: if you're a fast and effective 'production unit', then you're going to be a valuable asset wherever you are, and your value will be set accordingly. We'll see which of the two competing economic forces wins out.

<comment><USER10> I think it's not an issue with off shoring per se but with hiring. For some reason when off-shoring companies use a different hiring process. As if they were doing this to cut costs so they already start with a lower expectation and no longer look for the perfect match.

The off shore remote devs that will make it work exist but you have to look for them with a functional remote hiring mechanism.

</comment></comment><comment><USER11> Am I the only one who goes crazy when they see someone write "outsourced" when they mean "offshored"? These are different concepts.

<comment><USER12> I'd become numb to the distinction at this point, but yes, now that you reminded me it's bothering me too. Similar story with "disclaimer" and "disclosure", "third world" and "developing world", etc.

</comment></comment><comment><USER13> Speaking only for myself, most of my friends (from after university) are people I have worked with. And these are the people I usually get a coffee with, brainstorm ideas on whiteboards, go out for drinks or talk tech with them. WFH permanently would definitely impact this. Even though I might get along great with another dev from 100+ miles away, I cannot possibly do most of the things I mentioned above with them.

Edit: Friends also include people working in other teams like project managers, marketing, HR, support teams etc.

<comment><USER14> Definitely not with that attitude. I've seen people organize virtual parties with video conferencing software, have personally played games with my team members while having drinks, had brain storming sessions with digital whiteboards or a shared screen, and talked tech with friends for hours in one- on-ones.

If you go in with the mentality of "this won't work", well mate, then it won't.

<comment><USER13> He said it better than I could: [http://www.youtube.com/watch?v=Jtk6DtkJhrA&t=4m23s](http://www.youtube.com/watch?v=Jtk6DtkJhrA&t=4m23s)

</comment></comment></comment><comment><USER15> There are difficulties in communicating over the phone and video.

I don't know if others agree but during the Pandemic I've felt that the quality of TV news programming has gone down considerably because of the people interviewed or the reporters are behind a zoom etc. connections of questionable quality. It is often hard to understand what they say and it is not so pleasant trying to listen to them over the lo-fi connection.

So I think that until the tech of virtual meetings improves considerably there's an extra cost for not being able to communicate in person.

<comment><USER16> Even on a fairly high quality connection, you still lose a significant proportion of the non-verbal communication on a video call. The fidelity just isn't the same when you're looking at the slightly laggy image of a person on a small screen vs being able to see them in real life.

It presents real challenges working with people that you don't already know in person.

</comment></comment><comment><USER17> Come gather 'round people, wherever you roam

And admit that the waters around you have grown

And accept it that soon you'll be drenched to the bone

If your time to you is worth saving

Then you better start swimmin' or you'll sink like a stone

For the times they are a-changin'

<comment><USER15> Finally

</comment></comment><comment><USER4> A job that is done alone at a desk _can_ be outsourced.

Given the massive incentives for outsourcing and offshoring in the last couple of decades, I think the fact that most of the HN audience has pretty good jobs suggests that much of the tech world is more nuanced than that.

</comment><comment><USER18> The easiest part of my job is writing code. the hardest part is understanding it. Distance doesn't help.

<comment><USER19> but does it hurt?

<comment><USER20> Yes. It's much harder to perform complex communication across cultural, language, and timezone gaps.

<comment> globular-toast I've worked with non-native English speakers my entire career. Time zones just aren't as big of a problem as you think. You have to learn asynchronous communication. That means writing which, by the way, is by far the best way to perform complex communication.

<comment><USER20> I don't really understand the response. I'm not talking from a position of ignorance here; time zones are exactly as big of a problem as I think, because what I think has been informed by my experience working across time zones. There are certainly mitigation strategies, which improve the situation far beyond the naive solution of "let's pretend they're locals who just never come to the office". But no mitigation will make communication with someone you've never met in Shanghai as easy as it is with your buddy in the desk across the hallway.

<comment> globular-toast I've worked across time zones too. Your problem is that you try to fight their existence and want it to be like "your buddy across the hallway". But it's not. If you accept that then you can find effective ways to communicate with them.

</comment></comment></comment></comment></comment></comment><comment><USER21> People are focusing on the good side of working from home but the fact is that a job from home can be done anywhere in the world. Why would a company limit it self to the U.S? Work from home jobs are going to go to the people that are can do the work and are willing to work for the least compensation.

<comment><USER17> > Why would a company limit it self to the U.S

So this is essentially protectionism. I mean if engineers outside the US are just as talented and productive, why should a company only use US engineers, especially if their marketplace is global.

<comment><USER22> They should. The concern US developers have is that devaluation doesn't always occur in sanitary ways.

Let's say you have the same talent elsewhere, it's likely a company will pay _considerably_ less, and may also take advantage of other factors due to that person's desperation - overwork, given less input/influence, less career growth/trajectory, more contract based, less full-time, less or no benefits, less laws, etc.

So the quality of life for the US developer will not only come down in terms of pay, but also for all those other reasons. The worst part about it is that the off shore workers won't regain those improvements for awhile (as in, until their standards go up over time - higher pay, etc).

It's a regression on more fronts than compensation, with the enemy not necessarily being developers (where ever they are), but more so the ruthless market mechanisms.

</comment></comment></comment><comment><USER23> It's already happening

Call centers, office assistants are both getting outsourced to other countries or to AI

> If your job can be done from anywhere, that doesn't necessarily mean you get > to keep your job with the city salary and have a sea change.

It also means a "you" will get a chance at a remote job that you didn't have access in your region

</comment><comment><USER24> The submitted title ("A job that can be done from home, is a job that can be outsourced") broke the site guidelines by editorializing. Accounts that do that eventually lose submission privileges, so please read [https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html) and please don't do that.

</comment><comment><USER25> If the newspapers are to be believed, my job has been in danger of offshoring to countries overflowing with "engineering" talent for 20+ years. This is nothing new.

<comment><USER8> ^^^ - too true

</comment></comment><comment><USER22> For a tech oriented site, I feel like some old hats need to drop some knowledge on the first age of outsourcing. How did it go down, what were the results?

<comment><USER26> _How did it go down, what were the results?_

The fundamental problem is a misalignment of incentives. Noone has yet figured this out and there is no technological solution to it - it may actually be impossible and it ALWAYS bites the company that outsources on the arse eventually.

If your "trusted solutions provider" bills by the hour... they will seek to maximise billable hours. If they charge per ticket, they will drown you in paperwork for the most trivial changes. Anything that your in-house staff used to do that isn't explicitly stated in the contract, they either won't do or will charge a punitive rate for. If your needs change faster than the length of the contract, well you are stuck on the old contract. If you need some code written the spec you will need to write will be so detailed that it might as well be the code.

And after a few years you will have lost the capability to take the work back in-house and they will hold you to ransom, because they can, and because that was the plan all along...

But the short-term incentives to do it are strong, as a manager in a large corporation or a government department, you can sign that outsourcing deal and it will look good enough in the first year to win you your next promotion, or a guaranteed sinecure in the outsourcing company when you retire, by the time the truth comes out you will be long gone .

</comment></comment><comment><USER26> The particular vulnerability of programming jobs to outsourcing and offshoring has been well known for a long time. And yet we feign surprise that people would rather do medicine or law or other jobs that can't be.

<comment><USER27> Lawyers can practice remotely. Outside counsel is probably more common than (outside) contract engineering.

<comment><USER26> True but it is not easy to pass the relevant bar remotely, so there is that moat. Same with practicing medicine.

<comment><USER27> The bar exam is probably easier to sit remotely than software development.

</comment></comment></comment></comment><comment><USER28> I have to say, the gently threatening tone of this article really rubs me the wrong way. And it coincides with "stay in town or take a pay cut" rhetoric that was on my feed earlier today, from Facebook.

(Despite this development, the article doesn't mention it... maybe it will be updated at some point.)

I guess, after reflecting, what I don't like about it is that it tells us to be afraid. Be afraid about a way of working that might work really well for some, or even a lot, of people.

Here's a fun quote from the article:

> But the big challenge is how to safely move thousands of people to and from > the suburbs of Sydney and Melbourne and their offices in George or Collins > streets.

They're saying this in the context of a population living under pandemic protocols, but it's true all of the other time anyways. It takes a massive amount of engineering and energy to move masses of people from one place to another. Doing it needlessly is insane.

I suspect that what it comes down to is control, or apparent control, of a workforce. Especially in large organizations. We're going to see who is comfortable with giving up control and who isn't, and that's going to be interesting.

<comment><USER22> It also boils down what should be an accommodation for worksite preference to a sudden demotion to 'you are no longer worth it on a global scale and that changes everything'.

Companies are what they are, and they will optimize for this. To remain optimistic, we have to continue believing in the startup space to continuously innovate software standards, where users are unwilling to accept substandard software experiences. That will keep this field moving forward versus a wage race to the bottom.

Entrenched corporate companies would love tech to stagnate, that way they won't ever have to innovate and can cost optimize developers down to the bone.

</comment></comment><comment><USER29> back in the early 00's when I was in university I did some research assignments concerning the outsourcing and offshoring of it jobs in small and medium sized businesses. The main gist of it is that most ended up switching back to local or hiring back part of the staff. The main problems I was told were:

Cultural (mostly with eastern Europe) which introduces additional overhead due to cultural things not being equal especially for custom software for things like school districts as an example. You have to specify more which you assumed was common knowledge. Also things like meeting hours are somewhat more flexible in other places than in N/W Europe.

Timezones: many things have been said about it but it didn't work for most companies combining Europe with (South)-East Asia. Ukraine and Russia was not a problem.

Management capability: a managerial job locally is not the same remotely and most could not adapt or required more people offsetting the cost savings. This tended to count more for smaller businesses.

Emotional: lack of affinity with the product, remote workers are counting just hours spend and thus the TCO was ignoring the hours local people spend in their own time working on the product. This is not a given for all but was one of the reasons companies switched back.

These are just some examples. Most companies, especially the smaller ones, switched back to local hiring. Some made it work but not at the expected savings. The most successfull were the bigger businesses and my guess would be they had/took a longer time to prepare and hired the correct people for the job instead of relying on already hired employees which is not something you can always do in smaller businesses due to continuity.

I really wonder what is going to be the effect this time as everybody is forced at least to train all the soft skills like remote communication and scheduling.

</comment><comment><USER30> You know people keep talking about how to "recreate Silicon Valley at location X" \- here is an idea for making your small country / tourist hotspot way less reliant on tourism

\- implement world beating virus testing and tracking \- invest so heavily in child care and education it makes your head swim \- build out offices for rent and look at your building codes

Now start saying you are a remote work friendly area.

My wife and I choose to sit in London's metro area for one main reason - <USER2> schools. A second one is the health system. Nothing to do with commute times (god no) or access to job choices.

Want to replace tourism as an industry - focus on office based remote workers.

</comment><comment><USER31> If someone can do your job better than you, they should have your job.

<comment><USER32> The law of comparative advantage disagrees.

[https://en.m.wikipedia.org/wiki/Comparative_advantage](https://en.m.wikipedia.org/wiki/Comparative_advantage)

</comment></comment><comment><USER33> I know this doesn't contribute anything meaningful to the conversation at hand, but that comma in the title is super unnecessary.

Edit: my work here is done.

<comment><USER34> In case this comment is puzzling anyone, the title was changed by a mod (dang, [0])

[0] [https://news.ycombinator.com/item?id=23269061](https://news.ycombinator.com/item?id=23269061)

</comment></comment><comment><USER35> I find a difference (dare I say flaw) with Zoom type communication for unstructured team conversations is that without physical cues and without the ability to break into quick subconversations, chats among say 9 people are more likely to be dominated by a handful of leaders/big talkers than in real life.

<comment><USER36> In my opinion this is a benefit, subconversations during a meeting are distracting. If it doesn't need to be discussed by the entire group, that conversation should happen after the meeting.

Zoom meetings where everyone is on a separate camera/mic make that basically impossible, which keeps the meeting more focused.

</comment></comment><comment><USER37> Yes. And that's a good thing. It makes the world a little less unequal.

<comment><USER38> Yeah I find the dismissive attitude towards outsourcing, in particular in tech hilarious. Disruption is not so fun if you're the person being disrupted.

Bringing down employment borders would be one of the most transformative things we could do to raise global standards of living. Everyone should go and read Bryan Caplan's recent book on open borders.

<comment><USER39> It's not only the hopefully temporal inconvenience of being replaced. There is a finite amount of resources. If more people have the means to access them, less is available for those who currently consume them.

Having 9 billion people sharing the earth will lead to huge improvements in replaceable resources. E.g. movies will be better, appliances will be better, etc. However, if you want to see the Mona Lisa, it's most likely that you will never get close.

</comment></comment></comment><comment><USER40> Facebook have already indicated they will cut pay for their remote work force, as their cost of living goes down. It seems a cynical ploy to cut costs more than a drive to home working.

<comment><USER41> Well that's capitalism right?

<comment><USER42> >Well that's capitalism right?

It is indeed. It's quite a risky hand to play at a time where

\- cynicism about the tech industry has never been higher

\- there's a magnifying glass on the inequality wrought by the hands of hardheaded capitalism in the West

\- globalism is being rejected

\- million of jobs have been lost

\- everything is being propped up with fiscal stimulus, in the coming months the true cost of the financial damage on the economy will come to fruition

But if Tobias Lutke, Zuckerberg, etc. want to be the face of moving jobs out of the US and Canada in these times then by all means they can go ahead. Hopefully, if there's consequences to bear then it'll be on their heads too, and not the staff they've decided to drag along for the ride.

<comment><USER41> Facebook already has R&D centers around the world, you're talking as if this hasn't already happened.

\- cynicism about the tech industry has never been higher

Exactly. That's why I don't think the average American is going to care about 250K a year jobs in Silicon Valley moving somewhere else. I don't see a lot of empathy to the millionaire class in the U.S

<comment><USER42> >Facebook already has R&D centers around the world, you're talking as if this hasn't already happened.

Developed at a time where they could feasibly say there just wasn't the volume of talent in the US to support their ambitions. With all the startups going bust, big companies laying people off, etc. the same argument won't be feasible. In my eyes at least.

>Exactly. That's why I don't think the average American is going to care about 250K a year jobs in Silicon Valley moving somewhere else. I don't see a lot of empathy to the millionaire class in the U.S

That's one way of looking at it. Another is that FB and others moving the work away will directly impact the large swathes of people who work for these companies not in tech roles, and also decimate the local economy around their offices by dropping demand for local goods, services and by extension the jobs that exist to provide and fulfill the same.

All it takes is a couple of Democratic figureheads to get on TV and say that Zuckerberg et al's actions will dis-proportionally throttle the lives of minorities and the socially disadvantaged, which is true, and presto, you further the ire toward them at a time when they can least afford it.

<comment><USER41> They don't even have to fire anyone. They can just freeze hiring and hire more outside SV (which seems to be what they're doing). In 10 years 90% of their developers would work outside SV.

</comment></comment></comment></comment></comment></comment><comment><USER43> s/from home,// \- ftfy

Any job or task can be outsourced, but there are tradeoffs involved. It's nothing new.

</comment><comment><USER42> I'm a strong advocate for remote, and retain huge admiration for the fully- remote companies that existed and drove thought leadership in the area prior to this mass experiment we now find ourselves in, but the speed at which things appear to be moving draw an uneasy cynicism in me at this point.

I'm not worried about jobs being outsourced across the world to places like India. It's just not going to happen en masse. I'm sure some companies will try it and learn the same lessons others have previously about engaging in this behaviour.

It would also be political suicide, if not worse, if world leaders saw their companies trying to move locally held jobs outside their borders and did not drop the hammer on the companies involved. We're already in a time of mass unemployment, in which:

\- many were already heralding the death of globalism

\- we've learned very dear lessons about outsourcing the supply chain

\- people are become so much more aware of the risen inequality that even formerly hardline capitalist media are running headlines about the billionaires profiting massively off the pandemic while millions register for unemployment checks

So if the leaders want to prevent large scale social unrest, a gun which I believe we're already looking down the barrel of, then they would be wise to keep a very close eye on the movements carried out by companies within their governance.

That said, I do foresee some significant, lasting consequences should there be a mass shift to working from home or even a 4 days at home / 1 day in the office situation:

\- Today's cities face a real threat of ending up as Detroit did.

\-- People will flee for better pastures immediately. Why live in a shoebox in the sky when your money gets you far better elsewhere?

\-- Commercial real estate, and nearby residential real estate, has the potential to collapse and rapidly so. If I were living in Toronto, working for Shopify and owned a condo now I would likely be putting it on the market ASAP.

\-- Taxes, be it through indirect sales taxes or VAT, land transfer taxes, property taxes, etc. would crater leaving a massive hole in public funding through which no reasonable measure will fill.

\-- Public transit fare revenue will evaporate.

\-- Small businesses, restaurants, retail etc. will be wiped out with the lack of casual foot traffic

When the above potentially happens, those who wanted to live in a city for the benefits of living in a city would also likely flee.

\- A permanent, wide scale loss of jobs in retail, food & beverage, etc.

You could argue that those jobs will be replaced by new demands for new supply in the new areas previous city dwellers move to, but if everyone is WFH I would imagine they won't be frequenting such stores in anywhere near the volume that was provided previously.

Given the above I believe it's pretty self-centered and reckless of large companies to be making such announcements as Shopify, Facebook et al. have made at a time where cities and countries are merely trying to ward off a pandemic and are already facing an employment and budget crisis as a result of it. They're essentially doing one of the most damaging, potentially impactful things they could be by doing so right now.

These announcements, for me personally and despite my previous hopes of seeing more remote opportunities become available over time, are only furthering my cynicism toward this industry in general at this point.

</comment><comment><USER44> Sounds like the writer is scared that their bullshit job is on the line.

People have tried outsourcing programming forever. The last visible result was planes falling out of the sky. These jobs are highly paid not because they are done in the CBD, but because they require traits most people don't have.

</comment><comment><USER45> Or a job that can be automated by an AI program or robots.

<comment><USER46> I don't see how that follows. If I can do _X_ in any location, that doesn't mean _X_ is something that is a target for automation.

Offshoring and automation are partially orthogonal. Offshore the jobs that can't be automated, automate the jobs that can't be done elsewhere, retain the humans for high-skill or high-touch jobs that resist the first two threats.

</comment></comment><comment><USER47> True. And if I move to another country I'll let my employer know.

Until then, good luck finding another me.

<comment><USER48> The graveyard is filled with indispensable men.

<comment><USER49> I had never heard this quote before:

[https://quoteinvestigator.com/2011/11/21/graveyards- full/](https://quoteinvestigator.com/2011/11/21/graveyards-full/)</comment></comment></comment>


Intro.js – Step-by-step guide and feature introduction for your website - <AUTHOR> http://introjs.com/ <comment><USER2> One problem I have with libs like this is you have to chase the "Ok" and "Next" buttons around. It would be much better IMO to have a fixed footer with the action buttons but have the lightbox w/description move around the page as it does now.

In addition, I would prefer not to have the entire rest of the UI blacked out. Lightboxes don't seem appropriate here since I want to see the "big picture" all the time and just have relevant sections highlighted. This would actually be a much simpler implementation, just fade the background of the div to highlighter-yellow.

<comment><USER3> Have you seen Desmos's tours? There's no Ok or Next buttons at all, it automatically detects when you've actually interacted with the feature that you're being taught: [https://www.desmos.com/calculator?tour=sliders](https://www.desmos.com/calculator?tour=sliders)

(Disclaimer or whatever: I helped design and implement them.)

<comment><USER4> Oh wow!! This is so cool! Where was this tool when I was in high school ;)

</comment></comment></comment><comment><USER5> Another similar technology (with slightly less jumpy scroll, but I'm sure you could tease out some bugs if you tried):

Zurb Joyride - [http://foundation.zurb.com/sites/docs/v/5.5.3/components/joy...](http://foundation.zurb.com/sites/docs/v/5.5.3/components/joyride.html)

Last time I looked into building an introductory FTUX with intro.js, I found myself trying to make the choice between these two (intro.js and joyride)

<comment><USER6> Another good one for the road is HubSpot's Shepherd: [http://github.hubspot.com/shepherd/docs/welcome/](http://github.hubspot.com/shepherd/docs/welcome/)

Just had a really great experience creating walkthroughs with it.

<comment><USER7> Thanks. I just opened all the examples mentioned in the comments and this one feels the most elegant.

</comment></comment></comment><comment><USER8> While this might feel like a great way to introduce people to features in an application, needing it is probably a sign that your UI isn't really working. In my experience learning a UI is a chore, and people hate trying to do it. If the interface is well organised, uncluttered, and laid out in a logic manner that doesn't hide things there's no need to have an intro.

<comment><USER9> What you're overlooking is the users who type website addresses into google, and compose entire emails in the "subject" field. Some people just require a little hand holding no matter how obvious the UI is to the power users.

<comment><USER8> Do you really think a 5 step animation is enough to overcome UI problems like navigational searching or a fundamental misunderstanding of how email works?

Those things are _examples_ of the UI failures I'm talking about. Blaming the person using the app for doing something wrong is how "fixes" like putting in an "intro" happen. What we need is to design human-centred experiences that work the way we automatically think they should, not patching things up with instructions that'll be forgotten within 5 minutes.

This library puts the developer first (driven by thinking like "I told my users what's in the app so now they know what's there.") rather than putting the person using the app first (driven by thinking like "The users mustn't be afraid to experiment by clicking on stuff.").

<comment><USER10> > The users mustn't be afraid to experiment by clicking on stuff.

They shouldn't be, but they are. Nothing is "wrong" with non-technical users, but learning new software, especially complex software, is something they automatically believe they can't do, and so they are afraid to experiment by clicking on stuff.

I feel like Facebook's design is pretty intuitive, but I still have my mom asking how to do simple things that are hidden behind menus (for example, change privacy settings). Nothing's wrong with her, she's just afraid of breaking technology, and it's unlikely anything will change that fundamental belief. However, adding an intro could ease these types of users into the software, and perhaps even lead them to experiment a bit more.

</comment></comment></comment></comment><comment><USER11> This is a neat project but also replicates the functionality of many libraries (I'm not going to dig it up but there are more than a few of these floating around with varying amounts of maintenance). Anyone needing to use these seriously will quickly end up forking it. Also - I think the goal of EVERY UX designer should be to prevent the need of these kind of things AT ANY COST. It obviously varies by product and industry but any designer worth their salt will do everything they can to make it painfully obvious what a user should do at every step of the way.

<comment><USER12> There are complex system where something like this is inevitable, because it's complex by nature.

</comment></comment><comment><USER13> It's certainly a nice enough script for what it does, and I can see it being useful in some instances, but... a lot of this 'feature tour/tutorial' stuff strikes me as more of an excuse to not worry about ease of use. A good site design should be intuitive in of itself, not something that requires a guided tour to figure out.

It's a bit like the modern gaming world to be honest. In the old days, games like Super Mario Bros was designed to teach you through level design, so you'd figure out how things worked without getting a huge info box thrown in your face.

Like this:

[https://www.youtube.com/watch?v=ZH2wGpEZVgE](https://www.youtube.com/watch?v=ZH2wGpEZVgE)

On the other hand, games since got obsessed with tutorials detailing every minute thing about the title, leading to such over the top insanity as this 'help DVD' included with Super Mario Galaxy 2:

[https://www.youtube.com/watch?v=PbDLuDRgnkM](https://www.youtube.com/watch?v=PbDLuDRgnkM)

These tutorials on sites and in mobile apps remind me all too much of the latter. It's like 'we won't use our product to teach people in a natural way, so we'll put in a guide for complete dummies instead'.

</comment><comment><USER14> Kudos, cool project. However, this behavior is extremely easy to abuse and often leads to a poor user experience. If you want to do something like this, present one tip at a time and specifically when they are most useful. Don't present an entire manual during the onboarding experience.

Further reading: [https://www.nngroup.com/articles/mobile-instructional- overla...](https://www.nngroup.com/articles/mobile-instructional-overlay/)

</comment><comment><USER15> While some of the effects are smooth, the scrolling is jumpy and unexpected.

</comment><comment><USER16> To add to the list of similar/related tools, LinkedIn's Hopscoth: [https://github.com/linkedin/hopscotch/blob/master/CONTRIBUTI...](https://github.com/linkedin/hopscotch/blob/master/CONTRIBUTING.md)

</comment><comment><USER17> Unfortunately this implementation isn't accessible and doesn't work with screen readers.

</comment><comment><USER18> I hope websites would stop using "Raleway" for content text. I can't understand half of the letters because they're so thin.

<comment><USER19> It looks like he's using the Skeleton library and I think its font is set to Raleway by default.

</comment></comment><comment><USER20> [http://bootstraptour.com/](http://bootstraptour.com/) is very similar to this.

</comment><comment> martin-adams Does anyone know of any good research backed studies as to whether these guides improve usability or not?

</comment><comment><USER21> I use this library on couple of my projects, I love it.

Here's the HN discussion from the original post 3 years ago: [https://news.ycombinator.com/item?id=5380056](https://news.ycombinator.com/item?id=5380056)

<comment><USER22> I've used it too. It's nice and unobtrusive.

</comment></comment><comment><USER23> I like the idea behind this.. but its kind of jarring. Does anyone have a good wordpress plugin with similar functionality? I want to have an overlay that shows how I'm doing the data collection on my sites.

</comment><comment><USER24> This is pretty nice! It seems to need some work on mobile though: [http://i.imgur.com/6DcJOxu.png](http://i.imgur.com/6DcJOxu.png)

</comment><comment><USER25> It's a neat little project, but it's been around for 3 years.

<comment><USER26> I've seen a site completely made with reveal.js (and maybe pandoc).

[http://lab.hakim.se/reveal-js/#/](http://lab.hakim.se/reveal-js/#/)

<comment><USER25> How does this relate to introjs?

</comment></comment></comment><comment><USER27> I'm definitely going to look into this to help with on-boarding for my web app. I wish this had existed weeks ago.

</comment><comment><USER28> Feels misnamed, brought back thoughts of flash intros in 2002. Should be renamed to something like grandtour.js

</comment><comment><USER29> Done this a year before they've done theirs – [https://github.com/oskarkrawczyk/tour.js](https://github.com/oskarkrawczyk/tour.js) (never got traction unfortunately, probably because it's MooTools-based and 2012 was the year when everyone who used MooTools started dropping it in favor of jQ).

</comment><comment><USER30> It is great! I love that it is small, and love the hints feature.

<comment><AUTHOR> Cool! Yeah the hint feature was added recently.

</comment></comment><comment><USER31> Adding to the mix, we've also considered Onboard Tips ([https://onboardtips.com](https://onboardtips.com)) because it allows someone non-technical to create in-app tours.

</comment><comment><USER32> I'd use this... except my interface is in React.

</comment><comment><USER33> A similar library if you're using jQuery is [http://trentrichardson.com/Impromptu/](http://trentrichardson.com/Impromptu/)

</comment><comment><USER34> The step 3 -> 4 is jarring on Chrome 48 OSX.

<comment><AUTHOR> Hmm, I don't have this problem. Are you talking about the introjs.com demo page?

<comment><USER34> Yes

</comment></comment></comment><comment><USER35> onboardx.com is a similar tool with no coding implementation and segmented guides

<comment><USER36> While intro.js has some faults, this one is even worse. It doesn't even feel like it tries to fit into the webpage.

<comment><AUTHOR> Yeah I know the issues with the Introjs but I will release a new version this month, trying to address all existing problems. Cheers.</comment></comment></comment>


Happy halloween everyone (.mpg) - <AUTHOR> http://anybots.com/monty_halloween.mpg

<comment><USER2> I laughed and laughed. Beautiful. If that batch of kids doesn't produce a proper engineer (or two or three), then it will be the schools that have failed, because I think Monty has provided ample inspiration.

Of course, they _did_ seem at least as interested in the candy as in the giant robot delivering it, if not moreso. Maybe kids today are just completely jaded by technical miracles, since new ones seem to arrive every day. When I was a kid, we didn't have any of these Internets, or cell phones that can play movies or find out almost any fact in seconds, or robotic vacuum cleaners, or jet packs, or personal oxygen sleep chambers, like these kids today. We had to call people using phones that plugged into the wall, and if we wanted to send email our PC or C64 or Amiga had to dial up a FIDO or Aminet hub and do a batch send/receive. And we had to walk to school in the snow. Uphill. Both ways.

</comment><comment><USER3> Very disappointing. I watched the whole thing, all the while expecting that the robot arm would attack the children and leave bleeding and broken bodies everywhere. What kind of halloween is it without a dose of horror?

<comment><USER4> I didn't watch long enough to see that part.

<comment><USER5> Well, thanks for taking the time to add absolutely nothing to the conversation.

</comment></comment></comment><comment><USER6> My favorite part is how the kids are put in some kind of caged away from the robot for their own protection (or the robot's)...

</comment><comment><USER7> A frightening future. In each "pick" or "put" operation the robot was served by an eager human minion while most of the humans' needs were unmet.

</comment><comment><USER8> Here's something scary to show the kids -- giant spider eating a bird

[http://www.telegraph.co.uk/earth/main.jhtml?view=DETAILS&...](http://www.telegraph.co.uk/earth/main.jhtml?view=DETAILS&grid=&xml=/earth/2008/10/22/easpider122.xml)

<comment><USER9> My ~2 year old daughter was completely fascinated by that, when I showed her... of course, she thinks all big spiders are tarantulas; she calls the daddy long-legs spiders that.

<comment><USER8> My 8-year-old daughter was also fascinated, while my 12-year-old son was horrified.

I'm not afraid of spiders, but that looks like the biggest spider I've ever seen. I'm a little disturbed by the size.

</comment></comment></comment><comment><USER10> Really cool

</comment><comment><USER11> yay!</comment>


Friendly: NoSQL With MySQL in Ruby - <AUTHOR> http://jamesgolick.com/2009/12/16/introducing-friendly-nosql-with-mysql-in-ruby.html

<comment><USER2> Even if NoSQL solutions are new, there are no data durability concerns in my opinion.

I can talk for Redis but my guess is that's the same for the other datastores. In Redis to have full durability enable append-only-file: it's a text-only file, you can check by hand what's written inside your database.

In addition even while running in append only file mode, you can still ask Redis to "BGSAVE" (crete a copy of the dataset into an .rdb file in background). It's a single file and you can save it everywhere.

With the append only file, it's trivial to write a script to export ALL YOUR DATA into MySQL every 24 hours for instance. So you have your data in:

a) compact .rdb file you can backup easily. b) the append only file c) MySQL, where you can perform query to check if the data is sane.

In addition in the next months Redis will get a JSON export feature built-in, so you can also d) save data into a JSON file.

This new products are still younger than MySQL, and there are more bugs probably, but it's just a matter of being a bit more careful to use this DBs in production without concerns, experiencing a huge performance gain.

p.s. since the release of Redis 1.0 no critical bugs were found.

<comment><USER3> Actually, we do run redis in production, and it has been awesome for us. But, for this particular aspect of our application, we see about 20GB of data growth per week. So, an all-in-RAM db isn't really practical.

Either way, durability wasn't the issue for us. Mongodb has no write concurrency. So, when it got so overloaded with data that queries were taking forever, we had no way to purge older data without blocking all read & write operations for hours.

<comment><USER4> This is fixed now - large deletes won't block the world forever: <http://jira.mongodb.org/browse/SERVER-494>

<comment><USER3> Awesome. You guys rock!

</comment></comment></comment></comment><comment><USER5> just to point it out, postgres now has a builtin hash data type. This sees like it could be a better solution thatn serialized json <http://www.postgresql.org/docs/8.4/static/hstore.html>

</comment><comment><USER6> This reminds me of Elephant, a CL OODBMS that can use Postgres as a key/value store backend. OODBs, document stores, et al. are useful but using a complete RDBMS as a backend is both overkill and inefficient in my opinion.

<comment><USER7> Yeah, there is a reason Elephant is nearly an order of magnitude faster when run on Berkeley DB.

</comment></comment><comment><USER8> Looks great! I remember seeing the article about FriendFeed's datbase setup and thinking how neat it was. Very nice to see a simple implementation of that idea become easily available.

Now I just have to find a project where I'll need to use it!

</comment><comment><USER9> how is the cpu trade-off like in this type of setting? one more point is that i dont know ruby and i'm curious how can he add an index from that column?

</comment><comment><USER10> If only this was written in python...

A port anyone?

<comment><USER11> FriendFeed's Tornado has a database.py on github, maybe it will get you started: <http://github.com/facebook/tornado/tree/master/tornado/>

<comment><USER12> database.py isn't really in the same realm, but it could certainly be modified and enhanced to support this abstraction.</comment></comment></comment>


Reddit Considering Accepting Bitcoin - <AUTHOR> http://betabeat.com/2012/11/reddit-considering-accepting-bitcoin-as-payment-for-reddit-gold-subscriptions/ <comment><USER2> Just the fact that someone at Reddit is _publicly discussing the possibility of accepting Bitcoin_ is compelling anecdotal evidence that Bitcoin is on the usual path to wider adoption of all innovative products. It now appears to be expanding from a tiny number of innovators to a still-small but considerably larger number of early adopters.[1] I'm persuaded Bitcoin is likely to gain much wider adoption over the long haul.[2]

Kudos to Reddit!

\--

[1] [http://upload.wikimedia.org/wikipedia/en/4/45/DiffusionOfInn...](http://upload.wikimedia.org/wikipedia/en/4/45/DiffusionOfInnovation.png)

[2] I've written down my reasoning at [http://cs702.wordpress.com/2011/05/29/on-the-potential- adopt...](http://cs702.wordpress.com/2011/05/29/on-the-potential-adoption-and- price-appreciation-of-bitcoin-in-the-long-run/)

\--

Edits: added context; made my point clearer.

<comment><USER3> If a lay person were to ask how they get back their Bitcoins if they are stolen from their account, say due to a person gaining access to their computer, what do you tell them?

<comment><USER4> If a lay person were to ask how they get back their dollars if they are stolen from their wallet, say due to a person mugging them, what do you tell them?

<comment><USER5> To put their money in a bank. But there are no secure Bitcoin banks.

<comment><USER6> I'm willing to bet that I have more protection securing the contents of my Bitcoin Wallet than Wells Fargo has protecting my account at any level.

<comment><USER3> >I'm willing to bet that I have more protection securing the contents of my Bitcoin Wallet than Wells Fargo has protecting my account at any level.

Would you say this is true for most users?

<comment><USER7> Currently? Of course not. But when the idea of storing money on your computer (e.g. in a file) becomes more widespread, then the answer would be yes. Or at the very least it would be equally secure.

It's not that hard to provide people with a secure environment, as long as that environment is designed with with security in mind. A non-jailbroken smartphone is probably good enough, though a dedicated device might be better. And you can use live boots.

I can see a future where we use such (or similar) things to do money transfers.

<comment><USER8> Wells Fargo is a bigger target, but they also have many more layers of protections and an interest in providing customer service. My hunch would be that it would be easier for a criminal to profit off of attacking WF than attacking a personal bitcoin user, but that it would be easier to actually cause financial harm to a personal bitcoin user (even without profiting themselves). But to clarify that, I have a couple of questions about how storing your own bitcoins works, since I've never used it:

* How does bitcoin storage work with offsite backups? If someone compromised the backup, would that give them access to your money?

* If you lose the file (hard drive crash, home burns down, backup system fails, whatever), does neither you nor anyone else have that money anymore? I.e., someone wouldn't have to gain access to the money themselves to deprive you of it?

<comment><USER9> keep your wallet file encrypted and back it up to multiple locations on a regular basis (to update the backups with new private keys that are created by your client software).

there are also ways of generating bitcoin keys completely offline as well as producing signed valid bitcoin transactions completely offline. This way you can forward funds to keys that are not on a machine connected to the internet, or keys that are backed up only on paper (in multiple safety deposit boxes if you like). And also you can then put signed transactions from the offline machine onto a usb stick or whatever and then use a networked machine to forward those valid transactions to the bitcoin network.

Coinbase is doing something like this for their storage of customer funds. Coinbase seeks to be a bitcoin bank that wont get hacked or that if it somehow does get hacked ( _cough_ inside job, _cough_ ) that only very small losses could occur.

<comment><USER10> For what it's worth, Coinbase has no reported cases of fraud. We are tracking these kinds of things here:

<http://howdoyoubuybitcoins.com/>

For example, Bitfloor (which had 250K USD stolen): <http://howdoyoubuybitcoins.com/from/bitfloor/>

Verses Coinbase, which has no such history: <http://howdoyoubuybitcoins.com/from/coinbase/>

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> This really sounded more like Yishan was just acknowledging that he personally knew what bitcoin was. Notice that he didn't "distinguish" his comment as an Admin?

Generally when an admin is speaking "as an admin" they turn their username red, and get a letter A next to it.

Like this: [http://www.reddit.com/r/blog/comments/12v8y3/now_is_the_time...](http://www.reddit.com/r/blog/comments/12v8y3/now_is_the_time_to_invest_in_gold/c6yefec)

<comment><USER12> Yes, this is the same issue we had at Kiva when I and another engineer said we were going to explore accepting Bitcoin. I provided a lot of caveats but there were already two blog posts about it out on the interwebs the next day.

As I noted in the thread where Yishan mentions looking into Bitcoin, there are a lot of barriers to entry, barriers that we were not able to overcome at Kiva (at least for now). Maybe as a for-profit entity they will have better luck though as they have more options open to them, I'd just caution people about getting their hopes up.

<comment><USER13> The Bitcoin "community" really is it's own worst enemy.

</comment></comment></comment><comment><USER14> This is an odd coincidence, one of the Dead Man's Switch subscribers asked me if I supported Bitcoin, and I ended up adding a MtGox button on the site (it's pretty nifty, it even autoconverts to whatever currency you want and supports IPN):

<http://www.deadmansswitch.net/pricing/>

I've been warming up to Bitcoin, the same subscriber clued me into <https://blockchain.info/> and it looks pretty good. The only thing I'm worried about is that Bitcoin isn't very stable, but I don't really care since MtGox autoconverts all payments. There's no downside in accepting it, as far as I can see.

Plus, I can just keep them as Bitcoins and pay right away with very low fees.

EDIT: I also decided to dabble in a bit of MtGox trading with my newly- acquired 4 BTC, so I sold high and bought higher. It... didn't go very well.

<comment><USER15> "Autosell" is the term Mt. Gox uses as the name for the parameter in the shopping button API to sell immediately once the transaction has confirmed.

But that's the thing. Merchants are worried about the downside risk, and basically ignore that there is an upside gain possible as well. Sure, the merchant shouldn't be speculating with revenues that go to employee's salaries and inventory purchases, but if there's 10% profit, keeping 10% of revenues without converting that to USDs might be beneficial.

BitPay is a merchant processor that lets the merchant specify what percent of revenue to convert and remainder of bitcoins are paid out without conversion.

<comment><USER14> Eh, if I want to gamble, I'll go to a casino. I pay my employees in Euros, that's what I want revenue in. I even want it exchanged from dollars, so I get X Euros every time.

The autosell feature means I can add Bitcoin as a payment method with no hassle, I'm enjoying it greatly.

</comment></comment></comment><comment><USER16> Wow, an entire article based on a vague non-promising comment. Internet journalism at its best.

</comment><comment><USER17> These promises are incredibly vague, the most specific thing I saw was in another thread: "Alternative payment features are in the works" ([http://www.reddit.com/r/blog/comments/12v8y3/now_is_the_time...](http://www.reddit.com/r/blog/comments/12v8y3/now_is_the_time_to_invest_in_gold/c6yefec)).

Does reddit have the manpower to deal with a non-mainstrem payment system like bitcoin?

<comment><USER18> The manpower to support bitcoin payments is minimal. A single engineer could integrate with a bitcoin payment provider such as Bitpay (or Walletbit or Mt. Gox Merchant) in less than a day. These bitcoin payment providers have APIs that are very similar to Paypal and Google Checkout. After the initial setup, ongoing support would likely be minimal. These payment providers also offer to convert BTC to USD (or many other currencies) at the current market price. So, your company doesn't even have to deal in bitcoins.

Bottom line: if you're already accepting Paypal, accepting Bitcoin as well is easy.

</comment></comment><comment><USER19> For users who have never used Bitcoin, the cost and time involved in acquiring something as small as $3.99-$29.99 is quite high. It would likely cost as much as 10-50% of that cost in fees.

So I'm not sure if this will create new bitcoin users. But more vendors always helps.

<comment><USER10> Coinbase.com will allow you to buy up to $100 with a 1% fee. It's pretty much what I'm recommending these days to Americans.

<http://howdoyouusebitcoins.com/>

<comment><USER19> See my post below. I factor the deposit fee not just the wallet transfer fee.

<comment><USER10> I replied to your post, although if your bank takes 10% off the top to begin with, the issue isn't with bitcoins, it's your bank's deposit fee.

<comment><USER19> Cavirtex takes a flat rate of $3 when depositing money directly into their bank aact regardless of the amount, as I mentioned above. Not 10%...

<comment><USER20> Oh please, and then they charge you 3.0% when you do a trade on your first 100 bitcoins.

So if you buy $5 worth of bitcoins, it costs you 61.2% in fees.

If you buy $50 worth of bitcoins, it costs you 9% in fees.

If you buy $100 worth of bitcoins, it costs you 6% in fees.

<comment><USER19> Ouch yeah, another example of Canada lacking proper competition in an industry.

</comment></comment></comment></comment></comment></comment></comment><comment><USER21> I'll buy a Reddit gold subscription if I can pay in Bitcoin. Who uses Paypal anymore??

<comment><USER7> Many people use Paypal and I think it works fine. But yes, I'd be more likely to buy Reddit Gold in Bitcoin too!

<comment><USER22> coinbase.com (YC company) is like Paypal for Bitcoin.

</comment></comment></comment><comment><USER23> Reddit is not profitable? It feels like the difference between profitability and non-profitability is an extra ad on each page.

<comment><USER24> Who on earth would want to advertise on reddit, given the site's thoroughly unpleasant audience, the notoriety of which has been growing by the month?

Reddit _might_ be able to court the same sponsors (porn sites and such) that sustain the likes of 4chan, as long as said sponsors are willing to associate themselves with a far more misogynist, racist, child porn-defending audience. But this would likely inspire a revolt among redditors, who hold themselves in laughably high self-regard.

<comment><USER25> I think the contributors to racist, misogynist, and child porn-defending subreddits are only a tiny minority of Reddit's users. Most of the popular subreddits are fairly benign (AskReddit, IAMA, etc.)

Interestingly, all your recent comments seem to be about how you dislike Reddit. Consumption of a chill pill may be in order.

<comment><USER26> Subreddits like AskReddit do have a lot of racist and misogynist shit posted all the time, esp. those leading questions people post to AskReddit. The Reddit moderating system doesn't make Reddit a safe space for people, so I can see what the OP is getting at in terms of Reddit being an unsafe space.

</comment></comment></comment></comment><comment><USER27> How about an optional bitcoin miner running in the background while using reddit using webgl (so it can make use of the GPU) that accumulates bitcoin. As the user browses around reddit it will be active, slowly generating bitcoins as part of a pool. When the user has accumulated enough for reddit gold, it automatically buys it for them. Maybe it would take two years or so to accumulate a bitcoin to buy reddit gold but that's at least turning users into money.

Am I an idiot or did I just discover an alternative to the ad supported model for free content sites?

<comment><USER20> If ASICs are shipped in the next couple of months, then very soon GPU mining will do as little as CPU mining does today.

</comment></comment><comment><USER28> I'd love to be a fly on the wall with the upcoming Reddit / Conde Nast accountant meeting.

<comment><USER29> Condé Nast isn't their parent company anymore, they moved up to Advance Publications (the parent company of CN).

<http://blog.reddit.com/2011/09/independence.html>

</comment></comment><comment><USER30> I would also buy a reddit gold subscription with Bitcoin.

</comment><comment><USER31> I don't see much of a downside as long as Reddit gives a price quote in bit coins and convert it the main currency Reddit uses, probably USD. The main problem for main-stream business using Bitcoin I think is the volatility. Losing money because the currency fluctuates around 10% every 30 days is horrible.

<comment><USER32> There are services (e.g. Bitpay) which remove the currency risk for the seller. The seller can set their price in US $ or whatever and receive the payment in that currency while the purchaser has the option to pay in bitcoins at the going rate at the time.

</comment></comment><comment><USER33> Are there any privacy concerns with bitcoin? For instance, suppose I know the day someone got reddit gold. Could I then check the bitcoin transaction database (eg: bitcoin block explorer) to extract the ip of the users that created the account that day, and link this account to other transactions and try to pin down the individual?

This is just a far-fetched scenario, but I'm wondering what would change when money stopped being so anonimous as it is now..

<comment><USER34> You would be able to find their wallet address, but good practice is to use a different wallet address for each person you send payments to. The blockchain doesn't keep track of IP addresses to preserve anonymity.

</comment></comment><comment><USER35> Original source: [http://www.reddit.com/r/Bitcoin/comments/12vg6t/the_admins_a...](http://www.reddit.com/r/Bitcoin/comments/12vg6t/the_admins_are_considering_bitcoin_as_a_new/c6ykjba)

OP, _"Please submit the original source. If a blog post reports on something they found on another site, submit the latter."_ \- <http://ycombinator.com/newsguidelines.html>

<comment><USER36> -1. The Reddit comment has almost no context. The BetaBeat article explains Reddit, Reddit Gold, Reddit Gold pricing, Bitcoin, contains a full quote of the comment, and a number of other things that random Hacker News readers might lack complete context on.

<comment><USER35> Whatever helps you sleep at night.</comment></comment></comment>


Show HN: DVC 1.0 release. 5 lessons from 3 years of building open-source ML tool - <AUTHOR> Hey HN, creator of DVC here!
DVC (https://dvc.org/) is known as Git for data projects. Technically, DVC codifies your data and machine learning pipelines as text metafiles (with pointers to actual data in S3/GCP/Azure/SSH) while you use Git for the actual versioning. DevOps folks call this approach GitOps or more specifically in this case - DataOps or MLOps.
We've been working towards 1.0 since we started 3 years ago. What began as my pet project now has 100+ code contributors, 100+ documentation contributors, and thousands of users.
Our community has taught us a lot - here are some of the biggest lessons:
1. Users say the serverless and distributed nature of DVC (inherited from the underlying Git) is one of its "killer features".
2. To share ML projects within and between teams, it's not enough to track only files and pipelines. You also need metrics, plot and hyperparameter tracking. In DVC 1.0 we implemented hyper-parameter diffs, metrics and plot diffs right from Git history.
3. In DataOps, data transfer optimization is huge. Large deep learning models, millions of images in datasets, etc. We doubled down on optimizing 1.0.
4. ML pipelines evolve faster than data engineering pipelines and need to be easy to change. In 1.0, we've simplified the pipeline metafile format.
5. More and more teams use DVC as a part of CI/CD for ML and other MLOps tools. DVC is used under the hood in the CD4ML tool that was described in the canonical post on Martin Fowler's blog: https://martinfowler.com/articles/cd4ml.html. We built 1.0 with CI/CD users in mind.
More details on https://dvc.org/blog/dvc-1-0-release.
Happy to answer any questions here or at DVC Discord chat https://dvc.org/chat. <comment><USER2> How do you make money? Sorry to be that guy; I wouldn't ask this if you didn't claim "Happy to answer _any_ questions here".

<comment><AUTHOR> Good question! We build separate products (that use DVC) for monetization. No plans to monetize DVC.

Some analogy - Git is free for versioning, GitHub/GitLab as monetization.

</comment></comment><comment><USER3> Is it possible to use DVC within the new implementation of GitHub Actions? I checked it out on the website and apparently it looks like it supports it, but I wanted to know more about how you guys are getting ready for this new CI / CD feature?

<comment><USER4> We have a lot in store here- we are unrolling a new tool for CI/CD soon that works with GitHub Actions & GitLab CI. Adding run-cache to DVC 1.0 is just one way of preparing for more CI/CD uses of DVC. (FYI I am part of DVC)</comment></comment>


Real time customer support and here's how we do it - <AUTHOR> http://assistwindow.com/bwm/posts/real-time-customer-support-with-olark

<comment><USER2> My company uses Olark and loves the integration with gchat. As a developer, I like the Olark API: <http://www.olark.com/documentation></comment>


Send USBs to N.Korea to help spread freedom of information - <AUTHOR> https://flashdrivesforfreedom.org/ <comment><USER2> Great sentiment, poor design thinking.

With any technology used in the support of freedom, especially in oppressed countries, the technologist has to ask 'How can the bad guy turn this around?'.

USB has been a common vector for viruses. North Korea is rumoured to have a passing familiarity with creating cyberweapons. So they could intercept the USB sticks, infect them, and then let them get redistributed. Then all these people will use the sticks, the virii will call home with personally identifying information, and NK will either jail or closely watch the people you are trying to help. And that assumes that this effort isn't wholly an op by NK.</comment>


Spritz: A spongy RC4-like stream cipher and hash function [pdf] - <AUTHOR> http://people.csail.mit.edu/rivest/pubs/RS14.pdf

<comment><USER2> And a C implementation: [https://github.com/jedisct1/spritz](https://github.com/jedisct1/spritz)</comment>


Electronic Medical Records, Built For Efficiency, Often Backfire - <AUTHOR> http://www.npr.org/blogs/health/2014/11/07/361148976/electronic-medical-records-built-for-efficiency-often-backfire?utm_source=facebook.com&utm_medium=social&utm_campaign=npr&utm_term=nprnews&utm_content=202507

<comment><USER2> I'm a resident physician. EMRs are simultaneously awesome and awful. Awesome because most of the data and past records on a patient are a click away, just as promised.

Awful because they're invariably run by underpowered servers and delivered via laggy-as-hell Citrix windowing to crappy end-clients with smallish monitors.

Awful because they are all proprietary silos, and good luck migrating your hospital system off one and onto another. The idea of a portable electronic health record is largely a fiction at this point.

Awful because so much of the documentation you have to write has nothing to do with patient care, and more to do with defensive medico-legal CYA, and everything to do with the hospital getting reimbursed at the level it wishes to for each patient seen.

Awful because the software is legacy enterprisey garbage with a codebase written in MUMPS (look it up!) or some other esoteric language in the 1980s and carried forward with ongoing encrustation and decay since then. Awful because the UI actively gets in the way of our patient care workflow, and not the other way around. Need to look at lab data while writing progress notes on a patient? Too bad, you can't open both at once! Need to look at lab data or inpatient notes while discharging a patient from the hospital? Too bad, there are two layers of modal dialog boxes related to the discharge blocking access to anything else while you're discharging.

I've used the two market-leading big-enterprise EMRs in the US, Epic and Cerner, and they're both like this.

I used to write software for a living. The crappiness of even the most expensive EMRs (the ones that will set a big hospital system back a couple hundred million for an initial install, like Epic) completely floors me.

<comment><USER3> Do you have an opinion on or know much about newer / more user-friendly EMR's? I'm specifically thinking of PracticeFusion, but I imagine there are others.

Why doesn't stuff like this take-off? Is it the product of slow-moving hospital bureaucracy, or are there other reasons?

<comment><USER4> The CFO of large healthcare organizations is historically who decides what software is purchased and used. Why the CFO? Because most enterprise healthcare software is built to support the billing/accounts receivable department.

It's very difficult to find an EMR that focuses on patient care and communication. The cynical side of me thinks that there isn't an incentive to build this kind of software. Doctor's aren't paid to make you healthier; they're paid to submit to insurance companies the correct diagnosis and procedure codes.

<comment><USER2> Well, there isn't an incentive because typically it isn't physicians (or, at best, it is physicians who are primarily administrators and somewhat out of touch with clinical practice) who are making the purchase decisions.

I can tell you that most clinicians would be jumping for joy if software that was more physician- and patient-oriented became the norm.

</comment></comment></comment></comment><comment><USER5> Electronic Medical Record systems, and their non-healthcare counterparts in Enterprise Resource Planning systems, never delivered the cost savings and efficiency they promised because, at the end of the day, they're designed to streamline the billing/finance department, not the rest of the organization. The user interfaces are universally awful because they're designed to 'get finance/billing the information they need' rather than 'streamline the process for practitioners,' and at the end of the day the people buying the system aren't the ones who have to use it.

Someone once joked to me that an EMR installation was where you replaced all the people in your billing department with the same number of people, at a higher salary, in your IT department.

</comment><comment><USER6> I had a client that made an EMR that was built on the concept of disease management. The basic idea was that by giving the doctor the right data and questions to ask, the doctor would help the patient manage their chronic condition and help them not need to come to the office so often. Doctors loved it. Patients loved it. Management didn't buy it because it lowered per patient revenue.

<comment><USER7> _Management didn 't buy it because it lowered per patient revenue._

Another flaw of insurance covering normal doctor visits.

<comment><USER8> My insurer denied helicopter EMS for a patient due to lack of prior approval. Hmm, "On scene of an MVA, patient needs trauma center, call insurer".

Imagine this in auto insurance. "Sorry, your car accident coverage was denied because you didn't get prior approval."

</comment></comment></comment><comment><USER9> Speaking as a physician, I can tell you that EMRs range from "mostly not awful" to "warcrime."

<comment><USER10> did you ever do paper records or have you always done electronic records?

<comment><USER9> I have done all paper, all EMR, and a combination of the two. The combination is the worst. So far paper alone is the winner for most situations, although even crappy EMRs have advantages over paper. Most of the problems, as I see them, come from the general baggage of enterprise software (not sold to the people who use it, likely focus of dev time is on accounting/billing stuff I never see) and an attempt to slavishly emulate paper in digital form.

</comment></comment></comment><comment><USER11> EMR is not dysfunctional because it's EMR; it's dysfunctional because it's enterprise software for the Windows desktop.

It's very easy for the HN demographic to forget what it's like to use an "average" Windows machine. Startup and login times in the minutes to tens of minutes, extreme disk latency everywhere, nagging popups to update Java (has happened in my doctors' office more than once) and virus definitions, etc. Cheapass hardware is frustrating by default.

A surprising number of network-based Windows line-of-business applications are written with inexcusable naïveté - the application simply doesn't respond while waiting for network/server and a request that gets dropped or whatever will render the entire application inoperable for several minutes (or some other irrationally high timeout, if there even is one) unless you kill and reopen it.

Because Windows desktop management is so brittle/difficult/expensive and so many applications don't play well with MSI, updating is a manual process of walking around the practice and waiting for computers to be unoccupied so that you can load the new .exe onto them. In practice this sometimes just doesn't get done.

Add on the incredibly poor software quality that comes with lowest-bidder offshore development, zero attention to UX, and very aggressive and well- funded sales teams, and you get roughly the situation we have today.

I can't wait for somebody to take this on with a well-written web or Linux application. Even better if it targeted a domain-specific, stripped-down, locked-down Linux distro that could be netbooted by thin clients around the practice and deployed in a high-availability setup inside the firewall.

<comment><USER12> >Even better if it targeted a domain-specific, stripped-down, locked-down Linux distro that could be netbooted by thin clients around the practice and deployed in a high-availability setup inside the firewall.

I can hear it now...

 _" This looks great, but my staff only knows about how to use Windows computers; I'll have to retrain them to use these strange new Lee-nuks things. Can't you just make this run in a little box like on the other guys' systems, so I can keep my Windows PC workstations? I'd like that because our time-clock software is only available for Windows and that will save us having to keep two CPUs and screens on the desks."_

<comment><USER11> Maybe because it was their first (and only) EMR system, but in the practice I worked at, all the initial infrastructure was put in by Allscripts explicitly for EMR, with all other applications being secondary. They sold the boxes, set up the Windows domain, and managed everything. Only relatively recently did they start using it as their own infrastructure, instead of just the magic to make Allscripts EHR work. In that kind of scenario, it could just as easily have been Linux.

<comment><USER12> > In that kind of scenario, it could just as easily have been Linux.

In any of these systems, any general purpose OS would suffice. There is no real problem for people to move from one platform to another that can't be overcome in a short time.

</comment></comment></comment></comment><comment><USER13> The worst part of "billing" driven EMR's is that doctors get really good at recording procedures and diagnoses to fit what the insurance companies want to hear, not what the real medical condition is. This has the effect of distorting the patients reality polluting the medical data, rendering it mostly useless or of poor quality for any decent medical research work.

You got a throat ache, but that is not payed by the insurance? uhhmm immediate upgrade to a pharyngitis.

</comment><comment><USER14> There is a great chapter in the book "Hacking Healthcare" that addresses the challenges met with writing software that bests paper. I have linked it below for any software engineers interested in this problem space!

[https://www.safaribooksonline.com/library/view/hacking- healt...](https://www.safaribooksonline.com/library/view/hacking- healthcare/9781449309602/ch04.html)

</comment><comment><USER15> This isn't a problem with EMR. This is a problem with poor UX in the software that the doctors are using. Here's the thing about paper records: they're costly as well. Paper records need to be photocopied, faxed, paged through, sorted, filed, unfiled, etc. God forbid the doctor needs to physically move a wall of patient records, or a disaster in the office. EMR are theoretically just O(1) to access, update, save, and send. Not true with paper records.

I obviously don't know the true statistics, but I'd hasten a guess that although EMR costs doctors a few extra minutes per patient but cuts the overhead of maintaining and handling those records by an order of magnitude. If the biggest complaint is information overload (as the article seems to suggest), the whole system could be completely "fixed" with a simple OS X system preferences-style quick search box.

<comment><USER16> Don't forget about the inherent difficulty in doing retrospective analyses on paper records.

Want to see how drug X affected lab levels of Y on your organization's patients for the past Z years? Good luck with paper records.

<comment><USER17> Reminds me of my employers Oracle Financials rollout, which makes submitting a travel voucher a 30 minute chore that takes 3x longer to process.

Sure, somebody could analyze our 150,000 annual vouchers and figure out some sort of insight. But that's unlikely, as the people who have access to the data care only about payment KPIs, and accessing any data in the system is a nightmare for anyone else.

The scenario you describe is problematic as well. The liability associated with drawing conclusions about the efficacy of a drug based on unscientifically collected data would probably make it difficult to use it to drive decisions.

</comment></comment></comment><comment><USER18> If these medical records were in an open/standard/unencumbered format there would be startups all over this.

<comment><USER16> Here's an open-source one in Canada that has about a 20% market share in Ontario: [http://oscar-emr.com/](http://oscar-emr.com/)

Built by an engineer-turned-physician.

<comment><USER7> Thinking about it, it is probably no coincidence that this was done in Canada which has public healthcare.

</comment></comment></comment><comment><USER19> Pre-EMR, I thought most docs dictated their notes on tape that were then transcribed to the medical record by clerks. Why are doctors having to keyboard this stuff now?

<comment><USER20> Because no hospital system wants to pay for transcriptionists when they can mandate the MD do the data entry themselves. You hear the big backlash from the private practice MDs who still used a combination of paper chart and dictation that are just now being forced to move to EMR through MU 1 and 2. And with the cost of buying a whole new EMR rollout and support contract, find they can't afford to pay the transcriptionists too.

</comment></comment><comment><USER21> I'm a nurse/programmer and working with EMRs are maddening. The problem is the requirement that EMRs are built to protect against every potential thing that could go wrong. As a nurse, I'm spending a minimum of an hour a day clicking boxes and copy & pasting CYA lingo for all of my patients to protect against the 1% scenario.

</comment><comment><USER13> The issue with most (if not all) EMR's is that they are driven by the business side of things, not for the medical providers.

</comment><comment><USER22> what do the equivalent systems look like in countries with nationalized healthcare systems?</comment>