The Programming Interview Experiment - <AUTHOR> https://sourcegraph.com/blog/programming-interview <comment><USER2> At this point in my career (professional developer since the late '90s), I am extremely picky about who I work for. The interview is a two way street, where the prospective employee also gets to ask questions to the potential employer.

In the described workflow, the candidate must spend several hours before talking to anyone in depth at a company. This scenario would simply not work for me. I will not spend hours of my time for a company that I potentially might not want to work for.

<comment><USER3> Would your rather that they have you come on-site and perform the coding challenge for half the day, then continue their normal interview process for the other half the day?

Or worse, go through a normal 5-6hr interview loop doing random questions on the whiteboard?

I think their approach is much more flexible and approachable. The overall time commitment is the same either way.

<comment><USER2> You can achieve a mutual understanding during the initial technical (not HR) phone screen. This approach eliminates the candidate's ability to ask questions before a multi hour commitment. The traditional phone screen is important. Most questions cannot be answered by HR.

I have a much better understanding a company after the question portion of a half hour phone screen than I can get from searching the web for information about a company.

</comment></comment></comment><comment><USER4> To start, I think your interviewing practices are significantly better than the stock whiteboard-based version (and your criticisms thereof are spot-on).

That said, some concerns:

Leading with the source graph challenge seems a bit much. My experience has been that it's necessary to have at least _some_ filter for the unwashed masses before requiring that much work of someone. At my last company, we had a 30-minute non-technical talk with a founder (later replaced by our well- trained recruiter/office-manager). This helped filter out the people who weren't what we were looking for at a really high level (e.g., an ops engineer applying for a dev role, a really junior person applying when all we have room for right now is a senior one, a dev who only knows pascal, etc) before making them invest the time required for an in-depth take-home test.

Also, what's with the 24-hour limit? It seems too long to prevent cheating, but short enough to be annoying. For similar challenges, I've given people as long as they want (up to a few weeks), but asked for a blow-by-blow writeup of how it went and how long it took them. We may have gotten some liars, but we never had someone arrive on-site who claimed it took them 2 hours when, based on their skill, it probably took them much longer. People seemed reasonably honest about it, and the convenience seemed to be appreciated.

Asking to see source code for a significant project is tricky. I know good developers who wouldn't have any code to show for that because everything significant they do they got paid for, and can't in god conscience show to someone else. For example, I worked pretty long hours at my first job and built some interesting stuff, but I didn't have much time for side-projects because of it. A friend of mine works normal hours now, but does contracting on the side. Plenty of good code, but nothing he could show to someone else. That said, asking to see some source is very high-signal, so it may be worth the tradeoff of only selecting people with significant free side projects.

That said, it's still a way better process than teasers and whiteboard algorithms. If I were looking for a job, I'd definitely apply. :)

<comment><USER3> But what if the ops engineer can pass the coding challenge? He maybe trying to change roles.

Or the dev that only knows pascal - if they can pass the challenge using pascal then I'd think they could easily learn whatever coding language your team uses. Learning a language is easier than learning coding.

That's what I like about coding, and about these blind challenges - it's purely skill based. If they want to apply and can pass the test, they they're worth talking to. If the wrong people are applying (and passing) then your job description and challenge are wrong and need to be calibrated.

<comment><USER4> It is possible that someone with little dev experience could complete our challenge. We were mostly trying to avoid wasting time: theirs to complete the challenge and ours to review it, if we thought it was extremely unlikely they'd complete it.

However, when I referenced the hypothetical ops guy, I was referring more to a misalignment of goals: someone looking for a job where they'd be doing more devops stuff vs our need for a dedicated developer.

Regarding the hypothetical pascal developer, it really depends on your need. I've been in the position to hire bright people who can learn our tool set on the job, and that's great. I've also been in the position where we can't afford a month or two while a new dev learns our language an framework before they become productive. When in the latter position, it makes sense to weed people you're not interested in out early.

I would love it if a well-written job description would prevent unqualified people from applying, but my experience has been that it just doesn't. That's why some people use fizz buzz ([http://www.codinghorror.com/blog/2007/02/why- cant-programmer...](http://www.codinghorror.com/blog/2007/02/why-cant- programmers-program.html)). That's why we used a short phone screen.

<comment><USER5> > However, when I referenced the hypothetical ops guy, I was referring more to > a misalignment of goals: someone looking for a job where they'd be doing > more devops stuff vs our need for a dedicated developer.

If someone were looking for a devops-oriented job, why would they be applying for your non-devops-oriented position?

<comment><USER6> Do you think it's a bad idea to have a high-level conversation between the company and the prospective employee before assigning a take-home test?

<comment><USER5> That depends on the attitude you go into the conversation with and whether your take-home test can be automatically scored well enough for a coarse filter to reject the people who obviously cannot do it. It also depends on how many applicants you have. Spending half an hour per candidate gets onerous when you have hundreds, so cutting to the chase in a semi-automated way may be a better means of whittling the pile down than having a recruiter, founder, or hiring manager filter it based on conscious or unconscious biases.

I was highlighting that line because I think it indicates a subtle bias towards people based on background and assumes people are not making reasoned decisions about what they are applying to. I understand that there are tons of unqualified resume blasters out there, but I figure nearly all of them will either ignore the take-home or submit such schlock that it will be easy to automatically reject them.

</comment></comment></comment></comment></comment></comment><comment><USER3> "Bring the source code—we'll ask to take a look at it."

Seriously?? What actual company do you know that would allow its IP to walk out the door? And possibly to a competitor?

Additionally, projects are usually the work of a team, and are heavily iterated upon. "your contribution" is usually hundreds of little changes building upon other changes, not "this is the application I wrote".

<comment><USER7> This is a good point, and one I forgot to address.

We would never ask to see proprietary source code. Hopefully, a candidate would have a side project or open source work they're willing to share. If not, we'd definitely work around this. For team projects, we'd ask them to describe their contribution. So far, this hasn't been a big problem, but we're still iterating and making the process better!

<comment><USER8> The article is definitely not clear about this. In fact it says "bonus points if it's open source" implying that you expect to see proprietary work. It was a bit of a red flag for me while reading.

<comment><USER6> I interpreted that as "bonus points if it's contributions to a big official Open Source project, as opposed to your personal pet project with no exposure or users".

</comment></comment></comment></comment><comment><USER9> In 12 years and more than 10 clients I have yet to run into any developer so incompetent technically as not to be able to perform his/her job effectively. However, what I do run into are individuals who cannot conduct themselves appropriately, those who cannot work in teams, and those who overlook delivering the value for their client/employer that they were hired to deliver.

All of these individuals can code, no question. However, they are impediments to accomplishing the business goals that my teams are tasked with solving.

I find it interesting that everyone I know who has been terminated or laid off has been done so for reasons not related to technical ability, however technical ability is the majority of what's interviewed for.

I think it is much more important to hire people who can work with others and deliver the value desired, but yet this is not what most employers interview for. In my opinion, if I can get a credible recommendation that a candidate is technically competent, then I don't think spending an extra minute on technical questions is important. I would rather be sure they they are competent in all other aspects of the role.

</comment><comment><USER10> As someone who completed grad school and went through many interviews not too longer ago - I can tell you that this type of process is a much, much more enjoyable than the typical phone + whiteboarding interviews.

I think that some companies become so intensely focused on hiring the best candidate that they sometimes lose touch with the fact that the candidates are human too. Interviewees are rational and trying to find a job that has a best fit. By taking the time to give them a project to work on, it allows them to get a taste while demonstrating that you have respect for them.

My favorite interview was where a company gave me a sample data set, and basically said "Do anything with this in python". I ultimately didn't get that job (not enough experience), but I remember that company and the guy who I went over my creation with, and now respect them more than the dozen other companies who made me feel like I was on a conveyor belt.

</comment><comment><USER11> What about candidates who don't have several hours to spend on a programming assignment? Everyone seems to want one nowadays, and I don't have time to do one for everyone who asks.

I've done a bunch of programming tests, without progressing to the interview stage after doing them, and now I usually refuse.

<comment><USER12> Agreed.

While I do think that there is plenty wrong with traditional programmer interviews and that new approaches are needed, it isn't really a scalable solution on the candidate's side to spend hours to days on challenges for individual companies if everyone is asking for one. It becomes sort of like everyone wanting their own app store, or their own always-running polling service running on a desktop or mobile or whatever. If there are just a couple of people doing that, no big deal, but once more than a couple of entities start doing it, the system falls apart. If you're Google, Apple, Github, or some other prestige employer you can probably get away with this system even despite the scalability issue; otherwise good luck with it.

As a practical compromise, companies should be willing to waive the specific challenge if the candidate already has their own existing shareable source code they can point to, thus allowing a candidate to reuse the same example as a portfolio piece with multiple companies.

However, few companies are likely to do this since the same basic ego problem that breaks traditional programmer interviews comes into play ("Well we do things very different here, we're very special, so we must make the candidate jump through hoops X, Y and Z").

<comment><USER11> What makes it even more insulting is that, I do the assignment, and still no interview.

If you give a programming assignment, then you should at least have the decency to give an on-site interview to everyone who gives a solution that compiles and runs and gives the correct answer. That's why I usually refuse to do them now.

Not only am I spending a couple hours on someone I never met, the conversion rate is so low that I'm convinced it's a waste of time.

Also it's insulting. You're asking me to waste a couple hours of time for the privilege of maybe getting a chance to talk to you? When "maybe" turns out to be "almost never", I've really soured on the pre-interview programming test.

</comment></comment></comment><comment><USER13> Why ask to see source for a project above and beyond the work-sample test you've already administered?

I think work samples are a wonderful way to screen candidates (really, the best way) but to presume that all qualified candidates will have an additional sample of code (to the scale of a "significant project," even) available upfront for you without requiring an additional commitment seems disingenuous.

You end up applying an implicit filter for "developers who feel confident and comfortable contributing to OSS and/or possess copious free time for a personal project," which probably isn't really what you were trying to select for and tends to unfairly filter out certain types of candidate.

</comment><comment><USER14> Doing real code is very important. I'm curious if the OP has had any instances of catching cheating. I've seen instances of this with work products in other areas.

</comment><comment><USER15> This is a great post and I agree with everything you say and I am glad you guys are doing this.

I just wish you had more information about your company online (Angellist/Crunchbase etc). I think few people would be willing to put in 3-4 hours on a coding challenge unless they had a good idea about your funding, size etc.

I might have got a coding challenge from you a few weeks back but skipped it because I didn't have enough information. Ah well.. might do it this weekend.

Hope this helps!

</comment><comment><USER16> Not in a single interview ever have I been asked about the runtime of an algorithm. On the other hand it is mentioned in almost every article about interviews for programming jobs. How many companies do actually ask such questions? If your future job is mainly building standard business applications, web sites or whatever, no one cares about theoretical runtimes. If your future job is somewhere in research, game, database or operating system development or some other fancy stuff, no one will care to ask for trivial things like asymptotic runtimes. Personally I think most of the time people - interviewers and article-about-interviews writers - just mention Landau notation because they think it makes them look smart without realizing how trivial it actually is.

<comment><USER3> All those companies should be care about algorithmic runtime, it's part of performance and scalability. If you're not thinking about that then you're not writing production software.

I ask about runtime at every interview (part of "coding, data structures, and algorithms"). It is most definitely not a trivial concept - how do I define the correct solution to a problem when there are multiple approaches?

The requirements for a correct solution (even simple coding problems) include style, correctness, performance, and scalability.

<comment><USER16> I meant that Landau notation is a trivial concept, not finding a good algorithm to solve a given problem or determining the runtime of a given algorithm.

</comment></comment></comment><comment><USER17> What would happen if an interviewee 'flipped the script' \- gave the interviewer a coding challenge? Maybe under the guise of assessing the skill level of his potential co-workers? Is there any way an interviewer takes the bait?

<comment><USER17> Or, what if the interviewer explicitly challenged the interviewee to come up with an interview coding challenge? And perhaps even worked the challenge, with the interviewee hinting, helping, and evaluating? Maybe it would be a good way to demonstrate complete command of a subject, or the ability to think on one's feet, or even to evaluate the candidate as a teacher/leader of younger developers?

</comment></comment><comment><USER18> This part is the key, to me:

>The second thing is the strong focus on topics taken from >classroom algorithms and the lack of concern about >practical programming ability. A candidate might somehow >remember that the way to implement an optimal string >suffix matching function is to use a suffix tree, but >where is the question that gauges the ability to create, >test, and deploy a complete application in a reasonable >amount of time? Which skill is more important to you?

<comment><USER19> I thought that was a an unfair comparison. One is a small piece of trivial knowledge, the other is a huge amorphous amount of voodoo and experience. But I agree that practical knowledge trumps theory.

</comment></comment><comment><USER20> I had an interview once where I did not have to write a line of code - it felt very refreshing to have my capabilities respected, although I just somehow lucked out it turned out.

<comment><USER4> Interesting- I have the opposite reaction. I would (and have) refused to work for a company that doesn't ask me to code in an interview. It may be nice for me, but it means they probably didn't ask my potential coworkers to code either, and I don't want to work on a team picked like that.

<comment><USER20> Well, my GitHub is public (and was probably convincing enough), and everyone but me at the company was asked to code during the interviews - there was evidence on the TV from someone who coded for an interview that morning even. I was found from a local AngularJS meetup, and someone at the company was trying to gently prod me towards interviewing with them over the course of months...which wasn't so much of an interview process as seeing whether I was interested, since they clearly wanted me for my expertise.

</comment></comment></comment><comment><USER21> When I get asked things like: "Write a function that finds the longest positive-sum subsequence in an int array and runs in O(n) time."

I really wonder if those places ever find people that can do this. I never get offers from the places that ask this. I always get offers from the ones that have a similar process to the one in the article. I don't think I've ever even met someone who could answer these questions on-the-spot, with a marker on a whiteboard.

<comment><USER11> "Write a function that finds the longest positive-sum subsequence in an int array and runs in O(n) time."

That's a horrible interview question. If you know the trick (saw the question before), it's easy. If you don't know the trick, it isn't reasonable to expect someone to figure it out in 5-15 minutes.

That's a brainteaser question disguised as an algorithms question.

<comment><USER22> But there isn't really a "trick" here, though. The point of these questions is to figure out if the candidate has enough CS fundamentals to construct a reasonable solution to a problem that they've never seen before.

<comment><USER11> There is a trick. Loop over the array, remembering the maximum and minimum. If you've seen it before, the question is MUCH easier, making it a bad question.

</comment></comment></comment></comment><comment><USER23> Anyone interested in doing more complicated questions during the phone screen should check out [https://coderpad.io](https://coderpad.io) \- you get a live REPL or compiler (depending on whichever language you're using), and can run code instantly from your browser.

The best part is the environment is shared between you and your candidate, so you can both see the code being written and execute, live.

</comment><comment><USER24> I would add that the respect for candidate time is critical. I applied to one place and they requested a similiar coding project but this was just before the christmas holiday and they gave me a deadline of christmas day which just struck me as a little arrogant to think that I am really going to be doing their challenge on christmas.

</comment><comment><USER25> Whenever a post like this comes up I'm always curious why so much of developer interviews are focused an algorithmic performance. Is it because it can be mathematically proven while so much of what otherwise makes "good code" is subjective?

</comment><comment><USER26> Asking candidates to write actual code for your real app could be legally problematic.

Legally, the candidate will have the copyright to any code he writes unless there is a "work done for hire" agreement in place.

</comment><comment><USER27> This interview process assumes candidates don't have anything else to do. I think they should pay the candidate for time spent looking at their code and making suggestions for improvements

</comment><comment><USER28> It's a total nit and pet-peeve for me, but one does not "Architect" code. You architects domains.

You "design" code.</comment>


The Birth of ZFS [video] - <AUTHOR> https://www.youtube.com/watch?v=dcV2PaMTAJ4 <comment><USER4> rsync.net supports ZFS send and receive to their cloud storage platform:

[http://www.rsync.net/products/zfsintro.html](http://www.rsync.net/products/zfsintro.html)

<comment><AUTHOR> How have you dealt with the issue of byzantine streams? This is something that we have wanted to solve for a while, but it's a really nasty issue -- and at the recent OpenZFS Developers Summit we concluded that it wasn't practical. Instead, we're looking to tack more towards signed ZFS streams[1] -- though that work is only in the design stage. How has rsync.net solved this?

[1] [https://github.com/joyent/rfd/blob/master/rfd/0014/README.md](https://github.com/joyent/rfd/blob/master/rfd/0014/README.md)

<comment><USER4> I _think_ I know what you're referring to, but we've never called them "byzantine streams".

Are you talking about snapshots that have been broken by extended attributes bugs, or "disappearing" snapshots ... and then what happens when you send/recv them ?

Actually, either way, would you email email@example.com so we can chat over email ?

<comment><AUTHOR> I'm talking about a ZFS stream that has been deliberately corrupted and then sent to you to be received as a filesystem -- when it will in fact cause ZFS to panic (or worse). So I mean "byzantine" in the "characterized by deviousness" sense: someone actively trying to do harm to you. Or perhaps I have misunderstood the functionality that you offer?

<comment><USER4> Ok, that is not what I was thinking of. We have made no specific provisions for this.

 _However_ , we were scared enough about "unknown unknowns" that we set up this aspect of our service such that every ZFS send/recv customer gets their own zpool inside their own bhyve.

So it might be a DOS attack, but I don't think there is a data attack against rsync.net here. Comments ?

Unrelated: I'm curious - when I speak of snapshots corrupted by extended attributes or "disappearing snapshots", do you know what I mean ? Maybe we should talk :)

<comment><AUTHOR> I don't know enough about bhyve or your implementation to be able to comment, but if you assume that any system that has ever received a ZFS stream from a user is entirely compromised, then perhaps you can work around it that way. I'm not aware of a disappearing snapshot problem; has there been something sent to the ZFS list about this? Would have been a good topic of discussion for the OpenZFS Developers Summit a few weeks ago... ;)

<comment><USER4> Yes, that's exactly the model - these users have their own zpool inside their own VM.

</comment></comment></comment></comment></comment></comment></comment><comment><USER5> So, are Oracle still developing their ZFS separately, or they switched to OpenZFS?

<comment><USER6> Separately. However their changes and OpenZFS changes are now effectively incompatible.

Funnily enough is that OpenZFS is now the more advanced filesystem minus the clustering features Oracle integrated (shared disk clustering that is, not shared-nothing).

<comment><USER7> I've not followed closely but got any references for that ("the more advanced filesystem")? From what I can see Oracle's ZFS still has some features that OpenZFS lacks (on disk encryption support being one of the most obvious) and there's been lots of development in the last few years. [https://blogs.oracle.com/zfs/entry/welcome_to_oracle_solaris...](https://blogs.oracle.com/zfs/entry/welcome_to_oracle_solaris_11)

Disclaimer: I work for Oracle on Solaris (but not ZFS).

<comment><USER8> >on disk encryption

Given you could just run LUKS on top of (open)ZFS, it's probably the better security position to run an audited, established encryption product, than to consider a layer thrown on top of ZFS a feature.

<comment><USER9> There are major benefits to be had from moving the encryption layer on top of the volume management/storage EDAC layers which ZFS provides: in particular, it'd be nice to be able to scrub a locked dataset. I think (but haven't seen this firsthand) that Oracle's implementation offers that benefit.

<comment><USER10> You can still scrub a dataset with GELI ([https://www.freebsd.org/cgi/man.cgi?geli%288%29](https://www.freebsd.org/cgi/man.cgi?geli%288%29))

GELI creates a virtual block device that works great with ZFS, you get it all, self-healing, checksumming etc.

<comment><USER9> It looks like GELI goes below zfs just like dm-crypt. That doesn't allow you to scrub a locked dataset (one where the key is not in memory, possibly unknown). You could use zvols, of course, but that loses some (but not all) of the full-stack zfs benefits.

<comment><USER10> No, you create virtual block devices that needs to be "unlocked" before you can start using ZFS. If you need to scrub the dataset, you need to unlock all the virtual block devices, start ZFS and then run the scrub command.

The whole point is to run ZFS on top of virtual block devices which are encrypted with GELI and "unlocked".

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> Using zfs in production virtualization it´s fast and provides good integrity. You can basically get some of the hyped features of Docker the copyonwrite filesystem part with ordinary Linux virtualization.

<comment><USER12> I've been using it at home via FreeNAS after losing a bunch of old files to bitrot.

<comment><USER13> FreeNAS looks interesting, but I found the hardware requirements a bit prohibitive for home use. Renting storage in the "cloud" feels more cost- effective.

<comment><USER14> The hardware requirements are sometimes exaggerated. 8GB (non-ecc) memory on a cheap processor is fine for a home user with FreeNAS.

Whether or not it's cheaper depends on your use. Comparing to something like Glacier, FreeNAS is definitely cheaper if you need to store more than a couple hundred GB and/or access it frequently.

<comment><USER15> You are quite right that 8GB memory and a cheap processor is good enough for home use. However I thought the consensus is that one cannot skip ECC with ZFS because a silent bitflip can corrupt an entire zpool. In any case the extra cost is minimal so why not.

<comment><USER14> Running without ECC is no worse than doing so with any other filesystem [as far as I understand].

The thing about corrupting the entire zpool looks to be a bit of an urban legend ([http://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill- yo...](http://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-your-data/)). Flipping bits can of course cause problems, but doing a scrub after a bit is flipped in memory doesn't hose your whole pool. There's been endless discussion and flame wars on the topic, but I've never once heard someone who actually had it happen (because it can't really happen). You _can_ lose some data due to bit flips, you _won 't_ however lose everything in the whole volume like some people claim (unless you have something like an entire chip on the memory fail, then you're definitely screwed but neither ECC nor a different FS will help you there).

ECC is a _good idea_ and it's not hugely expensive, but there is one very big difference. That is, most people who might want to build a NAS have a decent bit of spare hardware lying around that they might be able to repurpose. That hardware is probably an old desktop machine and doesn't have ECC, so if you insist on using ECC then you can't use that old machine and have to buy a new mobo+RAM at least. I've been running ZFS for years without ECC doing weekly scrubs and never once destroyed anything.

Edit: the counter-argument/source of the theory that you absolutely need ECC is primarily this post [https://forums.freenas.org/index.php?threads/ecc-vs- non-ecc-...](https://forums.freenas.org/index.php?threads/ecc-vs-non-ecc-ram- and-zfs.15449/) if you want to the other side for yourself.

<comment><USER15> Thanks for the response. I think JRS's account has some serious issues with the writer conflating hardware failure (stuck bits, unreadable blocks, etc) with random error that is more common and inevitable, not to mention undetectable in principle without ECC[0].

Bit flip has been shown to cause both data loss/corruption as well as filesystem failure in ZFS[1]. The scenario they presented in the paper is probably less relevant for a home lab where most data are accessed infrequently and the real chance of catastrophic failure is probably below 0.1% per bit flip event, but the risk is real and will only get higher as storage densities go up.

People on FreeNAS forum tend to play on the extreme side of caution - <USER2> all if one is willing to cut corners on hardware other unsafe practices will probably follow, hence it is better to make it clear that certain acts are not worth the risk.

[0]:DRAM Errors in the Wild: A Large-Scale Field Study [http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf](http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf)

[1]: End-to-end Data Integrity for File Systems: A ZFS Case Study [http://research.cs.wisc.edu/adsl/Publications/zfs- corruption...](http://research.cs.wisc.edu/adsl/Publications/zfs-corruption- fast10.pdf)

</comment></comment></comment></comment></comment></comment></comment><comment><USER16> It would be interesting to see comparison between ZFS and Btrfs, ReiserFS in terms of features and development effort. Any idea?

<comment><USER17> RaiserFS is dead as far as I've read.

btrfs is a shitshow - sorry to having to resort to such words but I will refrain from using it ever again if I have a choice.

If you look at the btrfs wiki there a lot of features but most of these are horrible broken or implemented in a way that's surprising.

It's full of bugs and not-yet-implemented stuff - some stuff that comes to mind:

* RAID is utterly broken: \- [http://www.spinics.net/lists/linux-btrfs/msg48561.html](http://www.spinics.net/lists/linux-btrfs/msg48561.html) \- [http://www.spinics.net/lists/linux-btrfs/msg47845.html](http://www.spinics.net/lists/linux-btrfs/msg47845.html)

* Lot's of ENOSPC bugs (just look at the mailing list)

* You need to run the latest kernel (don't try anything below 4.2) \- if not: Data corruption bugs, deadlocks

* Quota is utterly broken and continues to be breaking (just look at the mailinglist)

* It's useless for databases or VM images (disabling datacow is a ugly hack IMHO) \- e.g. see here: [http://blog.pgaddict.com/posts/friends-dont-let-friends-use-...](http://blog.pgaddict.com/posts/friends-dont-let-friends-use-btrfs-for-oltp)

ZFS is not perfect either especially on Linux it's lacking tight integration into the VM subsystem and at least a few versions ago was prone to deadlocking, too. However it's not eating data and compared to btrfs it's rock solid.

<comment><USER18> > * RAID is utterly broken: - [http://www.spinics.net/lists/linux- > btrfs/msg48561.html](http://www.spinics.net/lists/linux-btrfs/msg48561.html)

 _sigh_ From the message that you linked:

"FWIW... Older btrfs userspace such as your v3.17 is "OK" for normal runtime use, assuming you don't need any newer features, as in normal runtime, it's the kernel code doing the real work and userspace for the most part simply makes the appropriate kernel calls to do that work.

But, once you get into a recovery situation like the one you're in now, current userspace becomes much more mportant, as the various things you'll do to attempt recovery rely far more on userspace code directly accessing the filesystem, and it's only the newest userspace code that has the latest fixes."

If you look, you see that the guy reporting the problem was using a _very_ new kernel, but a _one-year_ -old btrfs-tools package. It makes no sense to use tools that are _substantially_ older than the kernel you're running. He _might_ have had much better luck with recovery (that is, he might have been able to get a rw volume with all his data, rather than an ro volume - <USER3> all his data_ \- out of the process) with a recent tools package.

> [http://www.spinics.net/lists/linux- > btrfs/msg47845.html](http://www.spinics.net/lists/linux-btrfs/msg47845.html)

Maybe I'm reading _this_ one wrong, but it looks like the summary of the issues are:

* For multi-disk arrays with disks of varying sizes, each smaller disk does not receive data until all larger disks have as much or less free space than the smaller disk in question. [0]

* The read scheduling algo is currently naive, using PID evenness to determine the disk in a pair to read, rather than current IO load on the disk.

Did I miss something, or misunderstand something? If I didn't miss anything, I don't see how you can get "RAID is utterly broken" from "RAID has less-than- optimal write and read scheduling".

> * It's useless for databases or VM images (disabling datacow is a ugly hack > IMHO)

I can't agree. I use btrfs on spinning rust to store my multi-terabyte Postgres 9.4 DB. It works well enough for my low-to-medium-volume [1] workload.

> * Lot's of ENOSPC bugs (just look at the mailing list)

I don't run into those, and haven't for literal _years_.

> * Quota is utterly broken...

Yes, quota is a difficult problem that continues to be unsolved.

> * You need to run the latest kernel...

Yes. Agreed.

> ...if not: Data corruption bugs ...

I haven't run into any, _ever_ on the handful of systems on which I use BTRFS.

> ...if not: ... deadlocks

True. There _was_ that unlink/move deadlock in 4.1. I hit it from time to time when updating my copy of the metadata from the Gentoo Portage tree. Happily, it only blocked operation on the affected file. The remainder of the system kept running along, and a -ugh- reboot unblocked the operation. [2]

Look. I'm not saying that btrfs is feature-complete, suitable for _every_ workload, or that any given person _should_ use it for _their_ workload. I'm just saying that the situation is _far_ less dire than you're making it out to be. I've been using it as the rootfs for at _least_ one of my daily use systems for the past 5.5 years, and haven't had _any_ trouble out of it [3] in the past 2+ years.

[0] That phrasing was abysmal. Here's an example: Given an array of one each of 3GB, 2GB, 1.5GB devices. the 2GB device will get data after 1GB has been written to the 3GB device. The 1.5GB device will get data after 0.5GB of additional data has been written to each of the 3GB and 2GB devices (making a total of 1GB additional data written to the array before the 1.5GB device starts getting data). AIUI, if you're running in RAID1 mode, then the same logic applies, but with paired devices.

[1] Most definitely _not_ web-scale workload.

[2] I don't know if a unmount/remount cycle would have _also_ worked. The wedged operation happened on my rootfs, so I had no way to test this.

[3] Except for the unlink/move deadlock in 4.1.

<comment><USER17> Hi, sorry for my tone - it's beyond rational discussion - however I still think it's not totally coming from nowhere:

[RAID]

>sigh From the message that you linked:

The part below your quote is interesting.

> General note about btrfs and btrfs raid. Given that btrfs itself remains a > "stabilizing, but not yet fully mature and stable filesystem", while btrfs > raid will often let you recover from a bad device, sometimes that recovery > is in the form of letting you mount ro, so you can access the data and copy > it elsewhere, before blowing away the filesystem and starting over.

I have hotplug and online-replace with ZFS and mdraid. btrfs wants a special degraded remount that might or might not be able to recover to rw someday... replacing a disk online is not really possible. Handling a failing drive is not really implemented.

Don't call it stable or don't call it RAID. It's nice that I'm able to pull of the data of my ro-mounted degraded array when a disk fails.. however all of this stuff is not really documented and was very surprising to me. If you have production machine e.g. on btrfs RAID10 it sucks to not be able to replace a broken drive on the fly online.

ZFS/mdraid just do it..

[RAID II]

> * The read scheduling algo is currently naive, using PID evenness to > determine the disk in a pair to read, rather than current IO load on the > disk.

> Did I miss something, or misunderstand something? If I didn't miss anything, > I don't see how you can get "RAID is utterly broken" from "RAID has less- > than-optimal write and read scheduling".

Walking is a less than than optimal way to move from Moscow to Berlin. If I take a train I expect a train - btrfs is a train that occasionally stops and says: Well, just walk... okay, you see I'm kind of emotional invested in this shit. However it's not what's usually expected as RAID and it's not documented.

We have here a lot of applications that do exactly that: a single PID reads lots of stuff very often sequentially... so RAID1 doubles throughput. Well not on btrfs.

[ENOSPC]

> I don't run into those, and haven't for literal years.

4x4TB RAID10 (90% free) -> ENOSPC - this was somehwere between 3.16 or 3.19 - how can this even happen?

[Corruption]

> I haven't run into any, ever on the handful of systems on which I use BTRFS.

Good for you. Was hit by this nasty undeletable directory bug: [http://www.spinics.net/lists/linux- btrfs/msg35789.html](http://www.spinics.net/lists/linux-btrfs/msg35789.html)

And then this: [http://www.spinics.net/lists/linux- btrfs/msg45108.html](http://www.spinics.net/lists/linux-btrfs/msg45108.html)

Umounting and running btrfs --check repair is not really cool if you want some uptime...

[Deadlocks]

Currently btrfs deadlocks reliable in a OOM situation... With 4.2 things got better, before running some medium heavy Hadoop job or ElastichSearch indexing killed 20% to 50% of the machines due to deadlocks (with 3.19 IIRC)

> Look. I'm not saying that btrfs is feature-complete, suitable for every > workload, or that any given person should use it for their workload. I'm > just saying that the situation is far less dire than you're making it out to > be. I've been using it as the rootfs for at least one of my daily use > systems for the past 5.5 years, and haven't had any trouble out of it [3] in > the past 2+ years.

It's probably fine for personal use. But don't think about using it for something that is demanding. You can google the presentations that say: Use it! It's stable! Enterprise! It's not.

Maybe in 5 years. I just saw it breaking in too many strange ways to bother anymore with it.

<comment><USER18> > Hi, sorry for my tone - it's beyond rational discussion...

No worries. I don't give a shit about tone, just about content. Tone is next to impossible to accurately judge through text anyway, so I don't presume to understand someone's intent unless I have a pile of evidence. :-)

> ...btrfs wants a special degraded remount that might or might not be able to > recover to rw someday... replacing a disk online is not really possible.

But, from the email you quoted:

"...while btrfs raid will often let you recover from a bad device, _sometimes_ that recovery is in the form of letting you mount ro, so you can access the data and copy it elsewhere, before blowing away the filesystem and starting over."

And [0] indicates that you _can_ live-replace a failed drive in a btrfs RAID array. (The man page makes it seem that you might want to also pass the -r flag to the replace operation.)

I've never had the money around to amass the disks required to run my btrfs array in RAID1 data mode, so I can't test the claim of that Stack Overflow post, but I _do_ use it in _single_ data, _RAID1_ metadata and system mode. I've had a drive in that array suddenly drop out due to a firewire controller hiccup, and I suffered _no_ data loss. (The volume did get forced into ro mode, so future writes _were_ lost, but all applications _knew_ that those writes were lost.)

Additionally, I've been able to live-add and live-remove drives from my array with _no_ hassle (other than the expected speed degradation due to shuffling the data around) whatsoever.

> ...so RAID1 doubles [read] throughput. Well not on btrfs.

Maybe I'm out of the loop, but I've _never_ heard anyone say "use RAID1 to double your read throughput". That sounds like a _very_ nice thing to have, but far from essential. Its absence _certainly_ doesn't make btrfs's RAID1 implementation _broken_. :)

> Umounting and running btrfs --check repair is not really cool if you want > some uptime...

Agreed, _very_ much so.

One thing bothers me about that guy who was reporting the bug on the 4.1 kernel. He claimed that his mount options were:

        /dev/sdb /media/storage2 btrfs ro,noatime,compress=lzo,space_cache 0 0

but the issue he was reporting _smelled_ like it could only happen on a rw volume. Wonder what was up with that.

It's _also_ interesting that noone replied to his messages. Both linux-btrfs and #btrfs have always been quite responsive when I've sent messages.

> 4x4TB RAID10 (90% free) -> ENOSPC - this was somehwere between 3.16 or 3.19 > - how can this even happen?

Dunno. It looks like the Global Reserve got introduced in ~3.16, so it's not likely related to that. Maybe you hit a hard-link limit or had _really_ , _really_ large xattrs attached to files or something? (I _do_ know that btrfs has (had?) a _surprisingly_ low limit on the number of hard links supported in a volume, but I have _no_ idea how hitting that limit presents itself.)

Do you remember if you let the system idle for a few minutes and tried your operation again? (Not that that's a reliable [or even reasonable] solution, mind, but does provide a useful piece of diagnostic info.)

In the _distant_ past, running a balance on empty chunks (in order to free them and free up otherwise unused metadata space) was one recommended way to work around mysterious ENOSPC issues. This isn't _supposed_ to be required anymore, but it's a last-ditch cargo-cult thing that can't do anything worse than waste one's time.

> Good for you. Was hit by this nasty undeletable directory bug...

That _seriously_ sucks. However, I don't call that data corruption. Inconvenient, bordering on unacceptable? Yes. Data corruption? Nah. Your _data_ is still present and _correct_ , but you can't get rid of some directories. [1]

> Currently btrfs deadlocks reliable in a OOM situation... With 4.2 things got > better...

Is it _any_ OOM situation, or just _particular_ ones? I know that I've had google-chrome + Hangouts run away and nom all my memory on _multiple_ occasions, and had my btrfs volumes function just fine after the OOM killer had its way with Chrome.

Also, do you have a handle on _roughly_ how much better it has become in 4.2, and is this problem something that the btrfs folks are aware of?

> [btrfs is] probably fine for personal use. But don't think about using it > for something that is demanding.

That's a _far_ softer statement than "btrfs is a shitshow ... I will refrain from using it ever again if I have a choice.". :)

Anyway, I _do_ _REALLY_ hope I soon get the budget to set up a btrfs testbed. I'd _really_ like to see (and help diagnose) all these failures that people keep talking about... in a carefully controlled environment, with data that I don't care about -naturally-. :)

[0] [http://superuser.com/questions/685364/can-a-failed-btrfs- dri...](http://superuser.com/questions/685364/can-a-failed-btrfs-drive-in- raid-1-be-replaced-live)

[1] And -for the record- I saw an _eerily_ similar thing happen _every other month_ at a Windows-only programming shop I used to work for. One or more files and/or directories on the NTFS-backed Windows 2003 file server would _inevitably_ become undeletable. [2] The sysadmin would have to take the whole thing down for a half hour to smear chicken guts in the right places to get rid of the offending files/directories. Despite these _regular_ issues, we never decided to stop using NTFS. ;)

[2] And no, _noone_ who was using the file server had admin access. Unprivved user access was all that was needed to create these undeletable files and directories.

<comment><USER17> > And [0] indicates that you can live-replace a failed drive in a btrfs RAID > array. (The man page makes it seem that you might want to also pass the -r > flag to the replace operation.)

Good catch! I have to test on a rented server I'm running anyway in the next days - adding a an additional drive, running replace and removing the old drive seems to work - I'm about to find out if - deleting a device, rebooting and trying to replace an non-existent device also works - I guess it's likely that I have to fumble with the provider-rescue system (so now control about the kernel version :/)

I thought that the -o degraded dance was still necassary, looks like I was wrong. Let's see how the disk replacement on that servers turns out...

> One thing bothers me about that guy who was reporting the bug on the 4.1 > kernel. He claimed that his mount options were: /dev/sdb /media/storage2 > btrfs ro,noatime,compress=lzo,space_cache 0 0 but the issue he was reporting > smelled like it could only happen on a rw volume. Wonder what was up with > that.

That guy was me :)

Basically the disk got remounted ro after the failure and I just did a grep on /proc/mounts for the fs before posting (not very wise). This has been a bug that was fixed with 4.2 (didn't saw it again at least) and btrfs check --repair fixed it.

> It's also interesting that noone replied to his messages. Both linux-btrfs > and #btrfs have always been quite responsive when I've sent messages.

I had no time to hang around IRC at this time. Yes. Usually it's very nice there. I also want to say I'm not ranting against any of the people involved. I'm just unhappy with the current state of the technology.

[ENOSPC] > Dunno. It looks like the Global Reserve got introduced in ~3.16, so it's not likely related to that. Maybe you hit a hard-link limit or had really, really large xattrs attached to files or something? (I do know that btrfs has (had?) a surprisingly low limit on the number of hard links supported in a volume, but I have no idea how hitting that limit presents itself.)

Actually I don't know - the filesystem in question ran a while with the 3.13 kernel and got later upgraded to something less old. Maybe that's part of the problem. ENOSPC happened while a misconfigured Hadoop NameNode wrote tons of data 4x without checkpointing on the disk and when restarting the checkpointing server it all crashed. It's now running 4.2 and fine so far...

> In the distant past, running a balance on empty chunks (in order to free > them and free up otherwise unused metadata space) was one recommended way to > work around mysterious ENOSPC issues. This isn't supposed to be required > anymore, but it's a last-ditch cargo-cult thing that can't do anything worse > than waste one's time.

Yeah did that balance dance and it somewhat worked.

> Is it any OOM situation, or just particular ones? I know that I've had > google-chrome + Hangouts run away and nom all my memory on multiple > occasions, and had my btrfs volumes function just fine after the OOM killer > had its way with Chrome.

Never got around to nail down this one (would love to being able to file a bug on this) - but at the time I was not really able to do much on the machines - it's a NUMA system and basically ElasticSearch and Hadoop allocated far more than 64GB memory - OOM kicked in and _after a while_ the logs where full of hanging tasks related to btrfs - at that time I've found something related on the mailing list in regards to allocations and OOM however I'm lacking the intimate knowledge to be sure if it's related.

I've screamed at everyone to avoid OOM situations so I'm not sure if it's still a problem :)

> Also, do you have a handle on roughly how much better it has become in 4.2, > and is this problem something that the btrfs folks are aware of?

So far I did not see it again. I had not enough data to file a bug or investigate further. It was something lock related IIRC.

> Anyway, I do REALLY hope I soon get the budget to set up a btrfs testbed. > I'd really like to see (and help diagnose) all these failures that people > keep talking about... in a carefully controlled environment, with data that > I don't care about -naturally-. :)

Good luck! You can try running Hadoop and some heavy jobs on it. Lot's of threads that read and write lot's of data. There are some benchmarks like [https://github.com/intel-hadoop/HiBench](https://github.com/intel- hadoop/HiBench) to stress the system. Should also work on a single system.

> [1] And -for the record- I saw an eerily similar thing happen every other > month at a Windows-only programming shop I used to work for. One or more > files and/or directories on the NTFS-backed Windows 2003 file server would > inevitably become undeletable. [2] The sysadmin would have to take the whole > thing down for a half hour to smear chicken guts in the right places to get > rid of the offending files/directories. Despite these regular issues, we > never decided to stop using NTFS. ;)

Yes. There is worse stuff out there. I was using ZFS before with lots of really fucked up disks in a kind of "don't do this, you are stupid!" setup and ZFS just did it's job and never complained, same Hadoop workload + disks with thousands of bad sectors. Maybe it's unfair to compare btrfs to this, however it was a ride full of disillusionment to think btrfs compares to ZFS.

Maybe it's getting there and I was in at a bad time... I'm not so sure through.

</comment></comment></comment></comment></comment></comment><comment><USER19> Can we not post videos ?

its really hard to skim the material and decide if i want to read more in depth.

<comment><USER20> I actually was coming here to say that it was nice to have links to interesting stuff on Youtube. I agree it can be jarring when you're trying to read stuff and get hit with a video - maybe a HN youtube channel would be cool or something.</comment></comment>


Think FaceApp is scary? Wait till you hear about Facebook - <AUTHOR> https://www.wired.com/story/faceapp-privacy-backlash-facebook/ <comment><USER2> This article is FUD, and click bait, when the real people violating the rights to your photographs are probably hiding in government buildings, and they will use these photos against you... because you thought the threat was a Russian startup or Zuck.

The media should stop pretending they're lawyers. IANAL either but I was a pro photographer and I know there can be no FaceApp nor Facebook without you granting them rights to use your photos. If you refuse to grant them the right to use your photos: you get no FaceApp or Facebook because you'll sue them if they do whatever with your photos (which is normal). No right to publish your photos? they can't post it to Facebook without you giving them the right to it. No perpetual rights? you could sue them after 2 years for using your photos all that time, because they're still on Facebook.

Of course they need you to give them these rights. And no, they won't do bad things with your photos. Despite "owning" billions of photos, Facebook is still selling ads, not your photos.

<comment><USER3> It's fair that they need some of those rights to make the photos available. Doesn't make it untrue that facebook will use whatever means that can now and specially in future to extract values from those photos and not care about screwing your rights/privacy along the way.

<comment><USER2> "screwing your rights/privacy": do they? Where's the million of victims? Where's the class action?

Politicians and the media are screwing us and stirring up the air.

Facebook can do its thing and show me ads for apartments in the neighbourhood I've visited (and published photos of) this afternoon, it's always more relevant than trying to sell me Viagra. I'm also gluten intolerant, so I'd rather not see ads for pizzas. I DO want more clever ads, who doesn't?</comment></comment></comment>


WebSocket benchmarks - <AUTHOR> https://github.com/ericmoritz/wsdemo/blob/master/results.md

<comment><USER2> "I expected Go to kick Erlang's ass in the performance department but the message latency was much higher than Erlang's latency and we had 225 unhappy customers."

Go is neat and I hope to see it succeed and thrive, but it is simply unavoidable that Erlang has existed for much longer, has been getting tuned for much longer, and this sort of thing is its monomaniacal focus whereas Go is spread a bit more thinly at the moment. Go is trying to be a systems language, Erlang very much isn't.

<comment><USER3> What was surprising was how much faster the Java implementation was than Go. I expected it to be competitive but I didn't expect it to perform much better.

<comment><USER4> his reporting is very misleading. If you look at the number of messages and connections, Java only held ~5k connections, while Go held just shy of ~10k. In the same amount of time, Java was only able to facilitate the transmission of half the number of messages, so... it depends what you mean by "fast".

<comment><USER5> Interestingly, Go's "connection time" was by far the lowest.

He also tested with m1.medium instances, which are single cpu. No mention if the instances were 64bit or 32bit (this may matter for Go, as the GC has some issues under 32bit currently).

Tests are hard. Still nice to see a real-world-like comparison between some popular stacks.

Edit: discussion on go-nuts: [https://groups.google.com/group/golang- nuts/browse_thread/th...](https://groups.google.com/group/golang- nuts/browse_thread/thread/6dc8468d6237f676)

<comment><USER6> I used the m1.medium instance on 64bit Ubuntu 12.04. I will update the results.md to note that.

I will also rerun all the tests on 2 core 64bit instances tonight.

</comment></comment></comment></comment></comment><comment><USER7> When benchmarking language performance results, it's a good practice to explain why is one language faster than the other. In other words, profile and figure out the bottleneck.

If you don't know why, then it's better not to post them until you figure out the bottlenecks. There could be factors in your environment that can affect one language and not the other which you are not aware of. Publishing such number will confuse people.

<comment><USER8> My guess would be that the poor performing implementations are using the naive select() solution, instead of one of the more logarithmic solutions like epoll or kqueue.

Pretty surprised to see node.js behave so poorly. We had discussions recently at where I work as to what server solution we should use for a multiplayer platform, and as I had recently built a socket/connection handling server in C they asked what I thought about node.js, and my only concern was that if the underlying code uses select(), then it's going to be a poor choice...but I don't think anyone got around to testing it.

search for: C10K for more information

<comment><USER7> Poller is a very important factor but there are other things to consider as well depending on how fast you want to go. But sometimes 'fastest' is not what you should look at though.

Solutions like node.js, python, etc.. copy strings and buffers all the time and this can slow down the server considerably. They don't have an honest iov layer that won't copy data before passing it to writev and you cannot manage your own memory and create memory pools. These factors can have a bigger impact on performance than selecting the right poller.

<comment><USER9> That doesn't sound right to me. There are 10k clients, each sending (and the server echoing) one small timestamp record per second. An AWS medium instance has access to half a DRAM pipe (frankly this test should fit entirely in L3 cache, but let's be conservative). The back of my envelope says that the RAM bandwidth can handle about 100kb of data copying per send or receive event. That's a _staggeringly_ large amount of copying, literally thousands of times larger than the size of the record.

Honestly I think CPU or syscall overhead is a more likely culprit.

<comment><USER7> You'd think.

Write a web server in C and a web server in python/erlang/go/node.js. You'll notice that the one written in C (or C++) runs twice as fast. If it was a syscall bottleneck then they'd achieve the same performance.

zero-copy request handling is not possible in high level languages without hacking.

For a quick test, run httperf against the dumbest node.js web server that returns 204.

Then run the same test against an nginx server that does nothing but return a 204

location / { return 204; }

Compare the difference.

</comment></comment></comment></comment></comment><comment><USER10> This node.js version is most certainly faster [https://github.com/Weltschmerz/wsdemo/blob/master/competitio...](https://github.com/Weltschmerz/wsdemo/blob/master/competition/wsdemo.js)

Wondering how much!

<comment><USER11> Hm, what performance impact does using arguments have?

<comment><USER10> Using .apply({}, arguments) is actually a bit slower, but either method is so fast that it won't be a bottleneck. I use it in this case for elegance, because I can. Note that in your original implementation this is impossible, because you have to access the "type" variable and use one of two methods (sendUTF or sendBytes). `ws` has only one method (send) and the parameters of this method match the arguments to the callback for on('message').

</comment></comment></comment><comment><USER12> Not surprised to see Erlang come out on top. As I understand it, this is precisely the type of problem Erlang is good at - doing lots of little lightweight things with minimal overhead.

<comment><USER13> Erlang is quietly doing heavy lifting all over the place. Amazon's SDB, Facebook's Chat, Various High Speed Trading Implementations, Ejabberd, CouchDB, RabbitMQ... and far more places quietly and privately.

The good news is, it finally seems like other languages are starting to tool out the way Erlang has (Akka, ZMQ, etc).

</comment></comment><comment><USER14> Python network programs, especially ones with many connections, should use Twisted.

A benchmark of anything but the best library/tools in each language is pointless.

<comment><USER15> One of the good Twisted WebSocket libraries is called txWS: <https://github.com/MostAwesomeDude/txWS>

It wraps an existing TCP factory, and makes it work with WebSocket.

The entire API is a single function.

<comment><USER6> Excellent. I wanted to write a twisted implementation. care to take a crack at it and submitting a pull request?

<comment><USER15> Should I have time, I'll think about it.

<comment><USER6> I can implement the server. Is there anything I should know about tuning Twisted for this kind of benchmark?

<comment><USER16> If you haven't yet, hit up #twisted on freenode for any tips you need. Lots of heavy hitters there who are happy to help.

</comment></comment></comment></comment></comment></comment><comment><USER17> I'd love to see numbers for my hellepoll on that configuration.

<https://github.com/williame/hellepoll>

[http://williamedwardscoder.tumblr.com/post/18200335569/the-h...](http://williamedwardscoder.tumblr.com/post/18200335569/the- history-of-hellepoll)

[http://williamedwardscoder.tumblr.com/post/13590981677/perfo...](http://williamedwardscoder.tumblr.com/post/13590981677/performance- lessons-for-http-sockets)

</comment><comment><USER6> This initial release is not meant to be definitive. I welcome pull requests for better implementations in your language/platform of choice and any suggestion to better tune Linux for these tests.

</comment><comment><USER18> Pretty neat. I'm going to fork it and throw it against my C# implementation (<http://www.alchemywebsockets.net>) on mono and see how it compares.

<comment><USER19> would like to see that, please post it

</comment></comment><comment><USER20> Looking at <https://github.com/ericmoritz/wsdemo> it seems he's using his own erlang ws implementation and using a third party library for all the others?

If this is correct, then it's no big surprise that he does well in his own benchmark that he's developing against.

<comment><USER21> No. He is using Cowboy[1], which _is_ the mainstream websocket Erlang implementation (uh.. cowboy actually is a socket acceptor pool which happens to have awesome HTTP and WS handlers).

[1]: [https://github.com/ericmoritz/wsdemo/blob/master/src/wsdemo....](https://github.com/ericmoritz/wsdemo/blob/master/src/wsdemo.erl) [2]: <https://github.com/extend/cowboy>

</comment></comment><comment><USER22> did I miss something or the methodology lacks a warm up step?

What I mean is, I'd expect a first run of 5 mins to warm up the servers, than another one to actually get the data.

Otherwise, while averages will be mostly the same, numbers like the dropped connections _could_ be misleading.

</comment><comment><USER23> With websocket connections, provided the latency is low enough, memory usage is a bigger concern.</comment>


Generally Accepted Accounting Standards (GAAP) - <AUTHOR> http://avc.com/2016/04/generally-accepted-accounting-standards-gaap/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+AVc+%28A+VC%29 <comment><USER2> General accepted accounting _principles_.

Geez, what did they think the P was for??

<comment><USER3> I suspect Mr. Wilson was referring to the fact that GAAP is incorporated by reference into financial reporting regulations as a required standard. The article is about the standard [i.e. GAAP] not providing a complete picture of some companies' finances.</comment></comment>


Picat – A new logic-based programming language - <AUTHOR> http://picat-lang.org/ <comment><USER2> Looks like an interesting project.

The license or terms-of-use seem weird: _" Picat can be used for any fair purpose, including commercial applications. The C source code is available to registered developers and users free of charge. The project is open to anybody and you are welcome to join, as a developer, a sponsor, a user, or a reviewer. Please contact firstname.lastname@example.com"_

Why make things complicated with these conditions? If you are going to allow source access, why not use a standard open source license. If you aren't, if you want to control of the language for some purpose, make it clear this is a commercial product.

The GPL (and other standard open source licenses) are a real commercial licenses at this point. Roll-your-own software licenses seem a lot like roll- your-own device device drivers - might have made sense in yesterday's world but not today.

<comment><USER3> From [1]:

The Picat C-source code is distributed under the Mozilla Public License ([http://mozilla.org/MPL/2.0/](http://mozilla.org/MPL/2.0/)). You are assumed to agree to the terms and conditions as described in the license if you submit this request. Please provide a valid email address.

[1]: [https://docs.google.com/forms/d/188dDBVrNrrshZtO8smEBgskOrMl...](https://docs.google.com/forms/d/188dDBVrNrrshZtO8smEBgskOrMlCxhXGM6fvfhwDImo/viewform)

<comment><USER2> Well good then. It still seems silly to have the text I quoted on the main page - especially since someone who'd loaded the code is then legally free redistribute it without requiring registration or an email address.

</comment></comment></comment><comment><USER4> Here's an article explaining it's features with more mainstream terms.

[http://www.hakank.org/constraint_programming_blog/2013/09/a_...](http://www.hakank.org/constraint_programming_blog/2013/09/a_first_look_at_picat_programming_language_1.html)

</comment><comment><USER2> Another language which involves both constraint programming and imperative programming is Alma-0[1]. While Alma-0 doesn't seem that much more than a toy language, I always liked the backtracking + standard-loops feature. It seemed offer clarity plus incredible power.

I would wonder if Picat has similar feature (to implicit backtracking in imperative loops)?

[1] [http://en.wikipedia.org/wiki/Alma-0](http://en.wikipedia.org/wiki/Alma-0)

<comment><AUTHOR> Yes, ideas behind Alma-0 look similar to the ones behind Picat.

Loops in Picat are compiled to recursive calls. And there is automatic backtracking, like in Prolog.

</comment></comment><comment><USER5> I used to love Prolog, but Prolog is not general purpose. This has 'pi', 'cat', some imperative, and prolog taste in it...most interesting.</comment>


Something Just Smacked Jupiter and Here's the Photo to Prove It - <AUTHOR> https://www.space.com/jupiter-impact-flash-photo-august-2019.html <comment> chupa-chups Also interesting (3 years ago):

[https://www.nasa.gov/feature/jpl/jupiter-s-north-pole- unlike...](https://www.nasa.gov/feature/jpl/jupiter-s-north-pole-unlike- anything-encountered-in-solar-system)</comment>


Show HN: JAWS – A JavaScript and AWS Stack - <AUTHOR> https://github.com/servant-app/JAWS <comment><USER2> Thanks for sharing. I've just started work on a server-free AWS architecture, but instead of using the AWS API Gateway feeding into Lambda, I'm working on static pages with AWS SDK for Javascript (served from S3) talking directly to Cognito, RDS and S3 with lambda for async processing:

[https://aws.amazon.com/sdk-for-browser/](https://aws.amazon.com/sdk-for- browser/)

I'm surprised I haven't seen more discussion of SDK for Javascript since it removes a lot of the server requirements. Have you considered using AWS SDK for Javascript, or do you explicitly want a REST API as an abstraction layer? The API certainly provides some advantages if you ever want to migrate off AWS. In my case, the lambda functions get invoked by particular events on RDS, S3, or manual/browser triggers instead of AWS API Gateway invocation.

<comment> at-fates-hands I'd like to see this when you get finished. I'm looking for a new way to do static webpages that's fast and hassle free.

<comment><USER2> Unfortunately my client is closely coupled to a venture I'm doing on the side. I won't be open sourcing it. If it's just static web pages you need without mutable data, you can do that directly off Amazon S3. There are lots of blogs and open source projects for that:

[http://stout.is/](http://stout.is/)

[https://sysadmincasts.com/episodes/48-static-sites-using- aws...](https://sysadmincasts.com/episodes/48-static-sites-using- aws-s3-cloudfront-and-route-53-1-5)

If you sprinkle in AWS SDK for Javascript in the browser, you can handle mutations directly into various persistence services without needing EC2. I'm still very early in my exploration of this approach so I was hoping to see some case study myself. If there's no direct way to invoke lambda from a service event (e.g., an RDS mutation), you could invoke it from the client after returning from the RDS request.

</comment></comment></comment><comment><USER3> First of all nice work!

Second there is already a product out there called jaws: [http://www.freedomscientific.com/Products/Blindness/JAWS](http://www.freedomscientific.com/Products/Blindness/JAWS)

The last time I checked (and my information is very dated, 5+ years) this was a fairly popular piece of software in the blind community. I also do not know the status of trademark around the term. If you were going to change names, its early days for you now would be the good time to do it.

<comment><USER4> JAWS the screen reader is still quite popular, though an open-source one called NVDA ([http://www.nvaccess.org/](http://www.nvaccess.org/)) is on the rise.

I doubt that FS will come after this project for trademark infringement, since it's not a threat to them, not even in the same field.

<comment><USER3> I agree with you "minimal" risk assessment. However we have all see companies do stupid things. I don't remember off hand if you are REQUIRED to defend a trademark for it to be enforceable.

<comment><USER5> You are REQUIRED or you lose it. It is what has caused a lot of lawsuits in the first place.

</comment></comment></comment></comment><comment><USER6> Note that you should be wary of AWS Lambda limits. We blew through them on our first day directing traffic on a system we were testing with Lambda. They were able to increase our limits a few days later, but something to think about before you base your whole stack around it.

[http://docs.aws.amazon.com/lambda/latest/dg/limits.html](http://docs.aws.amazon.com/lambda/latest/dg/limits.html)

<comment><AUTHOR> Those will be raised soon.

AWS introduced API Gateway because they want back-ends being built on AWS Lambda. They will raise the limits, but this is all bleeding-edge tech and they are likely moving forward with caution.

<comment><USER7> They've already increased it from the originally published concurrent requests per function limit (originally 50, now 100) [1]

We'd like the "Total size of all the deployment packages that can be uploaded per account" increased. Since you have to "require in" (as you put it) all your dependencies with each published function it's plausible to hit that limit pretty quick.

The average node module is 1.6MB [2]. If your functions have 3 dependencies, you're near 5MB. That limits you to ~300 functions. That initially seems like a lot until you realize you may have to start breaking up complex logic into multiple functions to hit time limits. You also might need to maintain multiple versions for backwards compatibility (think v1 endpoints). And if you use Java 8, you're just screwed. That size limit can be hit quick.

[1] - [https://web.archive.org/web/20150315092544/http://docs.aws.a...](https://web.archive.org/web/20150315092544/http://docs.aws.amazon.com/lambda/latest/dg/limits.html)

[2] - [https://www.npmjs.com/package/npmjs- stats](https://www.npmjs.com/package/npmjs-stats)

<comment><USER8> It might be possible to write a wrapper function that pulls a larger function from S3. I think you get up to 500mb of /tmp storage per function. Obviously the initial startup time would be atrocious but if you have a high enough TPS it wouldn't be hit too often.

I could also see Lambda adding a paid-for option to keep a function "warmed up" at all times.

</comment></comment></comment></comment><comment><USER9> Great work - looks like a really useful piece to help you quickly get going using the AWS stack and Lambda/JS. The biggest issue I've always found is the extra steps between writing your code and getting it onto a server.

The "no server" movement definitely seems to be gathering steam, with Firebase and Parse on the data side and Lambda (augmented with JAWS and others) on the computation side. Am excited to see what this area will bring, in particular due to the combination of scalable hosting and containerisation.

(Plug - we're working on StackHut ([http://www.stackhut.com](http://www.stackhut.com)) in the "no server" space that provides a language-agnostic Lambda-like platform (currently JS/ES6 and Python with Ruby soon) that's powered by Docker and is accessible via JSON-RPC over HTTP. Would love to hear your thoughts on this and the "no server" space in general.)

<comment><USER10> Tried your public demo's, they don't seem to work.

<comment><USER11> Hi Jake! Sorry you're having issues.

I'm presuming you're on Sky in the UK? They are currently blocking HTTPS on various CloudFlare endpoints (such as ours!) We're attempting to get this sorted ASAP. Let me know if this isn't the case!

Cheers -- Leo

<comment><USER10> I am on Sky in the UK, looks like it is an ISP issue then. I'll keep an eye on you guys though, I'm a big fan of being able to just run functions on-demand like this. I used to use PiCloud extensively before it was bought out.

</comment></comment></comment></comment><comment><USER12> This seems very convenient...

Is there some blog/resource that provides an estimate of the pricing when we go this route, instead of a seperate server for the APIs.

I would like to know the rough pricing in an order of magnitude level?

Would it be twice as expensive as a django app for an application with light usage, etc?

TIA!

</comment><comment><USER13> Love this idea, but I wish there were an official integration between Lambda and RDS; I don't want to use DynamoDB.

<comment><AUTHOR> The plan was always to add RDS integration, specifically Aurora support.

That plan hasn't changed! Just need to find more time for open-source projects :P

<comment><USER14> I believe to support RDS in Lambda you'll either need to dynamically add the Lambda instance's public IP address to your RDS security group (and remove it once complete) on each run or make the RDS instance open to a large number of AWS IP ranges^1

[1] [http://docs.aws.amazon.com/general/latest/gr/aws-ip- ranges.h...](http://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html)

<comment><AUTHOR> Hmmm, I'll look into this. Either way, I'm sure Amazon is aware that RDS in Lambdas is a good thing, and they will continue to make this and their other tools easier.

We'll get there!

</comment></comment></comment></comment><comment><AUTHOR> Just added a section in JAWS Optimization on the AWS Pop-Up Loft because it's a tremendous resource and I don't think many people know about it. Since this page is so popular, I'll paste the section in here:

AWS Pop-Up Loft

AWS has an office/loft in downtown San Francisco where you can meet with an AWS engineer for free to discuss your project and your use of AWS services. It's an incredibly valuable resource, but I don't think many people know about it.

All you have to do is schedule a meeting in advance. Meetings can last up to 1 hour and can be scheduled M-F during normal business hours.

Further, their office and loft is also a free co-working space, as long as you are using AWS services. They have some nice couches and desks and it's a great way to network with other developers who use AWS services.

Again, all of this costs nothing. If you're in SF, or traveling there, check it out!

</comment><comment><USER15> This is jawsome. I feel like you're cheating on the "no server" bit with some AWS semantics but you're not the first so, whatever, nice job.

<comment><USER16> It's no server because Lambda is not a server like VPS that you get it up and running 24 hours, it's a service that gets a call and runs your function. You don't have a server up it's a function on the backend. It's being called by the JS SDK on the browser, therefore no server, plain static + function calling architecture.

Best,

<comment><USER17> > It's no server because Lambda is not a server like VPS that you get it up > and running 24 hours,

What you say makes no sense. There is a server , you just don't manage it yourself.

> it's a function on the backend

And where is the "backend" ? that's right , on a server.

> It's being called by the JS SDK on the browser,

What does the SDK call ? a server when your functions lie.

Lambda + api gateway are a BAAS. You might have some logic running in the browser but the lambda functions run @AWS. So there is a server.

It's a bit like Couch Apps by the way,except that you can scale at the 'design document level', not just at the 'node level'.

<comment><USER18> It's "no servers" for the developer in the sense that there is zero infrastructure to manage, configure or deploy. This is popularly known as a "PaaS" or platform as a service, ala Heroku and similar providers.

</comment></comment></comment></comment><comment><USER17> It's a bit what Couch Apps where supposed to be, without the vendor-locking part...

> no server

But you have to rely on AWS infrastructure. So what you meant is no server management.

<comment><USER19> the pricing difference is huge too, AWS's PaaS Elastic Beanstalk took away the "server management" part for the most part but this allows a lot of power when you need it and absolutely no costs when you don't.

</comment></comment><comment><USER5> I've been trying to find a core example of how Jaws works without having to download anything. Could you include an example in your readme of an example API using lambda? Even some pseudocode would help me understand.

<comment><AUTHOR> Yes, I'm working on an overview video now.

</comment></comment><comment><USER20> I've been looking for something like this within a ClojureScript workflow. With the new JS compiler it seems not too far off.

This is very cool. I would love to see more aws resources exposed as well through JAWS like dynamo or RDS.

<comment><AUTHOR> JAWS uses DynamoDB. Aurora (RDS) support is coming soon!

<comment><USER20> sweet!

</comment></comment></comment><comment><USER21> This is interesting, thanks for sharing. Was a bit confused about lambda.

So this would be cheaper than just having an EC2/DO instance running flask with a couple of endpoints?

<comment><AUTHOR> Absolutely.

You pay for EC2/DO even if your app isn't used. With JAWS, you pay only when your app is used.

Here is Lambdas pricing FYI: [https://aws.amazon.com/lambda/pricing/](https://aws.amazon.com/lambda/pricing/)

<comment><USER22> The examples are very interesting, as it's not necessarily cheaper, because if you could handle the same 3 million requests per month with a t2.micro for ~$10 per month, Lambda approach would be close to 2x more expensive.

However, the benefit here is that you don't have to provision to peak capacity needs + (some safety fudge factor), thereby having paid for, but idle capacity.

It's a very intriguing proposition for new services where you may not be able to optimize costs as much for fear of losing up time, but I'm sure it makes a lot of financial sense for AWS. Nothing stopping this from being a win/win though.

<comment><USER23> Here's a problem with cloud economics. People don't compare apples. Essentially, I am not short $10 but I could use some time back in my week.

Getting bogged down in managing servers, or a fleet of instances, or the devops stack to automate a fleet of instances; it's all a time sink with either an actual cost or at the very least an opportunity cost, orders of magnitude beyond that $10/month.

This is especially true for organisations where provisioning an instance requires an epic chain of approvals "because it's a server". Unfortunate but all too common.

Of course if you were planning to become (or already are) a competent devopsy admin then provisioning instances is not necessarily a waste of your time. For everyone else, platform-as-a-service such as Lambda can be an huge win. Sure, there's still a tipping point but it's much greater when time, rather than simply price, is considered.

On the other hand, and contrary to the Amazon-is-scalping-me gripes I hear on HN all too frequently, my experience has consistently been that AWS want you to minimise costs due to design. Every well-run business knows the long-term virtuous cycle of satisfied & continuing customers far outweighs a short-term revenue grab.

ObDisclosure: I am a former AWS architecture manager but do not speak on behalf of my ex-employer.

<comment><USER22> Couldn't agree more. The $10 example was in response to the EC2/flask comment earlier in the thread.

</comment></comment></comment></comment></comment><comment><USER24> Is there a way to use a file system-backed storage with this? For example, I would love to create a web frontend for ledger [1]. Ledger only reads text files but for a frontend to be useful it would need to write entries to files.

1\. [http://www.ledger-cli.org/](http://www.ledger-cli.org/)

<comment><USER19> 'getObject' and 'putObject' using the JS AWS S3 SDK on Lambda, triggered by the API Gateway: [http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.ht...](http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html)

<comment><USER24> Is there an append function? Otherwise it seems like it will have to send the entire file for each incremental write.

</comment></comment></comment><comment><USER25> Really cool... Now we just need longer Lambda execution times (current limit is 60s) and a proper crontab. For a complete auth flow you can also check:

[https://github.com/danilop/LambdAuth](https://github.com/danilop/LambdAuth)

It would be nice to integrate some ideas from LambdAuth into JAWS.

<comment><USER26> Hi, I am the author of LambdAuth and would be more than happy to help! JAWS is both interesting and promising!

</comment></comment><comment><USER27> Great experiment here. Lambda has been intriguing me for a while. More and more, as I scope out work for clients, I'm finding bits that could probably be run as Lambda processes instead of requiring an entire EC2 instance. This is kind of like taking a REST interface to a natural extreme.

<comment><AUTHOR> Agreed. I think of it as "cloud computing, fully realized".

</comment></comment><comment><USER28> I was looking into using lambda functions called in response to an S3 event. Specifically, I wanted to create a record in my SQL database after a file is uploaded to S3. Users are uploading directly to S3 using pre-signed requests, so I needed some way to step in and create a database record after they actually perform the upload to S3. S3 supports this via several methods, one of which is the lambda functions, but when looking into lambda functions there was no way to have the code access my SQL database internally, I would have had to make my SQL database open to the world. I didn't find any IAM rules etc to enable lambda functions to access my SQL database resources. I'd be curious to see how this is handled by this JAWS work flow.

<comment><USER29> I've also needed to have Lambda functions access databases on an RDS instance within a VPC and haven't been able to do so without opening the database up to everyone. Seems like a feature they should think of adding soon.

</comment></comment><comment><USER30> You may want to reconsider the name since JAWS is widley known as the defacto application for accessibility screen readers.

As far as legal stuff goes the american federation of the blind has been known to litigate against technology companies.

</comment><comment><USER31> Ha, been working on something similar. If you get a chance, take a look. Maybe we can collaborate.

[https://github.com/johntitus/hocus](https://github.com/johntitus/hocus)

</comment><comment><USER32> I'm presently about three quarters done with writing one of these myself; Lamdba plus SQS plus arbitrary functions. I have a different take on deployment, going with CloudFormation. Version updates are something you have to think about quite hard with this sort of system.

The real challenge is how to manage within the 60 second, 1024 thread limits in a sane way, however. You have to approach application decomposition in a much more rigorous way.

<comment><USER33> exratione, please consider posting a github link of your approach.

</comment></comment><comment><USER34> And the move towards Google App Engine continues...

<comment><USER35> If App Engine had some neat AWS features early on, it could have done really well. I think App Engine was one too many hops away from 'mainstream dev', but is probably very close now.

</comment></comment><comment><USER36> Is it really true that Lambdas can take 5 minutes to start up? Doesn't that rule them out as useful for things like running a website (instead of just background jobs)? Or does the article mean 5 minutes from the first time you deploy a new one to the time it's available, but after that new lambdas can be spun up in milliseconds to server each request?

<comment><AUTHOR> No, that is not true. While AWS Lambda has cold-start issues, they can be mitigated.

While AWS continues to improve Lambda performance (it's still early tech), I've started an "Optimization" section in the wiki, detailing how everyone can get better performance out of JAWS.

Further, JAWS is optimized for AWS Lambda performance.

<comment><USER36> Oh, okay, I must have misread the article then. Sounds good :-)

</comment></comment></comment><comment><USER16> Great job, grabbed my attention, I really liked this, but I'm doing so great with gulp that if I used this I really would still need to use gulp because of the optimizations, I'm also using S3 and Lambda, but I like the idea of running aws lambda locally before uploading and the "run a server locally" before upload.

<comment><AUTHOR> Ramon,

Could you share some of your gulp optimizations so that I can incorporate them into the project?

You don't have to share their code if you don't want to, but if you could write down what the gulp tasks are, I could incorporate them into the JAWS Command Line Interface.

Thanks for the positive feedback btw :)

<comment><USER16> I really liked the concept of your project and it's in my bookmarks already. I can share, can you add me in twitter it's ramonck, I'll write down a paper in Medium sharing the details, I already wrote down the concept of a mobile product I did with Gulp and Lambda I'm about to share in Medium as well, I'll also be glad to share the indepth details of the technology, but I think it will be more productive for you and for others to have it in a single spot. Add me there I'll share in the next week or so.

Best Regards,

<comment><USER37> Looking forward to this. Added you.

</comment></comment></comment></comment><comment><USER38> Does AWS API Gateway support Websocket endpoint? Would be interesting to created a realtime websocket API, backed by lambda.

<comment><USER39> You could probably do this with Fanout.io, which has a WebSocket->HTTP gateway.

</comment></comment><comment><USER40> Nice work on the project! Big fan of the graphics also :)

Reminds me of Divshot [0], a heroku-like service for hosting static sites. It's great that you can roll your own site, and don't need a middleman to host for you.

[0] [https://divshot.com/](https://divshot.com/)

</comment><comment><USER41> Sounds interesting. What kind of apps make the most sense for JAWS?

<comment><USER14> I've got to implement a simple site this week for our customers to be able to upload photos to S3 and then I need an API endpoint to list these photos. JAWS seems like a great solution for this.

</comment></comment><comment><USER42> this is really cutting edge stuff, great job. Would be event cool to include polymer webcomponents (on client side) , AWS Cognito for authentication along with this.

</comment><comment><USER43> Jaws already exists as a software project, for the blind.

Perhaps I am exposing ignorance about the needs of the blind community, but it would seem to me that a primarily important piece of software for them could be more confusing to find and update and so on if another piece of software appears with the same name. Maybe you should consider changing the name of your software before it's too late? It's possible that your Jaws may, in the long run, cause considerable confusion for the users of the pre-existing Jaws software.

I'm making a case based on empathy rather than trademark law and so on.

</comment><comment><USER44> Woohoo! It's the second coming of stored procedures and PL/SQL! Party like it's 1988!

Seriously though, these kinds of architectures come at a cost. The website's claims are either misleading, or the authors haven't understood the problem domain. E.g.:

 _Each of your API URLs points to one of these Lambda functions. This way, the code for each API Route is completely isolated, enabling you to develop /update/configure/deploy/maintain code for specific API urls at any time without affecting any other part of your application(!!!)._

Even if you add more exclamation marks, you'll still end up with front end code depending on the interface exposed by your backend code, meaning you'll have to do versioned rollouts, except you no longer have a single deployable but a zoo of store proce^W^Wlambda functions, with unclear interdependencies and (usually) no proper versioning or tracking.

This might be the right architecture for certain apps, but it's not the easy win/win that's presented here.

<comment><AUTHOR> Up next, I will add the ability to define your entire JAWS API in JSON via Swagger.

Swagger JSON can be imported to create/update your REST API instantly via AWS API Gateway.

Meaning, you can still do larger, versioned roll-outs by simply updating the JSON in your Swagger file.

This will give beautiful structure and simplicity to the JAWS workflow.

The single deployable server you advocate is a single point of failure.

<comment><USER44> I'm sure you'll further improve this, good luck!

 _The single deployable server you advocate is a single point of failure._

I'm not talking about a deployable server, I'm talking about a deployable as a unit of deployment, e.g. a war archive. A deployable is a compile time artefact, a single point of failure is a runtime concept. In that sense, saying a deployable was a single point of failure is like saying a line of code is a single point of failure.

</comment></comment></comment><comment><USER45> Anyone else immediately think of the screen-reader

</comment><comment><USER46> This thing is awesome, thanks for sharing!

</comment><comment><USER47> Excellent work, thanks for sharing!

</comment><comment><USER48> Seems great for cloud applications. Only thing I would worry about is vendor lock in.

</comment><comment><USER49> There's a server here -- it's just that it is not yours.

<comment><USER50> While I understand the need to give a snappy comment, does this really contribute any value to the discussion? It's obvious that OP means develop apps without dealing with servers. I don't think anyone would imagine that the entire app is running without any servers at all.

Please reconsider leaving this comment here.

</comment></comment><comment><USER51> beautiful, this will be fun to try out!

</comment><comment><USER52> Is this an april fools joke in August?

<comment><USER53> What's so funny about this?

<comment><USER54> the name combined with the shark!

</comment></comment></comment><comment><USER55> Yes! Yes! Yes! Now I just wish there was Python support.

</comment><comment><USER56> this idea is terrible

<comment><AUTHOR> Thanks :)</comment></comment>


Betting Against Tesla? Be Prepared to Lose - <AUTHOR> http://allthingsd.com/20130913/dont-bet-against-tesla-says-musk/

<comment><USER2> I'm amazed when an article describes an odd price/earnings ratio without even suggesting the possibility that that might allow the company to expand rapidly with the resulting capital, which would eventually make the P/E radio more reasonable, and ultimately benefit those who trusted that it all made sense long-term. Obviously that outcome isn't guaranteed, but it should be mentioned.

Tesla has significant room for expansion right now, but to do that, it needs operating capital. A high P/E ratio provides it.</comment>


Perl Rocks Latin American App Competition - <AUTHOR> http://mdk.per.ly/2011/12/06/perl-rocks-latin-america/

<comment><USER2> > There is often a description, a type, bandied at Perl that you cannot build > an application in the language in a short period of time…

I have…never heard this, and in the absence of anyone else's opinion I'd assume the opposite. When I want to get something done quickly, Perl is one of my go-to languages. Of course, the final product might not be as maintainable as it could otherwise have been, but that's to be expected when you focus on working code rather than pretty code.</comment>


15-year-old facing jail time for downloading 24 movies - <AUTHOR> http://www.geek.com/articles/news/15-year-old-facing-jail-time-for-downloading-24-movies-20110824/

<comment><USER2> I'm amazed that a teacher would do that to a student? I though their job was to educate not incarcerate?</comment>


Megaupload Points Out That The Feds Want To Destroy Relevant Evidence - <AUTHOR> http://www.techdirt.com/articles/20120406/12172918409/megaupload-points-out-that-feds-want-to-destroy-relevant-evidence-its-case.shtml

<comment><USER2> Techdirt (of course) acts like the forfeiture of MU's assets is unprecedented, but SCOTUS established more than 20 years ago that the direct proceeds from illegal enterprises can't be used to fund the legal fees of the accused:

 _A defendant has no Sixth Amendment right to spend another person's money for services rendered by an attorney, even if those funds are the only way that that defendant will be able to retain the attorney of his choice. Such money, though in his possession, is not rightfully his._

And, of course, the case against MU is far more straightforward than Techdirt implies.

Stories like this (the Slashdot "Your Rights Online" stuff) are the only things from Techdirt that routinely make it to the front page of HN, but it's worth noting (and I'm not the only person to note it; I got tipped off to this by 'tzs, who went to law school) that Techdirt _sucks_ at legal analysis.

<comment><USER3> > _that the direct proceeds from illegal enterprises can't be used to fund the > legal fees of the accused_

I like how the determination that something is an illegal enterprise did not require a trial. Our gov't must save a lot of money that way.

<comment><USER4> No, this guy quoting things is wrong. In the case he's referring to the defendant plea bargained and admitted guilt. They said he couldn't use the money, because it was an established fact (to which he himself confessed) that it was illegally gotten.

Whenever you see someone quote SCOTUS without providing the case, assuming they're taking shit out of context to prove a point. This guy even altered the quote slightly to make it harder to find, but the case itself is here -[http://bulk.resource.org/courts.gov/c/US/491/491.US.617.87-1...](http://bulk.resource.org/courts.gov/c/US/491/491.US.617.87-1729.html)

</comment></comment></comment><comment><USER5> [http://arstechnica.com/tech- policy/news/2012/04/megaupload-e...](http://arstechnica.com/tech- policy/news/2012/04/megaupload-erasing-our-servers-as-the-us-wants-would-deny- us-a-fair-trial.ars) seems like a better source

</comment><comment><USER6> I don't quite understand something. If I'm not mistaken, defendants in an US court have the right to know what evidence will be used against them in a timely manner. It's called discovery. So if MU were to destroy the evidence now that would me both the prosecution and defendant would have no evidence, right? So I'd assume the prosecution has a copy of any evidence and _that_ would be disclosed to MU's attorneys before the trial. So MU wouldn't hold on to the evidence but it'd be out there somewhere.

Right? I feel like an idiot. I feel like I've noticed something so simple that somehow it was already explained but I'm still not getting it.

<comment><USER7> Once the only evidence is in possession of the prosecutor, they are legally required to provide it all to you. On the other hand, things can often get 'lost,' 'misplaced,' or 'forgotten' that are crucial to the defense's case. I assume that MegaUpload wants access to the servers to extract the evidence themselves.

<comment><USER2> I gather that what makes this problematic is the expense of recovering evidence from the servers, as the hosting company is owed millions of dollars for them. MU does not have millions of dollars anymore.

<comment><USER8> >> MU does not have millions of dollars anymore.

They do not have _access_ to it.

Whether they still have millions of dollars is as yet undecided.</comment></comment></comment></comment>


Rob Daviau and the legacy game revolution - <AUTHOR> http://www.slate.com/articles/arts/gaming/2016/08/rob_daviau_s_legacy_board_game_seafall.html <comment><AUTHOR> I had never heard of this "legacy board game" concept before. Has anyone tried that?

What stops players from writing actions on paper instead of altering their game in order to play more than once?</comment>


Ask HN: Are there truly "Head Hunters" in the software industry? - <AUTHOR> I occasionally get recruiters saying they are head hunters looking for software engineer candidates for a specific position. I know in these cases they are not true head hunters. But, it did make me wonder if there truly are head hunters out there.<p>How would you know if a recruiter is a true head hunter? <comment><USER2> What distinction do you draw between "recruiter" and "head hunter"?</comment>


How we f***ed up our product development process and what we did to fix it - <AUTHOR> https://pilot.co/blog/building-a-product-team/ <comment><AUTHOR> Curious how different teams on HN settled on their product development process.

For us it was a painful journey—we weren't able to get much done at first, everything was always 'almost ready'. We ended up having to part ways with a PM who could've been great but wasn't experienced enough to put the right processes in place.

Things pickup up as soon as we stripped our process down to bare minimum, measured our productivity and worked on improving it every week.

<comment><USER2> > Doing a MVP

> having a PM

made me chuckle.</comment></comment>


If you quote VirusTotal, you should be fired - <AUTHOR> http://kasperskysucks.blogspot.com/2012/08/if-you-quote-virustotal-you-should-be.html <comment><AUTHOR> Disclaimer: Views expressed in the linked article are not necessarily my views. I post the link because I feel the writer of the article provides a series of interesting counterpoint arguments to the conventional wisdom that VirusTotal works well in all circumstances. That is, I present the linked article for well-rounded debate around VirusTotal. It's pros have already been argued, this article argues its cons. (Also, please remember that there's a no-shooting the messenger rule in effect! <g>)</comment>


The Ethics of Performance - <AUTHOR> https://timkadlec.com/remembers/2019-01-09-the-ethics-of-performance/ <comment><USER2> Not to mention global warming and unnecessary energy wasted because of badly written software.</comment>