Map Showing Ratio of Median Software Engineer Pay to GDP/Cap for 50 Countries - <AUTHOR> http://maps.ramiro.org/income-software-engineers-countries/ <comment><USER2> If I'm interpreting this correctly, this means a median coding salary in USA/Canada/EU is about/slightly below average for the economy, but in the developing world (particularly India) its waaay above average?

This kind of makes sense, although it sure shatters the illusion that software jobs are well paid in the developed world.

<comment><USER3> You are forgetting about profits which in the industrialized world is about 35% of the surplus, only 65% is wages. Therefore an average salary would only net a ratio of 0.65 in salary to gdp/capita. In other words the 0.9-1.1 ratios in the EU are pretty good.

E.g the ratio in Sweden is 1.03 and if an average salary is 0.65, it means that the average developer salary is 58% higher than the average salary. It matches my own anecdotal data pretty well.

<comment><USER4> I don't think i understand what you profits/surplus you are referring to. Care to elaborate?

<comment><USER5> GDP per Capita is significantly higher than median income.

</comment></comment></comment></comment><comment><USER6> It would have been better to compare the median developer wage to the normal median wage. But median wages can be hard to get (often you get bullshit like household median wages). Anyway, in the US median wage is a lot lower than average, so the comparison does not really say much there. In other countries the gap is often smaller, but still.

<comment><USER7> You mean median wage relative to GDP/Cap? I doubt that it is below average overall!

</comment></comment><comment><AUTHOR> I created this map based on data from Bloomberg/PayScale. For more info on what is displayed click on the i-button (3rd to right) in the top menu or ask me here.

</comment><comment><USER8> Software engineer covers a huge range of roles from low level java bot to CTO. You can't just blob it together like this and expect meaningful insight.

</comment><comment><USER9> What's going on with Norway? Does anyone who's familiar with the software industry there care to comment?

<comment><USER10> From the information blurb on the page:

        "The highest median income in absolute terms is earned by software engineers
        in Switzerland (ratio 1.21) followed by Norway (81,400 USD) and the
        United States ($76,000 USD). Norway has the highest GDP per capita (99,574 USD)
        in this list with software engineers median income at about 82% of it."

Due to oil incomes and other stuff, Norway tends to be a serious outlier in many cases when it comes to economic metrics.

<comment><USER11> Although Norway does have high GDP per capita, [http://www.bloomberg.com/visual-data/best-and- worst/highest-...](http://www.bloomberg.com/visual-data/best-and- worst/highest-paid-software-engineers-countries) said 99,574 USD is Average annual income. That's unfortunately not true according to [http://ssb.no/en/arbeid-og- lonn/statistikker/lonnansatt/aar/...](http://ssb.no/en/arbeid-og- lonn/statistikker/lonnansatt/aar/2014-03-20?fane=tabell&sort=nummer&tabell=168713), where average annual earnings last year for all sectors is 489200 NOK, i.e. 77824 USD. (Meanwhile, the earnings of programmers, according to [http://ssb.no/en/arbeid-og- lonn/statistikker/lonnikt/aar/201...](http://ssb.no/en/arbeid-og- lonn/statistikker/lonnikt/aar/2014-01-16?fane=tabell&sort=nummer&tabell=158312), is 52900 * 12 NOK = 101021 USD.)

It's kind of unfair to compare annual earnings of programmers directly to GDP per capita. If you actually compare it with the annual earnings of all sectors, the ratio for Norway is close to 1.3.

</comment></comment></comment><comment><USER12> What's up with Bulgaria vs Romania? One's at the top and the other at the bottom.

</comment><comment><USER13> Isn't this nearly a heatmap of poverty, or mean income?</comment>


Nevermore: an experimental Emacs interface for Notmuch - <AUTHOR> https://github.com/tjim/nevermore <comment><USER3> Just last week, some of us were reminiscing in Slack about reading mail and Usenet via emacs. It's amusing and a little unsettling to find the emacs/vi debate has been abandoned because you've gotten to the age where your colleagues have never experienced either... :)

<comment><USER4> That's something I don't get: _why_ don't they use either? For decades new programmers were exposed to one or both, and they sorted themselves more-or- less by their affinity for one or the other style of text editing. But now they don't seem to be exposed to vi & emacs, and chase Big New Things™ which inevitably don't deliver either a rich, full-featured editing language (as with vi) or absolute power & extensibility (as with emacs).

Is this related to the proliferation of JavaScript libraries and hip new languages? I honestly don't know.

<comment><USER5> > That's something I don't get: why don't they use either?

The other way to look at that question is to consider who the developers were back then and what alternatives they had. Emacs or vi compares very well at first glance to a tool like ed... but maybe not so much when side by side with Eclipse, etc. It's a harder sell these days, and that's only more true when you consider all the language/ecosystem integration you get from a modern IDE.

Emacs and vi also tend to be the kinds of tools that reward long-term study and practice. I don't know this for a fact, but I'd guess that with the massive increase in the prevalence of developers over the last 30-40, that it's brought in huge number of people that have little to no interest in learning the vi editing language, Emacs Lisp, etc.

> absolute power & extensibility (as with emacs).

As a 25 year Emacs user, I question the 'absolute power' bit... there's a lot of magic that's easy out of the box in IntelliJ or Eclipse that's quite difficult in Emacs... Maybe the situation could be different in theory, but it's not right now and many people have many good reasons for focusing on the 'right now'.

<comment><USER6> meh, give me ed over vi and emacs any day

"modern" seems to be a curse word these days, all the time I find it being used it's used to describe something despicable like the systemd ecosystem.

</comment></comment></comment></comment><comment><USER7> I wish the Github page said something (anything) about how this is different from/better than Emacs' existing Notmuch client...

<comment><USER8> Hopefully this is actually a MUA, and not a search interface with message-mode sprinkled on top?

I tried using notmuch on emacs as my primary mail client for several months, but it falls short on so many things (just to name one: cannot delete messages, cannot have copies in multiple folders, cannot apply a tag to an individual message without passing by the threaded view first, no notion of folders, and so on...). You can work around most of those, but it's just a work-around.

It's right that it's "not much" of a client. Notmuch combined with afew is incredibly powerful as a backend. But it's not a particularly good mail client.

You can use notmuch efficiently with neomutt, which is a great combination (excellent MUA with fast backend).

But for something closer to a real client on emacs itself, mu4e is much better. It supports identities out of the box, and has no silly limitations imposed by blind message-id indexing. Unfortunately, mu (the backend) has no concept of batch tag processing, which made certain operations very convenient [one thing I did was killing threads I didn't care about - <USER2> impossible to do with mu alone].

<comment><USER9> I share your frustrations with Notmuch as an Emacs client. But I still find it better than Gnus and Mu4e. And I have tested all 3 extensively aside from using Mutt for a decade.

I think making the Notmuch interface "pure" is a great design decision. By pure I mean that it just operates on tags, but never deletes or moves files and it doesn't even know about folders.

This gives it great flexibility. You can implement virtually any logic in a Maildir backend by remapping tags to actual files. Any arbitrarily complicated workflow, silencing, automated scoring like what Gnus offers, etc. is possible while the UI will be the same. Just tagging.

The problem is that Notmuch developers do not bundle any tools for implementing such a backend with notmuch, and third party stuff like afew [1] is lacking. It reminds me a bit about the situation with address completion. For a long time, mu offered this, while notmuch relied on hacky third party scripts.

[1] [https://github.com/afewmail/afew](https://github.com/afewmail/afew)

<comment><USER8> I still think notmuch is a great backend, but a poor client. Despite using mu4e, I still find Gnus superior in terms of pure reading efficiency (I _still_ use Gnus regularly with gmane). Scoring and thread handling is just superior.

There's a lot to gain from adaptive scoring alone.

In fact, I also switched over mu4e after trying a search backend for Gnus, and I've been oscillating from mutt/gnus/mu4e a lot over more than a decade.

Tagging is powerful, but I find it too constrained as a single tool to manage everything.

<comment><USER9> Yes, right now Gnus is probably the best option.

But with good logic to implement tag-based rules, something like notmuch could be marvelous.

</comment></comment></comment></comment></comment><comment> racer-v Cool, will this work well with proportional fonts? I'd like to go back to using Emacs for mail but I've gotten used to reading the mail body in Thunderbird or the browser, and going back to monospace is a bit jarring.

<comment><USER6> Eh, you can use a fixed-width font in GUI apps, too. In Kontact, for example, you just press "x".

FixedMisc is still the best font

</comment></comment><comment><USER10> It looks like it's abandoned. :/</comment>


Systems Software Research Is Irrelevant (2000) - <AUTHOR> http://doc.cat-v.org/bell_labs/utah2000/ <comment><USER2> Pike's pessimism in a way reminds me of the Bill Gates quote:

 _We always overestimate the change that will occur in the next two years and underestimate the change that will occur in the next ten._

Still some of his pessimism has been spot on. Linux is Unix and RISC-V is, well, the fifth RISC from Berkeley which was even then only codifying some principles Seymour Cray laid down (now with Cray vector instructions!).

But LLVM is more pliable and experimental (and slower) than GCC. Didn't see that coming. No one even remembers Netscape but Chrome and Firefox are not simply Netscape.

He's dead right on the over emphasis on measuring. I never even read the measurements section in papers. I'm only interested in the ideas. That said, recent conferences attach a badge to a paper which submits a verification artifact. That's useful, especially if I really want to dig into the paper.

<comment><USER3> > I never even read the measurements section in papers. I'm only interested in > the ideas.

Genuinely curious, how do you identify if an idea is of any practical use without caring about measurements?

<comment><USER2> An idea doesn't have to be practical and it certainly doesn't have to be immediately practical. I think _useful_ is a locally greedy optimization and that is not in itself at all bad. I just think that an idea might be able to take you a little further, sometime, just not right now.

<comment><USER3> That's a fair point. I probably overemphasized the word practical in my original comment. I didn't mean immediately practical. (I assume you meant "doesn't" in your comment.) And I do agree that even ideas or systems which perform poorly could be imminently useful. But I personally find looking at measurements helpful to give me some sense as to whether or not the approach being proposed is suitable for solving problems I care about.

<comment><USER2> For example, if you are reading the classic papers in a seminar, are you really even interested in their measurements? Well, if you are looking forward at new papers, I think the ideas come first. Remember, there's a ton of stuff to read. A buddy said he read two papers a day in grad school.

<comment><USER4> If you're reading classic papers then presumably they've already proven themselves worthy (by being considered classic)

<comment><USER3> Exactly. The thought process I go through in evaluating a new paper is certainly different from when I am reading older, proven work. But in both cases, I still personally care about the measurements.

</comment></comment></comment></comment></comment></comment></comment><comment><USER5> Rob Pike wrote this because of a disappointment in the decline of system software research, specifically in operating systems. In the 90s there was a lot of great work but the momentum, he is stating, was on the decline.

When you read what he writes (at least from my perspective), it's not a sincere call to stop doing systems research, but that everyone else thinks it's less relevant so what the hell is the point of pursuing it anymore?

One supporting point to this is, the conferences like ACM SOSP [1] and USENIX OSDI [2] have sessions now that have nothing to do with operating systems or systems software. For example, bug finding, "big data", web apps, RDMA (high- performance networking), machine learning, security (this does have relation to systems, but the research presented often is tangential, e.g., with GPGPUs, or a secure chat system), and graph processing (??).

[1] [http://sigops.org/s/conferences/sosp/2015/current/index.html](http://sigops.org/s/conferences/sosp/2015/current/index.html)

[2] [https://www.usenix.org/conference/osdi18/technical- sessions](https://www.usenix.org/conference/osdi18/technical-sessions)

</comment><comment><USER6> > Instead, we see a thriving software industry that largely ignores research, > and a research community that writes papers rather than software.

I have actively followed the NSDI and SIGCOMM community, and this is, for the most part, true: Research venues have become a hiring billboard for big companies (Microsoft specifically?), and that's all that is left of research.

Most papers published from these companies (and academia) are flawed at the core and primarily story driven (at least in the two conferences that I mentioned). Companies publish with data that is inaccessible to anybody but them---MSR, specifically, takes this to an extreme. The scientific contribution of most papers is close to nil. Writing and storytelling dominate the field. Experiments are cherrypicked, are rarely reproducible, and the software is seldom useable.

People rarely are willing to think outside the box and spend more than a year on a paper. Most people pick an idea from an outside field, apply it, and publish a paper. That's the end of it. Most people that I have talked with publish to get their name out, and rarely care about any scientific agenda.

Imagine, in a system's research community, a large portion of academic advisors cannot develop proper software---they can, however, pitch stories and write text for days.

The problem is not that people are evil... it's that the system/community has decided to take this path. And for one, I cannot fathom why. It is not even rewarding to publish a paper in these conferences anymore ... except to enjoy the trip and the dinners.

You hear stories on how people try to optimize their chances of getting in, e.g, I have heard and seen from good researchers that you should not register your paper early because you will get a two-digit paper number, indicating that this is a resubmission, and lowering your chances of getting in, etc. There are many such hidden gems there.

</comment><comment><USER7> It's not clear what is the main problem this talk/slides want to solve.

There's a lot of research and advancement in the things that matter:

\- Rust is showing how linear types can get a nice balance between safe systems and great resource usage.

\- GPUs and TPUs are the next-gen somewhat general purpuse computer hardware, but none of them came from operating systems research

\- containers partially solve the resource sharing problem, though Plan 9 what the author refers to is probably a better system design for containers. At the same time there's a huge pressure at the big companies to improve resource sharing (like the performance and UX of lambda functions) while minimizing the side channel vulnerabilities, so I don't think there's no money put into it (this is a multi-billion dollar problem for these companies, and the CEOs know it, as they have to keep building the datacenters).

These are just a few examples from all the advancements that were happening, I don't see any reason why universities would do so much better than companies.

<comment><USER8> > It's not clear what is the main problem this talk/slides want to solve.

Basically, Linux's design sucks, and the products that surround it suck. Except for a tiny few applications, nearly all of the industry for the past 18 years has gotten almost nothing innovative adopted by the majority of the industry.

Oh, well, containers, a combination of package manager and resource isolation. But that basically already existed as jails/zones, just without the package manager and remote backend. And microservice orchestrators, those are new, and useful. But neither of those make a dent in the laptops and mobile devices which the majority of the planet uses.

Just as one comparison, it's been 27 years since Plan 9 was first released, and we _still_ don't have most of its incredibly useful, novel features in any operating system. We're actually poorly re-inventing distributed operating systems, but as incompatible, over-designed, kludgy distributed applications.

<comment><USER9> Well, if "Linux's [non-]design sucks" is the problem, there is no solution. The Linux community is simply not interested in design, design reviews, reviewing interface designs, and so on. They are not setup for it and don't want to be. They are only interested in code.

So you get things like epoll.

On the other hand, Linux won. Maybe because of this, or maybe in spite of this.

</comment></comment></comment><comment><USER10> Rob is way smarter than I am but maybe this opportunity just doesn't exist any more. How much is there to be done in systems research that has a chance of influencing the industry but isn't just another layer of abstraction on top of existing well agreed upon standards?

<comment><USER11> A lot, I hope. We're entering a time where chip fabrication tech is commodicized. That's an opportunity for new architectures. Further up the stack there are a lot more inefficiencies that can be removed. In my laptop here is a CPU that can do more than two billion operations per second, but it struggles to render many websites. What we have now can't be the end-all of system's software.

</comment></comment><comment><USER12> It sucks that many of the great innovations from Plan9 never became widely adopted, but systems research still goes on! Containerization, for example, and virtualization, have given rise to a whole new field of systems research. And a lot of it is being done in Pike's own language, `go`

<comment><USER13> Au contraire ;) Innovations of Plan9 landed in mainstream systems: Unicode, Namespaces => Containers. If you will, Docker is the result of a mainstream implementation of a Plan9 feature.

I was recently thinking, if you want to create a popular system feature, you just need to copy something from Plan9. IMHO the only reason nobody uses Plan9 is its bad hardware and userland software support. Also I've given up hope this will ever change, however it's a nice innovation testbed somehow.

<comment><USER14> Some of us are still using Plan 9. I'm glad to see the connection between Plan 9 namespaces and containers mentioned, most people have missed that step in following the evolutionary pathway from unix chroot through BSD jails to Linux containerization. There is actually a semi-secret modern Plan 9 container-like service platform, but the Plan 9 community doesn't usually try to publicize/commercialize its work so almost nobody knows about it.

<comment><USER15> I don't see the connection that clearly. You forget about OS/390 virtualisation. That would be what comes closest to what Docker is today, as the earliest innovation I can think of.

</comment></comment></comment></comment><comment><USER16> > "But now there are lots of papers in file systems, performance,security, web > caching, etc.," you say. Yes, but is anyone outside the research field > paying attention?

uh... yes.

<comment><USER9> The past 19 years show just how much. The industry is absolutely abuzz with research and its applications.

Systems research sure felt like a dying field in 2000. But 2000 was the start of its revival. So much good has happened since. I wouldn't know where to start. GPUs? The Cloud? iOS and Android? ZFS and DTrace? Rust? I'm afraid of all the things I might miss in making a representative list!

<comment><USER17> Note well his definition of "is": "Now, not ten years ago, and I hope not in another ten years." Maybe you're actually agreeing with him?

<comment><USER9> Where is that? Anyways, I'm not disagreeing about the state of play in 2000. I'm disagreeing with the pessimism.

<comment><USER17> It's in the "Definitions" section. Interestingly, however, it's _not_ in the PDF - the words are there, but the definitions of the words are missing.

</comment></comment></comment></comment></comment><comment><USER18> I've heard talks by Alan Kay where he also decries that lack of research in Universities: They aren't exploring new hardware or new OSes. He talks about how at PARC they built computers that were 5 to 10 years ahead of their time in terms of speed and functionality, and could therefore explore a different set of ideas w.r.t the interaction with the user.

</comment><comment><USER19> I could not agree more! To me the big picture is that the industry is aligned towards servitude to MBAers, marketeers/salespeople and advertisers.

Let's go sell more and grow x% per year!!! Until the next disruptive fast-food junk product puts us out of business.

In the meantime we've helped making our end-users (the rest of the world) stupider.

How proud are you code monkey?

</comment><comment><USER20> > OLD stuff. Compare program development on Linux with Microsoft Visual Studio > or one of the IBM Java/web toolkits.

If visual studio and tools of the same design philosophy so great, then why are sublime text/atom/vs code such a massive success?

<comment><USER21> MS VS was actually way closer to Sublime that to its current incarnation. I used VC++6.0 as a replacement for Notepad for many years, because it started quickly and handled different code pages (encodings) semi-decently (while most everything else on Windows died on BOM or sth).

</comment></comment><comment><USER22> The polemic aged remarkably well

<comment><USER9> Is that sarcasm? Cause I think it must be.

Pike says (well, said, in 2000) there's been nothing new in GUIs. Fine, he was right in 2000. Then the iPhone and iOS happened just a few years later. Oops.

Or how about this:

> It has reached the point where I doubt that a brilliant systems project > would even be funded, and if funded, wouldn't find the bodies to do the > work. The odds of success were always low; now they're essentially zero.

Oh really? As he wrote that... Microsoft was about to release Windows 2000 and Active Directory. Sun was investing (or soon would start) on things like ZFS and DTrace, which have had massive impact on the industry. Apple was recovering and soon would give us the iPhone and iOS. The cloud was a long way away, though, again, Sun had its "grid" just a few years later, and now the cloud has been a huge driver of innovation up and down the stack. GPUs were soon to be a big thing. Microsoft, Amazon, Google, and many others, have invested huge amounts in research and innovation. Tensor processing and AI are all the rage in ways the AI researchers of the 80s could not have imagined. Linear types are being applied in real-world programming languages (Rust) to finally find a better way than garbage collection to alleviate manual memory management.

Pike's pessimism was wrong. I'm sure he sees this now. I'm sure he's ecstatic at how things turned out.

In his defense, 2000 capped a bleak decade in the software industry.

As Keith M Wesolowski used to sign his emails:

      Keith M Wesolowski "Sir, we're surrounded!"
      Solaris Kernel Team "Excellent; we can attack in any direction!"

As an industry, we attacked in all directions.

2000 was the start of beautiful things.</comment></comment>


Google's algorithm shows prestigious job ads to men, but not to women - <AUTHOR> http://www.washingtonpost.com/news/the-intersect/wp/2015/07/06/googles-algorithm-shows-prestigious-job-ads-to-men-but-not-to-women-heres-why-that-should-worry-you/?tid=sm_tw <comment><USER2> It may very well be that the algorithm is completely objective, but not necessarily "fair". This is the problem with profiling algorithms.

</comment><comment><USER3> TLDR: Some algorithms sometimes provide output that is offensive to some people. Maybe the programmers are to blame? Who knows!</comment>


Don't buy China's story: The coronavirus may have leaked from a lab - <AUTHOR> https://nypost.com/2020/02/22/dont-buy-chinas-story-the-coronavirus-may-have-leaked-from-a-lab/ <comment><USER2> Is this what passes for journalism nowadays?</comment>


Calamine: Excel file reader, written in Rust - <AUTHOR> https://github.com/tafia/calamine <comment><USER2> How neat!

Here's a little Excel-file writer in PHP; Excel will read these files, but this library won't:

        <?php echo '<?xml version="1.0" encoding="utf-8"?>'; ?>
        <?php echo '<?mso-application progid="Excel.Sheet"?>'; ?>
        <Workbook xmlns="urn:schemas-microsoft-com:office:spreadsheet"
            xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet"
            xmlns:o="urn:schemas-microsoft-com:office:office"
            xmlns:x="urn:schemas-microsoft-com:office:excel"
            xmlns:html="http://www.w3.org/TR/REC-html40">
          <Worksheet ss:Name="Sheet1">
            <?php $data = array(array(4,5,6),array("dude","whatever","m<om")); ?>
            <Table>
              <?php foreach($data as $row): ?>
              <Row>
                <?php foreach($row as $cell): ?>
                <Cell>
                  <Data ss:Type="String"><?= htmlentities($cell) ?></Data>
                </Cell>
                <?php endforeach; ?>
              </Row>
              <?php endforeach; ?>
            </Table>
          </Worksheet>
        </Workbook>

They're "technically" Microsoft Office XML-format[1], and not "Excel files", but users don't tend to be amused by programmers being technically right so I tend not to argue about that sort of thing. I only bring it up here in case someone wants to search for a specification on it.

[1]: [https://en.wikipedia.org/wiki/Microsoft_Office_XML_formats](https://en.wikipedia.org/wiki/Microsoft_Office_XML_formats)

<comment><USER3> Also, Excel will read the old BIFF files (XLS), CSVs, SYLK, and whatnot. You are doing, in reverse, exactly that which you oppose: "unless this library can open each and every and any file that Excel can, bug-for-bug compatible, it's Wrong!"

(While it is true that users will toss anything at you, it's quite pointless to expect one library to handle all of it: we used one to sniff the content, another to parse CSVs, yet another to parse XLSX, and in-house code for this obscure abomination which sadly still gets about 10% of use)

<comment><USER2> > it's quite pointless to expect one library to handle all of it

Libraries don't just come with implementation, but also a use model. Perl, for example has:

Spreadsheet::XLSX

Spreadsheet::ParseExcel

These have very different user (programmer) interfaces -- so different that now there's a Spreadsheet::ParseXLSX which just allows programmers (and programs) used to one interface, consume the new files with very little changes.

This rust library has it's own model, and if it's a _good_ model it makes sense to implement other file formats with this model -- and indeed this is a good way to test if it's a good model.

Other things like CSV are so fundamentally different from XLS/XLSX that they're _going_ to have a different model, and so it makes sense that these require different libraries.

</comment></comment></comment><comment><USER4> Which C# libraries were used in comparison? The 2 I know of, EPPlus and ClosedXML, do much more than just read cell values, so it's unfair to compare against those. For example the additional complexity of handling styles adds overhead.

Disclaimer: I'm a current ClosedXML contributor.

</comment><comment><USER5> For me, the gold standard of these libraries is Apache POI and NPOI. Porting to Rust would be perfectly possible, but it would be _a lot_ of work. Until you support that level of functionality, it's pretty much irrelevant what your "performance" is. You can always improve performance by dropping features.

<comment><USER6> actually apache poi is extremely slow while reading/writing large excel files. so there might be the need for some libraries to process them fast.

<comment><USER7> The streaming versions of poi (SXSSF) should be really fast if you're working with large documents.

<comment><USER6> only if you know the column size ahead of time. and it's still slower than other streaming solutions and at some point it buffers to disk.

</comment></comment></comment></comment><comment><USER8> There was a time I had to read XLS (BIFF formats) in PHP. Poked around, found some libraries (slow as hell), decided to port[1] one of them (python's xlrd).

What followed, even though it was almost 1:1 port, was an inevitable loss of huge part of my life, I'll never get back. However, I learned a lot about what a mess that format is. Since then ANY xml format Microsoft uses is godsent.

[1]: [https://github.com/laacz/xls-reader](https://github.com/laacz/xls- reader)

</comment><comment><USER9> Was there a particular reason that this library was written? The only thing I can glean from the repo is that it seems to be faster than C# alternatives.

<comment><USER10> Somebody needed to read excel files from Rust, possibly not from Windows? That would also be the reason why openpyxl and xlrd exist for Python: [http://www.python-excel.org](http://www.python-excel.org)

e.g. importable data using real spreadsheet files is much more reliable than CSV, and easier for clients to generate.

<comment><USER11> > e.g. importable data using real spreadsheet files is much more reliable than > CSV, and easier for clients to generate.

It can also be more a pita since cells can have different formats.</comment></comment></comment>


DynamoDB Is Not a Database - <AUTHOR> https://medium.com/@dvassallo/dynamodb-is-not-a-database-8e9729bee3d3 <comment><USER2> I'm afraid there are some innacuracies 1\. Aggregations can be stored within DynamoDB, you just need to setup how those are calculated, usually with DynamoDB streams and Lambda. You do not need to do it within the application at request time. 2\. "Local Indexes" isn't entirely accurate. Its called a Partition key and you can't avoid using a partition key, its kind of the pivotal query mechanism of a table. 3\. 10GB is not the limit of a partition. AWS's own documentation states that once a partition approaches 10GB it will be split.</comment>


Robots are trying to take our job. Let's acknowledge they're already here - <AUTHOR> https://www.propellercrm.com/blog/sales-a <comment><USER2> I'm all for transparency. To me, AI feels like another avenue in which those who are truly engaged in their jobs will excel and those who aren't will get found out. In most cases, I think AI will be a rising tide that lifts all ships and it's smarter to start figuring out how it works for your business now rather than scrambling to catch up when it inevitably becomes the "norm."

</comment><comment><USER3> I love this. I keep hearing AI will "took or jorbs!", but I am 100% for embracing it and making it work for us.</comment>


Law Students Lose the Grant Game as Schools Win - <AUTHOR> http://www.nytimes.com/2011/05/01/business/law-school-grants.html

<comment><USER2> eh, but I thought that was the point of a merit scholarship? Rich children pay big money so that they can go to school at good (as determined by average quality of the pupil) school. I thought the idea of a merit scholarship was to get the high-achieving but not rich children in to the school?

I mean, the quality of a school, I thought, was largely determined by the quality of the students admitted, and if you are letting rich kids in regardless of ability (and let's be honest, you are. someone has got to pay for the new football stadium.) you are going to have to be pretty damn selective with everyone else to get a good average.

I wasn't good enough to get a merit scholarship, and my parents weren't rich enough to send me to a reasonably good school anyhow, but I don't see how I got any more screwed than in a system where all schools are mediocre; I mean, that's what you get when you let guys who perform like I did out of high school in; a mediocre school, and I had plenty of opportunity to go to a mediocre school as-is. Hell, for good portions of my life, my employer would have paid for the venture.

If anything, more merit scholarships (rather than just filling those slots with the next class down of rich kid) seems like a net win to me. Hell, people like me probably wouldn't get as much out of college as someone who, you know, actually did the homework anyhow.

From the article:

"57 percent of first-year students — more than 150 in a class of 268 — have merit scholarships."

Personally, I think that's wonderful. Those are all kids who would otherwise have been too poor to go to that school; so the best and the brightest of the poor get to go for free, on the backs of the rich kids who want to go to school with smart kids. Seems like a good deal all around.</comment>


Watch and learn: How music videos are triggering a literacy boom in India - <AUTHOR> http://www.boston.com/bostonglobe/ideas/articles/2010/09/19/watch_and_learn/?page=full <comment><USER2> Seems like a good companion to the "how to raise boys who read" article.</comment>


Is my design too simple? - <AUTHOR> http://organimal.com

<comment><USER2> I think having the 'ORGanimal is' below the fold could be an issue.

I kinda got that it was a calendar for pets, but wasn't sure exactly what the deal was.

I don't think your design is too simple. But I think you might want to take a closer look at the flow. What people see and how they understand what they are supposed to do.

</comment><comment><USER3> It's cutting it close. I generally like simple, spare designs but many people don't. One problem I think are the typestyles. I think a sans-serif font would look better (Arial, Helvetica, Tahoma, Verdana, Trebuchet, etc.).

</comment><comment><USER4> Typo: "an application that let's you manage all your pets in one place" should be "an application that lets you manage all your pets in one place" (keyword: let).

</comment><comment><USER5> I didnt look on the content, just the design. I like it how simple it is, but the background annoyed me a bit. I got distracted from it.</comment>


Ask HN: Thoughts on idea... - <AUTHOR>

After spending some time looking at different CRM software options, I very much felt that individual freelancers, or even small web design teams, we're largely being left out of the current software offerings.  They all, for the most part, seemed to appeal directly to very large businesses in which collaborating and talking to each other was not that easy or aimed specifically at (traveling) salesman who need to maintain a gazillion different contacts/relationships.
It got me thinking about a CRM created largely with the freelancer in mind.  Users create a form of sorts to embed on their site, potential clients then fill it out and it's funneled and organized in the app.  I think this could be a really great tool, especially for people who get a lot of requests.
I'm mainly wondering if:
1)  Do you think this is a good idea or does email suffice in managing new and potential clients?
2)  Can you think of any other features to go along with this idea that would also benefit freelancers (besides the obvious ones like invoicing and time tracking as those two spaces are very crowded)?
3)  Would you pay for a tool like this?
And also, on that note, what about a workroom of sorts (similar to Elance) where you can pass back and forth all communications? <comment><USER2> 1\. to me crm != profitable. if email suffice, why not use gmail with labels & filter instead, isn't that enough? you setup filter for each customer/group, labels to things to do/follow-up, etc.. i am not really a fan to use crm as a tool to generate/convert new sale, there are too much other things we need to do just to convert a lead to a customer.

2) not really. this space is really crowded.

3) we know that the money lies with your existing customers and mailing lists, if you have a crm that can milk $$$ from these 2 sources, or perhaps an entire new channel/platform to generate $$$, probably i would pay for it. e.g. exchange, co broke, lease, etc.

</comment><comment><USER3> 1\. I think you'd have to take into account how the majority of freelancers actually do business and interact with their clients. If you're dealing with 2-5 clients a month a CRM would be nice but unnecessary, at least for the amount of communication you'd need.

2\. That crowded space you speak of is most of the tools an average freelancer would need. Maybe it's the high volume freelancer then that you'd need to address?

3\. Probably not, unless you have a very very compelling reason for me to.

</comment><comment><USER4> 1) email is enough

2) there's already a lot of people who focus on this space, and even in specific verticals like photographers

3) meh.</comment>


Andrea A. Rossi Cold Fusion Generator - <AUTHOR> http://peswiki.com/index.php/Directory:Andrea_A._Rossi_Cold_Fusion_Generator

<comment><USER2> It smells like hoax, it looks like hoax...

I can't help hoping for the best, with a heavy dose of skepticism. All we have to loose is some time.

<comment><USER3> Yeah; I certainly won't be buying the first unit--I'm kinda surprised they're selling a 1MW installation. Most buyers for that would be institutions which would be willing and able to sue if it's a hoax. I'd expect them to sell the 10kw versions over the internet.

<comment><USER2> Unless the big buyer is part of the scam.

Also, there is an imbalance of neutrons when transmuting nickel to stable copper, I wonder where these come from.

Unless they are using 62Ni... but its natural abundance is only 3%. And they mentionned 30% of nickel changed to copper. Are they enriching nickel?

</comment></comment></comment><comment><USER4> I've never been a cold fusion "believer" but this is starting to look a bit interesting.

Here's a NASA chief scientist who thinks it might be for real: [http://www.evworld.com/evworld_audio/dennis_bushnell_part1.m...](http://www.evworld.com/evworld_audio/dennis_bushnell_part1.mp3)

The neat thing is there's actually a theory of how low-energy nuclear reactions might work, which uses known laws of physics. Something to do with the weak nuclear force. It was published in 2006. Since then NASA's been looking into it, and they're about to start experiments.

Here's a 30-minute interview with one of the theorists. The interviewer is annoying but the scientist is really interesting. [http://www.blogtalkradio.com/sandy- andrew/2010/04/17/widom-l...](http://www.blogtalkradio.com/sandy- andrew/2010/04/17/widom-larsen-theory--energy-revolution-with-lewis-)</comment>


In Defense of Electron - <AUTHOR> https://journal.standardnotes.org/in-defense-of-electron-e7d5b5229b70 <comment><USER2> Sure, wasteful RAM and CPU is a big problem with Electron, but bigger problems not addressed in the article include:

\- Total disregard for the user interface experience of the host operating system, as native UI toolkits are often not used

\- Since native UI toolkits are not used, knowledge that the user already has about using their computer and how user interface elements should behave becomes useless every time they download a new Electron app with a different "design"

\- Electron app developers are rarely qualified to design good usable UI elements from scratch and are often really bad at it

\- Complete lack of accessibility integration - don't even bother trying to use an Electron app with a screen reader or other assistive tech

Yes, Electron helps you to rapidly prototype and release something faster, and it might even lower the barrier to app development, but it also encourages JFDI mentality and somehow the poor end-users get forgotten about.

<comment><USER3> > Since native UI toolkits are not used, knowledge that the user already has > about using their computer and how user interface elements should behave > becomes useless every time they download a new Electron app with a different > "design"

This is definitely not just a problem with Electron apps. I've seen a _lot_ of OS X apps written using cross-platform toolkits rather than Cocoa, and _every time_ it throws me for a bit of a loop.

> Electron app developers are rarely qualified to design good usable UI > elements from scratch and are often really bad at it

See above. This isn't solely a problem with "Electron app developers"\- more often it seems to be any developer trying to use a cross-platform toolkit.

> Complete lack of accessibility integration - don't even bother trying to use > an Electron app with a screen reader or other assistive tech

Do you have any more details about this? I haven't looked much into a11y under Electron- does it prevent screen readers from detecting ARIA directives?

</comment></comment><comment><USER4> I like that the author decided to take a different approach to thinking about electron, but his argument that everything is fine for him on his macbook that has 16gb of ram seems rather weak. Many users may have more economic laptops that have a realistic 4 - 8 gb of ram. If he were to try using a dev machine with specs more along those lines, I would be more willing to agree with him.

<comment><USER5> Agreed.

I work with many schools, and you may be surprised how many workstations with 4gb of RAM are still out there. When I first started at this company I scoffed and considered them edge cases, but now I know better.</comment></comment>


Diskhash – Disk-based, persistent hash tables - <AUTHOR> https://metarabbit.wordpress.com/2017/06/30/ann-diskhash-disk-based-persistent-hash-tables/ <comment><USER2> One of the nice things about using mmap'd data structures is that you don't have to slurp the entire thing into memory to work on it. The OS's virtual memory system will only load a few pages at a time for whatever's needed. This means that in practical applications you're likely to be within an order of magnitude of RAM speed if you're using an SSD.

Secondly, the other nice bit is that you can use structures larger than the physical RAM on the system. It's kind of like getting a system with enormous amounts of slow-ish RAM.

Thirdly, you can build a whole bunch of these for different purposes and just open the ones you need, and not bother with the rest, again like having even more RAM.

Finally, you can multi-process trivially on these as the OS sorts out the mess and suddenly it's like having a slightly slow multi-million dollar multi-tb shared memory supercomputer from 5-8 years ago at your disposal. Works great on Beowulf clusters if your disk storage is shared on fast links and can handle the IOPS.

The number of applications you can do with a 5-8 year old super computer are vast. I've seen 20tB classifiers built on more or less commodity hardware (<$100k for a cluster of cheap hardware and one big disk) that were previously thought to require a $5 million dollar supercomputer just because of the large shared memory. You can probably build an equivalent machine for <$10k these days off of Newegg and some decent NAS boxes.

<comment><USER3> > One of the nice things about using mmap'd data structures is that you don't > have to slurp the entire thing into memory to work on it.

Right, but one of the not-so-nice-things is that you can't do the I/O asynchronously, so you can end up with poor performance depending on the access pattern. (I guess you _can_ , with another thread running in the background touching pages, but it's more of a pain and you're not really guaranteed the data will stay in memory until you use them.) [Edit: Actually I guess if you touch by _writing_ to those pages rather than just reading from them then they'll have to stay in memory... though do note that I'm assuming no swap here.]

<comment><USER4> At some cost to portability, IIRC, on Linux you can ask the operating system to keep pages in memory. Other systems will probably have similar functionality.

<comment><USER5> They can remain paged in, but unlike on FreeBSD, you can't control when dirty pages are flushed to the backing filesystem on Linux. Specifically, the MAP_NOSYNC option doesn't exist on Linux (see [https://www.freebsd.org/cgi/man.cgi?sektion=2&query=mmap](https://www.freebsd.org/cgi/man.cgi?sektion=2&query=mmap) for a description).

</comment></comment></comment></comment><comment><USER5> How does it compare to mdbm and lmdb?

mdbm: [https://github.com/yahoo/mdbm](https://github.com/yahoo/mdbm)

lmdb: [https://symas.com/lightning-memory-mapped- database/](https://symas.com/lightning-memory-mapped-database/)

<comment><USER6> It sounds like djb's cdb to me [http://cr.yp.to/cdb.html](http://cr.yp.to/cdb.html)

<comment><USER5> cdb isn't memory mapped, and you can't update individual keys.

</comment></comment></comment><comment><USER7> The project's readme on Pypi reads _" The wrappers follow similar APIs with variations to accomodate the language specificity."_.

Beautiful. There are so many times in my career that I've had to use poorly hobbled together wrappers for other languages that I now really appreciate it when someone takes their time to make their API "Fit" with the language.

</comment><comment><USER8> While this is pretty neat, it reminds me a _lot_ of Perl tie, which is kinda considered an anti-pattern in modern Perl (though there are places where it still makes sense and is still used). Any data structure in Perl can be connected to a wide variety of backing stores using tie: [http://perldoc.perl.org/perltie.html](http://perldoc.perl.org/perltie.html)

It was introduced in Perl 5.0, so it's been around for about a quarter century, and it's generic in the sense that there are tons of backends that use the tie interface. BerkeleyDB and gdbm and flat files and probably any DBD SQL database are among the backing store options. But, again, a lot of people think it should be avoided in the general case, even though it _seems_ like a really nice simple solution to a common problem (I've got these hashes/arrays/whatever, and I want to save them on disk so they're persistent and shareable).

This implementation can't delete keys (yet?) which seems to limit its utility for general purpose tasks. I also wonder about reliability. When I first started reading I assumed it'd be used for small things, but he's working with a 32GB file shared across multiple processes. That's a lot of opportunity for data corruption.

<comment><USER9> You insinuate without providing the reasons behind it. I say this because diskhash and tie look like extremely useful things used carefully. This sort of thing solves exactly a problem I have right now.

> Perl tie, which is kinda considered an anti-pattern in modern Perl ... a lot > of people think it should be avoided in the general case, even though it > seems like a really nice simple solution to a common problem

Why? Any links to blog posts?

> When I first started reading I assumed it'd be used for small things, but > he's working with a 32GB file shared across multiple processes. That's a lot > of opportunity for data corruption.

I've used something similar and data corruption because of size is low on my list of concerns. If your storage layer fails in this manner silently, you have much bigger problems.

<comment><USER8> It's not an opinion I hold strongly, it's just something I've noticed. I've seen more than one talk where someone apologized for implementing something with 'tie'.

Here's one blog post about it (still opinion):

[http://www.skrenta.com/2007/05/tie_considered_harmful.html](http://www.skrenta.com/2007/05/tie_considered_harmful.html)

And, this article from the Modern Perl site itself has mixed opinions on the subject, but comes down on the side of not using it, in the general case, but knowing how and when to use it can be really valuable (Tie is the last subject covered on the page):

[http://modernperlbooks.com/books/modern_perl_2014/11-what- to...](http://modernperlbooks.com/books/modern_perl_2014/11-what-to-avoid-in- perl.html)

I'm not trying to say there's no place for it. Tie in Perl has been used for really cool stuff.

<comment><USER9> Appreciate the response and reasoning!

</comment></comment></comment></comment><comment><USER10> Interesting. I used a disk-based hash table very similar to this one in DOS Point-Of -Sale applications in 1989. Goal was to have consistent lookup time when scanning barcode labels. Of course my average lookup times were much slower due to low seek times.

</comment><comment><USER11> Nice and simple. I tend to use a wrapper around BSD tree.h and SQLite for persistence. Longer insert/update times but quick access and you get the SQL language and tools for free.

<comment><USER12> Quick access? Tree lookup is O(log n) on a rb-tree. Hashtables are O(1). And this Diskhash thing claims to scale to billions of entries. Not sure your options are comparable, therefore.

<comment><USER13> Claims is the operative word. Features generally aren't free. If mmap'ing files was some sort of performance godsend don't you think most DB manufacturers would just rip out all their custom caching code and replace it with mmap?

<comment><USER12> Absolutely correct! I guess a comparison with various levels of "rows" and concurrent access might be needed before judging any way...

</comment></comment></comment></comment><comment><USER14> I usually use gperf created tables or cdb tables for that purpose.

<comment><USER12> Neither of those seem to easily scale to billions of keys, though.

<comment><USER14> For gperf I had to add >int support by myself. [https://github.com/rurban/gperf/commits/long_keys](https://github.com/rurban/gperf/commits/long_keys)

cdb is limited to 2GB, yes.

<comment><USER12> Thanks for the work on and link to gperf! Looks interesting, to say the least.

</comment></comment></comment></comment><comment><USER15> This is a similar idea to what nbs uses internally:

[https://github.com/attic- labs/noms/tree/master/go/nbs](https://github.com/attic- labs/noms/tree/master/go/nbs)

... but the idea is developed to support fast mutations as well as other related datastructures (sets, lists, etc).

</comment><comment><USER16> What happens if the process can't shut down cleanly and writes are in flight? Is there a way to check for hash table corruption? Is there a plan to support deletion/compaction/defrag? Why would I use this over something like LevelDB?

<comment><USER17> I think the use-cases are different here. This library is set up for write- once, then read-only access later. LevelDB is all about mutability. I think the use-cases here is for constant-time lookups on fixed-size data sets. LevelDB is a tree-based map, where this is a hashmap, which may give better performance for the author's purposes.

From the second paragraph:

 _My usage is mostly to build the hashtable once and then reuse it many times. Several design choices reflect this bias and so does performance. Building the hash table can take a while. A big (roughly 1 billion entries) table took almost 1 hour to build. This compares to about 10 minutes for building a Python hashtable of the same size._

</comment></comment><comment><USER18> This is such a clean concept.

If you have a tiny application, doing a tiny thing, do you need redis? Do you need sqlite? do you need to write some custom-text-file-ma-bob? nah. I feel like simple simple little tools like this are in short supply.

<comment><USER4> OP here.

Use redis if you can. Use something like diskhash if you cannot afford the extra overhead of sqlite/redis.

In my case, the alternative we were using before is an in-memory hash table built at startup from a text-based representation of the data. It was pretty good and worked very well up to millions of entries, but at the current scale we work with, it takes least 10 minutes at startup were necessary to build up the thing and it used ~200GB.

</comment></comment><comment><USER19> Without going into to the code itself I'm always wary of these hobby projects that then get used at work when they are by the same author. Who now really owns the IP?

<comment><USER4> Look at the source code license, it's MIT licensed.

We are a publicly funded research organization, so even our "work code" is open source.

</comment></comment><comment><USER20> Ok, I know this doesn't help the conversation, but I kept reading "Dickhash - Dick-based, persisted hash tables".

I was so confused.

</comment><comment><USER21> sqlite gives you a bit more.

<comment><USER4> Yes, it does give you a lot more and is great.

Unfortunately, on my initial tests, it just was not fast enough.</comment></comment>


The 'Of Course' Principle of Design - <AUTHOR> http://daringfireball.net/linked/2012/04/06/of-course <comment><USER2> Probably would be better to link this to the original post (<http://om.co/2012/04/05/the-of-course-principle-of-design/>) rather than a short Daring Fireball excerpt.

</comment><comment><USER3> A copywriter friend of mine, with years of experience, once told me: "if the ad's too good, you hurt the product. In order to be maximally effective, it has to be good, but also somewhat average and familiar ." A similar principle seems to work in UI design.

<comment><USER4> The same applies for good movie soundtracks - the ones that fit so naturally that you don't notice them consciously but they impact you anyway.</comment></comment>


Internet connectivity in Sub-Saharan Africa - <AUTHOR> https://angani.co/blog/its-not-how-much-fibre-you-have-its-what-you-do-with-it/

<comment><USER2> _Going back to David's post, edge caching, in particular, while useful for static content hosted which is generally consumed (e.g. video) does not help with most Cloud applications. Cloud applications largely require two-way sending and receiving of data, so if we are to see more businesses being able to run interactive applications on the Cloud, more local POPs are required which carry the full application stack. For example, while your Google Photos viewing may be cached, for someone to upload the photos – you still have to upload them to a datacenter in Europe. In David Weekly's tests – Google and Apple content is actually served off a local edge cache in Nairobi, however that does not serve DNS traffic for example from Google's famous 18.104.22.168 IP address which is interactive (send-receive). Another tricky problem is if people use 22.214.171.124 – a CDN provider such as Akamai will see your request as coming from Europe, and will serve the request out of a European datacenter as opposed to the node in Nairobi._

CloudFlare will be rolling out edge hardware in Africa this year to address the latency problem. We will also be rolling out DNS in Africa as well so that DNS queries will not need to leave the continent (or even country in many cases).

For dynamic content we will be supporting our Railgun technology to help alleviate the backhaul latency from these edge machines to origin servers.

These problems are solvable and Africa has not been forgotten (at least by us). We have many clients in Africa and, of course, many people accessing CloudFlare managed sites from African countries.

<comment><USER3> > Another tricky problem is if people use 188.8.131.52 – a CDN provider such as > Akamai will see your request as coming from Europe, and will serve the > request out of a European datacenter as opposed to the node in Nairobi.

[https://tools.ietf.org/html/draft-vandergaast-edns-client- ip...](https://tools.ietf.org/html/draft-vandergaast-edns-client-ip-00) was designed to handle this exact problem.

<comment><USER2> Correct. EDNS0 is useful, but CloudFlare doesn't need to rely on that. A name like news.ycombinator.com resolves to the same IP address wherever you are in the world. We then route that IP address depending on the location of the user using BGP Anycast. This the 18.104.22.168 problem is irrelevant for us (might not be for legacy providers).

</comment></comment></comment><comment><USER4> > Bandwidth Costs

These costs trickle down to end users. Even in "progressive" (heh) South Africa 3G can cost as much as ~$150 per GB (so called "out-of-bundle").

I was attempting to set up IMAP in Thunderbird for my parents (who are forced to use 3G due to location) - the obvious merit there is the cloud backup of their emails. To conserve bandwidth they previously had POP3 set up to download headers only. This is impossible to set up with IMAP because, according to Thunderbird HDD space is more expensive than bandwidth and hence re-downloads items if you choose to have headers only. This is a well-known piece of software.

Keep the high costs of poor little Africa in mind when designing your software. Many of us are lucky enough to have access to "high-speed" 10Mbps uncapped copper but there are significant amounts of people who are outright extorted by the cellphone networks.

Please design your software to be kind with bandwidth. Latency isn't the only issue and, in fact, of all the people I know only gamers care about latency (because we have the bigger bandwidth/speed problem to deal with). The average African is quite happy to wait 1s for a website to load, they aren't happy if that page _costs_ them $1 to load.

<comment><USER5> On South Africa, it's more like 50-150 rand per GB (~4 - 13USD) [1]

Mobile data in South Africa is actually pretty cheap in absolute terms, though of course it's still extremely expensive compared to average income.

[1] [http://www.vodacom.co.za/vodacom/shopping/data/prepaid- data](http://www.vodacom.co.za/vodacom/shopping/data/prepaid-data)

<comment><USER4> > On South Africa, it's more like 50-150 rand per GB (~4 - 13USD)

        USD(Out Of Bundle Rates Per MB) * 1024MB = USD(2) * 1024MB = $164 ~ $150

Worst case scenario is the worst case scenario.

</comment></comment></comment><comment><USER6> In Ghana and Kenya, Open Learning Exchange tackles the bandwidth cost and latency issue by utilizing Ground Computing hosting based applications that require 2-way data interactions and adhere to Eventual Consistency. What Eventual Consistency means is that the apps are hosted in several communities on Ground Servers but "eventually" the Ground Servers connect to the Internet and sync with a central node on the Internet where other Ground Servers are also syncing to. This isn't so great if the latency of the 2-way data you care about is between someone in Nairobi and Atlanta but it is a big improvement between for 2-way interactions between people in the same community using the same Ground Server. Yet, there are certain kinds of 2-way data where gaps between syncs matters less as is the case of book and movie ratings in our Open Source Learning Management System that we built and host in schools, libraries, and community centers.

</comment><comment><USER7> > Another tricky problem is if people use 18.104.22.168 – a CDN provider such as > Akamai will see your request as coming from Europe, and will serve the > request out of a European datacenter as opposed to the node in Nairobi.

This is a tricky problem but there's a solution for it: edns-client-subnet, which allows the DNS provider to pass along your subnet for services which don't return the same response for everyone:

[https://tools.ietf.org/html/draft-ietf-dnsop-edns-client- sub...](https://tools.ietf.org/html/draft-ietf-dnsop-edns-client-subnet-00)

Google's Public DNS team announced that support had been enabled for Akamai last December:

[http://googlewebmastercentral.blogspot.com/2014/12/google- pu...](http://googlewebmastercentral.blogspot.com/2014/12/google-public-dns- and-location.html)

<comment><USER8> Thanks, we'll update the post to reflect this.

<comment><USER7> Thanks – it's definitely worth mentioning as a general concern since it's an important feature to look for in a DNS service no matter where you live.

</comment></comment></comment><comment><USER9> I find it interesting that the assumption is that the cloud needs to be pushed out to the edges, as opposed to making the endpoints smarter and able to speak directly to each other. To me this looks like a large factor in favour of (for example) dedicated mobile apps with their own caching logic and background processing over a one-size-fits-all HTML browser approach.

<comment><AUTHOR> There's a reason Amazon/Google and the other major providers tend to push their availability zones to the edge. It's because this matters. You can't always cache in the background especially for multimedia content. It's simply too heavy.

Not all content providers will be able to run the infrastructure that Google does, which is why it makes sense for cloud service providers to do the same.

</comment></comment><comment><USER10> Interesting read. Surprising to know that the entire continent accounts for just 3% of web traffic

</comment><comment><USER11> But I thought Google will provider coverage via balloons for free because they are such good people?</comment>


Ask HN: Where Can I donate a working computer? - <AUTHOR>

I have a 2011 iMac and MacBook that I would like to donate. I feel like giving by it to Goodwill it may go to waste. Are there any charities along the lines of "setup hackerspaces for underprivileged members of the community" that would like slightly used but very functional computers? <comment><USER2> Depending on where you are, you may try emailing Fred Wilson (@avc). I know he works with a lot of the schools in NY bringing technology in and may have some ideas. You can contact him pretty easily through his blog ([http://avc.com](http://avc.com)).

</comment><comment><USER3> Why wouldnt someone running the charity you imagine focus on getting new standardized hardware rather than fussing around with a hodgepodge of old machines?

There's probably a thrift shop who will take it to sell and put the money toward charity. It even could be sold yourself on Craigslist and the procedures used to support whatever cause exceeds Goodwill in merit.

</comment><comment><USER4> Macs are tough to donate - a lot of institutions want windows machines...

So first think of places that appreciate Macs like art schools, video or music programs, and other creative organizations. Once you find one of those I think you will have no problem.

<comment><AUTHOR> Great idea, thanks for the suggestion.

</comment></comment><comment><USER5> Depending on your area freegeek could be a great place to donate. They give away computers to people in need or let people earn a computer by volunteering and learning how to build their own!

</comment><comment><USER6> For the laptop anyway, you might take a look at: [http://www.kidsoncomputers.org/needed](http://www.kidsoncomputers.org/needed)</comment>


We Rejected Dan Shipper (And What I Learned About Hiring) - <AUTHOR> http://www.quintenfarmer.com/2012/05/02/we-rejected-dan-shipper-and-what-i-learned-about-hiring/ <comment><USER2> OP uses the word "rejected" 9 times.

A-level hiring is not about rejection. A-players aren't looking for a "job" and A-managers aren't trying to build an assembly line. It's about finding a mutual relationship where the whole is greater than the sum of its parts, and that requires a lot of things to come together.

> As a startup, if you have the opportunity to bring on an A player in a low > risk role (like an internship), you do it

All circumstances are different, so maybe he should have found room for Dan, or maybe not. Engineering quality (and talent in general) is on a power law distribution. When you consider engineering quality to be a competitive advantage (not all startups, even YC startups, need this), your early hires are your future managers, and will tend to influence the future hires. Its tough to attract and close people smarter/better than you.

I imagine myself now at age 26 (pretty decent engineer, doing functional programming and stuff), and compare to age 21 (cocky, overconfident) and while I was smart and driven and all that, it doesn't mean I can contribute to a rockstar team. I would drown and get left behind. World class teams at the top of the power law distribution, say the Clojure team[1], at age 21 I'd be a few orders of magnitude behind them and just not worth paying. at age 26, who knows, probably still at least 1 order of magnitude behind, not worth hiring. Square peg, round hole. At least I can see my trajectory of exponential growth over the last 5 years, and if that keeps up another decade, I can build or join a team like that.

[1] <http://thinkrelevance.com/team>

<comment><AUTHOR> (OP here)

I should probably note that the use of the word "rejected" was done to emphasize the humor of the situation (we rejected someone who is now being publicly recruited), and to draw attention to what a mistake it was.

I would completely agree that A level hiring isn't about rejection, job descriptions, or assembly lines. That's why I am such a fan of what Jason Freedman is doing at 42floors.

To your second point, I remain a firm believer that early stage companies (less than 10 people) in this hiring environment should be open to bringing on A players, even if they don't fit into an immediately defined role.

</comment></comment><comment><USER3> I know nothing about "Wheremyfriends.be, a simple webapp " and it is interesting to see how a brand is being built around one of its three developers. However, I hope we see less of these hero-worship posts on the front page (or at least limit the number of hero-worship posts until something more substantial has been accomplished)

Btw self-seeking flattery of powerful, influential people is generally considered as sycophantic.

Is there a good term for flattering people like the guy described in this post :)

 _[edit] for chc and anyone else who didn't get the sarcasm, yes, the intern wasn't powerful. That is precisely why the smiley-question asked for a new term_

<comment><USER4> I thought we were talking about a college sophomore who wrote something cool and gained recognition for his skills. Who's the powerful, influential person that Mr. Farmer is flattering?

I believe the term for flattering people like Dan Shipper is "paying a compliment."

EDIT: You're right, I didn't get the sarcasm. In the context of a comment that described the OP as "hero-worship," it doesn't come across as sarcasm. If we acknowledge that the guy is neither powerful nor influential, it seems pretty obvious that the OP's "flattery" is merely paying a compliment to somebody he respects. I think we could use more of that and less "So-and-so is full of shit" articles.

</comment></comment><comment><USER5> _But in a review between myself and the company cofounders, one sticking point kept coming up: Not enough relevant development experience. We were trying to build a core front end team that had an existing passion for mobile/tablet, and Dan was a square peg in that round hole._

It's hard to comment accurately without knowing much more about both sides involved here, but generally speaking, rejecting a (skilled+proven) developer who's willing to learn a new technology _because_ he doesn't have experience in it is just...a mistake.

<comment><USER6> What are the boundary conditions here? I'm a young developer (23) who's made over half a million USD$ from web apps while in school - would many companies be interested in hiring even with programming languages I have little to no experience with?

<comment><USER7> The startup I work for would! Assuming, at least, that you have experience in a couple moderately dissimilar languages (e.g. not only Java) it's really not a big deal. It's important to have _someone_ who is a _real_ expert on a language you're using, but most people can just be good at it, and you can get good at most languages quickly with sustained effort.

What is a big deal is being smart and motivated and knowing a lot about at least something useful. Everything else is gravy.

</comment></comment></comment><comment><USER8> Naming names in these "Why we didn't hire this person" always comes across as showy posturing. If you want to talk about insight gained in the hiring process, you don't need names. You only need names if you want people to be impressed by all the famous people you turned down.

<comment><USER9> They're trying to ride the Dan Shipper bandwagon. I thought that would have been clear; there's no other reason to write this post.

</comment></comment><comment><USER10> > it was clear that Dan had the mentality and intellect of an A Player. He was > young, driven, and extremely intelligent.

Young has nothing to do with mentality or intellect. Nice to hear someone who is in charge of hiring be honest and say it out loud though.

<comment><USER8> Whoa there. The quote can be interpreted two ways; the way you read it ("young, driven, intelligent" = "mentality and intellect of an A Player") and the way I read it ("mentality and intellect of an A Player. _In addition,_ he was young, driven...")

<comment><USER11> There's no legal upside to publicly being interested in young people. The best you can really do is look for inexperienced people with low salary requirements.

</comment></comment></comment><comment><USER12> It's funny, when I read that 42 floors offer, I thought Dan Shipper was a name they made up for a hypothetical developer who "Ships" (produces) code.

So I thought it was an open letter to all good developers that they wanted to hire them.

Granted, I only read the first half of the post...

</comment><comment><USER13> I think the whole thing is kinda ridiculous, making a blog post to hire somebody.. Maybe I'm just outgrowing HN or it's because I'm from Europe that I don't get the fuzz.

More on topic, I quite liked this blog post & think it was much more informant than the one from 42F

<comment><USER14> You're conflating HN with a single blog post.

</comment></comment><comment><USER15> Who is Dan Shipper?

<comment><USER16> I believe it's referring to this: <http://bit.ly/IHTk05>.

<comment><USER17> Actual link the above obfuscated link refers to:

[http://42floors.com/blog/posts/consider-this-a-job-offer- to-...](http://42floors.com/blog/posts/consider-this-a-job-offer-to-work- at-42floors)

</comment></comment></comment><comment><USER18> Hiring is random. There have been numerous studies that show people are highly influenced by interviews despite the fact that it's not a good indicator of ability. Would you have hired Systrom or Krieger (instagram) as engineers? AFAIK, they had no experience building large-scale sites, yet put together instagram. Would you have hired Paul Bucheit when he applied to Google? AFAIK, he hadn't done anything spectacular yet.

A-players are by definition people who have done something big. Who hired these people to give them a chance to do something big? What about the larger group of people whose startup flamed out before they could demo their skills? It's all a crapshoot.

Hiring is easy. A small startup is receiving resumes from a self-selected group of people who want to work there despite the low pay and lower job security. Most of the people reading HN could build an instagram-type system. So once you filter for ability with some stupid programming quiz, just pick the guy you like. If your company succeeds, he will magically be considered an A-player.

</comment><comment><USER19> writing one sided stories about your business relations in public, like this, is not ethically correct, in my opinion.

<comment><AUTHOR> Dan reviewed this post in its' entirety before it was published. I gave him final say on whether or not it was published or submitted here.

I agree that personal stories like this should have the approval of everyone involved before they are published.

<comment><USER20> I strongly suggest you add this to the post itself. If not early on, then at least at the end. My thought that "they probably let Dan sign off on this, but it would be good to know for sure" distracted from focusing on the broader point of your article.

<comment><USER21> I strongly agree with this point. I was also very distracted during this very good post wondering if Dan signed off on it before it was published. By the end I assumed that was the case, but the message would have been better served without the distraction.

</comment></comment></comment></comment><comment><USER22> I'm waiting for the michaelochurch analysis of this situation.

<comment><USER11> Did Google also reject Dan?

(Kidding, I have really enjoyed Michael's recent spurt of blog activity.)

</comment></comment><comment><USER23> What is an "A Player"?

<comment><USER11> It appears to be a reductive staffing methodology invented by this book: [http://www.amazon.com/Topgrading-Leading-Companies- Coaching-...](http://www.amazon.com/Topgrading-Leading-Companies-Coaching- Keeping/dp/1591840813)

The key points:

\- Hire people who are awesome

\- Fire people who aren't

<comment><USER24> The term A-player originated in the sports world a long, long time ago. It is literally a player of the sport who is considered A-grade material. This phrase, like many other from the sports world, has been adopted into the business lexicon.

</comment></comment></comment><comment><USER25> This feels like a reach to prove superiority and save face. Surprised Dan approved it.

</comment><comment><USER26> TL;DR:

HR at some no-name start-up gets it into their head that some kid can make all their programming dreams come true. They made a blog post to lure him.

Later, HR at some other no-name startup makes a blog post proudly announcing that the kid in question applied there first, also making some annoying backhanded compliments about how their standards were too high or some lame shit.

What next? Nothing at all.

<comment><USER27> Not only do I hate TL;DR meme (I don't believe it fits the community at HN), but that isn't even an accurate summary of the article or the situation.

<comment><USER28> Agreed. I thought the parent was ill served by editing the post and moderating the language, which for me more accurately described how I felt about the article and situation.

</comment></comment></comment><comment><USER29> _Always hire A Players_

Words cannot describe how tired I am of hearing these sorts of weird sports metaphors used to describe tech talent. Are we really so immature as an industry that we have to constantly rely on sports metaphors? It's as bad as talking about ninjas and rockstars.

Also, I'm really sick of all these mindless, droning, uninsightful posts about hiring in general. Who cares? Tech interviewing and hiring is a joke, and everyone here knows it. Larger companies act like you're saddled with the new hire forever, even though in most places reading HN it's extremely easy to fire people. Small companies act like they absolutely have to find the best talent in the universe, even though what they actually need to do is get some product shipped. It's all just ego stroking and an attempt to codify insecurities.

How about more awesome submissions about hacks, ingenuity, and product success, and less B.S. about name-dropping, baseless pontification on interview tactics, and overuse of bad metaphors. While were at it, let's drop the bubble FUD and ridiculous guesses about big company tech strategy.

/rant</comment>


ObjectIdColumns: Transparently Store MongoDB BSON IDs in a RDBMS - <AUTHOR> https://swiftype.com/blog/objectid-columns.html

<comment><USER2> We do the exact same thing at Buildy, but in Python. Here's the class:

[https://gist.github.com/irskep/9099262](https://gist.github.com/irskep/9099262)

<comment><AUTHOR> Nice, Steve! Always good to see validation of the idea in both directions. :)

</comment></comment><comment><USER3> As we gradually migrate data from MongoDB to MySQL, this code has been hugely helpful. It is making the transition as smooth and efficient as possible. I hope others can get some value out of the gem as well.</comment>


Stellar, Uber for private jets - <AUTHOR> https://www.stellar.aero/ <comment><USER2> What about surge pricing? Will you give me my first fare free? How often do you screw over pilots by raising your rates? Are the planes older models? This guy doesn't know where the Bermuda Triangle is, and his GPS stopped working. What should I do?

</comment><comment><USER3> They tout their integration with Apple Pay and TouchID. Is it really wise to trust your phone with the ability to authorize transactions in the tens of thousands of dollars?

<comment><USER4> I don't understand this concern.

I walk into a jeweler's with a bit of 3 year old plastic and walk out with a six figure watch.

Is it really wise to trust this bit of plastic with the ability to authorize transactions in the tens of thousands of dollars?

Why is adding my fingerprint to the transaction worse?

</comment></comment><comment><USER5> Nothing new here, On-demand charter flights and empty-leg market arbitrage as been going on for years, the UK as a very strong market.

The fundamental problem is lack of inventory on the supply side and high costs.

</comment><comment> david-given None of the 'book' links on their page work? Is this actually running yet or are they just touting for planes at this point?

</comment><comment><USER6> This was a horrible idea to put online, when it's not functional and appears to be broken.

</comment><comment><USER7> flight sharing a private charter between east and west coasts seems feasible, right?

</comment><comment><USER8> How many of these things have been shut down by the FAA so far?

<comment><USER8> Some quick googling suggests 'AirPooler' and 'FlyteNow' have both been disrupted by the FAA, FlyBlade and Bluestar Jets and Blackjet may or may not be doing something different to stay within the bounds of the law.

<comment><USER9> Stellar says they're "the world's first commercial operations system for Part 135 business jet operators", which means their flights have commercially licensed pilots working for regulated carriers, not private pilots illegally carrying passengers for hire, like FlyteNow's pilots were. So it sounds legal.

</comment></comment></comment><comment><USER10> NetJet anyone?</comment>


Owning the stack: The legal war to control the smartphone platform - <AUTHOR> http://arstechnica.com/tech-policy/news/2011/09/owning-the-stack-the-legal-war-for-control-of-the-smartphone-platform.ars <comment><USER2> The desktop computer market has been extremely innovative and robust over the last two decades without much of this kind of legal turf-grabbing. This kind of thing is ultimately only going to hurt consumers.

<comment> guard-of-terra During the windows rule, hardware specs went up but not much happened otherwise. Hard drives and RAM became 100 times bigger, but there was the same Start button, the same IE5 that became IE6 (sigh), the same games and videos on CDs. The only big innovative thing born in those years was 3D acceleration.

In the last few years we've seen a tremendous explosion of modern web, content streaming services; in the same time, rich mobile devices caught traction. They failed to do so for the whole 00s despite everyone predicting them to each year.

Basically, that's what having a platform monopoly or having no platform at all gets you.

<comment><USER2> Things would have been more interesting without the Windows monopoly, I agree. But in that time we got cheap but extremely powerful personal computers with robust, mulitasking operating systems and a huge ecosystem of very capable applications. Maybe there haven't been many radical innovations in the WIMP interface but these things go in cycles and the this one served consumers very well.

</comment></comment></comment><comment><USER3> "Everyone wants to be part of a winning stack, but even better is to be the bottleneck in a winning stack so that everyone else can join in only on your terms—and at your price." That's one cool line!

</comment><comment><USER4> Thanks - a brilliantly written article, as is customary at Ars...</comment>


Startup Quote (Anniv. Ed.): Bruce Wayne, ceo & chairman, Wayne Enterprises - <AUTHOR> http://startupquote.com/post/4234103411

<comment><AUTHOR> Everything's impossible until somebody does it.

\- Bruce Wayne

<http://startupquote.com/post/4234103411></comment>


Ask HN: How do you serve Torch models in production? - <AUTHOR> I have been using Tensorflow for my latest project. I have been very happy with it but there is a piece of the project in which it would be easiest to use Torch, for the pretrained models. Only thing I am stuck on is how people are serving Torch models in production. Tensorflow has a nice serving system, but I can't seem to find any recommendations on how to best serve Torch models in a low-latency application. Thanks! <comment><USER2> I heard, at Facebook, they use Torch to train a model and they have a tool to convert the result to a caffe model. And use caffe for inference.

<comment><AUTHOR> That's really interesting, tensorflow is really leading the pack with their production system</comment></comment>


And You Thought Java Was Oracle's Biggest Security Blunder… - <AUTHOR> http://www.tripwire.com/state-of-security/incident-detection/and-you-thought-java-was-oracles-biggest-security-blunder/#.VVHrL9tWDsg.hackernews <comment><USER2> This is grossly irresponsible on Oracle's part. I cannot fathom what they hoped to gain by releasing a product they never even intended to patch.

I don't use it myself so I wouldn't have standing, but I can readily see lawsuits resulting from this.</comment>


Show HN: Podfio – Lending Library - <AUTHOR> Hello!
We are building a service where people can rent just about anything from locals - be it cameras, skates or a tent. Additionally, if you have something to share, you can easily submit it to us for review and make extra cash by lending your item out.
Here is a link: https://podfio.com
This is a very rough version but we just started and want to see people's response.
Thank you! <comment><USER2> Great concept! good luck!</comment>


Surveying 1000 APIs - Think APIs are for Tech Startups Only? Think Again - <AUTHOR> http://www.3scale.net/2012/06/surveying-1000-apis-think-apis-are-for-startups-only-think-again/

<comment><USER2> The same author recently presented at Gluecon about the future of APIs. The presentation can be found here and is an interesting read: [http://blog.programmableweb.com/2012/06/05/reaching-a- millio...](http://blog.programmableweb.com/2012/06/05/reaching-a-million-apis- and-what-to-do-when-we-get-there/)

</comment><comment><USER3> Is great to see a lot of companies releasing APIs but if even better as a developer start a project and being able to easily use external services and even integrate your app with bigger services. Imagine creating an e-commerce an integrating your catalog with stuff from macys, amazon, etc.

<comment><AUTHOR> I think this will end up happening - Amazon already lets you suck out it's catalogue and push affiliate sales back to it. You could likely build very sophisticated niche virtual retail sites if all the bricks and motar data was there also.

Interestingly in the survey the number of ecommerce APIs was actually a lot lower than we thought it would be. This might be a blip in this "generation" of APIs, or that it's simply hard to do still.

<comment><USER3> Amazon has a great vision, others are just afraid about people "stealing" their content. Probably companies need to be educated in this area

<comment><AUTHOR> I guess the model needs to be right so everybody along the way gets paid, then people may step up.

</comment></comment></comment></comment><comment><USER4> I think that Twitter has absorbed some of the long tail - it's pretty easy to use a twitter account as a proxy for read and write commands for a low scale device or web site, of course you can't make it a chargeable interaction!

<comment><AUTHOR> Pretty interesting way to think of twitter - kind of a message bus.

<comment><USER5> Its an information network for humans and otherwise. Its simplicity + #hashtags makes work well for distributed messaging.

</comment></comment></comment><comment><USER6> Glad to see the takeup of APIs by the public sector. Will be interesting to watch whether emerging API management solutions can help the open data movement by enabling smarter ways of accessing government data.

<comment><AUTHOR> The US definitely seems to be leading here - makes more sense to open data than to try to provide more and more different UIs that people need (iPhone, Android, ...). Of the APIs on programmable web in that period almost all were US, a couple were UK and a couple we Canadian.

Further back in time there are other countries also though - and a lot of the Scientific APIs were European.

</comment></comment><comment><USER7> Very interesting analysis and segmentation of APIs! I am just wondering which factor has been so the most prominent for not having the long tail represented (or badly)?

</comment><comment><USER8> "The long tail is still missing in action" - plug&play arriving onto the API scene is going to change that fast.

<comment><USER9> What is plug&play? Is it a concept similar to Microsoft's driver system, or is it a startup, or is it a standard for API designers?

<comment><AUTHOR> I'd say platforms like stackmob which create mobile APIs + wordpress, drupal etc. which make it easy to add an API to those + then platforms like 3scale, mashery etc. which help manage access.

</comment></comment></comment><comment><USER10> We need more hard data like this on the API ecosystem. ProgrammbleWeb is a great source.

<comment><USER5> agreed there is not enough metrics on what is out there, where the opportunities are? I know John and the team @ PW is working hard to track on all of this, but its good to see more detail get pulled from someone external.

If anyone else wants to help slice and dice and make sense of it all, ProgrammableWeb has an API....

<comment><AUTHOR> Yep - here: <http://www.programmableweb.com/api/programmableweb>

</comment></comment></comment><comment><USER11> 1000APIs is a small sample size.

<comment><AUTHOR> Yes, agreed - but the total number of APIs listed is a little over 6000, so it's a reasonable sample to some extent. There are a lot more APIs out there which aren't public / listed - so it's hard to account for those.

</comment></comment><comment><USER12> Great analisys!</comment>