Amazon DSSTNE: Deep Scalable Sparse Tensor Network Engine - <AUTHOR> https://github.com/amznlabs/amazon-dsstne <comment><USER2> At a glance:

      - Only supports fully connected layers for now. No convnets or RNNs.

      - Requires a GPU. No option to run on CPU, not even for development.

      - Setup instructions for Ubuntu only. No Mac or Windows.

      - Uses JSON to define the network architecture. Which limits what you can build.

      - Takes in data in NetCDF format only.

      - Very little documentation.

      - The name is bad. I'm not going to remember how to spell DSSTNE.

It seems like a very early proof of concept. I wouldn't expect it to be useful to most people at this point. Built-in support for sparse vectors is interesting, but not a strong selling point by itself. I hope Amazon continues to develop it. Or, even better, contribute to one of the existing more mature frameworks.

<comment><USER3> It's more than that, and it's in use in production at Amazon. 8 TitanX GPUs can contain networks with up to 6 billion weights. As Geoffrey Hinton once said:

"My belief is that we're not going to get human-level abilities until we have systems that have the same number of parameters in them as the brain."

And you're right that it's a specialized framework/engine. But IMO making it more general purpose is a matter of cutting and pasting the right cuDNN code or we can double down on emphasizing sparse data. Amazon OSSed this partially IMO to see what people would want here.

<comment><USER4> > "My belief is that we're not going to get human-level abilities until we > have systems that have the same number of parameters in them as the brain."

An interesting quote.

Replicating functioning of the brain, or some major subsystem of it, is no doubt going to require far more than just billions of parameters. The cortex contains >15 billion neurons, but there are also the neurons contained in all the other brain structures. Furthermore, neurons connect via dense dendritic trees, the human brain having on the order of 100 trillion synapses.

Adding to the complexity, neurons have numerous "communication ports", including numerous pre- and postsynaptic neurotransmitter receptors, and a wide range of receptors for endocrine, immune system and other types of signals. Message propagation typically involves as well the layer of complex intracellular "second-messenger" transformations.

While it's highly probably future NNs will be developed that do even more amazing things than now possible, I think the challenge of equaling what real brains do is to say the least enormously daunting.

Somebody smarter than me could probably figure out the magnitude, how many nodes or weights it takes for a NN to function like the brain, though I imagine it will be a really impressive number.

Edit: typos

<comment><USER5> _> Replicating functioning of the brain, or some major subsystem of it, is no doubt going to require far more than just billions of parameters._

Maybe, but we shouldn't forget that computers do not suddenly lose their capability to function as exact, deterministic, programmable machines just because they happen to run an ANN.

What I mean is that there may be shortcuts to reduce the number of required nodes dramatically.

If you take the state of an ANN after it was trained to perform some specific task, you can ask the question whether there is a simpler function, i.e. one with much fewer parameters, that approximates the learned function.

Sort of like a human with the Occam's razor gene. I think the fact that the number of neurons does not correlate perfectly with intelligence in animals is an indication that there is room for optimization.

<comment><USER3> Absolutely 100% agree, but at the same time, I think we will ultimately need to build and evaluate models that can span the memory of more than one processor. I don't think a single GTX Titan X, GTX 1080 or even a server is enough here.

Additionally, data parallelization and ASGD broadly disallow these larger models (yes I know about send/receive nodes in TensorFlow, but they're not general or automatic enough for researchers IMO) while ASGD makes horribly inefficient use of the very limited bandwidth between processors. All IMO of course. There are hacks and tricks here, but I think those should be late stage optimizations, not requirements to achieve scaling.

Finally, I'm a stickler for deterministic computation as someone who spent a decade writing graphics drivers before joining the CUDA team in 2006, but that's pretty much a "hear me now, believe me later" opinion of mine after tracking down too many bizarro race conditions late into the night in that former life :-). Of course, one person's race condition can sometimes be an ANN's regularizer, but I digress.

I also agree we'll do some amazing things with far fewer neurons and weights than an actual human brain, but I'll bet you good money we end up needing more than 12GB to do it. AlphaGo alone was 200+ GPUs, right?

</comment></comment></comment></comment></comment><comment><USER6> Amazon is turning a new leaf. They stopped publishing to any major conferences after their last significant paper, DynamoDB.

My perception of Amazon is that they take everything from open-source but don't actively give back. Amazon and open-source never went hand-in-hand. Making their deep learning frameworks open-source is cool. Kudos to the team which managed to do this. I am sure internally, it must have been a huge struggle to get the approval from execs.

[Edit: Grammar]

<comment><USER6> For a second, a thought crossed my mind that Amazon is actively trying to change its external perception after the NY times article and is trying to cozy up to developers. I found this on Glassdoor. Apparently, it will take a long time for them to make their culture less toxic.

===From Glassdoor===

Cons

====

The management process is abusive, and I'm currently a manager. I've seen too much "behind the wall" and hate how our individual performers can be treated. You are forced to ride people and stack rank employees...I've been forced to give good employees bad overall ratings because of politics and stack ranking. Advice to Management Don't pretend that the recent NY Times article was all about "isolated incidents". The culture IS abusive and it WILL backfire once stock value starts to drop. I'm an 8 year veteran and I no longer recommend former peers to interview with Amazon.

== [Edit: Formatted to make it clear what was pulled from Glassdoor]

<comment><USER7> I just joined AWS ProServ and I really don't see any of these things. Pretty amazing team and one of the best work life balance I've seen in a tech company so far. I have 4 other friends who work at AWS and all seem very happy so far. I found the glass door comment and it seems to be from an engineering manager. I have a friend who manages one of the AWS products and he seems to be pretty happy.

I just joined so I really am not a statistically significant case but so far it's no where near what was in that NYT article.

Edit: I can't read apparently :) thanks heuving for clarifying and the commenter for reformatting

<comment><USER8> I suspect an inverse survivorship bias in the public representation of the company by ex-employees. Those of us with positive recollections tend to say very little, and (in my case at least) that's due to respect for Amazon's culture.

<comment><USER9> Eh, I had a pretty good ride there myself. I believe every incident in that NY Times article happened.

If you were at Amazon for any length of time and didn't notice the existence of toxic teams and the random chance element of being hired into one of them, you weren't paying attention.

</comment></comment></comment></comment></comment><comment><USER10> First TensorFlow and now this. Tensor is quickly becoming a mathematical-term- that-sounds-familiar-to-developers-but-most-don't-know-what-it-is-actually.

Another example is topology =)

<comment><USER11> Other one is isomorphic. Anything that sounds sciency or mathy will be adopted. There is no other way ;-)

<comment><USER12> my new programming language has isomorphic tensors built in as a first class language feature :-)

<comment><USER11> Sold!

I'll use it write microservices for my new IoT application.

</comment></comment></comment></comment><comment><USER3> Lead author of DSSTNE here...

1\. DSSTNE was designed two years ago specifically for product recommendations from Amazon's catalog. At that time, there was no TensorFlow, only Theano and Torch. DSSTNE differentiated from these two frameworks by optimizing for sparse data and multi-GPU spanning neural networks. What it's not currently is another framework for running AlexNet/VGG/GoogleNet etc, but about 500 lines of code plus cuDNN could change that if the demand exists. Implementing Krizhevsky's one weird trick is mostly trivial since the harder model parallel part has already been written.

2\. DSSTNE does not yet explicitly support RNNs, but it does have support for shared weights and that's more than enough to build an unrolled RNN. We tried a few in fact. CuDNN 5 can be used to add LSTM support in a couple hundred lines of code. But since (I believe) the LSTM in cuDNN is a black box, it cannot be spread across multiple GPUs. Not too hard to write from the ground up though.

3\. There are a huge number of collaborators and people behind the scenes that made this happen. I'd love to acknowledge them openly, but I'm not sure they want their names known.

4\. Say what you want about Amazon, and they're not perfect, but they let us build this from the ground up and now they have given it away. Google hired me away from NVIDIA (another one of those offers I couldn't refuse) OTOH blind- allocated me into search in 2011 and would not let me work with GPUs despite my being one of the founding members of NVIDIA's CUDA team because they had not yet seen them as useful. I didn't stay there long. DSSTNE is 100% fresh code, warts and all, and I think Amazon both for letting me work on a project like this and for OSSing the code.

5\. NetCDF is a nice efficient format for big data files. What other formats would you suggest we support here?

6\. I was boarding a plane when they finally released this. I will be benchmarking it in the next few days. TLDR spoilers: near-perfect scaling for hidden layers with 1000 or so hidden units per GPU in use, and effectively free sparse input layers because both activation and weight gradient calculation have custom sparse kernels.

7\. The JSON format made sense in 2014, but IMO what this engine needs now is a TensorFlow graph importer. Since the engine builds networks from a rather simple underlying C struct, this isn't particularly hard, but it does require supporting some additional functionality to be 100% compatible.

8\. I left Amazon 4 months ago after getting an offer I couldn't refuse. I was the sole GPU coder on this project. I can count the number of people I'd trust with an engine like this with two hands and most of them are already building deep learning engines elsewhere. I'm happy to add whatever functionality is desired here. CNN and RNN support seem like two good first steps and the spec already accounts for this.

8\. Ditto for a Python interface, easily implemented IMO through the Python C/C++ extension mechanism: [https://docs.python.org/2/extending/extending.html](https://docs.python.org/2/extending/extending.html)

Anyway, it's late, and it's turned out to be a fantastic day to see the project on which I spent nearly two years go OSS.

<comment><USER13> Thanks for sharing your story!

Let me comment on file formats as someone familiar with both netCDF and deep learning.

I agree that netCDF is a sane binary file format for this application. It's designed for efficient serialization of large arrays of numbers. One downside is that netCDF does not support streaming without writing the data to intermediate files on disk.

Keep in mind that netCDF v4 is itself just a thin wrapper around HDF5. Given that your input format is basically a custom file format written in netCDF, I would have just used HDF5 directly. The API is about as convenient, and this would skip one layer of indirection.

The native file format for TensorFlow is its own custom TFRecords file format, but it also supports a number of other file formats. TFRecords is _much_ simpler technology than NetCDF/HDF5. It's basically just a bunch of serialized protocol buffers [1]. About all you can do with a TFRecords file is pull out examples -- it doesn't support the fancy multi-dimensional indexing or hierarchical structure of netCDF/HDF5. But that's also most of what you need for building machine learning models, and it's quite straightforward to read/write them in a streaming fashion, which makes it a natural fit for technologies like map-reduce.

[1] [https://www.tensorflow.org/versions/r0.8/api_docs/python/pyt...](https://www.tensorflow.org/versions/r0.8/api_docs/python/python_io.html#tfrecords- format-details)

<comment><USER3> Thanks for that! And boy, I wish I had the resources the TensorFlow team has to build standards like this and also to write their own custom CUDA compiler.

I do want the multi-dimensional indexing for RNN data though. Maybe support HDF5 directly is the path forward.

Thanks again!

</comment></comment></comment><comment><USER14> Deep Learning systems are becoming C++11's halo projects. Here are some deep learning libraries from the Internet Big 4.

Amazon DSSTNE - [https://github.com/amznlabs/amazon- dsstne](https://github.com/amznlabs/amazon-dsstne)

Google TensorFlow - [https://github.com/tensorflow/tensorflow/](https://github.com/tensorflow/tensorflow/)

Microsoft CNTK - [https://github.com/Microsoft/CNTK/](https://github.com/Microsoft/CNTK/)

Facebook fbcunn - [https://github.com/facebook/fbcunn/](https://github.com/facebook/fbcunn/)

They all utilize C++11 or later. Just as Hadoop pushed Java in the big data, map-reduce realm, I think these libraries will push C++11 in the Deep Learning realm.

</comment><comment><USER15> I get the acronym is easy to pronounce with the suggested word, but why not just use the suggested word (destiny) as the name instead of the acronym. So much easier to read and write. They could explain the name's origin in Readme.md

<comment><USER16> "Destiny" would also be ungooglable.

<comment><USER17> Meanwhile, DSSTNE is completely unmemorable, so even if you wanted to google it, you're going to end up typing "amazon destiny machine learning" or something

<comment><USER18> I don't know about you, but I'm much more likely to be googling a project as I'm already working with it as opposed to for general information purpose. In that context, "DSSTNE [problem keywords]," seems more useful to me.

<comment><USER17> That's true... I was mostly thinking about the case where you aren't using it, but remember hearing about it or want to check it out.

</comment></comment></comment></comment></comment><comment><USER19> Maybe someone who works on deep learning could comment on what this provides vs other open source systems like theano, tensorflow, torch, etc.

<comment><USER20> They claim it's twice as fast as tensorflow, which is not blow-you-out-of-the- water (compare to like 50x speedup from GPU on most places), but it's a solid speedup.

It's easily parallelizable on GPU's, or so the claim goes.

Its configuration language is much, much shorter than caffe's, but upon inspection it looks like that the configuration language is also much less flexible than caffe's and they implemented a damn sight less stuff. No recurrent anything, for example, or LSTM, no gating stuff that you would need if you were doing LSTM, no residual net stuff, just off the top of my head.

It looks like much, much less complete docs in comparison to TF and Theano and things. Note the probability of dropout given in the user docs, but the actual documentation for dropout feature is hidden away inside the repo.

The important thing, however, is that they claim that there's a significant improvement on doing training on extraordinarily sparse datasets, like recommender systems and things like that. It seems very specialized for that specific exact purpose: see only accepting NetCDF format data, which is common enough in climatology-land but less common in machine learning-land proper.

The test coverage... To a first approximation, there is no test coverage. It seems quite research project-y.

</comment></comment><comment><USER21> Soo... what is the application for this (other than buzzwords)

<comment><USER22> srsly? all the discussion above 10hrs+ before your comment and that's your question?

RTFM: [https://github.com/amznlabs/amazon- dsstne/blob/master/FAQ.md](https://github.com/amznlabs/amazon- dsstne/blob/master/FAQ.md)</comment></comment>


Rapportive (YC S10) Raises Over $1 Million From Gmail Creator & Others - <AUTHOR> http://techcrunch.com/2010/08/02/rapportive-funding/

<comment><USER4> I was confused about some of the jargon in Paul Graham's "The future is no fixed amount, no fixed closing date, and no lead" quote so I looked them up best I could:

no lead - A lead investor is basically someone who commits, and says "we're in" at the start of the round. They commit organizational and negotiating resources. They also typically take a board seat. See <http://www.avc.com/a_vc/2007/10/the-lead-invest.html>

mass syndication - Instead of having a lead investor and a big closing event, you drum up interest from lots of parties who commit based on the understand that other investors are also buying in with the same terms. See [http://blog.rafaelcorrales.com/2010/05/mass-syndication- is-p...](http://blog.rafaelcorrales.com/2010/05/mass-syndication-is-party- round.html)

no fixed amounts - This seems to mean that a valuation is set, but the actual amount of money invested is not. I imagine it's sort of like a line of credit, but the startup takes as little as they need. Does anyone else know more details about this?

no closing - I'm reaching here a little, but I think what's implied here is that instead of getting everyone to commit to one big agreement, which is signed at once with great fanfare and large checked being signed, the startup just gets interested parties to sign on one at a time, and they go until they have sufficient funding. This is obviously more of a leap of faith for investors, so you have to set your valuation below market. See <http://venturehacks.com/articles/no-lead>

Anyone else have details on what these things mean? I'm just clicking around reading blog entries.

<comment><AUTHOR> Your definitions are pretty close.

no lead = there's no one investor who particularly stands out (e.g. by taking >30% of the round); a lead would usually drum up the other investors; with no lead, it's the entrepreneur who drums up the others.

no fixed amounts, no closing: with convertible notes (as opposed to straightforward sale of shares, aka priced round) the entrepreneur can be a lot more flexible. The round doesn't technically have a valuation, but it does have a valuation cap: see my write-up [http://www.yes-no- cancel.co.uk/2010/05/05/valuation-caps-on-...](http://www.yes-no- cancel.co.uk/2010/05/05/valuation-caps-on-convertible-notes-explained-with- graphs/) for an explanation. The amount raised can also be flexible.

This doesn't require any more of a leap of faith or any lower valuation than a priced round, and the paperwork is simpler. That's why convertible notes are very popular for seed rounds these days.

</comment></comment><comment><USER5> So in about 18-24 months the Rapportive team should have had time to acclimate to the new corporate culture and integrate these features into Gmail natively. I'm definitely going to use it then.

</comment><comment><USER6> Rapportive looks cool but I wouldn't be surprised to see something similar or 'good enough' appear in Labs at some point. It reminds me of url shorteners on twitter.

<comment><USER7> Looking at the set of investors who endorsed their vision, I think they probably have a lot more to come which may define why its more interesting beyond the mashups they have right now.

</comment></comment><comment><USER8> Not to be disgusting...but where is the revenue generation play?

Or is this a str8 sell to Google play?

<comment><USER9> I would probably pay $5 for it right now. It's really fucking useful.

<comment><USER10> For entrepreneur 'networking' linkedin types I can see the value. For everyone else, not sure...

<comment><USER11> Actually I can see the value for any random middle-manager at a big company whose job involves a bucketload of email correspondance with lots of people outside of the company. That seems like a big enough market to me.

<comment><USER10> Are random middle-managers at a big company using GMail though? Wouldn't they be using outlook/exchange?

<comment><USER12> Yeah, these guys are competing with Xobni.

</comment></comment></comment></comment></comment></comment><comment><USER13> Is that legal to replace the ads? I would think it is against the TOS of the site.

<comment><USER14> Users chose to do it to themselves. So presumably its just as legal as adblock

<comment><USER13> Yes they do it but I just question how viable a company is that relies on blocking ads to display their data.

<comment><USER15> If it drives more people to the online version of Gmail then that's still a win for Gmail in some sense (if not monetarily). I currently use Gmail via IMAP so never see any ads but Rapportive could tempt me back online.

<comment><USER16> Yep, especially if it gives them a lot of useful information about you to target ads to you in the rest of the googleverse.

</comment></comment></comment></comment></comment><comment><USER17> congrats to an amazing Open Angel Forum company!

proud to be an investor.

<comment><USER18> proud to have you on the team. let's crush it.

<comment><USER19> How do you get away with removing the ads from Gmail? is that not against their TOS?

<comment><USER20> We believe the context we provide is a much more valuable use of the limited sidebar space than the ads it replaces; and we think that adds value to the platform, both for Google and for its users. People evidently agree, as (nearly) nobody complains about the loss of the ads, but lots of people love the utility we provide - and indeed have switched to Gmail to get it.

We're not interested in cheating Google out of their ad revenue, although we suspect the Gmail ads aren't making that much revenue anyway.

We have a good relationship with Google, and we were a launch partner for their contextual gadgets platform: [http://googleenterprise.blogspot.com/2010/05/putting- email-i...](http://googleenterprise.blogspot.com/2010/05/putting-email-in- context-with-gmail.html)

<comment><USER21> That didn't really answer the questions.

<comment><USER22> Actually it does. Google thinks replacing the ads is fine, because they are adding more value to Google's email product than they are taking away.

</comment></comment></comment></comment></comment></comment><comment><USER23> Not to be biased, since I know the guys, but eTacts (YC w2010) is a much more powerful solution for making my inbox life better.

<comment><USER24> I think that Raplets are what makes the difference between the two services, because it let developers easily integrate their application in it, so for example if you have a to-do list app, you could integrate it with Rapportive to show any open to-do item related to that person.

<comment><USER20> We're really excited about the potential of Raplets: they make users' lives easier by integrating repetitive workflows into their email, and they help websites increase engagement by being right there in their users' inboxes.

For example, we have an internal admin page allowing us to fix and diagnose bugs, showing us what browsers someone uses, etc. It took us about half an hour to build a Raplet that integrates that into our email, so that whenever we get a support email, we're one click away from the admin panel for that user.

We also have some great integrations with web services like Crunchbase - <USER2> an investor's portfolio as soon as they email you - and MailChimp - when a customer emails you, see whether they read your newsletters and what links they've clicked on.

</comment></comment></comment><comment><USER7> Rapportive is awesome because its simple and easy to understand. Everyone gets it without any confusion. Would be interesting to see how they make money, though. Thats a toughie, especially considering the target market is Gmail users.

Also, anyone know how they are getting Twitter, Facebook, LinkedIn and Skype info for email addresses? Are they using RapLeaf? If so, are they paying for every such lookup? That would be a huge cost to deal with.

</comment><comment><USER24> Congrats to Rapportive! They are awesome, especially when it comes to help other developers to build on their platform.

</comment><comment><USER25> I wonder how many people install this just to get rid of the Google ads. I did.

</comment><comment><USER26> congrats guys, you guys have a great product, glad to see the news.

</comment><comment><USER27> literally crushing it

</comment><comment><USER9> Rapportive is one of those things that I didn't have a need for, but now that I have it installed, I wouldn't want to be without.

<comment><USER28> Are you getting lots of e-mails from people you don't know? In general I'm getting e-mails from people I already know (if its business related, I'm not too fussed by their facebook profile) and if they're friends, well - <USER3> contact me on fb anyway and not e-mail.

Any companies I get e-mail from, I already know about so the company profile feature isn't that useful idea.

I'm just wondering what the value prop is if you can enlighten me?

<comment><USER20> The core idea is that there's a lot of useful context about the people you email, but email clients right now don't give you any of that context. We serve it up right next to your email, so you don't have to hunt around for it. We want to help you engage more meaningfully with the people you email.

We've found different people find it useful for different reasons. Founders, journalists and BD people get a lot of mail from people they don't know, and we've had a lot of love from that demographic because we help them put a face to the email. But as you say, a lot of people get mail mostly from people they already know.

We've got some ideas coming up for that which we think are going to be incredibly useful. Right now, if you get an email from a friend, you can see their recent tweets right there in your email - I've found this really useful, as I can refer to their tweets while replying to their email, or see who else they've been talking to. There's a lot more coming along those lines.

Is there anything we could do that would make it indispensable to you?</comment></comment></comment>


Ministers pledge to end 'poor doors' in new build housing - <AUTHOR> https://www.bbc.co.uk/news/uk-49053920 <comment><USER2> So I know in London any new housing development has to allocate part of the properties for shared ownership or social housing, To be able to get planning approval.

Seems very reasonable because property developers make a lot of money selling luxury homes. And also in a lot of places the development is made after demolishing "unsightly" high density social housing.

Often with far fewer homes than there were there originally.

The social housing may be in a separate apartment block in a large development or the less prestigious lower floors.

If you pass by a development it's often obvious what the poor door is.

I've seen one where the social housing residents can't come in through the main entrance but instead goes through the utilities entrance, past the bin storage.

The justification I normally hear about it is that the private home owners will pay high service charges for things like concierge, swimming pools, gyms landscaped gardens, etc etc.

But honestly this seems like an excuse. Like people aren't asking for services for free.

The architecture of their home is telling them that they are inferior every time they enter their homes.

I find it hard to imagine the scenario where a developer is presenting their plan to the council and saying: "this is the door for the poor people"

But then again, I can imagine the arguments of "this is how it is always done"

<comment><USER3> > The architecture of their home is telling them that they are inferior every > time they enter their homes.

Rather is it telling them that the tax payers are paying for their home. The idea being that (for those without a permanent disability) social housing should not be something you stay in forever, rather only until you have the means to find a place to stay on your own. Is it necessarily bad that people are reminded of that?

<comment><USER4> > Is it necessarily bad that people are reminded of that?

Every time they have visitors, every time they go home, every time they get the post, every time their children see a playground they are not allowed to use [0]?

Yes.

[0]: [https://www.theguardian.com/cities/2019/mar/25/too-poor- to-p...](https://www.theguardian.com/cities/2019/mar/25/too-poor-to-play- children-in-social-housing-blocked-from-communal-playground)

<comment><USER5> I live in fancy expensive luxury housing, next to mine there's _even fancier_ , almost twice as expensive housing.

Is it bad that every time I have visitors, every time I go home, every time I get the post I see the wonderful infinity pools which I'm not allowed to use?

I'm not sure if there's a significant difference between my story and the one in the article.

Sure, I feel a tiny bit jealous every time I look out of the window. But that's just a reminder that I should work harder.

<comment><USER4> I think there is a difference, and the difference is that the developers of the property appear to have (at least, from my perspective) gone to some effort to actively separate these social housing tenants.

The law requires that a certain amount of this housing is made available for these people who are vulnerable (be it financially or socially or in health) but it seems that the property developers made an active effort to segregate these people from the rest of the residents based entirely on their financial status or ability to support themselves.

</comment></comment></comment></comment></comment><comment><USER6> There's a real misunderstanding in this thread of what this "social housing" actually is.

This is not council housing.

This is affordable housing. It's owned by a landlord and then privately rented. The people in it tend not to be the unemployed, because they can't afford it. The rental prices tend to be set higher than the Local Housing Allowance, and so Housing Benefit (or the housing element of Universal Credit) would not cover the whole rent.

The people living in this affordable housing tend to be teachers, nurses, police officers, fire-service personnel, public law solicitors and barristers, small business owners -- the people who help make society work. We need those people, and it benefits us if they live in or near the communities in which they work.

There's also a bit of misunderstanding about the other side of these housing developments -- some of them are luxury developments, but many of them aren't. They're just expensive but normal housing, because London happens to be stupidly fucking expensive.

<comment><USER7> The real problem is that these people aren't paid enough to afford "rich" housing which is a problem considering these people make society work.

However forcing "affordable" housing in "rich" areas is the wrong way to deal with it IMO.

Let the market deal with it. These people can switch careers (or not consider such a career in the first place) to something that's better paid. When the talent pool dries up the salaries for these careers will rise naturally because there's gonna be no other way to get police officers, nurses, etc.

<comment><USER6> We've seen that the market cannot deal with it. The UK housing market is broken.

You comments about people switching careers are incoherent.

</comment></comment></comment><comment><USER8> Mixed developments for which a "poor door" makes sense are unnatural because they force different people together. I have enough trouble in a medium apartment building with divergent interests between people owning very similar apartments on different floors (different heating expenses) or on different sides (different trouble with trees and other properties); I cannot imagine a place with different people getting different services.

The solution? Drive moderately rich people out of the city with brutal taxation of luxury homes (according to building types and straightforward measures like residents per total area of the property).

</comment><comment><USER9> Honestly, seems like a bad idea.

If the rich are willing to pay a premium for the top floors of a building forcing them to share the building with the poor will make them just build their own.

Better yet, have incredibly punitive taxation for this so it becomes even more of a social symbol. Then use that money to improve the lives of people living in the lower floors instead of doing meaningless gestures.

</comment><comment><USER10> This might be going in the wrong direction. To address wealth inequality, shouldn't we instead make it obvious? For example, by publishing tax records.

[https://en.wikipedia.org/wiki/Cobra_effect](https://en.wikipedia.org/wiki/Cobra_effect)

<comment><USER11> Why not both?

Poor Doors are arguably a realisation of the cobra effect; It depends if the goals of "affordable housing" are about reducing inequality (like, systemically, not just finding somewhere to live) or not, and if poor doors have an effect on that.

</comment></comment><comment><USER12> Ah, yes. They may want to talk to people responsible for selling parts of Vancouver olympic village for 2-3 mil per apartment in one building while the adjacent one was allocated to the welfare housing. Even in a presence of Chinese money. See how well it went.

<comment><USER13> How well did it go?

</comment></comment><comment><USER14> So, is the tax payer supposed to be paying for social housing, or social status?

</comment><comment><USER15> What does this have to do with hacker or tech news?

<comment><USER16> Your account was created 8 months ago, I suggest reading the guidelines, linked at the bottom. Usually it's much older accounts that ask this question... But it's a question that often gets asked so I don't blame you!

</comment></comment><comment><USER17> 'Poor doors' are just a natural consequence of putting social housing where it doesn't belong.

Making social housing more luxurious isn't solving any real problems.

Tax the luxury developments more and spend that money on social housing in a reasonable location instead.

<comment><USER18> Where does social housing belong then? Where rich people can't see it?

Society benefits greatly from mixed areas - race, class, etc.

<comment><USER17> Social housing belongs where the prices are reasonable, i.e. mostly not in central London luxury developments.

We're talking about the most desirable areas in the world here, trying to force in social housing is just weird.

It's not like there isn't a bunch of places within a very reasonable commute where they could put social housing. Fuck, I used to commute from Reading for a while!

<comment><USER19> And how is somebody earning £60 a day going to pay for a £23/day commute, and housing on top of that, smartypants? Oh, right, taxpayer money.

It's almost like they've modelled this out before making it policy.

<comment><USER17> What exactly is wrong about using taxpayer money to subsidize those commutes?

I think we should certainly impose heavier taxes on the luxury developments and the rich living in them.

<comment><USER20> No taxpayer money is being spent. That's the whole point of this system, to avoid spending on housing benefits by utilizing price caps.

<comment><USER17> Are the developers not taxpayers?

No spending is avoided here, it's merely obscured a little.

Taxing the developers for straight cash would at least give the government more control on how it is spend.

</comment></comment></comment></comment></comment></comment></comment><comment><USER21> What if the "poor people" become a nuisance? Such articles always frame things in a rosy light, as if the evil rich planners are the only reason people are divided.

What if the poor people are alcoholics, drug dealers, petty thieves, antisocial people? Why should other people be forced to deal with them?

<comment> em-bee do only poor people mess up?

why should i have to put up with that rich neighbors son who thinks he can do whatever he wants because his parents keep bailing him out?

this attitude is clearly discrimination and unfair.

<comment><USER21> The poor peole (on welfare) are mor likely to be trouble - they are on welfare for a reason.. Rich people pay good money to be shielded from trouble.

Do you have children? Did you get a cheap flat in the poor part of town? Many families pay higher rents, so that their kids have save streets and playgrounds to play in.

In my middle class neighborhood, I am not aware if a rich kid causing trouble. What trouble do you have in mind? Loud parties? Is the rich kid going to steal my kid's iPhones? I am talking about violence, pollution, syringes on playgrounds, stuff like that.

<comment> em-bee > The poor people (on welfare) are more likely to be trouble - they are on > welfare for a reason..

as someone who's family has lived on welfare too, i find that notion insulting.

the reason for people being on welfare most often has nothing to do with people getting in trouble. loosing a job, divorce, natural disasters, illness in the family. there are many reasons someone could need welfare.

i urge you to really meet these people and understand their situation instead of dismissing them like that.</comment></comment></comment></comment>


This Chart Shows Why Apple is the Greatest Tech Company of This Century - <AUTHOR> http://startupdispatch.com/opinion/this-chart-shows-why-apple-is-the-greatest-tech-company-of-this-century/

<comment><USER2> Given that we still have 88 years to go, isn't it a bit early to discuss who is the greatest of this century?</comment>


Show HN: GitBook WebEditor - <AUTHOR> https://www.gitbook.com/blog/releases/webeditor

<comment><USER2> I've been using this to write a book and really enjoy it so far (though I also enjoyed the desktop version, for what it's worth).

One thing I've noticed (and that I tweeted earlier) is that having my github hooked up to my book, every time I save (via cmd-s, which I do out of habit every minute or so) it automatically pushes to my repo. So without realizing it, I had something like 50 commits after working for 15 minutes. Not a big deal, but it might be nice to separate the two.

The other thing I've noticed is that saving also builds the book. So in this same 15 minute period of working, I went to the status page of my book and noticed I had 50 or so builds currently being processed, with almost all of them failing in a flurry of emails an hour later. (Side note, any reason for this?)

So the only thing I would say is that I'd like to see a separation of the saving, committing, and building, much like the desktop version had. I'd like to build my book (and commit it) at the end of the session, not on every save. But maybe that's just me.

</comment><comment> foreign-inc Am I the only one who thought this was a GitHub product from the LnF of their website?

<comment><USER3> They even send out an "Explore today on Gitbook" email which looks an awful lot like the "Explore today on Github" email.

It's a cool product, but they could use some original direction in their branding and marketing.

<comment><USER4> I'm pretty sure it's intentional, although they obviously wouldn't admit it.

</comment></comment></comment><comment><USER5> Has this company licensed the Git trademark? [http://git- scm.com/trademark](http://git-scm.com/trademark) (2.3)

</comment><comment><USER6> Err maybe I'm misunderstanding this but the pricing says 80% royalty? That means they take 80% of your profits if you sell a book authored in this tool? Is this normal in the book world?

Edit: I think I misread, the writer probably gets 80% royalty.

<comment><USER7> 80%-cost is higher than any other POD service I know, that's potentially quite good.

Through my current publisher I get 70% after cost, so it depends on what their fee is (and it irritates me more that they don't specify what that fee is).

I'm also more than a little uncomfortable with the monthly fee angle, especially with it being mandatory to have your books listed on other services. I'm a firm believer in 'money should flow to the author,' and it makes your book's public availability outside of a niche site with poor discoverability contingent entirely on continually providing said site with more money. Your book now no longer pays for itself, but rather must make at least enough to cover the cost or else it vanishes from the public eye.

It's a small fee, but having been through it and seeing what the long-tail of a book can look like, it's something that can add up and be problematic in the long run, and it also hampers the service's utility for those who wish to offer their book free in digital. You can also probably bet on the likelihood it will get bigger, not smaller, over time.

This is, in small scale, why vanity publishers were awful and why everyone jumped on board with POD in the first place.

<comment><AUTHOR> I'll add more details about what are the fee and that the author is keeping 80%.

</comment></comment></comment><comment><USER8> I've just wrote something for about an hour, then went to make a new outline point via "add article" on the left pane. As a result, the editor refreshed to the new empty outline-point and my previous written text disappeared, which is a big no-go, I guess.

<comment><USER8> One of the co-founders replied within three hours on a Sunday to this inquiry with a notification that a code-change has been pushed such that this issue can't happen again.

I am impressed.

</comment></comment><comment><USER9> This looks interesting. It reminds me of [http://prose.io/](http://prose.io/) which is basically a general-purpose Github-based web editor and that I used some time ago to edit my markdown files.

</comment><comment><USER10> Looks good. Any plans for Bitbucket support?</comment>


More Firefox Bloat? Say It Ain't So, Mozilla - <AUTHOR> http://www.wired.com/software/coolapps/news/2007/05/firefox_bloat <comment><USER2> The article phrases it in a way as if to indicate that Firefox has to choose between being slim and having good (meaning bug-free) plug-ins. Why not have official Firefox-supported add-ons, distinguished from those made by users, which add functionality such as spell-checking and page-caching? These add-ons could even be included in the default installation, with the ability to remove them if desired.

If they did this, the core of Firefox would be slim, with additional functionality provided by plug-ins, just like the original vision, and all the great functionality that has been added lately would not be sacrificed.

</comment><comment><USER3> I found it handy to do a binary chop on my installed extensions. Turned out that only 2 caused any significant memory leaks, so I disabled them until they updated them and fixed the leaks.</comment>


Show HN: Super Simple Static Site Compiler - <AUTHOR> https://github.com/rickr/site_builder

<comment><AUTHOR> Hiya HN,

I spent a few hours tonight to make a SUPER simple static site generator.

It basically concatenates a header, your body, and a footer. After the compilation there's a hook to execute a post hook.

This post hook gets passed the name of the file, which allows you to take actions on a per page basis.

This allows you to change things like nav classes to active etc.

</comment><comment><USER2> This is the first static site generator i have seen that actually deserves to carry the lable "simple".

Good job.

/edit: you might want to include a Markdown parser.</comment>


Google Fiber bait-and-switch: buyer beware - <AUTHOR> http://andstillipersist.com/2014/09/google-fiber-bait-and-switch-buyer-beware/

<comment><USER2> Dude, I'd love to get Google Fiber even if I had to wait for it. Get cable installed, make sure there's no contract and wait. There's a lot of stuff they have to build first and those of us stuck with crappy cable service will have a hard time feeling bad for you.

</comment><comment><USER3> This article sums up the very definition of "First World Problems"</comment>


Jam - The JavaScript package manager - <AUTHOR> http://groundcomputing.co.uk/code/jam

<comment><USER2> I've been building projects with Node.js (+ using npm), and here are my two cents:

\- One of the best changes to npm was to install modules locally instead of globally by default. I wish jam also did that, global installs are troublesome.

\- Isn't package.json going to conflict with my npm's package.json? I definitely don't want to polute my back-end's packages with front-end only stuff.

Basically, I'd like more sensible defaults. Change the package.json to some other filename and install packages locally by default. I'm also unsure why it would put files inside /jam, I don't want to change my configuration to use some obscur and badly named "jam" folder. I believe people like the vendor/javascript or public/javascripts conventions, that would be a better default in my opinion.

Otherwise, I like the initiative. In the past we've been using Jim (<https://github.com/quirkey/jim>) which works more like bundler. Now we just do it manually for lack of a better solution. I don't think this is it yet, but it's close.

<comment><USER3> Why does your server code and your UI code share the same base directory?

<comment><USER4> What exactly is wrong with that? In Rails, you have app/controllers and app/views. UI code and server code sharing the same base directory.

<comment><USER5> Does seem potentially problematic if you don't want to be exposing server- logic. Especially when the developer may not know what software will be ultimately serving the static files over http.

<comment><USER4> As a newbie webdev, what would you suggest? Making an API-based web application? (the backend would serve and receive JSON and the front-end would simply play around with it) Or am I completely misunderstanding this.

<comment><USER3> That's a reasonable way to build an application, and one that I personally prefer, but there isn't a silver bullet. But if you're building a Node app your server js and your client js don't need to commingle. Make a src/js/client directory, and stick your package.json in there (if you want to use this tool).

</comment></comment></comment></comment></comment></comment><comment><USER6> Another project to check out: <https://github.com/volojs/volo>

Volo is arguably more established and written by the same guy who wrote require.js. It's still early in development, but is already very good.

EDIT: From a quick glance, it seems like volo works better with existing javascript libraries too. Volo's add/install command simply uses github's search API to find any existing project and can install it (it also supports other URL formats).

</comment><comment><USER7> Kudos to the author but I don't like yet another package system (this one specifically hosted on packages.jamjs.org). What happens when that goes down, or the author calls it quits, or spammers take it over etc? IMO it's much better to adopt the existing npm ecosystem as Ender has done.

</comment><comment><USER8> Unfortunately choice of name, perhaps, as it clashes with this: <http://www.perforce.com/documentation/jam>

<comment><USER9> yeah, the jam build system is a great tool and has been around a long time. Seems like people should do some more research before naming their projects.

<comment><AUTHOR> You can call it JamJS if it makes you feel better ;) Obviously, I'd have preferred to avoid a naming conflict had I known about this project but it's getting pretty difficult to find unique names for JS projects these days. There's already a 'jam' in NPM!

</comment></comment></comment><comment><USER10> Every two years or so someone comes up with a new javascript package manager. Obviously something about the javascript ecosystem makes this hard because someone constantly attempts to solve the problem over and over again. What' wrong with the previous 3 or 4 attempts. I'm honestly curious. I don't live in the world of javascript very much anymore so I don't know what all the issues are.

<comment><USER11> Perhaps package managers are just fun to write? :-)

</comment></comment><comment><USER12> This looks very promising!

JavaScript badly needs a package manager and the community badly needs to agree on a module format (AMD). Do you think that will happen? How can we make that happen?

Every other language has package managers.. ruby has gems and bundler, node has npm, java has maven. That makes it so easy to develop any application or library that can be used by any other application or library. ClientSide JavaScript finally has an amazing way to manage dependencies and build modular applications (AMD). All it needs now is a good package manager for being able to split large applications into individual components and manage external library dependencies. I want to be able to configure jQuery, Backbone, jQuery plugins, Backbone plugins, any other JS library, my own subcomponents as dependencies, use them via AMD's require and optimize in a smart way when building (e.g. only build in a single copy of the given library and warn if multiple versions are required).

I'm really hoping Volo (<https://github.com/volojs/volo>) or Jam (or both if they become interoperable) will become _the_ solution for this stuff.

</comment><comment><USER13> I'm not sure I get it. This seems to run in your development environment, in order to manage your dependencies and output your JavaScript code to a minified version ready for deployment, but it's in JavaScript. What kind of development environment executes JavaScript? What kind of workflow is this geared towards? Is that a nice way to develop?

<comment><USER14> It runs on node, which executes javascript in a development environment.

<comment><USER13> Ah, I guess I had never really connected the dots, that to use node, you have to do your development in the browser. This is all starting to come together. ; )

[Or not, apparently.]

<comment><USER14> Well, not exactly. From wikipedia: "Node.js consists of Google's V8 JavaScript engine, libUV, and several built-in libraries." So node installs a version of the V8 engine in your dev environment, so it can run js on the server without a browser.

That's why jam requires node - it's written on javascript and uses node packages so it can work with the filesystem, and do other server/dev environment things that js in a browser can't do.

When you develop with node, it's just like doing development for any other language.

Edit: in case this wasn't clear - you aren't working in a node environment. In this case, node is simply used because it can execute javascript in a server environment. So jam can be written in js and do filesystem stuff if it runs using the node utility. Jam simply runs on node, but a project using jam doesn't necessarily need to be a node application.

<comment><USER13> Thanks for that explanation. One last question: why would you do 'jam install backbone'? Are people developing all their client-side code in the node environment as well?

[Or, it seems like maybe node in this case is just a dependency for jam, which is simply a tool used to make all your JavaScript development easier, without introducing any extra runtime dependencies.]

<comment><USER15> There are two sides to Node. One is the HTTP server environment you're thinking of. The other is just as a generic JavaScript execution environment, with packages for filesystem etc..

</comment></comment></comment></comment></comment></comment><comment><USER16> How is this different from ender.js (which has been around for a while)?

A short summary of pros / cons of each would help.

</comment><comment><USER17> Libraries like this are useful.

Ender sounds similar and has been out for quite some time: <http://ender.no.de/>

</comment><comment><USER18> is there a point to using this outside of node.js and npm? I'm not a big js person btw.

<comment><AUTHOR> Yes, definitely. You shouldn't need to know anything about Node.js beyond installing it. Jam is for managing _front-end_ code and would be useful in any application with a lot of client-side JavaScript.</comment></comment>


Salon asks ad-blocking users to opt into cryptocurrency mining instead - <AUTHOR> https://www.theverge.com/2018/2/13/17008158/salon-suppress-ads-cryptocurrency-mining-coinhive-monero-beta-testing <comment><USER2> This actually seems pretty smart. But even if you get everything right with the user experience, you're probably looking at a fraction of a cent in the time spent reading even a long article.

I've been mining a little Monero for fun lately, and even with decent hardware and the GPU fully engaged, I'm looking at a couple dollars a week from overnight mining.

<comment><USER3> Is it still profitable when you account for electricity cost?

<comment><USER4> Even if it is, eventually it won't be as profitable. Inefficient markets will quickly become efficient as equilibrium is reached. There is no free lunch.

</comment></comment></comment><comment><USER5> We should redouble our ad-blocking efforts.

</comment><comment><USER6> Remember, your contract with the network is you're going to watch the spots. If you don't want to watch the spots, technically by getting the content you're committing theft of service. And "the network" will find ways to detect thieves and punish them and/or collect their pound of flesh.

</comment><comment><USER7> duplicate: [https://news.ycombinator.com/item?id=16364919](https://news.ycombinator.com/item?id=16364919)</comment>


iOS Design Weekly - <AUTHOR> http://iosdesign.co

<comment><AUTHOR> Inspired by Dave Verwer doing iOS Dev Weekly (<http://iosdevweekly.com>) I created iOS Design Weekly. Most blogs about creating iOS applications are focused on development and not so much on design. Hopefully we can help show some of the great articles on designing better iOS apps.

If you have any articles you think I should include please contact me on Twitter: <http://twitter.com/nathanbarry></comment>


Sick systems: How to keep someone with you forever (2010) - <AUTHOR> http://issendai.livejournal.com/572510.html <comment><USER2> Could also be called the ideology of a sociopath.</comment>


Blazegraph 2.0 – GPU-accelerated distributed graph database - <AUTHOR> https://www.blazegraph.com/product/gpu-accelerated/ <comment><USER2> We've used Blazegraph at work for a project, and we have used a number of other graph databases.

Blazegraph is a very niche product and requires a lot of time for setting it up and adjusting it for your workload.

If Blazegraph peaks your interest then you should also look into Yarc platform by Cray.

Should you want to look more into graphs, but don't want to spend endless nights just trying to load your data then I would recommend Stardog, which has just been a pleasure to work with.

<comment><USER3> The promises they make are really tempting. However, when I experimented with Blazegraph last year, it ended up stuck more often than I could count, whether during bulk import or in answering some ad-hoc SPARQL queries.

I can imagine that with extensive tuning, BlazeGraph provides a good database. Just don't expect it to have the polish and convenience of a modern RDBMS or a shiny NoSQL store :)

<comment><AUTHOR> WikiData selected BlazeGraph to back its new Query Service:

[https://lists.wikimedia.org/pipermail/wikidata- tech/2015-Mar...](https://lists.wikimedia.org/pipermail/wikidata- tech/2015-March/000740.html)

Here's a spreadsheet showing WikiData's evaluation of each candidate graph database:

[https://docs.google.com/spreadsheets/d/1MXikljoSUVP77w7JKf9E...](https://docs.google.com/spreadsheets/d/1MXikljoSUVP77w7JKf9EXN40OB- ZkMqT8Y5b2NYVKbU/edit#gid=0)

<comment><USER4> Their selection "process"[1] was.. not what I'd choose to use, especially since they changed the priorities as they evaluated. But then they abandoned that process[2] so I wouldn't read too much into the evaluation.

Basically as far as I can see, the main reason BlazeGraph was chosen was this: _[they] had me out to their office (a house an hour and half from mine)_ [3]

I'm sure BlazeGraph is fine. We were doing a very similar evaluation at the same time, and the Titan situation screwed us over too. But we took a look at BlazeGraph after Wikidata chose it, and found it pretty rudimentary at that time.

[1] [https://phabricator.wikimedia.org/T90101](https://phabricator.wikimedia.org/T90101)

[2] "As you can also see we didn't finish filling them all out. But we've still pretty much settled on BlazeGraph anyway. Let me first explain what BlazeGraph is and then defend our decision to stop spreadsheet work" [https://lists.wikimedia.org/pipermail/wikidata- tech/2015-Mar...](https://lists.wikimedia.org/pipermail/wikidata- tech/2015-March/000740.html)

[3] [https://lists.wikimedia.org/pipermail/wikidata- tech/2015-Mar...](https://lists.wikimedia.org/pipermail/wikidata- tech/2015-March/000740.html)

</comment></comment></comment></comment><comment><AUTHOR> MapGraph -- the GPU-accelerated graph engine -- has been rolled into Blazegraph 2.0, and it looks like this means OLTP and OLAP can be combined into a blazing-fast, single OLXP system.

From: [https://www.blazegraph.com/product/gpu- accelerated/](https://www.blazegraph.com/product/gpu-accelerated/)

      The original work was funded by DARPA and presented at the
      2014 SIGMOD conference in a paper entitled, MapGraph: A
      High Level API for Graphs [1]. This work is available in
      open source. Later work, in collaboration with the
      University of Utah SCI Institute [2] and funded by DARPA,
      applied multi-core techniques running on over 750 M cores
      on the Titan Supercomputer to extend this to Multi-GPU
      traversal with Breadth First Search (BFS).  On a cluster of
      64 NVIDIA K40 GPUs, it demonstrated a throughput of 32
      Billion Traversed Edges Per Second (32 GTEPS), traversing a
      scale-free graph of 4.3 billion directed edges in 0.15
      seconds, which was featured in a presentation IEEE Bigdata
      Conference.

[1] [https://www.blazegraph.com/whitepapers/MapGraph- SIGMOD-2014....](https://www.blazegraph.com/whitepapers/MapGraph- SIGMOD-2014.pdf)

[2] [http://www.sci.utah.edu/publications/Fu2014a/UUSCI-2014-002....](http://www.sci.utah.edu/publications/Fu2014a/UUSCI-2014-002.pdf)

VIDEO: Blazegraph GPU and DASL at Super Computing 2015 ([https://vimeo.com/148519808](https://vimeo.com/148519808))

It's GPLv2 and now has support for SPARQL and TinkePop3/Gremlin.

See [https://groups.google.com/d/topic/gremlin-users/8fS_ak- tWNs/...](https://groups.google.com/d/topic/gremlin-users/8fS_ak- tWNs/discussion)

</comment><comment><USER5>
        ERR_CONNECTION_TIMED_OUT

<comment><USER6> Meanwhile, check out:

      https://github.com/blazegraph/database

Also down:

       https://blog.blazegraph.com/?p=977
       https://wiki.blazegraph.com/

</comment></comment><comment><USER7> "Your connection is not secure

The owner of www.blazegraph.com has configured their website improperly. To protect your information from being stolen, Firefox has not connected to this website."

<comment><USER8> We've seen cases where the newer Thawte root certs were not included in the trust chain. That might be the issue.

</comment></comment><comment><USER8> Got any questions on Blazegraph GPU? Just let us know...

<comment><USER9> What's the performance like on data sets that are too big for any single GPU's memory?

Can the GPU be used to accelerate shortest-path queries (e.g. dijkstra's algorithm) and if so, where can I read more about how that's achieved?

<comment><USER8> The graph does need to fit into GPU ram. We use graph partitioning for multi- node, Multi-GPU configurations.

Dijkstra's algorithm which, as mentioned by Davidson et al. [1], is a "sequential algorithm [that] is poorly suited for parallel architectures like GPUs that require large numbers of parallel threads for efficient execution."

Instead, we have variants of the algebraic formulation of the Bellman-Ford algorithm as given in Kepner and Gilbert's book [2].

[1] Andrew A. Davidson, Sean Baxter, Michael Garland, and John D. Owens: "Work-Efficient Parallel GPU Methods for Single-Source Shortest Paths." In Proceedings of the IEEE 28th International Parallel and Distributed Processing Symposium (IPDPS), 2014. [http://dx.doi.org/10.1109/IPDPS.2014.45](http://dx.doi.org/10.1109/IPDPS.2014.45)

[2] Kepner and Gilbert: "Graph Algorithms in the Language of Linear Algebra."

</comment></comment></comment><comment><USER10> > provides a Scala-based language to write graph and big data analytics and is > complementary to the Spark and Hadoop ecosystems

It'd be awesome to have Blazegraph as a backend for Spark's Pregel queries.

With Tensorflow bindings in place, and the BIDMach/BIDMat libraries, it is very nice seeing Spark getting some serious GPU attention.

<comment><USER8> We definitely see Spark + Scala + Blazegraph DASL to be a sweet spot for combining the ease of Spark with the GPU performance. Have a submission in to the Hadoop Summit in June on it: [https://hadoopsummit.uservoice.com/forums/344955-data- scienc...](https://hadoopsummit.uservoice.com/forums/344955-data-science- analytics-and-spark/suggestions/11664123-the-power-of-spark-the-speed-of-gpus- for-graph-an)

<comment><USER11> You guys should submit a talk to Spark Summit. Look forward to it.</comment></comment></comment>


D-Wave CEO: Our Next Quantum Processor Will Make Computer Science History - <AUTHOR> http://recode.net/2014/09/25/d-wave-ceo-our-next-quantum-processor-will-make-computer-science-history-video/ <comment><USER2> > "Our [widget] Will Make [industry category] History"

I'm pretty sure every corporation, every scientist, every inventor thinks this about their big project. While some actually disrupt, most intend to disrupt, and fail.

Disruption is not only about invention, it's also about marketing... One thing is for sure no one has cornered the market on disruption.

I don't know if I'm being that clear, in relating industry disruption and making history... An example is IBM's Watson made Computer Science history but they marketed the shit out of it, too, and by hosting a competition on jeopardy between Watson and the greatest Jeopardy players of all time, media attention was scored, public paid attention, and only then was 'history' made. Watson, the invention was not the history creator, the event involving Watson was the "history".

Another example is Nikoli Tesla, while he technically made history many times over, isn't very strongly remembered in the public conscious, Edison on the other hand wasn't as much an inventor as he was an excellent marketer. He didn't even invent the light bulb, but he sure did make it famous and by doing so "made history".</comment>


US Health Dept. Files Patent for Improved Ebola Treatment - <AUTHOR> https://greybmusings.wordpress.com/2014/12/11/us-health-dept-files-patent-for-improved-ebola-treatment/

<comment><AUTHOR> These are the CDR chains -

CDR1 comprising GFAFNYYDM/I/LH (SEQ ID NO:1) CDR2 comprising YINPGGGNTYYADSV (SEQ ID NO:2); CDR3 comprising QLYGNSFMDY (SEQ ID NO:3),

or

a heavy chain CDR3 comprising QLYGNSFFDY (SEQ ID NO:4), a heavy chain CDR1 comprising SEQ ID NO:1 or GFAFNYYDMF (SEQ ID NO:17), and a heavy chain CDR2 comprising SEQ ID NO:2, and

a light chain sequence, comprising a light chain CDR1, CDR2 and CDR3, wherein the light chain CDR3 comprises HYSTPLT (SEQ ID NO:5).</comment>


Scalable distributed b-tree - <AUTHOR> http://highscalability.com/paper-practical-scalable-distributed-b-tree new distributed systems primitive? <comment><USER2> This looks like it'd be promising for having SQL databases over a cluster of machines; how far along is this research?

<comment><USER3> How does this provide any more promise than similar distributed data structures? A distributed b-tree will not get around the CAP paradox and the necessity of client code to understand that it is running across a distributed service. Other than an efficiency gain how will this be any different than wrapping changes to the various internal nodes of a standard b-tree in a lot of paxos calls and sharing it across a cluster...

<comment><USER4> can you shoot up a link to the CAP paradox?

<comment><USER5> <http://devblog.streamy.com/2009/08/24/cap-theorem/>

[http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.20.1...](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.20.1495)

CAP is absolutely _not_ a paradox; it is just a conjecture or a theorem.

As far as CAP and distributed databases, it is worth noting that a lot of distributed DBs don't actually need to be partition-tolerant.

<comment><USER3> Not sure why I put paradox instead of conjecture there, but a quick answer to the "what is it" query that does not involve reading papers is that for a distributed system at any point in time you can only have two of consistency, availability, or partition-tolerance. "At any point in time" is a key item there, as can change the two characteristics chosen as the DB goes through its various states or at different layers in the transaction.

OTOH, saying partition-tolerance is unimportant is only partially-true if your db is only going to run on a single lan. This is often the case (i.e. basically saying that "distributed" is another word for "cluster") but if your db is going to run across a wan then sacrificing partition-tolerance can have unpleasant consequences. I do a lot of work in Erlang and its built-in distributed db (like the language itself) was designed to run in a phone switch among SBCs and blades that shared a common backplane. In this case a partition meant that something extremely bad was happening and graceful shutdown/failure in the face of a partition was a good choice. Now that a lot of us are using these tools out in the wilds of the internet we are discovering that sacrificing partition-tolerance has some unpleasant side- effects too: running mnesia across EC2 instances, for example, is not a trivial undertaking because you need to deal with inevitable network problems by shutting down auto-rejoin features of the db and effectively turn off availability for a short period during the rejoin.

</comment></comment></comment></comment></comment><comment><USER6> Tokyo Cabinet and Tyrant have B-tree support.

<comment><USER7> As far as I could tell, Tokyo uses B-trees within a single instance, not across instances as this article is talking about. Has this changed?

<comment><USER8> No.</comment></comment></comment>


Show HN (take 2): A basecamp alternative (private beta) - <AUTHOR>

NOTE: This is a private beta, to create an account use invitation code "hn2" - there is only 100 invitations, so hurry up :)
Hey everyone, thanks for all your feedback on http://www.ubercamp.net/ my basecamp alternative.
Tech stack = angularjs + twitter bootstrap + ruby + postgres + heroku
I have made lots of changes and would love you to take a look and give me some feedback.
Note: click on project title to go to project view (need a better ux/visual indicator)
Some of the changes are:
- improved ux - better "dashboard" (still lots of work to do here) - improved milestones view - better in-app support (click question mark in bottom right and say hello) - if creating new account, a demo project is created to give you an idea of what is going on. - several bugs were killed in this release.
Thanks. <comment><USER2> Tried signing up with the code, keep getting a 500 something went wrong error :/

<comment><AUTHOR> Thanks. It's fixed now.

PS: Never deploy a new version 1 hour before you go on vacation :)

</comment></comment><comment><USER3> I like the astronaut in the bottom right corner. Very cool.

</comment><comment><USER4> Looks cool, looking forward to trying it out. Good luck!</comment>


No one wants to review your code! Let's fix that. - <AUTHOR> http://www.codevolley.com/

<comment><USER2> "We have run out of code to review. Come back later!"

<comment><AUTHOR> A matter of critical mass ;)</comment></comment>


What Hokusai's Great Wave tells us about museums, copyright, online collections - <AUTHOR> https://medium.com/open-glam/the-great-wave-what-hokusais-masterpiece-tells-us-about-museums-copyright-and-online-da0f25bd4ed2 <comment><USER2> The copies of The Great Wave in collections around the world are not only different in color and tone. There are many reproductions of varying quality among them.

David Bull, a woodblock print maker in Tokyo, made his own reproduction in 2015 and explains some of the differences in his video series about the project.

This one's a summary, starting at 04:00: [https://youtu.be/BizndSI4Nfc](https://youtu.be/BizndSI4Nfc)

<comment><USER3> Geeee, thanks for sending me down this rabbit hole :D

</comment></comment><comment><USER4> The fact is, you can purchase a $100M Picasso but you'll have no right to put an image of it on your website unless you pay extra to the Picasso Estate. Alongside the reproduction rights discussed in the article, these two elements are so poorly understood that they seem universally exploited by rights holders, copyright agencies, artist's descendants and so on, all to the detriment of the public.

Globally, there's a ~70 year gap [1] in the comprehensive digital representation of modern art, where work from artists who died <70 years ago is subject to such fees or legal obstacles that it often can't be shown online, even by the current owners or gatekeepers [2].

On related topics, museums continue to think some or all of the following:

      1. Digital collections will prevent physical visitors from coming through the door.

      2. Making digital collections freely available will lead to a plethora of badly made t-shirts, bags and prints that the museum gains no revenue from.

      3. Digital collections offer a revenue stream that will save the museum from funding cuts.

      4. Museums should divert resources to build their own collection systems and web platforms.

      5. Our museum is so uniquely special that digital solutions used by others can be readily dismissed as irrelevant.

Rights and digital issues combined mean the representation of art in the digital age remains a complete mess, and as we see in other media a void is filled by legally-questionable and often short-lived solutions.

[1] [https://www.wikiart.org/en/pablo- picasso](https://www.wikiart.org/en/pablo-picasso) (scroll to bottom) [2] [https://webapps.fitzmuseum.cam.ac.uk/explorer/index.php?qu=&...](https://webapps.fitzmuseum.cam.ac.uk/explorer/index.php?qu=&Oop=OR&Fma=picasso&Fpp=&Fmt=&Fob=&Fds=&Fdst=AD&Fde=&Fdet=AD&Oaa=true&Oat=true&do=Search&size=10&dti=text)

<comment><USER5> Sure. But unlike Picasso, Hokusai has been dead for ages. How can anyone legitimately claim any copyright on his work today?

<comment><USER4> From reproduction rights, see main article.

<comment><USER5> I read it, but I must have missed that. Thank you!

</comment></comment></comment></comment><comment><USER6> Museums and archives like to claim (and perhaps hope) otherwise, but there's no copyright to claim in a scan of a 19th-century ukiyo-e print.

In the US anyway, copyright requires a modicum of originality and creativity not present in a simple faithful digitization. Though the caselaw is perhaps murkier than might be ideal.

<comment><USER7> The fact that this work is clearly out of copyright does not imply any obligation on part of these museums to provide anyone with copies. They may not be able to impose terms on the basis of copyright but they might be on the basis of providing a service.

<comment><USER8> Some museums do claim to impose copyright terms on simple digitalisations of public domain works. It is a ludicrous claim they mostly get away with: it completely goes against the idea of public domain, and claims creative protection on a work where there is as little creativity as possible involved.

</comment></comment></comment><comment><USER9> The Library of Congress print is even larger than reported in this article, a 146.1 MB TIFF [https://www.loc.gov/pictures/item/2008661021/](https://www.loc.gov/pictures/item/2008661021/)

<comment><USER10> I find it interesting that the only download options are a couple of jpegs that are less than 200Kb, or a TIFF that's 146 Mb.

<comment> DC-3 From the huge TIFF I exported and uploaded a 11.3MB JPEG here:

[https://github.com/DC-3/wave/blob/master/wave.jpg](https://github.com/DC-3/wave/blob/master/wave.jpg)

<comment><USER10> Thank you. I think I would have gotten around to re-scaling the image down to a more reasonable size eventually. This saves me that effort.

</comment></comment></comment></comment><comment><USER11> This makes me wonder: Are there any original woodblocks for the "great" ukiyo-e artists that survived?

<comment><USER2> Hard to tell. Since the prints were so cheap back then the individual blocks were used until they were worn down if they were popular, and then probably either cut down to be reused for a different carving or just tossed out.

Identifying an original would be near impossible too, since popular ones would be copied or recarved. Does a second edition from the same shop count as an original?

</comment></comment><comment><USER12> I hate that everything beautiful in this world is monetized. It's the root of all evil in my opinion. Greed and profits to feed ego and personal status. I think even in a Star Trek post scarcity fantasy the "Ferengi" will still be out there trying to squeeze personal wealth out of everyone. So much of life is behind a paywall and only available to a few. I dream of a world where we create and do things for enjoyment and the enjoyment of others. Where everything doesn't have to be priced or guarded to prevent others from pricing it. Naive I know...

</comment><comment><USER13> Does anyone find it mildly ironic that the platform used to _publish_ this was Medium?

</comment><comment><USER14> This article seems rather perfunctory tbh. Funding is quite a big deal for museums, and the assumption put forward that open access is inherently better than anything else the museum can do with the money it raises by licenses is completely unsupported.

<comment><USER15> Fubding may be a big deal for museums in the abstract, but _how_ much funding do these museums raise from selling "licenses" to reproduction of these public domain works? It can't be _that_ big of a deal, and museums can still request payment for _endorsed_ reproductions (basically a quasi-sponsorship arrangement targeted at _high-profile_ reusers of these works, driven by signaling goals - "we pay for the museum's endorsement, this shows we're serious!" and broadly-aligned incentives - "the more reproductions the better, for both the museum and the reuser") without any need for quasi-copyright claims.</comment></comment>


Marc Benioff Says 'Facebook Is the New Cigarettes' - <AUTHOR> https://www.inc.com/jason-aten/salesforce-founder-marc-benioff-says-facebook-is-new-cigarettes-heres-why-he-thinks-government-should-get-involved.html <comment><USER2> only person I know who abuses facebook is my mom</comment>


Should we be applying to YC - <AUTHOR>

Our creation is at: www.snailmailpics.com<p>In the past couple of months we have created two different applications of printing pictures on your cell phone, computer and drop box. We are trying to get some feedback on the product market fit. Would love the community's feedback.
Use the coupon code BoilerUp to get free printing from our DropBox app. <comment><USER2> This seems very similar to a YC S11 company - picplum.com - which shut down at the start of 2013.

What challenges did they face, and why are you better?

</comment><comment><AUTHOR> hi tomblomfield, thank you for the company link. We will look into them. Not sure if we are better, but definitely taking a different approach to printing pictures. We arent requesting users to upload pictures to a application or even downloading an application. We are integrating to where their pictures are already stored , ie. Dropbox and using SMS as an application to print pictures!. Do try us out..!</comment>


Show HN: RussianDict – English to Russian Translation for the Modern Web - <AUTHOR> https://www.russiandict.net <comment><USER2> It can NOT translate _" hello, dude!"_[0]

> We couldn't find what you were looking for in our dictionary.

> A few suggestions:

> \- Check your spelling.

> \- Try searching for similar words.

[0] [https://www.russiandict.net/transations?utf8=%E2%9C%93&searc...](https://www.russiandict.net/transations?utf8=%E2%9C%93&search=hello%2C+dude%21&commit=Translate)

<comment><AUTHOR> I fixed it. Now, you can search for "hello, dude!"</comment></comment>


The water is so hot in Alaska it's killing large numbers of salmon - <AUTHOR> https://edition.cnn.com/2019/08/16/us/alaska-salmon-hot-water-trnd/index.html <comment><USER2> Human being destroying the nature via Industrialization, one day the nature will revert Back!</comment>


Show HN: The New Kobra.io – "Google Docs" for Code, No Reg Required - <AUTHOR> https://kobra.io

<comment><USER2> I'm not sure if I'd want to work on the same code at the exact same time as someone else. I like pair programming but only with at most 1 writer.

</comment><comment><USER3> This is an excellent product built by a talented marketer, I wish it all the luck in the world. Great for a quick pairing session.</comment>


Ask HN: Running your own mail/calendar/newsreader - <AUTHOR>

I like the idea of having a synchronized mail server (a-la IMAP), calendar (that syncs with multiple devices), and newsreader (that keeps track of what I read on multiple machines). I like to be able to access them from multiple client machines and/or through the web. And google has it all for free! Well, except for Dropbox, which google doesn't have (but Dropbox does).
But I like to own my own data, usage habits, mail contents, etc. I'm willing to pay some $10-$20 a month to get that if I can't get it in free software.
But I can't find a reasonable solution, paid or free, that I can run on my own server.
I've tried, at some point or the other, the packages below, and I wasn't satisfied. Newer versions might be better - please list your solutions below so we can all enjoy them.
Services sought:
mail; calendar; newsreader; document store; bookmarks; mobile device synchronization of mail/calendar; dropbox-like
- Citadel/UX: only mail+calendar, no working device sync.
- Zarafa: clunky and slow, no newsreader or bookmarks
- SparkleShare: dropbox only, no windows support.
- newsblur: feed reader only, no mobile device sync
If you're running your own system outside of google/dropbox, what are you using? <comment><USER2> 1) <http://en.wikipedia.org/wiki/Comparison_of_mail_servers> 2) <http://www.gregwestin.com/webdav_for_ical.php> For Macs, I think this is what I did a few years ago 3) I know your fear, I use Instapaper.

</comment><comment><USER3> I admined Postfix with a bunch of web GUIs, some of them fairly "Outlook- like". I made my living doing that and I'm still happy to consult on email & deliverability.

Having said that, my professional advice to you is DON'T.

If someone else wants to admin your email for a fee or for free, LET THEM.

</comment><comment><USER4> Bit of a high price point but I've had very good experience with Kerio Connect. It's similar to Microsoft Exchange but has much better compatibility (Can be run on Linux, UNIX or Windows) and supports many more protocols and features

</comment><comment><USER5> mail: postfix (SMTP) + dovecot (IMAPS) + roundcube (webmail) + z-push (push email)

calendar: DAViCal</comment>


Show HN: Revenue Forecasting for SaaS Apps - <AUTHOR> https://saascast.io <comment><USER2> Bit of pricing advice: businesses making a lot of money don't mind paying for software. I think the $10 flat fee will come back and bite you in the future. SaaS metrics was so important for us and we were dropping $150 a month on ChartMogul when our MRR was only $3000.

<comment><USER3> Interesting. Did it help figuring it out your pricing?

<comment><USER2> I wouldn't say we used Chartmogul as a basis for our pricing. I do think a forecasting feature in Chartmogul would have helped us realise much quicker that our current pricing had us on an unsustainable path.

We got Chartmogul to be able to gauge the health of our startup, and for that it's been worth every penny. I can't sing their praises enough.

<comment><USER3> Interesting. Why it was an unsustainable path?

I am trying to figure out pricing for my own startup and it's no easy task.

</comment></comment></comment></comment><comment><USER4> a bigger player is [https://www.zodiacmetrics.com/](https://www.zodiacmetrics.com/), who have many peer-reviewed papers and are thus transparent with their (general) forecasting approach. i would love to see some model details given the maturity of other teams.

(disclosure: i took a class with the prof whose research drives zodiac)

<comment><USER5> Zodiac is definitely worth a look. They did a pretty good job of predicting our long-term user's future revenues. Also all of the academic papers around it are good reads.

That said, that didn't solve our problem because what we really wanted was predictions of our new customers. So we could quickly see whether or not their quality was changing. I just wrote something up about where we landed. [https://engineering.ezcater.com/early-prediction-of- customer...](https://engineering.ezcater.com/early-prediction-of-customer- lifetime-value)

</comment></comment><comment><USER6> One feature that I think will help smaller SaaS businesses: Revenue Calendar.

For example, on Jan 1, $500 will be charged, Jan 2, $250 will be charged, etc..

Sure, it's great that in a given month you will process X in MRR, but knowing when that ends up being charged is always a plus for SMBs.

</comment><comment><USER7> Does this do anything more than just chart some KPIs and extend a trendline from your revenue?

</comment><comment><USER8> Out of curiosity: Did you try to validate your hypothesis that people will find this useful?

No need to share juicy detials, I'm genuinely wondering. I'm very much thinking about "on top feature" small SaaS projects and Stripe is a prime candidate, but I wonder if I would use this.

General Stripe reporting solutions are there. I'm having a hard time seeing how nominal revenue predictions are actionable. In case things go up I should be able to gauge by actionable metrics (churn, sign up rate, conversion rates), and wouldn't care to end up with $ estimates (will be slightly off either way).

Would love to chat about the idea/ topic in general in case you're interested.

<comment><USER9> I believe there are already many solutions for this, [https://baremetrics.com/](https://baremetrics.com/) offers a paid service for reporting churn, sign up rates, conversion rates etc, [https://clearbanc.com/](https://clearbanc.com/) offers a free solution for the same metrics. I believe they both support stripe data as well.

<comment><USER8> Yup, I kno of baremetrics and the like; hence I would use them to check on general, actionable SaaS metrics and am wondering about the perceived value of Saascast for customers of those tools that already have a feeling where their business might be going in the next one to three months.

With changes in the product I'm not sure whether longer time horizons make sense.

</comment></comment></comment><comment><USER10> I believe ChartMogul are adding revenue forecasting to their product shortly.

<comment><USER2> Really?? That would be great! What's your source?

<comment><USER11> Yep, forecasting in ChartMogul is coming.

Source: I looked at our backlog :)

</comment></comment></comment><comment><USER12> Great name, rolls of the tongue.

</comment><comment><USER13> What problem does this solve?

</comment><comment><USER14> You should consider using a production build of react for your front-end!

</comment><comment><USER15> Staring at a blank screen, wondering why.

2.3MB JS on your home page...</comment>


Ask HN: How do I get started with robotics? - <AUTHOR>

Watching this Threes video has rekindled a desire to learn how to build computer-controlled robotics, but I know absolutely nothing about it (nor anything about circuits -- I didnt pay enough attention in High School Physics).<p>Where is a good place to start? <comment><USER2> In highschool we had a class where the teacher would provide LEGO Mindstorms for us and let us do anything we wanted with it. Using optic sensors, we made it find its way out of a maze which we set out on the floor with tape. These things are great fun, but probably most suitable for kids to play with.

Probably a better idea would be to get started with something like an Arduino. Not too long ago I read about Tessel on HN which seemed interesting, sort of an Arduino but you could program it with JavaScript, if that's your thing.

Also there's tons of articles and videos on the web of fun projects people built with Arduino's and Mindstorms, so I would definitely google around a bit to see if you find something you think is interesting. Anyway, good luck.

<comment><USER3> To add to what jeremyaarts said, last year Lego released an upgraded version of their Mindstorms set called EV3. I know there was a board that allowed you to enhance the older NXT Mindstorms set with an Arduino, though I'm not sure if this board is compatible with the EV3. One of the nice things about the EV3 is that it runs it's own version of linux, however you can also can run a custom version of linux on the EV3 if you want, opening the system up to a world of fun :-)

</comment></comment><comment><USER4> Arduino robot hacking is a great place to start.

An alternative or complementary approach is to work in simulation, and with that decision you can immediately start working with state of the art stuff, as follows:

ROS is the dominant Open Source robot platform. It works with the powerful Gazebo simulator. It's all free. Researchers, companies, universities, and hobbyists all use it, and there's an active community. The learning curve can be steep, but this is powerful stuff.

[http://www.ros.org/](http://www.ros.org/)

For a simpler start, take a look at the platform that was popular before ROS overtook it: Player/Stage

[http://playerstage.org](http://playerstage.org)

</comment><comment><USER5> reddit.com/r/arduino

robots.net

[http://www.societyofrobots.com/robotforum/](http://www.societyofrobots.com/robotforum/)

shoot me an email if you want to talk robots.

</comment><comment><USER6> check out instructables.com. search on arduino it's a micro controller. I've been thinking about popcicle sticks and mini servos. both are really cheap.</comment>