What's left of NoSQL? - <AUTHOR> http://use-the-index-luke.com/blog/2013-04/whats-left-of-nosql/ <comment><USER3> As usual, the next-big-thing-to-replace-the-old-crappy-thing has become nothing more than a useful addition to our toolbox.

Which is not a bad thing, but I wish we could do it without the overhyped nonsense. It really is time for this business to grow the fuck up.

The "web generation" has brought us lots of great changes, I would have quit IT back in the 90's if the internet hadn't happened. But this immature attitude towards everything from technology stacks to how to run a company is really starting to grate.

<comment><USER4> You are right. They are just tools nothing more, nothing less.

For those of us that have been doing software development for a long time we would have heard countless vendors hyping their products. It's what they do and will continue to do. Normal developers have always ignored it and chosen the right tool for the job.

I actually find the anti-NoSQL crowd to be the immature ones and are often quite patronising as though I am some idiot for choosing a particular database. It's quite bizarre.

</comment></comment><comment><USER5> This is an incredibly short-sighted piece for two reasons

1) the data challenges of yesterday's internet giants are the data problems for just about every 21st century enterprise tomorrow. We're at the start of / in the midst of an irreversible data explosion.

2) Established players not taking new technologies and upstarts seriously is never evidence of the threat not being real. We know that on the contrary our industry is characterized by constant disruption where established players for whatever reason do not consistently stay at the front line of innovation and tend to get leapfrogged. (as an analogy Google+'s lurch into social may be akin to Oracle's lurch into nosql. Does that mean social was not real?)

Bottom line: It all depends on what you're doing. But more and more of us will be dealing with more and more complex data in the years to come. Of that, I'm certain.

<comment><USER6> I agree that's a bit short sighted, but I think it's one of perspective. The mistake most people make is to believe that Oracle (and other relational DB vendors) are competing directly against the various NoSQL stores (this includes doc stores, scalable key-value stores, etc.). I don't think that's true. I think that relational databases are really competing against industry- specific SaaS. So instead of implementing their own database for inventory & sales, companies may opt to use a service. These SaaS companies, in turn, are more likely to adopt a variety of technologies (including NoSQL and relational). Since the SaaS companies serve more than a single company, they're also more likely to adopt easily scalable technology like Cassandra. So fewer businesses will need to purchase databases (of any sort), but the ones that do will have greater scalability needs.

<comment><USER4> NoSQL is absolutely competing with Oracle and other relational DB vendors. You are seeing it today with the valuations for companies like Mongo or DataStax. SaaS is hardly a big enough market compared to say every enterprise. And surely the companies listed on the client pages agree with me.

And people forget something important with SaaS. Data sovereignty. Here in Australia for example there are many enterprise companies who are forbidden from using ANYTHING that is hosted in the US due to the grey legal area e.g. Patriot Act. So in-house databases are absolutely still here to stay.

<comment><USER6> It's interesting, my experience with companies here in the U.S. (mainly SMBs) are that they feel much more comfortable using a SaaS compared to adopting an internal NoSQL store. Granted most of these companies aren't highly technical, but I suspect for them using a SaaS is easy and doesn't require much internal expertise. Adopting a new NoSQL store though may require training, etc. Of course I concede that things may be different in larger enterprises and companies outside the U.S.

I would bet, however, that most of the clients that do business with Mongo or DataStax still intend on using/maintaining a relational system. I haven't really encountered many companies that's decided to completely dump their relational systems in favor of something else.

</comment></comment></comment></comment><comment><USER7> I am wondering if the prolonged latencies from most of the Google services I am experiencing in the past few years are somehow related to using Spanner instead of the previously conventional "compute in advance offline"-"serve immediately" approach. The Spanner paper mentioned they sacrificed a bit of latency (from nanoseconds in DHT lookup to milliseconds). I understand it's more convenient for Google developers to have more predictable thought framework instead of making every single project a piece of art while spending most of the time fighting inconsistencies arising from eventual consistency. The question is if it was worth it? I remember time when Google services were amazingly fast - that time is unfortunately gone :-(

<comment> nevi-me In what context are you referring to the "compute in advance offline" part? Sometimes it's costly to compute things in advance and store them, when you might not need them. If storage space becomes a concern, then Google would have to write some fancy algorithms based on data access patterns that determines what to compute in advance, and what to compute JIT. Which complicates things because now they have to write and regularly run programs that tell them what to compute, which simplistically means that they are at least running 3 different tasks, instead of 1 on-the-fly computation.

<comment><USER7> I meant the traditional NoSQL architecture (like what LinkedIn was using) where you had offline batch processing based on Map Reduce executed regularly during the day and the results were stored in a distributed hash table with super low latency and eventual consistency. This made access super fast but inflexible, i.e. only what was precomputed could have been accessed.

</comment></comment></comment><comment><USER8> We switched from SQL to Mongo on [http://versus.com](http://versus.com) one year ago and are quite happy—we couldn't imagine to go back.

But I think it heavily depends on the specific use case what DB to use.

<comment><USER9> I agree, Mongo is easy and fast.

My problem with NoSQL is that you MUST know there won't be changing requirements:

Later you might _need_ to do the equivalent of joining 3-4 tables. Then you have trouble... But I'm a coward that look both left and right before crossing a street.

<comment><USER10> "My problem with NoSQL is that you MUST know there won't be changing requirements"

I thought the main advantage of document-oriented schema-less databases (I haven't used MongoDB but I have used CouchDB) was that they are supposed to give _more_ flexibility in the early stages of projects.

NB Personally, I've moved to PostgreSQL and its JSON storage as then you get most of the benefits of both worlds.

<comment><USER9> >>[NoSQL] are supposed to give more flexibility in the early stages of projects

I agree with what you wrote.

But my point is that _changing_ requirements can screw you, if you find that you _later_ need relational capabilities.

<comment><USER10> Well, that's why I gave up on CouchDB once I found that I was actually much happier with the hybrid approach of using JSON documents _inside_ a relational structure.

</comment></comment></comment></comment></comment><comment><USER11> An interesting article and it does seem like people rushed to embrace NoSQL and are now trying to force it into some kind of consistency after the event - not a bad thing, incidentally - there's a lot of interesting work here (Vector clocks, CRDTs, etc.).

One thing that surprised me though was the lack of a key player: Amazon. Their Dynamo paper was hugely significant and as a company they use eventually consistent stores for a whole swathe of products at scale.

Why mention Facebook and Google but omit this other major player, especially are their experiences tell a different story.

<comment><USER7> Amazon estimated that each 1 ms of additional latency costs them a few million dollars a year as well as decreases rate of returning users. For low latency there is still nothing better than eventual consistency, so they may be driven by the bottom line. Also, from my experience of being a merchant on Amazon with hundreds of thousands of items in inventory that need to be updated almost realtime as prices/stock changes all the time, leading to a few millions updates a day (in bulk), I can't envision how Amazon would be able to achieve fast import from a horde of merchants like me on any SQL system.

<comment><USER11> Yeah, the Dynamo paper is very clear about the rationale behind creating DynamoDB and its usage:

\- downtime (lack of availability) is very expensive to them, both directly and also reputationally

\- traditional ACID stores don't scale with writes whereas an eventually consistent store can achieve this

So I completely agree with you in short: Amazon have great reasons to pursue approaches based on eventual consistency.

</comment></comment></comment><comment><USER12> Relational databases don't scale horizontally. That's still as true today as it was a decade ago. Therefor engineers who need to cope with Big Data and Web Scale will continue to migrate away from relational database solutions towards persistence technology centered around distributed systems. It's as simple as that.

If you build your app around a relational database and you need to scale up big then at some point you're going to hit a brick wall in terms of scaling out storage and/or writes. You either have to build sharding logic into your relational db app from the beginning(which is a pain that NoSQL saves you from), or else you have to re-architect your entire app when the time comes that you need to deal with scale. Many shops end up borrowing VC money to build out a team to re-architect their systems to handle web scale, but this can be avoided by thinking about data access patterns from the beginning and choosing a technology that can handle your future needs.

[https://en.wikipedia.org/wiki/Scalability#Horizontal_and_ver...](https://en.wikipedia.org/wiki/Scalability#Horizontal_and_vertical_scaling)

<comment> workhere-io AdWords was run on MySQL up until two years ago. The vast majority of developers working on "web-scale" projects won't get to handle projects with larger requirements than AdWords. In that perspective the whole NoSQL trend makes very little sense, especially given the fact that e.g. PostgreSQL and MySQL have a lot of needed features that many NoSQL databases don't, and that the most "hip" NoSQL database a couple of years ago was a database that doesn't even scale that well (MongoDB).

<comment><USER4> You do understand how MySQL et al are used in those cases right ? They are treated as dumb key value stores and sharded horizontally with joins done in the application layer. They are NOT your typical SQL deployment and the features you talk about are often meaningless.

And you are 100% wrong about MongoDB not scaling well. The stories you hear of people switching are never going back to PostgreSQL or MySQL they are going to the next level in scalability e.g. HBase or Cassandra.

<comment><USER12> Like I said because the relational database doesn't scale horizontally you are forced to build a sharding layer into your application which introduces complexity into the application layer and limits your ability to use the relational features of the relational database. At that point you might as well just be using a key-value store that does the sharding for you and offers greater flexibility.

<comment><USER13> there is still transaction in one shard beyond SQL. I believe that ACID is the main benefit comparing with NoSQL store

<comment><USER12> My NoSQL store of choices gives me document-level ACID semantics along with eventual consistency, MapReduce, and replication.

Here's an example that explains how to get transaction-like guarantees from this kind of NoSQL data-store:

[http://guide.couchdb.org/editions/1/en/recipes.html](http://guide.couchdb.org/editions/1/en/recipes.html) [https://en.wikipedia.org/wiki/BigCouch](https://en.wikipedia.org/wiki/BigCouch)

<comment><USER13> Interesting, that isn't transaction, it is just a workaround, and I don't think you can design your app like that which treat document as a transaction log. And then using view to generate the real information.

<comment><USER12> that's exactly how I design the parts of my app that need a transactional nature. Map/reduce views make it a breeze to query for a consistent aggregate view of the world.

This is not something new it's a well established technique from the relational world called "event sourcing"

[http://www.martinfowler.com/eaaDev/EventSourcing.html](http://www.martinfowler.com/eaaDev/EventSourcing.html)

when everything is a write you don't have to worry about conflicts or locks, so that's nice, plus couch is really good at scaling write thoughput with small documents.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER14> I'm far from an expert on this topic, but I am a developer, and I am currently working on a project that leverages both MySQL and ElasticSearch for storage. MySQL handles all of the "boring" data such as users, profiles, comments, etc. ElasticSearch is basically a giant product database, no real relational data, just a normalized structure and easy to query. ElasticSearch is serving as a primary data store, there is no backing in a database because ES is just that good.

Whenever I see these "SQL vs NoSQL" arguments, I always have to wonder: Why one over the other? A lot of projects can benefit from both and there's no reason you absolutely HAVE to use one or another. It's perfectly reasonable (and probably ideal) to use more than one storage system in your projects.

If you have a bunch of nails to hammer in and bolts to tighten, you don't choose just a hammer or a wrench to do that job...you grab both and use each for what they do best.

<comment><USER15> > ElasticSearch is serving as a primary data store, there is no backing in a > database because ES is just that good.

And you've been running this application in production how long?

<comment><USER11> I agree that using Elasticsearch as a primary data store seems risky. The situation is improving though; the v1 release introduces easy backups (for example, to s3) which from my experience work very well.

Prior to this backing up was a messy process.

</comment></comment></comment><comment><USER16> NoSQL discussions always seem to conflate three very different things: storage engines, APIs and architecture. Where do we store the data ? How do we access it ? How do we make sure it scales ?

The "traditional" approach is to use Oracle/SQL Server/MySQL for storage, SQL and/or ORM as an API, and single-server tables-with-relationships as an architecture. Back in the early 2000s, everybody did this. Sure, there were a few performance-minded exceptions that went with sharding or master-slave architectures instead, but those were exceptions.

And single-server architectures tend to behave badly at medium loads. Spend the market rate for a genius DBA, and they still behave badly at high loads. The next step is a 32-core 128GB RAM monstrosity that costs an order of magnitude more than what eight 4-core 16GB servers would cost.

Most NoSQL solutions came with a new architecture. You had the MongoDB flavor of distributed storage, or the BigTable flavor of distributed storage, or the CouchDB flavor of distributed storage, and so on. Properly implemented distributed storage eats high loads for breakfast: just add more servers. This is a good thing.

My issue with the NoSQL movement is that they threw away the baby with the bath water. They threw away the single-server relational architecture, which was a nice change, and they also gave up the old battle-hardened storage engines and the highly expressive SQL language and replaced them with only- recently-experimental engines and ad hoc lean APIs.

It takes time for a storage engine to mature. To have all its performance kinks ironed out and all its bugs smoked out. I still remember the brouhaha around MongoDB persistence guarantees, or the critical data loss bugs in CouchDB.

And the lean APIs just forced back all the querying logic into the application, with all the filtering and the manual indexing and the joins and the approximate but ultimately incorrect implementations of whatever subset of ACID was required at the time. This wasn't an entirely bad thing: it certainly made many developers aware of the performance implications of some joins or transactions. But when you need to write a JOIN or GROUP BY or BEGIN TRANSACTION that you know will scale properly, and there's no API support for it ? Feh.

I'm a huge fan of the CouchDB architecture. Distributed change streams, with checkpointed views and cached reductions. But I have been burned by the CouchDB storage engine (can you say "data corÊ–NÑ %ñXtion" ?) and I see no point in bending knee to the laconic CouchDB API. So I took the CouchDB architecture and reimplemented it with a PostgreSQL back-end. It's _faster_ (don't underestimate the cost of those HTTP requests), I have trust that after PostgreSQL's decade-long history all threats to my data are long gone, and I can always whip out an SQL query when I do need it.

It's nice to see so many NoSQL solutions migrating back to an SQL-like API and gaining enough maturity to keep your data safe. In the near future, I expect them to be nothing more than "Architecture in a box" solutions for when you don't want to implement specific architectures in SQL. And I expect more and more "architecture plugins" to become available: with a library, turn a herd of SQL databases into a distributed architecture of type X.

<comment><USER17> Single-server relational backends can survive loads well in excess of most of us will ever see for our software. There are a handful of truly web-scale companies that need specialized engineering, and the time to solve for that is when you actually have that problem in sight. Conforming SQL engines can do some amazing things with simple and declarative code, giving that up in the hopes you might be Twitter-scale one day is, in my opinion, a quite poor tradeoff.

<comment><USER16> I certainly agree that most people who pick NoSQL solutions "for scalability" never add a third server to their cluster.

</comment></comment></comment><comment><USER18> How do people that abstract away their database via an ORM feel comfortable about not dealing directly with the data and accidentally dropping a db column via ORM? I know when I am dealing with the database I am a lot more surgical in my approach than when I am writing code.

<comment><USER19> You generally only let the ORM create/update your schema during early development. By the time you get to production (or heck, by the time your team is even seriously engaged), you will have disabled that feature. In pretty much every enterprise-grade ORM I've ever seen, schema modification is disabled by default, and must be explicitly turned on. In most Java shops where I've ever worked... unless it's a quick prototype, the database schema will be developed before you start coding anyway. Also, in a typical enterprise scenario, the username with which you configure the ORM lacks admin-level privileges to alter the database.

Most of the criticisms about ORM's come from people who have never used them, beyond maybe working through a Rails chat-room tutorial once upon a time. It really has nothing to do with "abstracting away the database".

As the name indicates, Object-Relational Mapping is merely about reducing the boilerplate required to map a relational schema to programming language objects. If you do that mapping by hand, then you have to make decisions when a table/object has relationships. Picture a CUSTOMER table, which has a foreign key relationship to an ADDRESS table. When your application loads a "Customer" object:

[1] You could "eager fetch", meaning that you go ahead and retrieve all of the ADDRESS rows related to that CUSTOMER, and attach the Address objects to the Customer object. Eager fetching is wasteful and leads to poor performance, because you're hammering your database for values that you often don't ever use.

[2] You could "lazy load"... meaning that your Customer object has an "addresses" field, but you wait until some code tries to use that field before you actually query the ADDRESS table to populate it. This is much better design, but complicates things. The lazy load logic has to go somewhere. You either have to ensure that every piece of code using that object is aware of the lazy load pattern, or you have to stuff database logic into the "getter" method for each lazy-loaded field.

ORM's give you highly-performant lazy loading, without the buggy boilerplate suggested by #2 above. Moreover, enterprise-class ORM's typically handle caching for you, to avoid hitting the database unnecessarily. Monitoring the state of objects to notice when they've gone stale due to changes on the database side, etc.

Lastly, for complex queries, most ORM's have a query language (e.g. JQL, HQL, etc) that is nothing more than a VERY THIN wrapper around SQL. It merely smooths out differences between various vendor dialects. You're not abstracted away from the database, you still very much need to understand SQL and the underlying structure of your data.

<comment><USER18> _you still very much need to understand SQL and the underlying structure of your data_

That's good to hear! Though in my experience of interacting with people who have learned development in last 3-5 years, many of them are very oblivious to basic database concepts. They mostly think in terms of the object in the code, relying on the framework to take care of the database. It might even generate the sql that they just need to execute. But anyone who's worked with production database from before the advent of frameworks like Rails etc. knows that the only thing worse than messing up an sql statement is running auto generated sql.

</comment></comment></comment><comment><USER20> >> NoSQL is nothing more than a storm in a teacup

There's been a difference with this "technology cycle" though, there have been some prominent, well grounded voices from the start of this cycle.

That's a good thing.

That wasn't the case for other cycles - the thin vs fat client cycle, the DAS vs NAS cycle etc. etc.

EDIT: i'm not saying NoSQL has no application. I earn a paycheck working with a huge graph db (and it's nothing to do with "social", yay!), and have previously been a heavy user of Cassandra.

</comment><comment><USER21> > NoACID

Perfect!

</comment><comment><USER22> NoSQL has always been a stupid term, especially as there are "NoSQL" datastores that support SQL, e.g. OrientDB.

<comment><USER23> The term was perfect. The spectrum of NoSQL implementations out there defies an easy name to cover them all. They aren't all key-value stores, they aren't all json stores, they aren't all big tables, etc.

Indicating that they were NOT within the current paradigm of SQL databases was the only way to go.

<comment><USER22> Why is it a good idea to lump such technologies together, when many of them are fundamentally different? Do we call road vehicles that aren't cars NoCars?

<comment><USER24> Ironically, we kind of do, at least in the US. We call the trucks. SUVs are a bit of an anomaly, but I've heard plenty of people colloquially call our explorer a truck.

<comment><USER25> You call motorbikes, buses, and bicycles trucks?

</comment></comment></comment></comment></comment><comment><USER26> Finally a good writeup that explains the strange phenomenon of NoSQL. Now, all that's left is to introduce a tax for "drinking the Coolade".

</comment><comment><USER27> We are stuck with NoSQL in HTML5, because Mozilla and Microsoft refuse to implement WebSQL ([http://en.wikipedia.org/wiki/WebSQL](http://en.wikipedia.org/wiki/WebSQL) )

IndexedDB is fine for storing JSON objects, etc. but a relational database with SQL query syntax, indexes, etc. more powerful and means less code to write. With IndexedDB one has to reinvent the wheel to just get basic query features.

WebSQL is not deprecation, the W3C Working Group Note actually says:

      'This specification is no longer in active maintenance
      and the Web Applications Working Group does not intend to
      maintain it further'.

WebSQL is only available in Webkit based Browsers (Safari, Chrome) which means most mobile browsers. As SQLite is in public domain, no company would "loose their face" if they choose to use it. They could fork off SQLite and change the SQL query syntax (parser) to whatever the W3C finds suitable. [https://www.sqlite.org](https://www.sqlite.org)

Mozilla Firefox and FirefoxOS both already ship SQLite for years and can be accessed by its internal JavaScript API. And several Microsoft products already use it anyway (e.g. Forza Xbox games). Microsoft has of course also various other SQL database libraries like MS Access JetRed, MS Outlook JetBlue and SQL Express.

We had a discussion about it recently: [https://news.ycombinator.com/item?id=7574754](https://news.ycombinator.com/item?id=7574754)

The new hip things is "NewSQL" ([http://en.wikipedia.org/wiki/NewSQL](http://en.wikipedia.org/wiki/NewSQL) ). For example Facebook, Google Ads, etc. are powered by MySQL's InnoDB database engine. I would go as far as count SQLite to this group.

We would need a movement to convince Mozilla to finally add WebSQL to Firefox and FirefoxOS.

<comment><USER28> There are good reasons why work has stopped on the WebSQL specification — everybody was using SQLite, and the specification can't be tied so closely to a single implementation of SQL. The specification even has the line "User agents must implement the SQL dialect supported by Sqlite 3.6.19"[1].

1\. [http://www.w3.org/TR/webdatabase/#web- sql](http://www.w3.org/TR/webdatabase/#web-sql)

Edit: here is Mozilla's rationale for not supporting WebSQL: [https://hacks.mozilla.org/2010/06/beyond-html5-database- apis...](https://hacks.mozilla.org/2010/06/beyond-html5-database-apis-and-the- road-to-indexeddb/)

<comment><USER27> SQLite implements the _SQL-92 standard_ ([http://en.wikipedia.org/wiki/SQL-92](http://en.wikipedia.org/wiki/SQL-92) , [http://en.wikipedia.org/wiki/SQLite#Features](http://en.wikipedia.org/wiki/SQLite#Features) ).

Firefox certainly ships with SQLite and even their IndexedDB implementation is based on top of it internally (the irony): [https://plus.google.com/+KevinDangoor/posts/PHqKjkcNbLU](https://plus.google.com/+KevinDangoor/posts/PHqKjkcNbLU)

Microsoft has MS Access JetRed, MS Outlook JetBlue and SQL Express SQL Databases that ship already with Windows and/or Office and are SQL-92 compatible. It should be trivial for them to figure out which SQL DB engine would fit best and integrate it in IE 12.

<comment><USER28> First sentence of [http://en.wikipedia.org/wiki/SQLite#Features](http://en.wikipedia.org/wiki/SQLite#Features) : "SQLite implements most of the SQL-92 standard for SQL but it lacks some features."

</comment></comment></comment></comment><comment> guard-of-terra If you're doing accounting, sure you would want ACID compliant database. There you have a limited amount of kinds of data to store with strict and rarely changing constraints. You can keep it on one big expensive server (plus backup) and it's better to bring the system down to allow inconsistency.

However, for most web development SQL is seriously not good. You end up with hundreds of loosely coupled tables which constantly change their structure for new features. Half dozen of joins on every request. Hundreds of lines of SQL. Constant pain. And it's not like you cared so much for the consistency - <USER2> once a year three comments disappear from your web site, so what?

And it's painful to make SQL multi-master.

For (the most of) web development document databases are so much better. MongoDB is pretty nice because it makes hundreds of lines of SQL with ten files of code redundant - all per one complex document.</comment>


When people land on your webpage, give them goals. (Growth Experiment – 2 Cases) - <AUTHOR> Here is a growth experiment I ran with a positive impact and wanted to share it with everyone here.
Google uses several signals to identify if a page they sent people to is high value or not.
One of the important signals Google uses is time spent on a page.
This experiment is tied to time spent on-page and has contributed to over 100% uptick in Google organic traffic in the last two weeks.
It is a straightforward experiment that does require you to have some programming knowledge and create webpages or a blog using code (e.g., HTML/CSS, Javascript)
I used this with a blog, but you can use it with any webpage.
The task is simple →
When someone lands on a webpage (or a blog post in my experiment) give them a few tasks to do. Typically, people like checking off tasks.
And while accomplishing tasks, they increase their time on your site.
Here are two examples:
1. Blog post with 101 Remote Work Stats with stat tracker. I created a simple tracker on the right side of the blog post that tells people how many stats they have seen out of the 101.
Before publishing this "Statistics Seen" tracker, on average, people spent 25 seconds on this blog post.
After incorporating the "Statistics Seen" tracker, they are spending about 1 minute and 15 seconds.
Here is the link if you want to check out the implementation https://www.prospercircle.org/blog/top-remote-work-stats
2. Blog post on Video Interviewing Tips with todos. Again, several websites provide such tips.
I created a list of 20 tips, broke them in three categories
The todo of seeing tips in each category is automatically tracked based on user scrolling.
Again, the time spent on this blog post increased. Initially, people only scrolled up to 33% of the blog post before the todo was on the site, now the average scroll is 80% of the page.
Here is the link if you want to see the implementation - https://www.prospercircle.org/blog/video-interview-tips <comment><USER2> wonder if switching one technique for the other performs better.</comment>


Nike's Vaporfly running shoes and tumbling records - <AUTHOR> https://www.reuters.com/article/us-athletics-shoe/nikes-vaporfly-running-shoes-and-tumbling-records-idUSKBN1ZN0MJ <comment><USER2> Olympics swimming did ban full-body swimsuits after it determined they gave an advantage. These shoes are being studied now.</comment>


Is it a Canaletto or a Bellotto? Don't ask an art historian - <AUTHOR> http://www.theguardian.com/education/2015/nov/15/art-historians-fail-spot-differences-paintings-canaletto-bellotto <comment><USER2> I almost never do.

To be less flippant, I'm not sure that defining the field of art history more broadly is a bad thing. I would hate to see the disappearance of those who can make fine distinctions between painters; there should be room for them too in the art history world.

I did guess correctly, by the way, but I couldn't tell you whether it was because I saw one of the pictures long ago at a Canaletto exhibition at the National Gallery of Art, from books, or just chance.

</comment><comment><USER3> Need people to classify paintings by idiosyncratic style?

This seems like a rather approachable problem for Google Brain and friends.

Heck, since the painters are all dead, you can solve most of the problem by just saving a copy of all the known authorships, and classify the rest.</comment>


5 Reasons Your Startup Should Have An Internal Hackathon - <AUTHOR> http://blog.fetchnotes.com/post/8209641270/5-reasons-your-startup-should-have-an-internal

<comment><USER2> dc_id</comment>


Mental Models to Make You a Worldly Wise Person - <AUTHOR> https://medium.com/swlh/100-mental-models-to-make-you-a-worldly-wise-person-19b9ca9d034c <comment><USER2> not a fan of this at all and I've seen this 'model based' arguing more and more recently.

The blatant problem with it is that just like the appeal to fallacy stuff that is very popular among certain groups of people it's not really intellect or 'wisdom' as much as a badly integrated bag full of tools that are too broad and easily misapplied and used as crutches.

It's a little bit like people who think if they read 20% more books they're 20% more wise but that's not really how it works because genuine generalised 'wisdom' or intellectual feat isn't just a lose connection of piecemeal ideas. The real intellectual effort is in the synthesis and the production of new ideas which by definition aren't on a top 100 list.

<comment><USER3> I would add to this that making lists like this is akin to reading an encyclopedic textbook front to back; theres no continuity or hydrating component that clarifies the deeper meanings implied. In fact, these lists are the scientific equivalent of motivational quotes people print on posters and then never look at again.

Mental models as I've always seen them are icebergs, unique to every person, closer in semantics (shape) for people who were trained similarly (via academia or apprenticeship), and wildly different for those who came to a model in a discipline from a completely different direction.

Used in lists like the one linked, we only see the tip of the iceberg, the written anchor that can be used by someone else who has a similar model. Otherwise we might as well hand this to a teacher in k-12 education and have them be as well off as rote-memorizing formula in place of learning what a subject is actually all about.

Mental models IMO should be formulated holistically, derived from examples, allowed to be explored so the value of the wisdom becomes apparent to the learner over multiple visitations. In this way we transfer and evolve the key insights across a large number of contexts rather than hand someone a page of ideas. The most common response to being handed a page like that is to say "cool, thanks", save it to the HDD, and never look at it more than once.

</comment></comment><comment><USER4> At the library last week, I saw this book on mental models:

[https://www.amazon.com/Super-Thinking-Book-Mental- Models/dp/...](https://www.amazon.com/Super-Thinking-Book-Mental- Models/dp/0525533583/)

<comment><AUTHOR> You might like his Medium post that preceded his book. The author is the guy that started DuckDuckGo. [https://medium.com/@yegg/mental-models-i-find- repeatedly-use...](https://medium.com/@yegg/mental-models-i-find-repeatedly- useful-936f1cc405d)</comment></comment>


Domain Storytelling: A DDD tool to visualize domain stories in the browser - <AUTHOR> http://domainstorytelling.org/ <comment><USER2> What are the advantages/disadvantages you've found over event storming? Is this just a good way to knowledge crunch with stakeholders to get to a good analysis model, or do you find it leads to specific learnings you don't get with any other method?

<comment><USER3> "event storming" and "knowledge crunch with stakeholders" are two phrases I've never heard before. Can you explain them?

<comment><USER4> I don't understand it in detail, but event storming is basically a workshop for sharing knowledge about a problem domain. You get SMEs into a room with whoever is executing (devs, designers, etc) on a project and, under the guidance of a facilitator, have a time-boxed, business-oriented discussion about the domain. You identify domain events, ie, all the interactions that occur between users and system components.

<comment><USER3> Thanks!

</comment></comment></comment></comment><comment><USER5> Interesting! With a domain model like this, a program could exhaustively test all paths for invalid states. The model would need to some annotations of invariants or could ask a human to review.

I highly recommend Eric Evans' book "Domain-Driven Design: Tackling Complexity in the Heart of Software" (2003) mentioned on this page. It is a bit of slog, but worth it.

Or for a quick introduction to domain-driven design, check out the (free download) of "Domain Driven Design Quickly" e-book from InfoQ:

[https://www.infoq.com/minibooks/domain-driven-design- quickly](https://www.infoq.com/minibooks/domain-driven-design-quickly)

</comment><comment><USER6> We changed the URL from [https://github.com/WPS/domain-story- modeler](https://github.com/WPS/domain-story-modeler) to the project page, which gives more background info.

</comment><comment><USER7> Very useful, I find that discovering domain knowledge can be a painful process when trying to build a relationship with a new client. Hopefully this will help

</comment><comment><USER8> I was looking for something like a while back. Bookmarked.

</comment><comment><USER9> Erm UML rediscovered? :]

<comment><USER10> Nope. UML is a language for describing software systems. It's software first. Domain-Driven Design is a focus on understanding the actual concepts of domain experts. So this is people first.

As Martin Fowler said, "Any fool can write code that a computer can understand. Good programmers write code that humans can understand." That's much more likely to happen if programmers start with the existing _human_ understanding of the situation.

As a contrast, let me describe two terrible code bases I worked on. The first was something built by roving gangs of consultants. It was about donations and corporate matching, but that was hard to tell by the code. It was basically a giant snarl of HashMaps with magic keys that got passed around all over the place, with bits of code fishing out things they needed. It mostly worked, but it was incredibly hard to sort out because the code didn't _mean_ anything.

Another is the typical Java enterprise mess. I was once called in to advise on an open-source product that was being built by consultants used to doing banking work. Between being fetched from the database and rendered on the screen, data was copied from object to object _7 different times_. ThisBean and ThatBean and DTOFactoryIntializerStrategy. Most of the code base wasn't really about the domain, it was cargo-cult rituals and architectural patterns that had no practical value. Worse, having the domain knowledge smeared out over a zillion objects means that their early domain understanding was locked in, so the code base and the way people talked about the domain drifter further and further apart.

At this point I'm a firm believer in Domain-Driven Design. If you really understand how people think about what they do and then build the system in those terms, you'll get a much better code base and a much better user experience. Domain concepts are generally very stable, so your design is less likely to go obsolete.

A good example is yesterday's article about building a real-time editing system. They got good code because they really thought about the human experience and then built in those terms: [https://news.ycombinator.com/item?id=18220020](https://news.ycombinator.com/item?id=18220020)

<comment> keithb- I'm not trying to nitpick, but UML is purpose-first, not solution-first. The language always allowed for varying levels of abstraction (i.e. linguistics, not layers), and it was always possible to describe the domain instead of classes+methods, but one is always creating diagrams not code. For example, a collaboration diagram can specify domain concepts and use plain language for activities and artifacts; it is not necessary to consider functions and arguments and exceptions. Oh my.

Your anecdotes are more common with object-oriented languages and poor object- oriented design approaches. Any fool can claim to have discovered a OOD pattern, but a good programmer will not claim to have discovered anything. Instead, she will use the software to express the user's activity and desired outcomes.

That being said, some software languages take more skill and experience to implement a solution than others. Sometimes it is necessary for developers to consider the choice of HashMap over List<?> or worse MyCollection which implements List<?> over an array. These considerations may have created to more types of diagrams (UML, OOD, etc.) that describe software in more an more detail, but what is necessary to express a solution is really subjective and rarely universally understood. You can't sit an inexperienced developer in front of a C++ program and expect them to understand that they are looking at code for a text editor anymore than you could sit them down in front of a collection of DDD diagrams and expect them to build a perfect representation in Java.

We (the royal "We") are looking for a proper balance in design and implementation. That is a work-in-progress: it is the point of this career and won't be captured by single checkbox on one's resume, e.g. "knows UML or not".

<comment><USER10> > UML is purpose-first, not solution-first

I don't believe that's the case. If we set the Wayback machine for 1999, it's pretty clear: "The Unified Modeling Language (UML) is the industry-standard language for specifying, visualizing, constructing, and documenting the artifacts of software systems. It simplifies the complex process of software design, making a 'blueprint' for construction." [1]

It's very clearly about software.

[1] [https://web.archive.org/web/19990127214541/http://www.ration...](https://web.archive.org/web/19990127214541/http://www.rational.com:80/uml/index.jtmpl)

</comment></comment></comment></comment><comment><USER11> Nice, might come in handy soon. Bookmarked.</comment>


New 12-core Mac Pro, starting at $3799 - <AUTHOR> http://www.apple.com/macpro/

<comment><USER2> Where "new" means a minor speed bump of the old 12 core Mac Pro but still on the same, last generation CPU tech, a 2.5 year old graphics card, no Thunderbolt, and no USB 3. (Though the latter 3 could all be solved for existing Mac Pros with the right add on cards if Apple chose to do so.)

<comment><USER3> And they're charging (as far as I can tell from finding the same/equivalent parts on newegg for the lowest end $2,500 model) 4-5x the actual hardware cost.

CPU: $260, 3x2GB ECC DDR3: $60, 1TB 7200 RPM HDD: $120, Radeon HD 5770: $120

<comment><USER4> what cpu model did you chose to price it a 260$?

six cores nehalem start around 1000$ according to wikipedia

[http://en.wikipedia.org/wiki/Nehalem_(microarchitecture)#Ser...](http://en.wikipedia.org/wiki/Nehalem_\(microarchitecture\)#Server_.2F_Desktop_Processors_2)

While probably far from "best bang for the buck", the 2 2.4GHz is probably closer to the real cost than the other models.

Best 6 cores for the buck seems to be this baby right now, although it doesn't seem to support multiple processors as it is not a Xeon DP. <http://ark.intel.com/products/63697>

</comment></comment></comment><comment><AUTHOR> Comparison with the previous models: [http://www.macstories.net/wp- content/uploads/2012/06/Mac-Pro...](http://www.macstories.net/wp- content/uploads/2012/06/Mac-Pro-Mid-2012_comparison.jpg)

</comment><comment><USER5> $15,416.00 for the fully pimped version. Sploosh. What kind of multicore Linux box could I build for that? (I love Macs, just not this range).

</comment><comment><USER6> This looks more like "we couldn't keep buying the old processor" than a real upgrade.

</comment><comment><USER7> Was this announced in the keynote? Why did they keep it out?

<comment><USER8> > Why did they keep it out?

Apple doesn't generally hold events or give keynote time over to simple spec bumps in product lines that aren't hugely popular, like the Mac Pros, or the Mac Minis.</comment></comment>


The Cathedral and the Bazaar - <AUTHOR> http://www.firstmonday.org/issues/issue3_3/raymond/ <comment><USER2> just read this for my CS class last week in regards to online cooperation and the open Source Movement. good stuff.

<comment><AUTHOR> It's old, but it's a classic. :)</comment></comment>


Twitter: The Discovery Engine Is Coming - <AUTHOR> http://blog.twitter.com/2009/04/discovery-engine-is-coming.html <comment><USER2>
       By default, the tweets you see on your home page are from sources of information
       that you have curated over time - in other words, the accounts you chose to follow.

I thought this announcement was just 'hey guyz we've moved the search bar' until I read this part. Authority search! Interesting.

<comment><USER2> OK, I can't read.

Don't mind me.</comment></comment>


Dealing with the Startup Police - <AUTHOR> https://leowid.com/dealing-with-the-startup-police/ <comment><USER2> I'm disappointed that "grow a spine and tell the Startup Police to fuck off" wasn't listed as a viable option.</comment>


Elixir Best Practices: Structs, String-Keyed Maps, and Atom-Keyed Maps - <AUTHOR> https://engineering.appcues.com/2016/02/02/too-many-dicts.html <comment><USER2> The conclusions reached here seem to conform strongly with what I've discovered "the hard way" in both Elixir and Ruby-land.

It also gives me a better idea of when to switch from "bag of mapped values" to a Struct. Great!

I'd further add that the "binary_to_existing_atom" function may make it possible to both take advantage of atom keys AND prevent DDoS attacks, assuming you are whitelisting the keys you expect somehow.</comment>


Climate change represents a near- to mid-term threat to human civilisation - <AUTHOR> http://www.climatecodered.org/2019/05/can-we-think-in-new-ways-about.html <comment><USER4> > _A doomsday future is not inevitable! But without immediate drastic action > our prospects are poor._

So, what you're saying is: Unless we (immediately and drastically) change our behaviour, it's a doomsday scenario[0,1,2]...

Doesn't that make it inevitable because it seems that nobody has or is changing their behaviour drastically enough - any time in the recent past or the nearest enough future - to prevent it (based on projections)?

I mean, it's not as if this realisation struck us just yesterday, yeah? We've had some time to prepare and/or give enough of a shit about this to try to affect it, right?

Also, you're expecting (almost) the entirety of an egocentric species - <USER2> that was entirely ok with the mass extinctions that it has caused so far - <USER3> suddenly, overnight give a rat's ass about the future state of the entire world and everything within it...?

They're so far up their own asses about 'x' trivial thing (compared to the literal death of the species/planet) that they're seeing teeth and _loving_ it.

I - for one - welcome the end times.

Fucking shake that etch-a-sketch, biology, let's try this shit again.

[0] - [https://www.metoffice.gov.uk/food-insecurity- index/](https://www.metoffice.gov.uk/food-insecurity-index/)

[1] - [https://www.theguardian.com/global- development/2013/apr/13/c...](https://www.theguardian.com/global- development/2013/apr/13/climate-change-millions-starvation-scientists)

[2] - [https://www.scidev.net/global/climate-change/news/put- agricu...](https://www.scidev.net/global/climate-change/news/put-agriculture- at-heart-of-climate-talks-says-rep.html)</comment>


Ask HN: Working on a Ground Breaking Project. Lost on how and where to execute. - <AUTHOR>

Excuse the rambling but I need to explain my history and my challenge. Ok, so, Im on my 3rd startup, first 2 were ecommerce startups that were acquired(small exits). I worked my butt off and built those businesses with less then $100. I read and discect EVERYTHING thats Silicon Valley related daily."Family thinks im possessed" but i just cant not analyze everything thats going on in silicon valley and tech and biz related. I live in Dallas(cheap real estate but no startup scene.) You may say that there is "SOME" startup scene but there is NO culture that empowers tech startup, there is minimal funding, and the talent/ desire to leave a good job and help someone create a huge market disrupting company is not here. Im currently working on a "social experiment" that will push the boundaries of whats possible in a . It has been bootstrapped by me, but I dont know what to do. Do I continue, launch with no financial backing, but be near homeless-ness?(with family) Or do I slow down progress and try to get funding in my Area? Or do I pack it all up and go to the Valley?
I love the valley been there about 5 times. It feels like its where I belong. But the gas prices, state income tax and real estate is so darn expensive its hard to live comfortable.
I need to community to help me with this one..
Help !! <comment><USER2> Austin is close and does have a fairly vibrant start-up scene/culture. Make the drive and go to some meetups or functions.

Also, don't be mysterious. Remember, even Dean Kamen thought he had a ground breaking invention. After all the secrecy and hype, it turned out to be an overpriced little scooter. Tell people honestly about your idea, they're too busy on their own things to "steal" it, but the feedback you get will be invaluable.

</comment><comment><USER3> As a purely subjective ?, this is going to come down to priorities, and as such there's no "right" answer. You'll be able to look through history and find plenty of examples of people who have bet it all and won, and they may have been right to make that bet depending on your perspective. However, there are also more people who have bet it all and lost (even when they were right - check out Charles Goodyear).

Personally, I think it's irresponsible to put your family in a situation you characterize as "near homelessness" for a "social experiment" - especially if you can get this done more slowly by pursuing funding more than you have. I also think it's a rough idea to move to a far more expensive locale on the hopes that it will help you with your project. You have a skill-set that can provide for your family and leave you the time and means to work on your experiment on the side. You live in a city that has a market for that skill- set, which means you have options at your fingertips. I think you ought to play it safe and put your family obligations ahead of your startup ambitions for the time being.

</comment><comment><AUTHOR> Thanks for the response. Spouse is from North Cali. And moving any further south in Texas will only make for more whining and complaining of the heat. Austin does have alot going on though. So I guess what im saying is that its either here or move..

</comment><comment><AUTHOR> near homelesness = broke.</comment>


How to live free, do everything you want and insure you'll never, ever be happy. - <AUTHOR> http://rym.waglo.com/wordpress/2010/01/08/how-to-live-free-do-everything-you-want-and-insure-youll-never-ever-be-happy/ <comment><USER2> I never knew insurance companies took on this kind of risk.

<comment><USER3> It's not misspelled. Look it up.

<comment><USER4> [http://www.google.com/dictionary?langpair=en|en&q=insure](http://www.google.com/dictionary?langpair=en|en&q=insure)

[http://www.google.com/dictionary?langpair=en|en&q=ensure](http://www.google.com/dictionary?langpair=en|en&q=ensure)

<comment><USER5> Historically, they are interchangeable. Insure has come to mean primarily _insurance_ , particularly in the UK, though the first definition given in the OED is still _"to make a person sure of a thing"_ (marked obsolete), but I still see _insure_ with that meaning regularly from Americans.

I like this quote:

<http://www.wsu.edu/~brians/errors/assure.html>

 _"Other authorities, however, consider "ensure" and "insure" interchangeable. To please conservatives, make the distinction."_

Looks like the Grammar Nazis have won, when we're giving writing advice intended purely to avoid their wrath.

OED ref, though I don't think that's a public link if you're not on a campus with a licence:

[http://dictionary.oed.com/cgi/entry/50118480?single=1&qu...](http://dictionary.oed.com/cgi/entry/50118480?single=1&query_type=word&queryword=insure&first=1&max_to_show=10)

<comment><USER3> It's not being a grammar nazi to point out that something is _not_ misspelled.

</comment></comment></comment></comment></comment><comment><USER6> I saw 3000 visits on my post this morning, I thought, wow, lots of spambots last night. But then I notice it was posted here. Fixed the spelling mistake, I usually check for that but sometimes one slips. English is not my primary language. Thanks for your comments.

</comment><comment><USER7> I fail to see any coherency in these points. Can someone enlighten me as to what unites them?

<comment><USER8> Most are a portrait of a certain kind of person that one wants to avoid (and especially, avoid being).

<comment><USER9> Avoid being honest? And what does that have to do with his price point comment? You can be honest and still charge someone more than something costs... I'd say it's almost dishonest not to. It's called charging for your time/work,etc?

</comment></comment></comment><comment><USER10> This is a fairly pointless post, just a collection of snarky one-liners without much insight. Less of this on HN please.

</comment><comment><USER11> Wow... I also don't understand why these traits (cognitive biases?) fit together, but somehow they feel like a coherent archetype. Surprisingly insightful.

</comment><comment><USER12> How is this post voted to the top in HN?. Beats me.

</comment><comment><USER13> _Wear out the reload button on your browser._ -what does this mean?

<comment><USER14> Okay, when reading various blogs (or checking facebook) I always get nervous that the page's owner will analyze the logs and call me out on just how many times and how rapidly i hit refresh waiting for new content to appear or hoping that someone has replied to my comment.

<comment><USER15> I'd have to say it would probably be really awkward if people could see what pages I clicked through to on something like facebook and when.

The other one is at work, sometimes explaining why I have something set up the way I do and how I'm using it is awkward.</comment></comment></comment>


Politician suggests an embedded chip to prevent use of welfare benefits abroad - <AUTHOR> http://www.metropolitan.fi/entry/finnish-politician-suggests-people-be-embedded-with-a-chip-to-prevent-misuse-of-social-welfare-benefits-abroad

<comment><USER2> Perussuomalaiset are increasingly becoming an international embarrassment. It seems like I only see our politicians in the international news when they've either said something racist again, or they're negotiating how best to put the screws to Greece next.

<comment><USER3> Don't worry, I don't think there's one country on earth where one can be proud of one's politicians.

Btw I'm French, do you feel my pain now ? ;-)</comment></comment>


The Evaporative Cooling Effect - <AUTHOR> http://blog.bumblebeelabs.com/social-software-sundays-2-the-evaporative-cooling-effect/#storycontent <comment><USER2> Last time this was posted <[http://news.ycombinator.com/item?id=1777665>](http://news.ycombinator.com/item?id=1777665>), Eliezer linked to a related article worth reading as well: Well-Kept Gardens Die By Pacifism <http://lesswrong.com/lw/c1/wellkept_gardens_die_by_pacifism/>

</comment><comment><USER3> I was hoping for some thermodynamics awesomeness.

<comment><USER4> I agree.

Here's one in lieu - the stillsuits in "Dune" could not work. They use evaporative cooling ("Perspiration passes through it, having cooled the body") but turning the evaporated water back into a liquid - which is condensative warming - requires more energy than was needed to cool the person in the first place.

<comment><USER5> They could work assuming you had both an energy source and a heat dump. For example the ISS could use a dehumidifier to collect sweat and process it because it has large solar panels and can dump heat to space. However, as described they are not going to work.

</comment></comment></comment><comment><USER6> Perhaps all this time spent worrying about getting into the inner sanctum would be better repurposed making cool shit? Making smalltalk with the Director of Design at Facebook isn't an end in itself.

This is one of the things which annoys me most about a certain strain of SF startup culture: too much worrying that there's a holier of holies you haven't managed to get into yet (Davos, seriously?) and not enough time nose down on change-the-world problems.

</comment><comment><USER7> Something I think the OP doesn't get is that this delineation into high and low quality _people_ isn't useful anymore. It was, in a time when books had to be copied by hand and almost everyone was illiterate, and when long-standing reputations (built over centuries, carried through blood) mattered because information traveled at 20 miles per day (if that) but that ended a few centuries ago. Ideas and contributions matter a lot more.

For example, Davos and the Bilderberg Group are relics of a feudal era that humanity is evolving out of. They don't belong in this century.

In fact, one of the most disappointing things about getting rich for a lot of people is realizing that "rich people" actually aren't more interesting, more creative, more intelligent, or even more energetic than people in general. Few people will admit as much, but a lot of the desire behind social climbing is the belief that "better" people hang out behind those closed doors. And yet, in reality the people don't get worse or better as you climb. The average quality stays (perhaps remarkably) the same.

The problem isn't, "How do we keep the hoi polloi out?" The whole point of the internet is that you can't. It's, "How do we keep the average quality of contribution high?" It shouldn't matter if those contributions come from a European prince or an African peasant. It's a big world and there are a lot of mind-bogglingly stupid rich, expensively-educated people, and an equally large number of very intelligent poor people with no formal education.

To prove that "high quality" people can produce low-quality content, just look at Autoadmit or some of the Wall Street-oriented career websites. These are some of the most educated people in the world, and yet the quality of content is very low. Or look at fucking UrbanBaby, which represents the average IQ among the Manhattan upper class as 82 and a third.

<comment><USER8> It's nothing to do with rich or poor, educated or not. It's all about the quality of the contributions. Some people produce consistently high quality contributions, and some people produce consistently low quality contributions. That's what's meant by high quality and low quality people.

</comment></comment><comment><USER9> Facebook is not a community, it's a medium of communication. A sixth of all humanity is not a very useful group for social purposes.

Calling it a community with warrens doesn't seem like a useful description, to me.

<comment><USER10> Isn't Facebook more of a paradoxical phenomenon? You go to Facebook to use Facebook to communicate with people on Facebook. My point being that it's both a community and a medium of communication at the same time?

<comment><USER11> You go on Hacker News to use hacker news to communicate with other users of Hacker News.

This also works for every other website in existence, I think.

<comment><USER9> Hacker news is somewhat more of a community though, because it's a group based on common interests.

Facebook is supposed to be for everyone, so it's not really a group at all.</comment></comment></comment></comment>


RIM: We don't need better products, just marketing - <AUTHOR> http://mobilesyrup.com/2012/01/30/rim-creates-4-cartoon-characters-to-spread-the-be-bold-message/ Wow. Just when you thought RIM couldn't get any more disconnected from reality, they come out with this. Four poorly-drawn cartoon characters to live the BlackBerry lifestyle. Just wow. <comment><AUTHOR> If you doubt that this is official, it's right there on their blog: <http://blogs.blackberry.com/>

</comment><comment><AUTHOR> Their new CEO said they need new marketing to combat their troubles.

</comment><comment><USER2> That's pretty embarrassing.

</comment><comment><USER3> In a way, I actually agree with this statement. Here's my reasoning: RIM's main manufacturing plant is in Waterloo Ontario. They also have a plant in Dallas, Hungary and Mexico. They offer good wages and excellent working conditions. That's more than I can say for Apple and Foxconn.

Just this morning a person in RIM management told me the costs on RIM devices are typically higher because they stay away from unfair labor practices.

That means a lot to me. The humane and ethical treatment of human beings trumps everything, including performance and features.

I have an iPhone 4 (I waited outside a Rogers store all night to be one of the first to have one in my city) but with what I read is going on at Foxconn, I can't support Apple anymore.

<comment><AUTHOR> I suppose you are boycotting everything else made in those factories from other non-Apple brands as well?

<comment><USER3> Is this your subtle and twisted way of rationalizing your conscience away? Will this make the purchase of your next Apple product easier?

Tell yourself anything you want, but you're going to have blood on your hands if you continue to support Apple.</comment></comment></comment>


Show HN: 1 click sharing of interesting stuff on HN with your team - <AUTHOR> http://teamgum.com

<comment><AUTHOR> And ofcourse anything else on web is also shareable with the team & open for discussion.

I am looking for your critical feedback on our work.</comment>


OpenStreetMap for the dyslexic - <AUTHOR> http://www.osm4dys.org/ <comment><USER2> As a dyslexic person, this actually makes things worse. I have a hard time reading caps, I believe it's because the characters have the same vertical positioning so there are less visual cues to go on. A similar thing with serif fonts, I understand that the tails are meant to aid readers, but I just find it as additional noise.

Switching that font to lowercase doesn't help much (didn't see the link at the top of the page initially, not until I went back to the page in fact). That font feels like my monitor is badly in need of calibration. I get that they're adding a horizontal rule to aid readers to follow each line, but it just looks weird! Maybe things would be different if I grew up with it. Plus I've put in tonnes of time and effort reading 'normal' fonts.

Pro tips:

-Line spacing greatly helps things (things get cryptic when it's removed).

-Small fonts make things harder to read (right now my browser is at 150%, have a 43" monitor so why not). I generally increase the font 20% (even when I have no eyesight problems).

-Contrast! Bright coloured/ contrasted backgrounds (such as black on red) make make things much harder to reader to read (I left that typo in, brain was still thinking about harder and read (er... did it again here) at the same time)). Feels as though there's too much info coming in simultaneously. Similar with black on white, but not as bad. That could just be an ADD/ ASD thing (there's a surprising amount of crossover) as I literally hate the white from some LED bulbs (10k+) - it's piercing.

-Avoid overly long paragraphs (tend to get a little lost in the wall of text).

Dyslexia is a funny thing, it affects people in different ways to different degrees (a friend of mine is a bank manager, yet he can barely write a text message). It's taken me a while to develop strategies, such as: I read any any [duplicate word] text I write at least three times, yet I still have errors. Often sound words in my head as I write, and some words I constantly get wrong. Such as 'field', have to remember it as 'fI-elD' (hard i at the start) - Kind of ironic as the secondary school I went to had field in its name (luckily it was embossed on my jumper:D) and now I constantly work with databases. At school my literacy was appalling, at college it was embarrassing (with tutor corrections on every line), wasn't on uni/ after that I started to get a handle on things. In my thirties I still feel I'm improving, people often judge intelligence in people's ability to write. I'm sure some of my teachers would bet good money that I'd never be able to touch-type (although I'd be lost without spellcheck).

If anyone knows a text reader that isn't a pain in the a __(able to read a paragraph or article, and not follow my cursor as I change applications). I 'm all ears! ;)

<comment><USER3> > _If anyone knows a text reader that isn 't a pain in the a (able to read a > paragraph or article, and not follow my cursor as I change applications). > I'm all ears! ;)_

Do you mean a text to speech reader, or a reading tool that helps with dyslexia more generally?

<comment><USER2> Mainly to read articles/ ebooks (I take more in that way), but also to proofread what I've written as things still slip through spell check.

I'm not that bothered about speech to text, as products I've tried in the past often end up more cumbersome than the problem they solve. I might try again as the has been an explosion in machine learning recently, so maybe technology has advanced to be more natural & accurate.

<comment><USER3> For articles/ebooks, I'd put in a plug for BeeLine Reader [1], which I created originally for use as a speed reading tool but learned is very helpful for many people with dyslexia. We are currently doing a couple studies with accessibility researchers to better understand how it helps and for which specific populations. Any feedback would be appreciated!

1: [http://www.beelinereader.com/individual](http://www.beelinereader.com/individual)

You can read Kindle books using our iPad app and browser extensions (via the Kindle Cloud Reader). Both tools also work for news articles or arbitrary websites.

</comment></comment></comment></comment><comment><USER4> > DYSLEXICS FEEL BETTER WHEN: ... SAME COLORS REPRESENT SAME THINGS, FEW > INFORMATION ARE ON THE SCREEN.

I was identified as being "dyslexic" as a child, I had difficulty learning to read - but at some point I had an "aha" moment that I still remember very clearly even though I was very young: _" english is full of completely illogical inconsistent nonsense, it's largely just a memory game combined with a bunch of very subtle rules"_, but of course as a child no one ever explicitly tells you this, I can only assume most people either just grok that or put up with it unconsciously without effort. As soon as I realised this it was much easier to progress and also more generally modulate my natural tendency to look for the _assumed_ presence of logic and consistency in other systems created by humans - basically the "accepting random shit" filter that the majority of people seem to have by default, I don't think I could have carved my own career into the obscurity of computers and technology without that skill that others seem to just have, but it's also handy to maintain my original perspective because it helps me see flaws, weaknesses, inconsistency etc in designs very quickly.

The above list reminded me of this, and although I wouldn't say it's "wrong", I am pretty sure everyone has a preference for those attributes: consistency (same colours representing the same things) and only relevant info on the screen etc - so in short perhaps it's just that dyslexics are more sensitive to bullshit and poor design, but being one doesn't make me an expert.

[EDIT]

Alternate hypothesis: I'm not dyslexic and was misdiagnosed, after all there is a whole multidimensional spectrum of non-neurotypicals and dyslexia sounds like a pretty arbitrary category.

<comment><USER5> >I was identified as being "dyslexic" as a child, I had difficulty learning to read...Alternate hypothesis: I'm not dyslexic and was misdiagnosed... dyslexia sounds like a pretty arbitrary category.

It isn't arbitrary, but the definition is so broad that it is almost meaningless. It isn't even a category of disorders that are grouped by symptoms, they are grouped by how those symptoms manifest themselves when trying to read. Basically you have dyslexia if you have trouble reading or learning to read, have normal intelligence, and don't have any vision issues. Therefore the only real way to be misdiagnosed with it is if there is some other more specific undiagnosed disability that impacts intelligence or if there is some undiagnosed vision problem like not identifying a child needs glasses. It also means it is possible that taking a different approach to learning "cured" you of your dyslexia.

<comment><USER4> > It isn't arbitrary, but the definition is so broad that it is almost > meaningless.

You are absolutely right, "arbitrary" wasn't really what I meant.

> It isn't even a category of disorders that are grouped by symptoms, they are > grouped by how those symptoms manifest themselves when trying to read. > Basically you have dyslexia if you have trouble reading or learning to > read... Therefore the only real way to be misdiagnosed with it is if there > is some other more specific undiagnosed disability that impacts > intelligence...

I've always been deeply suspicious of broad labelling through analysis of fairly non-specific cognitive symptoms. Even with the consideration you mention: excluding other physiological and neurological factors.

Reading the various definitions of different forms of dyslexia now which suggest certain causes - I don't feel like any of them applied to me. I think the failure in these attempts of classifying neuro-type or whatever is the expectation for little variation, and I don't mean in terms of a so-called "spectrum" I mean because the brain is incomprehensibly complex to us, is a composite of all kinds of "skills" that are subtly different in each individual, this is what I mean't by a multidimensional spectrum. It seems to me that even while trying to identify very specific manifestations of a symptom it's going to be easy to mix up completely separate underlying causes.

</comment></comment></comment><comment><USER6> The three most helpful things for me as a dyslexic are;

1\. Highlight the text I'm reading. That text is HORRIFIC when highlighted so I can follow what I'm reading 2\. Make my window thinner so my eyes have less travel distance from left to right. It's harder for my eye to wander to other lines etc 3\. Line spacing! For the same reason as above.

This site doesn't do 3, 1 has overlapping font with highlight making it harder to read and I think does a poor job of 2...

Like this honestly makes me feel like I'm a kid again and my mam is telling me something she read in some stupid magazine that's taking advantage of her desire to be a good mother. She wants to help me but the advice in the magazine about curing dyslexia with Angostura bitters isn't going to work and feels like a cruel joke played by whoever wrote that article... Just like that incident, there was ZERO evidence and the about section of the site has ZERO evidence as to whether any of this works

</comment><comment><USER7> Regarding the font used, OpenDyslexic, there isn't actually any evidence it helps readability (apparently): [https://s-e-o.org/which-font-is-best-for- dyslexic-users-the-...](https://s-e-o.org/which-font-is-best-for-dyslexic- users-the-science-reviewed/)

</comment><comment><USER8> Interesting choice of font. Does the font help with reading it to the dyslexic? For me it makes it actually harder.

(I get that all-caps might help, but that font?)

EDIT: Apart from that, it's a cool idea though

<comment><USER9> Are you sure that it's the font and not the clearly non-native English making it harder to parse? For me the font is ugly but still just as readable, whereas the hallmark signs of an ESL writer were definitely noticable.

Also the abysmally loose kerning and undifferentiated leading make everything run together into a scattered wall of text, which probably doesn't help.

<comment><USER10> I've put up a couple of PRs to fix some of the English and the line spacing. Preview here: [https://rjmunro.github.io/osm-for-the- dyslexic.github.io/](https://rjmunro.github.io/osm-for-the- dyslexic.github.io/)

I hope they merge them as it looks a lot better to my (admittedly non- dyslexic) eyes.

</comment></comment></comment><comment><USER11> Considering the existence of a two-years old thread at [https://www.reddit.com/r/openstreetmap/comments/4i5o1h/osm_f...](https://www.reddit.com/r/openstreetmap/comments/4i5o1h/osm_for_the_dyslexic/) this prototype is at least two-years old.

<comment><AUTHOR> I believe the project is at least 2 years, but the web map at [http://www.osm4dys.org/viewer/](http://www.osm4dys.org/viewer/) is newer. At the very least, I heard about this recently.

</comment></comment><comment><USER12> As someone with severe dyslexia I hate this font so much. It's so damn ugly and it HARDLY works (for me it's < 10% better). If someone could design an elegant front weighed so that an internal drop shadow can help the mind render the character more easily and accurately in 3D space, that would be a god send.

<comment><USER13> Have you tried BeeLine reader?

[http://www.beelinereader.com/individual](http://www.beelinereader.com/individual)

<comment><USER14> Damn this idea is really cool and works for me. I wish I knew this site existed much earlier

<comment><USER3> Fun fact: BeeLine Reader was launched via a Show HN!

[https://news.ycombinator.com/item?id=6335784](https://news.ycombinator.com/item?id=6335784)

</comment></comment></comment></comment><comment><USER15> Hey everyone - I'm working on a book reader that allows you to read text while it annotates using Wavenet.

I have dyslexia and I've found that I can read so quickly with high retention using this method, and having specialized software for it is nicer.

It might help anyone here who suffers from the same thing that I do.

Email me at firstname.lastname@example.com if you want to be an early user!

</comment><comment><USER16> It didn't work zooming in past city-level for me, I'm guessing that's a limitation of the prototype?

The flags were an interesting choice instead of country names. I can see how this could be easier to parse than denser country labels, but it does required knowledge of flags.

Other than that is the main difference just the use of the dyslexic font and a reduction in label density?

<comment><AUTHOR> I think they just have data in Switzerland and Italy. It get's down to neighbourhood level, but not any closer.

</comment></comment><comment><USER9> On Android 7.0 I can't get past the about screen. It says to click somewhere but I've clicked everywhere and nothing happens.

<comment><AUTHOR> try to long-click the icons. I had to do that on my laptop anyhow, it's strange.

</comment></comment><comment><USER17> It's funny how the site misspells "dyslexics" twice.

<comment><USER18> ironic maybe, but quite possibly the author is dyslexic themselves and made the mistake easily? I can say from personal experience its never funny for the author.

</comment></comment><comment><USER19> Nice</comment>


iPad 3 4G Teardown - <AUTHOR> http://www.ifixit.com/Teardown/iPad-3-4G-Teardown/8277/1

<comment><USER2> Hm... the battery numbers are surprising. The battery is 43.5Wh, which is huge. Over 9.5 hours us "use", that comes to 4.6W average. My Dell XPS 15 L501x (very much not a "light" machine!) has a 9-cell 91Wh battery and got about 5 hours continuous on it when it was new, for about 18.2W.

So the iPad 3 is drawing 25% the current of a beefy 2.9GHz Arrandale laptop? That's shockingly high, IMHO, especially considering that the screen area of the 15.6" laptop is twice that of the tablet.

Similar, the numbers they show aren't much higher than the range (30-50Wh, 6-9 hours) already achieved by a bunch of netbooks on the market.

<comment><USER3> With LED backlighting, isn't most of the screen's power draw driven by the switching? And the iPad has a lot of pixels to switch--almost certainly more than your Dell laptop.

<comment><USER2> Honestly I'm no expert. But I'd find it very surprising if there was much difference at the same lumens output. The backlight is not per-pixel in a LED- backlit display (maybe you're thinking of OLED displays?), though I understand it's often per-region to allow large dark areas to be under-lit.

And note that LED backlights are routinely put up as having overall _lower_ power consumption than CCFL panels like the one in my laptop.

Really my point is this: that's a laptop-sized battery in that thing, and it's seeing only slightly super-laptop power draw.

<comment><USER4> Your point misses the mark: with the 1/2 of the battery of your notebook iPad moves 50% more pixels for twice as long:

It's 2 * 1.5 * 2 = 6 times more efficient than your notebook. Of course, usage patterns are also different, but it's still wonderful piece of engineering.

<comment><USER2> More efficient only in your oddball unit of pixels-joules/second. I'm not interested in arguing about features or the design tradeoffs of an Apple device, that always leads to a platform flame just like this one.

I'm saying that the _actual power draw of the iPad_ is much, much closer to that of a laptop than most of us expected.

</comment></comment></comment></comment></comment><comment><USER5> And 1GB of ram confirmed. Apple's policy of not mentioning the ram made sense to me when they started, because it did seem like a amount of ram in the initial model (256mb). But they've doubled the ram with each new generation and made no mention of it. I guess apple has just made the calculation that 1) it doesn't matter to most consumers (the ipad is supposed to be magic), and 2) they'll be outspeced by the competition over the lifetime of the model, anyway.

<comment><USER6> The reason is because they view it as a consumer electronics device. How much RAM is in your DVD player?

<comment><USER7> RAM on a tablet affects performance in a very straightforward way. Why mention any specs if it's just a "consumer electronics device"?

I think the reason they don't mention RAM is because they don't want to get in a silly numbers based arms race. They want to say "here's the whole package, it's pretty great, and better than the other guy's stuff, even if some of his specs are a little better on paper".

</comment></comment></comment><comment><USER8> Wow, beautiful photographs. They always do a great job.

Has anyone looked closer at the A5X package in these pictures? Is that a glimpse of PC board under that can? Is there some kind of multi-die package (and not just POP)?

<comment><USER9> Although the heat spreader is oddly shaped, the A5X isn't big enough to justify more than one die. I wonder why they unstacked the RAM; I would imagine either for thermal reasons or for more memory bandwidth.

<comment><USER8> They added a picture of the chip with the heat spreader off. It's one massive die.

<http://guide-images.ifixit.net/igi/i6ZlJCQfdaHyktRY.medium>

<comment><USER9> Anand estimates 117 sq. mm. which seems reasonable to me. [http://www.anandtech.com/show/5681/apples-a5x-die-and- size-r...](http://www.anandtech.com/show/5681/apples-a5x-die-and-size- revealed)

<comment><USER8> The RAM is on the opposite side of the board. Go figure.

</comment></comment></comment></comment></comment><comment><USER10> I love these teardowns - so superfluous, and yet so compelling nonetheless.

<comment><USER11> The thing that I love about these teardowns, and which really stands out to me, is how well designed Apple's products are internally as well as externally. If you compare the teardowns between the iPad and the HP Touchpad ( [http://www.techrepublic.com/photos/cracking-open-the-hp- touc...](http://www.techrepublic.com/photos/cracking-open-the-hp- touchpad/6253940?seq=19&tag=thumbnail-view-selector;get-photo-roto) ), it's pretty clear why Apple has such a lead in tablet manufacturing efficiency and cost.

<comment><USER12> The reasons for Apple's leadership in battery life become clear, as well.

</comment></comment></comment><comment><USER13> I want to know if they successfully got it back together.

</comment><comment><USER14> From page 3 Step 18:

"Like the A5, the A5X system features a 1 GHz dual-core CPU. The upgrade that earns it an 'X' is the new GPU, which Apple claims outperforms even Nvidia's Tegra 3 processor."

Considering that even last year's A5 outperforms Tegra 3 on many benchmarks casting aspersions here seems meritless. You might quibble with Apple's 4x claim but that is not how this reads.

</comment><comment><USER15> oh my god, they're doing these in real-time, and they're server is suffering under the load too.

</comment><comment><USER14> Anand, estimating by die size, says A5X looks like 28/32 nm.

[http://www.anandtech.com/show/5681/apples-a5x-die-and- size-r...](http://www.anandtech.com/show/5681/apples-a5x-die-and-size- revealed)

Qualcomm's chip is confirmed as still being the 45 nm model.

[http://www.macrumors.com/2012/03/08/new-ipad-appears-to- util...](http://www.macrumors.com/2012/03/08/new-ipad-appears-to-utilize- qualcomms-mdm9600-baseband-chip/)

So for next year's iPad we can expect a tock architechture update (to A-15) for the SoC and a tick process shrink for the baseband chip. Not sure how many batteries they'll be able to throw out but hopefully some and it'll be thin again.

</comment><comment><USER16> Did anyone see how many WAN antennas it has?

<comment><USER17> Looks like 3 according to this picture: <http://guide- images.ifixit.net/igi/MIEhlvVuGWrPGUQw.large>

</comment></comment><comment><AUTHOR> "There isn't much difference between the old and new version of the A5: an "x" in the name and an extra 0.2 GHz clock speed." -- <http://www.ifixit.com/Teardown/iPad-3-4G-Teardown/8277/2>

<comment><USER18> GPU within it went from dual to quad-core. Maybe it also upgraded from SGX543 to SGX544 or SGX554. <http://en.wikipedia.org/wiki/PowerVR#Series_5XT> Hard to teardown a SoC.

<comment><USER19> Hard _er_ , to be sure, but definitely possible.

See [http://www.lps.umd.edu/MicroelectronicsIntegration/Microelec...](http://www.lps.umd.edu/MicroelectronicsIntegration/MicroelectronicsCOTS.html) and <http://www.break-ic.com/topics/break-ic.asp> for 2 overviews.

There is an entire industry built around decapping and reverse-engineering ICs, usually to make cheap clones in China or somewhere else that'll turn a blind eye to IP infringement.

There's also 'Dr. Decapitator' of ROM extraction fame: <http://decap.mameworld.info/>

Annoyingly, I can't find the article I remember reading, which was a look at the debugging / test labs at AMD or Intel. They had all manner of depackaging, imaging, probing, and even in-situ modification tools. They could cut traces with incredibly precise laser pulses, and use bond-wires or additional metal deposits to test faulty designs. Incredible stuff (and a nightmare for anyone who wants "tamper resistance")

</comment></comment></comment><comment><USER20> Question, the Samsung LCD display they linked to shows a 1024x768, not the 2048 resolution of the iPad, what gives?

<comment><USER21> I think they mean the model number, LTN097QL01-A02, is a samsung model number. The 1024x768 lcd in the linked document has model number LTN097XL01-A01.</comment></comment>


Little Data: How do we query personal data? - <AUTHOR> https://www.petekeen.net/little-data

<comment><USER2> Imagine if you had a heart-rate monitor and a sleep monitor.

The heart rate monitor would know when you spiked yourself with caffeine, combined with the sleep monitor that knows you've only slept 4 hours the past 3 nights.

An alert pops up on your computer: "hey, you need rest. Stop drinking so much caffeine. Eat an apple and take a 20 minute nap, you'll feel better."

Those are just two data sources.

Harvest (time tracking) + RescueTime (productivity) + Runkeeper would be super valuable too. You'd know that you're 10% more productive on days that you run.

<comment><USER3> Better to have a fridge monitor that pops up when you take the Red Bull out of the fridge and before you drink it. (Or coffee machine...or phone with GPS that recognises that you just entered starbucks...the trick is it should pre- empt the caffeine to affect your behaviour in the moment).

</comment></comment><comment><USER4> I'm working on some software to record as much data as possible, from as many sources as possible, into one unified form so that you can actually analyze it.

I want the data sources to include far more than just health data--GPS, title of active computer window, sms history, file modification times, device and sensor readings, snapshots from your webcam, etc--really anything that you can record.

If anyone is interested, feel free to email me!

<comment><USER5> What's the unified form?

<comment><USER4> I thought about it for a while, and this is the best I could come up with:

\- All data is just a stream of "events" from different "sources" for any given "data type"

\- Sources are things like my desktop, my laptop, my phone, my camera, etc

\- Data Types are things like global position, cpu usage, webcam snapshots, webpage text, etc. Anything really.

\- Events are just a timestamp + zero or more other "lightly typed" "fields".

\- By "lightly typed" I mean the field is either: binary data, searchable text, a float, or an integer

\- Fields give the actual data for the event. If there are no fields, it's a countable event (ex: imagine recording every heartbeat, all you need is a timestamp). For something like global position, you would have latitude and longitude fields. For another datatype called "semantic location" you have just a searchable text string with things like "car", "work", "home" or whatever other labels you have. Fields can also be binary data (like photos, videos, etc). I'll probably try to make lots of data types, each with very few fields, to make it easier to work with the data (ie, it's nice if there is only one numeric field in any given data type, then it's easy to graph over time)

I'm currently collecting all of the events via some scripts that are specific to each data type, and then emailing those events (serialized as json) to my own server, which just inserts the events into a sqlite database specific to that source/data type pair.

The nice thing about this approach is that anyone can easily send an email from any language, and if you don't want to use my backend, you can just send the events to your own email server and then do whatever you want with them once they're there.

Another benefit is that you and I don't have to agree about exactly what fields and data types there are. If you call your gps data "gps" and I call mine "global position", well, who cares? It's our own personal data. If this takes off, I'll make guidelines about what to call what information and how to format it for easier interoperability.

</comment></comment></comment><comment><USER6> We are working on solving some of this, specifically for health data, at [http://humanapi.co](http://humanapi.co) \- you can connect your various data sources, and we then give you a central personalized API you can query.

(full disclosure, I am the founder of this company)

<comment><USER2> Cool! Is this going to be available to consumers in general?

edit: fwiw, I understand if it won't be.

<comment><USER7> If not, you can (eventually) use Personable.me, but I'm still in the process of integrating enough sources that one could realistically use for health analysis.

<comment><USER2> That's cool. Are you building it?

<comment><USER7> Sorry, yes, I should have qualified that. Yes, I'm building that. It's only about a week's worth of development thus far, but I've got a fitbit and a WiiThings scale on the way to start integrating those into the API as well.

</comment></comment></comment></comment></comment><comment><USER8> My startup is working on this, analysing and making sense of personal data, check it out: [http://sympho.me](http://sympho.me)

It works by connecting to services you might all ready use etc.

(full disclosure, I am the Co-founder of this company)

<comment><USER9> Your product looks like it will be a neat service. Do you intend to also have an API to contribute data (in addition to your "import" tool?)

<comment><USER8> Thank you!

We have that planed down the pipeline:D

</comment></comment></comment><comment><USER10> [http://chai.it.usyd.edu.au/Projects/Personis](http://chai.it.usyd.edu.au/Projects/Personis)

Project Description

Personis supports an accretion/resolution approach to reasoning about people, places and devices. It was designed to support user control of both the information held about them and the way that it is used.

also [https://github.com/jbu/personis](https://github.com/jbu/personis)

</comment><comment><USER5> _In computing, linked data (often capitalized as Linked Data) describes a method of publishing structured data so that it can be interlinked and become more useful._

[http://en.wikipedia.org/wiki/Linked_data](http://en.wikipedia.org/wiki/Linked_data)

<comment><AUTHOR> Could you extrapolate a little? Maybe include some links?

<comment><USER5> It's the concrete implementation of the Semantic Web - using a common model (RDF[1]), to link different data sources / pools and query them in a standard way, using SPARQL and similar technologies, and even perform automated reasoning to discover new facts about it.

[1] [http://www.rdfabout.com/intro/](http://www.rdfabout.com/intro/)

<comment><AUTHOR> Sounds pretty cool. Most of the data sources that I'm interested are not interested in publishing their data using RDF, but maybe it would be useful to have RDF be a result of an ETL process.

<comment><USER5> Yeah, there are many tools already that can help you import different data formats into an RDF datastore, such as Virtuoso: [http://www.openlinksw.com/dataspace/doc/dav/wiki/Main/](http://www.openlinksw.com/dataspace/doc/dav/wiki/Main/)

</comment></comment></comment></comment></comment><comment><USER11> IPython Notebook is my go-to environment for munging small to medium data. Admittedly, it's not for non-programmers, but for personal hacks and hobby projects, it's great having Pandas, SciPy, scikit-learn, etc. at my finger tips in the browser.

</comment><comment><USER12> The problem is that companies will always have more data and can do more with that data than an individual.

</comment><comment><USER13> How do I collect all my personal data? (without a being a major pain in the ass)

<comment><AUTHOR> That's one of the big barriers to this actually becoming more than just a thought experiment. Lots of interesting data sources are locked up with either no export capability at all or no way to automate exports. For the power company example, the only way to get that data out is to login to their website and follow a convoluted path to a "download" button which you have to click twice for some reason.

<comment><USER14> While all auto is cool, it doesn't seem like having to download data once/month would be all too terrible, and it _certainly_ shouldn't stop you from the larger project. You could have the system prompt you to do it.

Like balancing your checkbook once/month. (You see, we used to have these little books called "registers," and ...)

</comment></comment></comment><comment><USER15> Why not use a backend as a service company for your "data soup"?

<comment><AUTHOR> I'd like to self-host it if possible. Letting a 3rd party possess all of that information at the same time seems risky to me.

<comment><USER16> Yahoo Pipes can be used for normalisation, but I don't know any datamining program for little datas...

</comment></comment></comment><comment><USER7> Warning: Shameless plug for project that isn't finished

This, among other things, is the reason I started building [http://personable.me](http://personable.me).

Ultimately, I see it as being the recepticle for WiiThings data, Fitbit data, etc., and I'm working on those now. I just finished up (but haven't deployed) Webhooks, with the idea that input + webhooks for output should allow a personal API to interact with other systems when API data is updated.

<comment><USER17> Nice idea. What worries me is having to log into what looks like a cloudy service which I imagine stores my personal data, accessed using other cloud credentials. I get that cloud is where the burgers are good at the moment, but my personal preference is for something I can put onto my own server. At home.

To that end I've been working on my own analysis tool for personal data. Most of the work has been on a Windows desktop app, and data import is limited to CSV. But data is synchronised to my phone.

[https://www.wittenburg.co.uk/Entry.aspx?id=0a505400-5bf6-4a6...](https://www.wittenburg.co.uk/Entry.aspx?id=0a505400-5bf6-4a6d-b107-6b4b797f33ae)

<comment><USER7> So, the source is open, if that helps... or, at least was, and will be again. I closed it down temporarily while I separate the 'Personable.com' source from the 'Personable' source, but soon, you'd be able to clone the repo and stand up your own instance of it and host the data yourself, if you preferred.

Interact looks neat, but I suppose that this has a slightly different objective. To me, for Personable, the aim is that the API interacts with other services you use. Perhaps for the purpose of making your life easier (e.g., enter a personal API endpoint instead of having to fill out a profile, and let the website read in the data you let them access), perhaps for making the lives of others easier (e.g., your friends want to know where you're at, but don't want to check Facebook, Foursquare and Twitter to see which checkin is the most current), but all of that interaction depends on at least some of that data being net-accessible.

I haven't gotten into what should be public vs. what should be private yet, so, thus far, the advice for Personable is to just treat it like you should treat every other cloud service, and not put anything there you wouldn't want the world to read, but privacy options are forthcoming as well.

<comment><USER17> That's fair enough and adds good perspective. Will keep an eye on your progress. Interestingly, open-sourcing Interact is still doing my head in. If I ever finish it I'd like to make some money from it. But at the same time I understand that I get zero security cred for closed source. Happily I can delay that decision for a while...

<comment><USER7> If all goes well, I see Personable as being a hosted API provider for those who don't want to bother with hosting it themselves, with the source code otherwise available for those who don't mind, or don't want to relinquish control -- similar to how Wordpress operates currently.

There's no inherent reason you can't profit from an open source service, especially if you're able to provide value to those willing to pay for the convenience of not having to run the service themselves, and/or those unable to.

Good luck either way. Closed source may not be the death knell you expect.

</comment></comment></comment></comment></comment><comment><USER18> I like the way this guy thinks :)</comment>


Show HN: CubeChat – Party in 3D - <AUTHOR> https://cubechat.io/ <comment><AUTHOR> After months of development, CubeChat is now available for public sign-up!

CubeChat is a web app for partying in 3D You are a cube with webcam video, and you can jump around and talk with other cubes like at an in-person party! Audio is spatial and drops off with distance.

Hopefully CubeChat can help add more fun to work and social life, especially with people being remote these days!

<comment><USER2> I like it, really neat idea!</comment></comment>