The dark side of NoSQL - <AUTHOR> http://codemonkeyism.com/dark-side-nosql/ There is a dark side to most of the current NoSQL databases. People rarely talk about it. They talk about performance, about how easy schemaless databases are to use. About nice APIs. They are mostly developers and not operation and system administrators. No-one asks those. But it's there where rubber hits the road. <comment><USER2> I've been migrating data from a NoSQL database into PostgreSQL for the past few weeks. Now data occupies about half the size on disk. And queries are up by a factor of 100. Especially the advanced ones. Note however, that I have a pretty good understanding of SQL, so I know how to optimize the queries.

One powerful aspect of SQL is that the database _can_ optimize the query. Some NoSQL databases enforces you to write joins manually in code. Unless you really know what you are doing and spend hours on it, the query will be slow. This is exactly what I see with many of our queries. Relational databases is not the answer to everything, but I would prefer to start with one rather than a NoSQL variant. You can always migrate when things begin to get too slow (ie, when you get mad success - which you probably never do anyway)

<comment><USER3> _I've been migrating data from a NoSQL database_

That's like saying "I live in a capitalist nation and capitalism is not what it's cracked up to be."

NoSQL is an ideology, and anyone you hear who uses that term is an ideologue. Please name the "NoSQL" you have been using and name the specific problems you have had with it so we can discuss matters more concretely.

Just as RDBMS users prefer one above all else, so do "NoSQL" users. I have seen my fair share of them and I thought a good chunk of them where "crap" (where crap means "does not fit my mental model of the solution"; mainly because I am too naive to study the field better and appreciate what the tools are capable of, so I dismiss anything that doesn't entirely model CL's Elephant.)

Last week I rolled out my own Memcached "replacement" in Common Lisp and I was spitting in bradfitz's general direction, that looney bloat monger! Yeah, my "memcached" replacement is an in-process memoizer for all the DB accessing functions. It worked until the server instance was overloaded and the load- balancer forked a second instance: then I had a problem. I wish I could suck- back all the spit I have sent in brad's general direction.

<comment><USER4> > NoSQL is an ideology, and anyone you hear who uses that term is an > ideologue.

You're sounding a little ideologue-ish yourself. :)

NoSQL is a term of limited use since there is such a huge variety of non- relational database technologies, but that doesn't mean using it makes you an ideologue any more than someone using the term "dynamic languages," which is similarly broad.

<comment><USER5> ...and similarly naive / nondescriptive / misfocused

<comment><USER3> btw, it's ironic that the author of the story was complaining about the lack of querying language, when, in fact, "NoSQL" stands for "No Structured Query Language" :-D

</comment></comment></comment></comment></comment><comment><USER6> I've had an unusual problem with couchdb. A bad query can flatten your host. The engine uses every bit of performance in pursuit of its answer, which might be the workload equivalent of a series of stupid open joins in SQL. Your bash consoles become unresponsive. The same mistake with a traditional relational database causes less damage because those databases are less effective at harvesting the power of the host.

There's a training obstacle. I know how SQL works and how to make SQL perform well almost by instinct. NoSQL requires me to relearn data querying. The lack of standardisation across offerings means that the knowledge I acquire will be less transferable what I've accumulated from Oracle/Postgresql/Openbase/Mysql/Sqlite.

<comment><USER7> You should think of CouchDB as the Web2.0 equivalent of Microsoft Access. It's good for running a simple web-app off, because you can store the HTML, JS and the data all in CouchDB (I think). So one use-case may be a _simple_ billing system for your local surf shop that is sure as hell not going to store a lot of data or handle a lot of load. I got this impression from a Damien Katz talk. (I think the use-case they use is a recipe app.)

<comment><USER8> Well with the load balancer, and continuous replication you could theoretically scale to any level easily.

<comment><USER4> No. CouchDB doesn't support partitioning, so all your replicas need to be able to handle the full write volume.

<comment><USER9> It will soon. We have an 8-node cluster running in PROD right now. A Dynamo- like layer provides consistent hashing, partitioning, read/write quorum constants and the like.

</comment></comment></comment></comment></comment><comment><USER10> One of the complaints that people have about the NoSQL label is that it attempts collect a huge variety of databases that don't fit the traditional label under one name. The problem with that is illustrated in this article, which does a great job of describing problems that some of the NoSQL alternatives experience – but the problems here don't apply nearly as strongly to document-oriented databases, or to graph databases. I think the title's overstating the argument, in other words.

Now if it were "the dark side of column-oriented databases," Bob's your uncle.

</comment><comment><USER11> This are real problems. With Redis I tried to mitigate some of this problems by providing an API that allows to retrieve the full state of the database just using available commands (without having to mess with the binary dump) including for instance reading the TTL of an expiring key. This allows to create scripts that are able to convert a full database into a CSV file, and restore back the CSV file into the DB, just using the net API.

Of course with key/value stores changing the structure of the database is harder compared to SQL databases, still designing a good data layout when building your application can help a lot to make it simple to add features to the application without to resort to scripts that rebuild the data in the key/value store.

<comment><USER12> i agree with data export point. @antirez if we could clone you and distribute to other KV projects then all KV datastores skyrocket in popularity. first and foremost issue in KV popularity is ability to try quickly. if there would be a small php script that converts an sql dump into a KV datastore (and vice versa) then much more people will be trying and they know that if it wouldnt work out they can easily opt out. i hope you'll continue to be thoughtful for others who are not C++ or Java wizards. thanks!

</comment></comment><comment><USER7> I'm one of the lead programmers of Keyspace, which is a key-value store. The main strength of Keyspace is the consistent replication, but right now I'll tell you what my experiences are building a web app (real-time search) using Keyspace. In other words, what happened when I drank my own kool-aid.

1\. Schemaless. This is a half-truth. In reality there is always some kind of schema, otherwise you'd get chaos. What works for us in Keyspace, where everything is a string, is to use "/" as a separator convention. So we have keys like: "/user/id:001" => <data>

More complicated example:

"/user/id:001/view/keyword:keyspace/"

2\. Listing. Since listing happens on a prefix basis in Keyspace (give me everything that starts with "/user/id:001"), if you just want to get a list of some properties of the user, you have to create the list explicitly, otherwise you'll get all the keys under the user. Eg.

"/user/id:001/views/list/keyword:scalien" => null "/user/id:001/views/list/keyword:keyspace" => null

3\. If you need to list by property X, you need to create a subtree for it, as above. This sucks if you figure this out at a later point, because you have to programatically parse your existing data and create it, besides adding it to the online code for new items. That's why RDBMSs are so good.

4\. Backups. No good backup strategy at this point, hot or cold.

..

So why should anyone put up with this? You have to realize there's lots of hype right now, in reality KV stores should only be used by a small fraction of DB users. If an RDBMS is Java/C++, then a KV store is (can be) Assembly. You have much more control over what's going on. (You can of course get the same control by having a giant key-value table in your RDBMS.) You can put a cache layer in your stack more easily.

So you should think of a good KV store as something that you use under certain hi-performance cases, and even then, the KV store should offer some plus over having a giant key-value table in My/Postresql. Usually, that plus is some kind of distribution or replication that usually works. In case of Keyspace it's replication.

The other big plus is, as I learned, is that programming in the KV model is _much much more_ natural then having embedded SQL statements in your code. A typical segment in my (Python) code is:

users = conn.listkeys("/user/id:%s/views/" % user_id) for (...):

This is much nicer then having SQL statements, and since you're using a KV store, it can't get more complicated than this. As I said, I assume you're doing hi-performance stuff, in which case some crappy ORM layer is out of the question anyway.

<comment><USER13> Have you made, or you aware of, any comparisons between a KV store such as this and the same thing implemented as an IOT in Oracle?

<comment><USER7> No and no.

Here's a link about IOT for others:

[http://www.oracle.com/technology/products/oracle9i/datasheet...](http://www.oracle.com/technology/products/oracle9i/datasheets/iots/iot_ds.html)

Personally I think there's a fairly large cultural disconnect between shops running open-source stacks and shops running Oracle, so such a comparison would not influence user choices much. But it would be very interesting for an implementor like me: if Oracle is faster, which is likely at this point, it would set a performance point to beat.

As a final fun fact, Keyspace is currently running on top of BerkeleyDB, which is an Oracle property =) But not for much longer, I'm very unsatisfied with BDB, it doesn't live up to its image of being an industrial strength btree store (it sucks).

<comment><USER14> What do you dislike about Berkeley DB? I've been using it for a little while for a pet project, and it seems decent. I'd like it to support range-based join cursors, but that's not a major complaint. Granted, I haven't yet had to deal with database recovery, so I don't know how it stacks up there.

Also --- what do you plan to replace it with? I know that AllegroCache started out based on BDB, and ended up not using it. The Franz guys said performance wasn't up to par, and I always wished for more details.

</comment></comment></comment></comment><comment><USER15> "NoSQL" is a completely meaningless expression. Please stop using it.

<comment><USER16> It's just a marketing/catchy term for the trend to 'next-generation' (read: 'different') database engines. Since there is (in almost all cases) no SQL (the language) involved, it is a pretty appropriate name.

<comment><USER15> There's a reason that Java is not called "NoPHP". While technically true, it's just not very descriptive or meaningful.

<comment><USER4> A better analogy IMO is Python, Ruby et al calling themselves "dynamic languages" to distinguish themselves from the popular static languages of the time.

<comment><USER17> But they called themselves dynamic, not "NoC++" or "NoJava". That is, it was described by its intrinsic properties instead of as not "a specific instance of a something that implements what they don't like."

SQL is a way to interact with a relational database, but it is not itself a database. But "NoRDBMS" still isn't as good as saying what the technology is - there are an infinite number of things that are not relational databases.

</comment></comment></comment></comment></comment><comment><AUTHOR> From what I can tell, it seems the "dark side" mentioned only really affects Voldemort and Cassandra, out of all the databases that fall into the "NoSQL" category?

<comment><USER4> Ad-hoc querying affects all distributed databases, which includes dynomite, hbase, and hypertable as well as voldemort and cassandra. Also all key- oriented databases, which include Tokyo Tyrant and redis. And some others like CouchDB (where computing a new view via map-reduce is basically equivalent to doing a query via hadoop against another system, so I don't consider that real ad-hoc query support).

Really the list of databases that do provide ad-hoc queries is much shorter. Neo4j (graph db), mongodb (single-master "document-oriented" db), the xml databases, probably some other minor projects.

TFA also mentions data export as a "dark side" but for projects that lack this it's really just a "nobody's needed it enough to write it yet" thing. So that's less interesting IMO than talking about ad hoc queries which is usually more of an architectural limitation.

<comment><USER18> Well, ad-hoc querying _can_ be implemented in Hypertable with HQL, it's just not a priority at the moment, so features are a bit lacking. I mean, it's not an inherent 'darkside' of distributed database. OTOH, I've done cap shell with HQL and grep/sort/whatnot a few times on a few TB of data and not exactly painful :)

</comment></comment></comment><comment><USER19> These have nothing to do with the concept, just the current state of the tools. There is no reason that a non-relational DB without a declarative query language can't have good support for data dumps, ad hoc queries and ad hoc "data fixing".

</comment><comment><USER20> Backups are a particularly serious issue for distributed variants.

<comment><USER4> Do you mean because they offer poor support, or because there's usually large volumes of data involved, or both?

Cassandra, for instance, provides an easy snapshot API, and you can back up the snapshots using whatever infrastructure you'd normally use to back up TB of data. (If you don't have such an infrastructure, yeah, that's a problem, but not Cassandra's fault. :)

Edit: I should point out that since Cassandra supports multiple data center replication already, I'm having trouble picturing a scenario where you want to do anything w/ the snapshots besides just leave them on the Cassandra nodes themselves (plan extra HDD capacity as necessary depending on how long you want to keep them). But some such scenarios probably do exist.

<comment><USER18> Replication != backup, if you/someone accidentally/maliciously screw up your data with the API, the screw ups are replicated as well, especially with Cassandra, where there is no builtin data versioning.

Backups are essential when you're doing major upgrades or data migrations/mangling that can fail.

OTOH, data export is not an inherent problem of distributed database either. If enough people want the feature, they can be built without too much fuss.

<comment><USER4> > Replication != backup

Obviously. The context was using snapshots as backups -- you can keep them around indefinitely, space permitting, and if you're using Cassandra's multi- datacenter features they're automatically "remote" as well.

> especially with Cassandra, where there is no builtin data versioning

At the risk of belaboring the obvious, versioning != backup, either. :)

</comment></comment></comment></comment><comment><USER21> It seemed to me like the article was more about particular "NoSQL" databases than it was about the genre as a whole. In other words, I don't think the author's pain is endemic to this admittedly broad class of data stores, just the few that do exist.

<comment><USER11> The "missing alter table" problem affects more or less any kind of schema-free DB.

<comment><USER22> How so. In most schema-free db's the missing alter table feature is not needed. maybe you meant the quotes to be in a different place? missing "alter table problem" perhaps?

<comment><USER18> With completely schema-free db, you have to encode 'column' in a key, which means you cannot delete or rename a column easily (in O(1) time).

<comment><USER23> You won't get O(1) since scheme-free dbs are not organized around columns/fields in the same way a traditional db is all about rows and columns, but just about every one I am aware of let's you create an index on a particular field so at least you get O(n). Given the infrequent use of this operation I am happy to leave it in the "O(n) is good enough" category in return for the other benefits an efficiencies that a schema-free db offer.

<comment><USER18> Well, you can get both with Hypertable, where you can alter table to add/drop column families in O(1) time, while still have schema free benefits with column qualifiers.

</comment></comment></comment></comment></comment></comment><comment><USER24> Hmm... there are different query APIs to all the NoSQL variants?

What stops you from doing a one-liner from shell (or a short throw-away program) instead of a SQL query/update? (Well, except for the power of SQL compared to the APIs, of course.)

Disclaimer: I don't know enough about these newfangled things and their tradeoffs to have a serious opinion. They do look intriguing. :-)</comment>


Explaining the problem under that "Wikipedia sized proof" – with a puzzle - <AUTHOR> http://www.youtube.com/watch?v=pFHsrCNtJu4&feature=youtu.be&a <comment><AUTHOR> The article about the "Wikipedia sized proof too big for humans to understand" was posted several times - here's the one that got most of the discussion:

[https://news.ycombinator.com/item?id=7264886](https://news.ycombinator.com/item?id=7264886)

Unfortunately, there seem to be many misconceptions about the whole thing. This video explains the problem amazingly clearly. For most people it's a subtle problem to get your head around, but this presentation seems to make it almost obvious.

What do you think - is this clear?</comment>


Ask HN: How do you reward yourself? - <AUTHOR>

<comment><USER2> food</comment>


First thoughts on Node.js - <AUTHOR> http://peachshake.com/2011/05/13/first-thoughts-on-node-js/ <comment><AUTHOR> A few of the links use our new project, OilyApe. Click and enjoy ;)</comment>


Full text: An epic Bill Gates e-mail rant - <AUTHOR> http://blog.seattlepi.nwsource.com/microsoft/archives/141821.asp <comment><USER2> Why is this surprising? Bill Gates doesn't need to fanboi Microsoft internally; it's _his_ company. He needs to be as critical as possible so the outside world gets a good product. It's _his_ company that looks bad when things like this happen, so of course he is going to "rant" about it. It's his job.

<comment><USER3> It's surprising because you might think Bill Gates would have some pull at his own company. Most of these issues he raises have been problems for years, and have gotten even worse in some cases!

<comment><USER4> Many of these problems are still prevalent in Microsoft. Confusing processes and the like.

So, it's not Gates. Gates is a good guy and he sounds like he gets it. Is it the corporate structure then?

<comment><USER5> When Microsoft started out it operated under a pretty flat organizational structure, similar to how Google is run. They've gotten fat in the middle (management) over the years and it's stifled any chance they've had of getting anything done.

See this blog post from a former MS employee about the year it took to design the shutdown menu in Vista: [http://moishelettvin.blogspot.com/2006/11/windows- shutdown-c...](http://moishelettvin.blogspot.com/2006/11/windows-shutdown- crapfest.html)

<comment><USER6> Interesting read.

</comment></comment></comment></comment></comment><comment><USER7> This reminds me of the bug report Linus Torvalds submitted when he couldn't figure out how to get youtube to work on linux.

<https://bugzilla.redhat.com/show_bug.cgi?id=439858>

</comment><comment><USER8> It makes me nervous that I didn't see it as a rant as all. Am I that callous?

To me, it looks like something constructive that every CEO should do -- spend time dealing with their product like their customers do, and make sure the troops know when it isn't up to standards.

</comment><comment><USER9> My favorite lines:

"Someone decided to trash the one part of Windows that was usable? The file system is no longer usable. The registry is not usable. This program listing was one sane place but now it is all crapped up.

But that is just the start of the crap. Later I have listed things like Windows XP Hotfix see Q329048 for more information. What is Q329048? Why are these series of patches listed here? Some of the patches just things like Q810655 instead of saying see Q329048 for more information.

What an absolute mess."

<comment><USER10> I also liked that one. But it also reminded me that I often use synaptic on Debian when I am looking for applications on my system which is rather similar to the "add/remove software" on Windows. The only place where I can do a sane search (including searching summaries what the applications actually do) and get an additional good categorization of the software. Funny how the package manager on both sytems seems to have better information about your applications than for example the typical startup menu (which is usually a mess after a few months - also on both systems).

</comment></comment><comment><USER11> "they told me that using the download page to download something was not something they anticipated."

And rightly so. Anyone who tries to use the download page to download something will give up and never try it again, so after a while everyone will ignore it!

<comment><USER12> Your logic is stunning raganwald... stunning..

I'm rolling on the floor...

<comment><USER13> How do you type while doing that?

<comment><USER14> Voice recognition?

<comment><USER15> Full-body keyboard.

</comment></comment></comment></comment></comment><comment><USER4> Looks like Bill Gates knows how to communicate his thoughts easily for the reader.

Neither I _nor_ the recipient of the email would have read that entire thing if it was in a large paragraph block.

<comment><USER16> I agree, it was a nice user story.

But, I'm damn sure that all the recipients would have carefully read every word of that email, even if it had been written in one large leetspeek paragraph.

</comment></comment><comment><USER17> And that's how a guy with "no taste" critiques. Imagine Steve Jobs' emails!

<comment><USER18> Steve Jobs said _Microsoft_ has no taste, not Bill Gates. I think that if you're a good software developer (and BillG certainly qualifies), you need to at least have some degree of taste.

<comment><USER19> It's pretty clear that BillG has business, management, and technical acumen, but I'm not so sure that he has taste in the aesthetic sense. If he does have good aesthetic taste, he hasn't done a very good job of implementing it.

<comment><USER20> Actually I consider some of the new Microsoft designs to be pretty nice:

XBox 360, Silverlight, Live Maps, even Vista, are at least comparably pretty to the competition, and some would vie, even prettier.

<comment><USER11> With respect, I do hope you realize that "taste" and "design" are about an awful lot more than "pretty."

I would say that MSFT products being "pretty" demonstrates how little they get it. They think that making something "designed" means smearing pretty on it after you're done, as if you can design a web application by building something randomly then hiring a graphic designer to make the page slook nice just before you go live.

<comment><USER20> Okay. But throwing out Vista and Silverlight, I contend that the XBox 360, Live Maps are both highly usable, well designed products that people want. This may be true of Silverlight too, if they ever wake up and realize that good webapp, flash and UI designers/developers tend away from Windows.

<comment><USER14> I found the Xbox360 UI to be atrocious, actually. Why does the green X button turn the system on and off? Why are there 6 different versions of the Marketplace, none of them consistent? Why are there ads _everywhere_?

I enjoy the games themselves but the design seems extremely ad-hoc.

</comment></comment></comment></comment></comment></comment></comment><comment><USER21> The difference between Apple and Microsoft is that this rant would come from Steve Jobs _long_ before the products were actually shipped.

</comment><comment><USER22> Sometimes I can't help but wonder if Gates saw the writing on the wall and jumped ship to preserve his legacy.

<comment><USER23> If he did, it's surprising he didn't go years before this.

<comment><USER24> errr... he did - he's been stepping out since 2001.

<comment><USER25> Exactly, you can't just jet at the end of a day if you want your 20% of the company to be worth something when you're gone.

<comment><USER24> I don't think he gives a damn about his 20% of the company as far as cash is concerned. It's more of a legacy thing. He created Microsoft - it's his legacy, it's what he'll be remembered by, and he really cares for it.... it's a part of him.

</comment></comment></comment></comment></comment><comment><USER26> too many cooks in the kitchen. MS needs a strong product management voice ala Jobs/Schiller to cut through the crap and insist on a coherent, consistent, and comprehensible product vision and user experience.

</comment><comment><USER27> It is kind of suprising how much disconnected he is with the flagship product of his company.

He should have been able to be a lot more specific and detalied about what was going on, and not complain in generic terms "downloaded more stuff, this popup here, that thing there".

It seems that back then he was not involved at all with the development of windows. It feels like comming from somebody that just started using it, and not somebody that build it (or gave guidance on building it). And this was in 2003, so Windows XP had been out for about two years.

<comment><USER28> I'm not so sure the level of detail is an indication that he's disconnected with his own company's products. The vibe I got was that he was trying to describe to the people who build this stuff what it's actually like to be an end user. I think the whole point is that they're the ones who are disconnected and completely out of touch. I seriously doubt that Bill Gates _needs_ to download and buy Movie Maker from microsoft.com like a customer would.

Even if he is the one who is totally disconnected I thought all of his points were excellent and a perfectly accurate illustration of how Windows has gone wrong. What I'm left wondering is what's been going on there the past several years. If these critiques by Gates are something new it's a real shame for Microsoft that he didn't start sooner. If his statement that it's his job to do this every day is accurate then I'd say they have even bigger problems.

</comment></comment><comment><USER24> You can't deny that this is someone who _gets_ software and knows how things are supposed to be. Now whether _Microsoft_ gets software or not is debatable, but BillG most certainly does. Anyone that's spoken to him in real-life (even some of the most fanboyish Apple users) has confirmed this fact time and time again - it's a real shame for the software industry he "decided" to pull back from his company in 2001.

</comment><comment><USER29> Duh, Bill:

<http://search.msn.com/results.aspx?q=download+moviemaker>

I thought I'd try to install it to see if they streamlined it, but it turns out it was part of SP2. I guess I never needed to know about it...

</comment><comment><USER30> """Then it told me to reboot my machine. Why should I do that? I reboot every night -- why should I reboot at that time?"""

Haha

% uname -a

Darwin Galactica.local 9.3.0 Darwin Kernel Version 9.3.0: Fri May 23 00:49:16 PDT 2008; root:xnu-1228.5.18~1/RELEASE_I386 i386

% uptime

19:08 up 23 days, 6 mins, 2 users, load averages: 0.87 1.02 1.07

<comment><USER31> I had the same thought when I read this. I used to turn off my computer every night, until I switched from Windows (98) to Linux. In grad school, my roommate and I had uptime contests with our Linux boxes, which he won with something like 6 months of uptime. The most common causes of lost uptime were electrical power outages and moving apartments.

Now, like you, I mainly run OS X, but it's on a laptop, which wreaks havoc with my uptime. Ah, well. I guess you can't have it all.

<comment><USER32> I'm always amazed by this. Why would you keep your computer running if you're not using it? just for comparing lengths?

<comment><USER33> Laptop:

10:56 up 83 days, 1:50, 7 users, load averages: 1.05 1.32 1.12

I don't reboot - but I do close the lid to let it sleep. My main reason for not rebooting is I generally have 100 things open at any time and that 'state' is important.

<comment><USER14> As memory and bulk storage become faster and faster and gain more and more capacity I think that the idea of losing application state just because your PC shut off will become a horror story to tell our children. Eventually it will become one large memory space backed by a variety of local and network stores.

</comment></comment></comment></comment></comment><comment><USER34> Its good that these sort of issues are exposed, even if by Bill. The scary thing is that MS QA does not pick this stuff up.

As more of this stuff gets exposed this will hopefully make MS work harder. And as Apple is becoming a REAL contender, hopefully MS will pull their head out of their ass.

<comment><USER35> MS's QA is oriented towards making sure that individual pieces function, not that the right thing happens. Their aim is 100% automated unit testing... but nobody ever _tests_ that doing something is intuitive.

The UX folks do, but that doesn't mean they get their way.

<comment><USER34> MS having unusable products is like Exxon not selling fuel. Very quickly users will abandon windows in favor of Apple if it gets too bad.

</comment></comment></comment><comment><USER12> Well... BG isn't the only one who get's frustrated with windows.

I gave up on that crap environment a long time ago and started using Mac's...... the only way to go. Definitely worth the extra money.

<comment><USER36> Equivalent macs don't really cost extra money.

<comment><USER37> Because of the extra value of the system just working?

<comment><USER36> no, they just plain don't. they all come with isights and firewire and DVD burners and such. comparisons i've seen with _equivalent_ PCs are roughly equal.

if you want lots of RAM or a larger harddrive, you probably have to buy that separately to not get overcharged, though.

<comment><USER37> OK -- I spec'd a Dell XPS M150 just now to be equivalent to the $2500 Macbook Pro. I got a better than equivalent config for $1678 -- it has 4GB of RAM, would be $1578 with 2GB of RAM.

It comes with a built-in 2MP webcam and bluetooth and N-wireless. It has HDMI rather than DVI out. A bluray burner is an option, as is a 1920x1080 screen. The only thing it is missing is firewire and I don't think anyone can argue that Firefire is worth $900, except people that need Firewire. The backlit keyboard is nice and generally indicative of the Apple touch, I guess, but that adds $900.

At one point, it was definitely true that you couldn't get a laptop with all the extras in a Mac. But this Dell Computer really seems equivalent. Perhaps not in build quality.

<comment><USER36> I guess it varies by when and what you compare. e.g.

[http://www.macworld.com/article/49403/2006/02/pricecompariso...](http://www.macworld.com/article/49403/2006/02/pricecomparison2.html)

has the macbook pro doing well.

or this

<http://kb.wisc.edu/showroom/page.php?id=3045>

has 150 more for macbook (not pro). they seem to have firewire and isight missing on the pc, and a modem missing on the mac.

\</comment></comment></comment></comment></comment></comment><comment>-

i just did a comparison of imac vs dell all-in-one.

imac = 1300 (add 1gig of ram)

the dell is 1300, but i need to change stuff like including a video card, which it doesn't look like i can do (dell's customization page sucks). anyway, apple seems to win.

the macbook pro isn't really for price-sensitive people, anyway. it's the imac, macbook, or maybe mini that i'd expect to price compare OK or well.

<comment><USER13> The biggest problem with Apple's lineup is for people who already have a monitor they like. Your options are the mini, or a ridiculously expensive Mac Pro. There's no middle ground.

Also, I don't think the mini is really price competitive even on the low end. I can get the same hardware specs for considerably cheaper elsewhere, though, of course, not with the awesome form factor/design.

I really like the mini in terms of hardware, but it's not a bargain.

<comment><USER4> MacBook or MacBook Pro in clamshell (closed-lid) mode.

<comment><USER38> Why is Mac's answer always to buy more? I want my monitor, so I should buy a laptop at a premium and -then- I can use my monitor? Never you mind that, of course, the Macbook & Macbook Pro have no docking station, so that's a minimum of manual connections every time I disconnect/reconnect. If I never use the laptop as a laptop, I've way overspent.

I had a similar conversation at WWDC about the 'no docking station' issue. Apple's response: buy everyone a laptop AND a desktop, and use roaming profiles.

Yeah.</comment></comment></comment></comment>


Show HN: We wrote a book to help data people build scalable analytics stacks - <AUTHOR> https://www.holistics.io/books/setup-analytics/ <comment><USER2> I'm a data engineer at a large corporation. At my current company, we use Pentaho to extract and transform our data from Oracle daily. The transformed data is loaded to a staging database where we then model it to a star-schema. Then the final results are bulk loaded to an on-prem DW. The process takes hours to complete.

I'm interested in moving to a model that continually extracts changed data to a data lake, then using the power of a cloud data warehouse to read those files and perform the transformations and modeling in SQL. I guess that's the ELT concept that you mentioned in the book's summary.

Goal being to reduce the latency and allow for the possibility of more frequent batches, as well as making the process more accessible to my team with strong SQL skills and being able to adapt faster to changing business needs.

This book looks like a good foray for me to get a glimpse into those new process. Thanks for putting it together.

<comment><USER3> I'm one of the authors of the book. Yes you're right. The book outline the transition from the "old world of BI" to the "new world of BI". If you read through the book, you'll see the biases clearly stated there:

\- ELT over ETL

\- Data Modeling is crucial as part of BI workflow

\- Cloud DW over on-premise DW (but this depends on your org's requirements)

\- SQL reporting (Redash, Metabase, Looker, Holistics) over non-SQL reporting (Tableau, Qlik)

<comment><USER4> I'd toss SeekWell[1] in the "new world" category, but we've taken a different approach. Instead of forcing people to use a whole separate platform for BI, we decided to tightly integrate with the tools people were already going to for data (e.g. Google Sheets, Excel, Slack, etc.). We've found teams stay better informed when the data is in places they're already hanging out.

[1] [https://seekwell.io/](https://seekwell.io/)

</comment></comment></comment><comment><USER5> Really well done! The content looks like a great balance. I'm an engineer working a bit in data engineering, and I find the content of the book relevant to me.

I like how you explain why the methodologies of the pre-cloud era still have lessons learned to apply to today, but implementation best practices have changed thanks to the cost model of the cloud.

The section that stood out the most to me, strangely, was not anything to do with the technology or analytics stack. It was in Chapter 3 – Data Modeling Layer and Concepts where you discuss the dynamic between the CEO and the data analyst and the data. This really articulated quite well how our own dynamic functions at our current company. Even with our current data warehouse, our BI team is a bottleneck, and it is something becoming more and more apparent to me. It is my primary motivation in seeking out how best to re-architect our analytics stack.

<comment><USER3> Thank you for sharing your thoughts! Yes the concept of data modeling and self-service analytics are interesting, yet few people fully grasp it. Chapter 3.1 is probably one of my most favorites.

</comment></comment><comment><USER6> This is just the book I need!

A little bit of context: I am a product manager and I have been working with data analysts and engineers for a few months, and even though I have tried to do a lot of research, sometimes I still don't understand what they said.

Terms are extremely difficult and varied depending on the site, and it seems like each company will have a different perception for one term.

So that's where this book comes in handy. It helped me visualize the big picture of the whole data analytics landscape. What's more, I understand what the role and challenges of the data analysts and data engineers in my team are. I was able to communicate with them in their "language", especially when I was explaining why we should use ELT instead of ETL (Chap 3, I suppose)

Anw, I think this book is great for non-tech people like me, but it requires certain experience in the tech industry to get started with. Definitely recommend for other PMs who will be working with data people!

</comment><comment><USER7> I've been looking for something like this for a while. Most of the time when I go online to search for resources on building analytics stack, most of the content is biased towards the vendor's preferred way of doing things. This looks like it will give me a high-level understanding to the why of the proposed approach.

<comment><USER3> This is exactly why we started out writing this book. We actually spoke with a lot of customers, and a fair share of them sharing the same thing: They found a lot of how-to on the web, but none of which is comprehensive, and goes deep into the "why" and the "history" of BI.

</comment></comment><comment><USER8> Amazing that we can get such high quality resources for free. That said, I'm not convinced by the example where the CEO uses the "data modelling layer" (essentially what holistics offers to build for you and what this book is an ad for). In my experience a good data analyst does far more than "translate" the business question to SQL. The exec's understanding is not only limited by not knowing SQL, but also by potential confounds or a billion other things that can make a seemingly meaningful result meaningless or dangerous.

I don't think simpler technology can give people the magic answers and easy data access that they crave any more than no code tools can let people build complicated _correct_ systems

<comment><USER3> You're right, and also right that we might be biased. We actually not trying to say that with the modeling layer the CEO can remove her reliance on the DA completely, that would be foolish. We think she can only reduce that reliance down when it _only_ comes to getting access to data. We are not talking about complicated analysis that requires a proper data mindset.

</comment></comment><comment><AUTHOR> Hey HN. This is something we've been working on for the last three months over at Holistics.

If you're a data analyst, data engineer, or a founder setting up a data analytics stack for the first time, this is a book that will give you a soup- to-nuts overview of an entire field.

Like most books about data analytics, this assumes some amount of technical competence.

Unlike most books in the space, this is mostly about first principles. About the ideas behind the tools, not the tools themselves.

The hope is to give you 'just enough to not get lost'. And the book is written to be read within 2 hours of reading — in some cases, no more than two sittings!

There's probably more than a hundred hours of research and writing that went into this. I'm looking forward to read your comments.

</comment><comment><USER9> Great book and spot on of the problem statement. One interesting note to point out is I didn't see any mention of testing your data models or version control? This should be part of the process of the modern analytics stack to ensure data quality.

A suggestion of approaches and tools could be useful. Whether it's via tools such as dbt for expected field values or with frameworks such as GreatExpectations. What happens to data that doesn't conform to expected values? How should you handle it? and how can the testing process be automated? This forms an important part of ensuring data quality and reliability of the analysed output.

<comment><USER3> Yes.. we did get a number of early feedback about this exact topic of data quality. But we eventually decided to cover it in another separate sidebar to the book. It's also not a simple topic to cover. And the book is supposed to "give you enough to be dangerous".

</comment></comment><comment><USER10> Nice, the illustrations look pretty good. I just took a look at the table of contents, it seems to cover a lot of my questions about data analytics for a product guy like me, will spend some time reading this weekend.

Sending to my data team btw, thanks for sharing

</comment><comment><USER11> Very nice drawings, what tool was used ?

<comment><USER3> We used iPad, Apple Pencil and Paper app :)</comment></comment>


NoPriv.py - Easy IMAP email backup to HTML archive - <AUTHOR> https://raymii.org/s/software/Nopriv.py.html

<comment><USER2> is this a good idea? shouldn't backups be in a _standard_ format (for the medium in question)? what if i want to import emails from this _backup_ into another mail program / mail provider?

it seems like a useful static(?) _interface_ to email, but as a _backup_ it's oddly restrictive.

maybe it's just confusion over the use of the word "backup". but for me, a tarball of maildirs, while not readable by a human, is closer to what i expect that to mean. perhaps this could be modified to provide both?

<comment><USER3> Not only does it generate a non-standard format, the HTML has significant problems:

\- It would be relatively hard to reconstruct the email with a script, e.g. the headers are formatted as a plain text with  for the labels, the data left unmarked in anyway and
 for line breaks (no ids or classes at all). From <http://sparklingnetwork.nl/nopriv/2012/12/26.html>:

        From: "Remy van Elst <email@example.com>"

        To: ""email@example.com" <firstname.lastname@example.com>"

\- It's invalid markup[1], with the email's HTML contents inserted wholesale, including the  and  tags.

\- Not sure if this is a problem with NoPriv.py, or if it has occured earlier in the chain, but something has screwed up character encodings, e.g. in the URLs here: <http://sparklingnetwork.nl/nopriv/2012/12/1.html> (The page is served with no encoding specified nor is there a <meta http-equiv> so it defaults to UTF-8, and these characters are invalid. However, even in Latin-1 or anything else, they are definitely not '=' which appears to be what they should be.)

(The markup is also horrible, with strange indentations and a lot of extraneous whitespace and one-cell tables.)

[1]: [http://validator.w3.org/check?uri=http%3A%2F%2Fsparklingnetw...](http://validator.w3.org/check?uri=http%3A%2F%2Fsparklingnetwork.nl%2Fnopriv%2F2012%2F12%2F26.html&doctype=HTML5)

<comment><USER4> Simple enough issues to fix, of course, since this project is open source and only 431 lines of Python.

<https://github.com/RaymiiOrg/NoPriv/blob/master/nopriv.py></comment></comment></comment>


Did John Steinbeck Spy for the CIA? - <AUTHOR> https://www.thedailybeast.com/john-steinbeck-did-nobel-prize-winning-novelist-spy-for-the-cia-in-paris <comment><USER2> Gloria Steinem worked for the CIA as well.

It becomes very difficult to understand what happened during the 50s and 60s, the more you look into it.

<comment><USER3> It used to be a conspiracy theory.

Now the only question is the extent to which she did that.

There is good evidence that the magazine was also paid for by the 3 letter agency.

<comment><USER4> > There is good evidence that the magazine was also paid for by the 3 letter > agency.

Do you mean New York or Ms.? I'm not familiar with this theory; can you link to more info?

</comment></comment></comment><comment><USER5> I can't imagine this helped with Hemingway's late (and justified) paranoia.

<comment><USER6> I always find the posthumous vindication of Hemingway to be interesting because people want to say justified paranoia, or ironically accurate paranoia

But shouldn't we just change the word to describe Hemingway? Paranoia _means_ unjustified suspicion/distrust and encroaches on mental health issues. It should say people _thought_ he was paranoid about a thing that turned out to be true. So in reality Hemingway was _suspicious_ of shitty spies.

<comment><USER5> Right, yeah I agree paranoia isn't the right word. I don't know what the right word would be though. While he must have been aware of being monitored by his government, I suspect it would still have come off as paranoia to the people who knew him or knew of him, justified or not.

<comment><USER7> > Right, yeah I agree paranoia isn't the right word. I don't know what the > right word would be though.

The word for thing that is like paranoia but instead of being irrational and pathological its entirely rational and grounded in fact?

"Justified concern"?

</comment></comment></comment></comment><comment><USER8> Both Geller and Randi worked for the CIA. Is there any way we can tell at this point what the "truth" was?

<comment><USER9> Geller probably told them "Why, yes, I _can_ spy on enemy bases with my psychic powers" whereas Randi was a consultant who advised them "That guy is full of shit, and so is anyone else who tells you they have psychic powers. To take them seriously is to waste taxpayer money."

</comment></comment><comment><USER10> Joel Whitney has done a lot of research on the CIA's remarkable involvement with writers and artists. His book on it is [https://www.amazon.com/Finks-C-I- Tricked-Worlds-Writers/dp/1...](https://www.amazon.com/Finks-C-I-Tricked- Worlds-Writers/dp/1944869131). There are good interviews with him about it online. It's amazing how extensive these involvements were and how major the writers were.

[https://news.ycombinator.com/item?id=10116113](https://news.ycombinator.com/item?id=10116113) from 2015 is on this topic, but not that great a thread.

</comment><comment><USER11> So effectively he probabbly just provided tidbits of observations about people he met?

<comment><USER12> That's spying at its fundamentals. Don't let the movies fool you.

<comment><USER13> Exactly, spy networks are about aggregating information from many people about many people.

Fundamentally, spying is very good for the world. It allows for nation states to understand each other's thinking and reasoning. The most dangerous situations come from misunderstanding, as we are now experiencing with North Korea.

<comment><USER14> > Fundamentally, spying is very good for the world.

'Spying' is not a well-defined term. I think you mean 'information gathering' but also under the rubric of spying fall subversion, sabotage, political interference, propaganda and disinformation, etc.

Even in the realm of information gathering the techniques used are often unacceptable - such as deliberately putting someone in a compromising position and then extorting them to provide access to information they posses.

I don't think the assertion that those things are good for the world would be widely accepted.

<comment><USER15> Publicly, the CIA claims that they don't use negative inducements.

<comment><USER14> As part of Edward Snowden's explanation of his radicalization he relates an anecdote of his work for the CIA in Switzerland.

From the wiki page:

 _Snowden described his CIA experience in Geneva as formative, stating that the CIA deliberately got a Swiss banker drunk and encouraged him to drive home. Snowden said that when the latter was arrested, a CIA operative offered to help in exchange for the banker becoming an informant._

 _Ueli Maurer, President of the Swiss Confederation for the year 2013, in June of that year publicly disputed Snowden 's claims. "This would mean that the CIA successfully bribed the Geneva police and judiciary. With all due respect, I just can't imagine it_

Everyone should feel free to apply their own judgement about the value of this anecdote.

Edit: I would regard kidnapping, rendition, and waterboarding as a "negative inducement", myself.

</comment></comment></comment></comment></comment></comment><comment><USER16> Yes, documented.

(Exception to Betteridges Law of Headlines)

</comment><comment><USER17> Tangentially related, but isn't the rule that when a headline ask a question, the answer is generally yes?

<comment><USER18> [https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...](https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines)

The answer is generally "no".</comment></comment>


Effective vs Ineffective Pair Programming - <AUTHOR> http://www.skorks.com/2009/07/effective-vs-ineffective-pair-programming/

<comment><USER2> Having recently been exposed (by a class forcing us to use pair programming) to this, I have a few insights on pair programming:

\- Both partners must be fairly versed in the language and methodology of the programming. I think its fairly explanatory, but there is a definite limit to how much of a knowledge gap there can be between the experienced and un- experienced programmers. If that gap is too big the only result will be slower coding, written at about the same quality as the more experienced programmer would write on their own, but with considerably more frustration on both programmers' parts.

\- Do not do a "test trial" of pair programming, working on simple projects. From what I has read about research into pair programming, the only gains come from working on complex problems which would be difficult to conceptualize entirely in one person's head. If work is only done on small projects, pair programming will look like a complete waste, since non-pair programming would have similar results considerably faster.

\- Pair programming has a sort of half-life in terms of knowledge. The lower- skilled of the pair will learn half the gap in knowledge at aproximately the same rate. While that rate will be different between people, from my experience with pair programming, the lower-skilled partner will learn half the knowledge gap in 3-4 days. This will repeat, and the gap is halved every 3-4 days of working together.

On top of that, it seems to me that Pair Programming is not a good fit for long term programming. It seems best suited when there is an individual who needs to learn about a specific system, and that using it long term lessens the effectiveness it has. Once the transfer of knowledge has slowed down, and both individuals know aproximately the same ammount about the system, it seems to be much less effective. However, since their (should be) a constant learning about the system, occasional pair programming sessions would be advantageous after this point.

If I were to engage in pair programming, this is roughly how I would set it up:

Assuming that there a 5, 8-hour work days, divided into 2 4-hour programming sessions (for a total of 10 sessions a week), when a new individual is introduced they should spend the majority of their time pair programming with the more advanced individuals. While I am just pulling this number out of thin air, I would think 8 sessions of pair programming the first 1-2 weeks, with 2 sessions programming alone. As time progresses, I would shift to steadily less pair programming, with a minimum of 2-3 sessions per week.

This is largely because I am not convinced Pair Programming leads to better coding. I have not seen any real evidence to support that, in fact, from what I have seen it tends to slightly more convoluted and difficult to debug code (although pair-debugging seems to be incredibly effective). I have, however, seen how useful pair programming is in teaching a person. The ammount a person can learn in a short time, while working in a pair is incredible, and can be simply astounding.

<comment><USER3> To me, your comment spells out quite conclusively that you have not been part of a good pairing environment.

It is good that you see the knowledge transfer benefits, but I much prefer thinking of it as knowledge _distribution_ rather than _transfer_. By this I mean that your setup creates continual knowledge gaps and then fills them in the pairing session. That seems much less valuable than actually distributing the knowledge throughout the team by pairing all of the time.

I recommend you keep an open mind about the other major benefits of pairing and that you actively seek good pairing opportunities. It is worth it and you will know it when you experience good pairing.

<comment><USER2> It was a terrible pair programming environment. Pair programming was simply being used to up the average grade of the class (which succeeded).

I disagree with saying that constantly distributing knowledge by always pairing is a better situation. It makes the assumption that everything that can be known about a system is already known, which doesn't often (ever?) hold true. By separating out, you can cover far more area and find more of those unknowns, while coming together for regular pair sessions helps spread that newly discovered information around.

<comment><USER3> _It makes the assumption that everything that can be known about a system is already known_

No, it does not. I'm not sure where you extracted that from in my comment.

Also, if you are aware that your environment was terrible, it seems presumptuous to assess the benefits of pairing in a good environment. I assure you there is more than teaching at play.

</comment></comment></comment></comment><comment><USER4> I used to be a big pair programming zealot, but I've backed off, since I've seen it fail with so many people at so many places. Having said that, the times I've worked in good pair programming environments have been the happiest, most productive times of my life (at work, anyway).

So then, what constitutes "good"? This article seems to focus on the code, which I think can matter, but I believe is much less important than the people having the ability to pair effectively, which in my experience consists of two things: (1) the ability to clearly articulate what one is thinking, in real- time, and (2) the ability to allow your partner's ideas to be better than your own. It also helps if you don't get annoyed by someone asking you questions while you code, but that's less important, because you'll get over that if you and your coding partner have (1) and (2).

<comment><AUTHOR> I agree with you, I suppose I tend to take those things for granted, possibly because I've been fortunate enough lately to work in environments where both 1) and 2) are pretty much a given (lucky me).

Pair programming is an agile practice and in my experience it is a LOT easier to do agile when you have good people on your team, not just good developers, but good human beings.

</comment></comment><comment><USER5> I've had some good, some bad experiences with pair programming.

The best experiences were when both of us were learning a new API and type of program. That's the sort of thing where you are likely to make lots of small mistakes, and it's a lot less frustrating when you have someone else to help catch them.

The worst experiences were when the programs were relatively easy, and there was just a lot of stuff to be done. The person watching gets bored, and the person coding gets distracted.

I feel like if one person is confident they can bang something out, just do that. If it's the sort of annoying programming neither of you want to do, it helps to pair. Hard to have general rules though for such a personality- dependent activity.</comment>


Downloading in bulk using wget - <AUTHOR> http://blog.archive.org/2012/04/26/downloading-in-bulk-using-wget/ <comment><USER2> Although it's a relatively recent arrival, wget has to be one of the top-ten command line tools of all time.</comment>


Brexit: France activates no-deal plan - <AUTHOR> https://www.bbc.com/news/world-europe-46906046 <comment><USER2> It is interesting to note how Ivan Rogers, someone who arguably understands Europe more than anyone else in government estimated it would take up to 10 years to come up with a satisfactory deal. But he was essentially pushed out as he was generally considered by the government to be too pessimistic about the whole affair..

[https://www.telegraph.co.uk/news/2016/12/15/post-brexit- trad...](https://www.telegraph.co.uk/news/2016/12/15/post-brexit-trade-deal- could-take-10-years-still-fail-warns/)

<comment><USER3> I can strongly recommend Tim Shipman's _" Fall Out: A Year of Political Mayhem"_ for an account of the post Brexit vote annus horribilis.

The only thing to look forward to with Brexit is reading his account, some time next year, of the current antics.

<comment><USER2> Agree. I have just finished reading it.

It has left me wondering, are all administrations this shambolic - we are all humans after all - and this book is just a rare glimpse into what goes on behind the scenes, or is this just an exceptional circumstance? Maybe a bit of both. It would be interesting to read about what goes on behind a really well run government - should one exist..

</comment></comment></comment><comment><USER4> I wonder if it would make more sense to instead of having the UK leave the EU, have England and Wales leave the UK with the UK (Scotland, Northern Ireland, Gibraltar) remaining in the EU.

That would certainly seem fairer to the Scots, who rejected their independence vote largely because they weren't sure that they would be able to remain in the EU if they left the UK.

<comment><USER5> "they weren't sure that they would be able to remain in the EU if they left the UK" \- to be more precise, they were 100% sure that they will never be allowed into EU, Spain would never agree for this, as this would open path for Catalonia to leave Spain and join EU.

<comment><USER6> The Spanish government have since signaled, back in 2017, that it won't veto any attempt by an independent Scotland to join the EU.

</comment></comment></comment><comment><USER7> I'm not sure if I've been stuck in an information silo and have totally lost the plot, but there seems to be substantial circumstantial evidence that Russia interfered in Brexit, which throws into question whether the result can be accepted.

I'd be interested to know whether people on here think I've strayed into a very plausible conspiracy theory or whether this is something real and of concern. I feel I could accept a result in which there was not some effective psy-ops campaign being run, but cannot accept a result which is "not the will of the people".

[Edit: spelling correction]

<comment><USER8> Is there any mechanism in Westminster politics, or any democratic political system for that matter, for invalidating election results if it is determined that an external state interfered with the process?

If it was found that Russia interfered in some capacity, would/could anything be done?

<comment><USER9> The Electoral Commission can void elections in the event of fraud: [https://www.electoralcommission.org.uk/__data/assets/pdf_fil...](https://www.electoralcommission.org.uk/__data/assets/pdf_file/0010/150499/Challenging- elections-in-the-UK.pdf)

One of their most high profile cases: [https://en.wikipedia.org/wiki/Erlam_%26_Ors_v_Rahman_%26_Ano...](https://en.wikipedia.org/wiki/Erlam_%26_Ors_v_Rahman_%26_Anor)

I don't think either of these apply for nebulous "external interference", but certain actions such as running unlawful electoral advertising should count.

Vote leave have been fined and referred to the police: [https://www.electoralcommission.org.uk/i-am-a/journalist/ele...](https://www.electoralcommission.org.uk/i-am-a/journalist/electoral- commission-media-centre/party-and-election-finance-to-keep/vote-leave-fined- and-referred-to-the-police-for-breaking-electoral-law)

 _BUT_ : these apply to elected positions. Referendums don't really have a strong constitutional place in the UK, and the Brexit referendum was "non- binding": the result did not legally oblige anything specific to happen.

Brexit is a choice of the government. No-Deal Brexit is also a choice of the government, made two years ago when sending the A50 notification without a viable plan.

</comment></comment></comment><comment><USER10> I can't belive there isn't a viable candidate taking up something like:

"I'm not going to drive us face first into the ground because of a referendum the promised somethings that can't happen with a brexit. We're going to come out with the short end of the stick here if we're not smart. We need to work out a new deal where we stay but address some concerns."

Hopefully the EU could be adult about it and throw them a bone or two and let things settle.

<comment><USER11> The referendum was a vote to leave; reasonable people can disagree about what form that should take but the referendum result clearly rules out remaining in the EU, in the most simple of terms.

May has negotiated the only practical set of terms. If Corbyn was as serious about ruling out a no-deal Brexit as he claims, his party would've voted for May's deal. The part I find incomprehensible is that the remain lobby refuses to acknowledge that if we vote down every possible deal then the result will be no deal.

<comment><USER12> I don't really agree with this analysis.

A vote was held, and the decision was to leave. That's fine - we should not have run that referendum in the form it was, but we can't deny the result.

The problem is, Brexit was an entirely empty vessel into which voters' fantasies were poured. The outcome in no way resembles any of the ideas that were articulated before the referendum. There is now a concrete plan that basically everybody hates, because it's the worst of both worlds.

There are three realistically possible options right now. The UK either exits with the agreed deal, exits with no deal in a disorderly fashion, or cancels Brexit and remains in the EU. The first option is bad because nobody wants that deal. The second is bad because it would be obviously catastrophic. The third is terrible because it directly contravenes the outcome of the referendum.

In short, all the options are fucking awful. There's no real democratic legitimacy for any of them. Under these circumstances, having a second referendum that says "this is what we were able to negotiate, now do you want to go through with it or forget the whole thing" is possibly the least shit option. I hope it will be cancelled, but at least if it's not I can be sure that it was voted for with knowledge of what was to come.

<comment><USER5> I am not sure why hard exit is going to be "catastrophic". There are many countries outside EU doing just fine and doing business with EU countries. Plus nothing stops UK from signing bilateral trade deals with EU countries and I wouldn't be surprised if Germany and France were first in the row do sign such deal (loudly crying at the same time how important is the EU solidarity).

<comment><USER13> It has been explained multiple times to several people (Donald Trump, one of them), that for EU countries is impossible to establish individual trade deals, they must do as part of a block.

[https://www.businessinsider.com/trump-trade-merkel- germany-e...](https://www.businessinsider.com/trump-trade-merkel-germany- eu-2017-4?IR=T)

It would defeat the purpose of a common market. What stops UK from doing so is EU laws.

But I agree it won't be catastrophic. UK will survive, the EU will survive.

But jobs will be lost in the process, economies will be affected, quality of life will be lowered.

And people voted for this to happen.

<comment><USER12> Both the UK and EU will survive, but I think that's an awful metric.

I have a startup in the UK. We employ EU citizens, sell products and services to EU customers, buy products and services from EU suppliers, and receive EU grant funding. A disorderly Brexit will affect us in unknown, possibly terminal ways.

Aside from that personal issue, I t's possible in the worst case that hundreds of people may die due to lack of medication. The economy will be affected to the time of 10% of GDP over 15 years, by the government's own prediction.

I think that counts as catastrophic.

</comment></comment></comment></comment></comment></comment><comment><USER14> I'm most disappointed in Labour and the Liberal Democrats here. The Conservatives are kind of locked in as the Euroskeptic party at this point so they're going to stick with the party line that referendum is binding and they'll crash out of the EU over not leaving.

But where is the opposition saying that the referendum was nothing more than a protest vote against immigration, 52-48% is a narrow split, we can't possibly have our cake and eat it too (eg on freedom of movement) and that pursuing some Norway like model combines the worst of both worlds (eg maintaining membership of the EU customs union and being subject to its rules without having the power to vote on and control the EU agenda as a member would).

The backstop is nothing more than kicking the can down the street and the ultimate result is probably reunification with Ireland, which is why the DUP in particular and the Conservatives in general don't want it.

The only alternatives are hard exit and remaining.

Why hasn't the opposition been more vocal about this? All I see is hand- wringing and non-committal rhetoric from Corbyn.

<comment><USER11> Corbyn doesn't want to remain in the EU, and even if he did, his whole narrative is that he wants a democracy that's more representative of ordinary people over Westminster elites. If he tried to offer a policy of technocrats overruling the will of the people he would lose all credibility.

The Liberal Democrats are pro-remain I think? But they're an irrelevance at least parliamentarily.

</comment></comment><comment><USER15> In a decade when EU economy start collapsing, British people will be for another vote but this time EU soldiers will be restoring ordnung and make sure the outcome is the only right one. It's the UK last chance.

</comment><comment><USER16> >less and less unlikely

I hate phrasing like this</comment>


U.S. Will Exceed Its Paris Accord Goals Despite Trump's Official Withdrawal - <AUTHOR> https://futurism.com/report-the-u-s-will-exceed-its-paris-accord-goals-despite-trumps-official-withdrawal/ <comment><USER3> Sounds like a success to me.

\- The US isn't obliged to pay other countries to change their actions - <USER2> they'd only have to do if they wanted to, because there was no enforcement on other counties behaviour.

\- The US federal Government has give more power of climate policy to the states

\- The states who could act, have and have voluntarily chosen to improve the environment.

Win, win, win.

<comment><USER4> Seems like the states took the climate control power, not that it was given to them.</comment></comment>


Nim Advocacy and Promotion Strategies - <AUTHOR> http://forum.nim-lang.org/t/671 <comment><USER2> Alright, then show me! For example, the beginner tutorial was linked a while ago ([http://nim-lang.org/tut1.html](http://nim-lang.org/tut1.html)) and I wouldn't criticize it at all, because it's well written, but it doesn't show me why _I_ would care about Nim when there are dozens of languages I could choose instead.

It looks like Python, with a little more types.

Fwiw, I'm also a language geek and my favorite language is Factor. If I wanted to evangelize it, I could show how idiomatic Python or Haskell code translated to Factor takes up less than half or a third in size. So that's its USP (Unique Selling Point) -- really short code. What is Nim's USP?

Julia: "fresh approach to mathematical computing", Rust: "safe resource disposal without GC", Go: "C successor backed up by Google", Swift: "Replacement for Objective-C", Clojure: "Scheme on the JVM", Nim: "no idea what to write here"

Is it the Why should I be excited? paragraph: "Nim is the only language that leverages automated proof technology to perform a disjoint check for your parallel code. Working on disjoint data means no locking is required and yet data races are impossible" If I understood what the hell that means, maybe I would be excited. :)

<comment><USER3> _I_ am excited about Nim because it compiles to C that can run on an ARM microcontroller. I believe this was a possibility at some point, but I don't know if this is a priority for Araq. So (personally) I see it as a (potential) "C replacement for embedded systems".

<comment><USER4> I'm just trying Nim for the first time. How do you compile to totally self- contained C?

I tried pulling the files out of nimcache/*.c, but it is wanting to include nimbase.h.

Is there a way to compile an entire Nim program to a single self-contained .c file (ie. an amalgamation, aka SQLite?)

<comment><USER5> You copy the nimbase.h file along with the nimcache/*.c files. That's it. The compiler doesn't do it because it's a waste of time/space copying a file which you most likely can include from somewhere else, and it's not a generated file.

<comment><USER4> Good to know. It would be nice if there was an option for the compiler to build a single .c file that was totally self-contained, for distribution purposes.

<comment><USER5> I don't know anybody who redistributes source code as a single C file, but you could try concatenating the files into one. Usually people redistributing source code don't mind having several files bound together by some make|build script. The nim compiler has the --genScript switch to make one. Most people willing to get something redistributable prefer a binary static|dynamic library or final executable.

<comment><USER4> > I don't know anybody who redistributes source code as a single C file

SQLite pioneered this approach, called an Amalgamation. It offers some noticeable advantages:

[https://www.sqlite.org/amalgamation.html](https://www.sqlite.org/amalgamation.html)

It is gaining some followers:

[https://github.com/vinniefalco/Amalgamate](https://github.com/vinniefalco/Amalgamate)

[http://stackoverflow.com/questions/2719311/tool-to-create- an...](http://stackoverflow.com/questions/2719311/tool-to-create-an- amalgamation-combine-all-source-files-of-a-library-into-one-fo)

[http://lua-users.org/lists/lua-l/2008-08/msg00594.html](http://lua- users.org/lists/lua-l/2008-08/msg00594.html)

> Usually people redistributing source code don't mind having several files > bound together by some make|build script.

A single .c file is easier to integrate into various build systems. For example a Visual Studio project, or another Makefile without needing a separate script. It's just cleaner.

> Most people willing to get something redistributable prefer a binary > static|dynamic library or final executable.

These aren't portable like a C source file is.

</comment></comment></comment></comment></comment></comment></comment><comment><USER6> As an engineer I would be persuaded by a) implementing something practical but simple simply, b) implementing something a bit more complex legibly and c) writing about the results which explain why they are neat.

The killer promoter is of course a book.

I usually wait before starting on new languages so I'm not slowed down by incomplete tooling, missing compiler features etc.

What really gets me going is a 1-2-3 tutorial which explains how to implement something non-trivial like calling c dlls.

<comment><USER7> The package manager Nimble is written in 800 lines of Nim: [https://github.com/nim- lang/nimble/blob/master/src/nimble.ni...](https://github.com/nim- lang/nimble/blob/master/src/nimble.nim) Maybe that would satisfy (a)

</comment></comment><comment><USER8> Lack of tutorials...

Learning the syntax/features of the language should be part of documentation, not a tutorial.

"Hello World" as the core/intro example with nothing else after that (or trivial command-line programs) will not get many adopters.

Show people how to build REST APIs, GUI programs (like a text editor), a small web-server and maybe some other scientific/math example.

Showing just a few features and then saying "go build" will not get normal people to adopt the language. Only "savants" will pick it up.

Give examples/libraries and people will build on those.

The syntax is also not _that_ close to Python. It is obscure in some places and not as readable. If you're going to compare it to Python, do a readability test:

Write the exact same application in Python and Nim and then ask somebody else to compare the 2.

I tried to pick up Nim, but I was fighting against it every time I tried to go beyond a trivial CLI program.

I'll take Pythons slower implementation with 3000 libraries over Nims "C-like" speed where I can't even build a REST API.

I like Nim, but if all the effort of promoters is just promotion, the language will stay where it is.

You geniuses working on Nim for a long time need to share your examples and show the noobs how you built things with decent screencasts/tutorials/lessons.

Something tells me that even just 3 decent examples where real applications are built (eg. a game like Snake using a GUI and game-engine/lib) will get +1000 Youtube hits in no time.

</comment><comment><USER9> Nim is very, very close to being the most competitive language in my toolkit. If JSON were handled more elegantly, like Python, I'd probably be completely sold.

Binding to Python would also be an absolutely killer feature.

<comment><USER5> Take a look at [http://forum.nim-lang.org/t/360](http://forum.nim- lang.org/t/360) for Python bindings, that should lead you to [https://github.com/micklat/NimBorg](https://github.com/micklat/NimBorg).

</comment></comment><comment><USER10> Perhaps they could fix the Wikipedia and Wikibooks presence? Has it really been "banned" from Wikipedia?

<comment> def- [https://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletio...](https://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletion/Nimrod_\(programming_language\)_\(2nd_nomination\))

> Lacks reliable independent secondary sources to establish notability as > required by WP:GNG. Every source is WP:PRIMARY. Every one of them. Googling > turned up posts to online discussion forums but nothing useful. > Additionally, I note that the decision to delete at the previous AfD was > unanimous for the same reasons.

> Perhaps think of it this way: a language becomes notable when people who > haven't been involved in its creation start writing about it. If/when this > language gets to that point you'll have no problem creating an article. At > the moment, though, there just hasn't been enough uptake to get the coverage > we need for notability.

<comment><USER11> Would the Dobbs article be enough to get it reinstated then? [http://www.drdobbs.com/open-source/nimrod-a-new-systems- prog...](http://www.drdobbs.com/open-source/nimrod-a-new-systems-programming- languag/240165321)

<comment> def- No, since it's written by Araq, the creator of Nim.

</comment></comment></comment></comment><comment><USER12> Maybe update the site design and logo? It looks very 2000's and could be a huge benefit.

<comment><USER13> An aesthetically nice website is a neutral or a negative signal when it comes to technical subjects, like computer languages. Content first, graphical design is completely secondary.

Just need to represent information about the technical subject clearly.

If the technical merits are there, adopters will come. Technical people are not baffled by $animal . $excrement. Instead, it puts them off.

Edit: I get downvoted a lot, but could someone prove me wrong? What technical concepts (like programming languages) got popular with help of nice graphical design?

<comment><USER12> That kind of thinking will only prevent adoption, not help it.

Aesthetics are important to technical subjects as well. It shows that a lot of time and consideration have gone into every part of the project, not just the technical side. It's also a good sign of community support/interaction. I will agree that content is the most important, but you shouldn't write off aesthetics so easily, especially when you want people to be interested and adopt your technology.

<comment><USER13> Try to google some popular languages. Use [http://archive.org/](http://archive.org/) to see how their respective pages looked like in the past. I find it hard to believe aesthetics helped their adoption.

Interesting article, related to web design and conversion rate:

[https://blog.kissmetrics.com/shocking-truth-about- graphics/](https://blog.kissmetrics.com/shocking-truth-about-graphics/)

<comment><USER12> Here's Ruby for example, before it was popular: (2004) [https://web.archive.org/web/20040211074335/http://www.ruby-l...](https://web.archive.org/web/20040211074335/http://www.ruby- lang.org/en/) and here's when/right before it (arguably) started picking up a community [https://web.archive.org/web/20070805190709/http://www.ruby-l...](https://web.archive.org/web/20070805190709/http://www.ruby- lang.org/en/) (2007)

Popularity being based on the Rails 2.0 release since that's when Ruby/Rails seemingly started to take off. It's a bit before my time so if anyone knows better feel free to correct me.

<comment><USER13> So we can probably agree that at least Ruby didn't get popular because of graphical design.

And here's Nim:

[http://nim-lang.org/](http://nim-lang.org/)

It looked pretty good when I saw it for the first time today. Told me what it is about concisely. I don't see how a better design could significantly improve Nim's adoption.

Edit: I did completely miss the carousel at top of the Nim main page. I think it'd work better, if it was just serially on the page instead of the widget. Same content, less "design".

<comment><USER12> You're missing the entire point. If they had a better design it would probably interest people more, pertinent information would be more visible, and it wouldn't look like it came out of 2002.

This is pretty much the "sales" page for Nim and it looks outdated and unmaintained. For comparison check out the Rust homepage. It's not "flashy" but it's simple and looks fairly nice. It gives you a simple example and tells you why you should be using Rust in as few words as possible.

[http://www.rust-lang.org/](http://www.rust-lang.org/)

Here's some other sites that have fairly good designs that show that the community cares about how they and their language come across.

[http://elixir-lang.org/](http://elixir-lang.org/) [http://rubygems.org/](http://rubygems.org/) [https://www.ruby- lang.org/en/](https://www.ruby-lang.org/en/) [https://new- www.haskell.org/](https://new-www.haskell.org/) [http://www.scala- lang.org/](http://www.scala-lang.org/)

All of those languages/tools show that the community/devs care about how they and their language look to others/potential adopters. I'm not saying that having a great site design (not graphical design, that's different) are going to make Nim the most popular language on the planet. I'm suggesting that it can, and will help if they move forward and pick up a more modern design and simplify the content to get their point across.

</comment></comment></comment></comment></comment></comment></comment><comment> vezzy-fnord I don't understand why Python comparisons are being thrown around here. From my limited experience with Nim (I liked the name Nimrod better, but oh well), it seemed a lot more Wirthian than it did Pythonesque. And of course: AST macros.

</comment><comment><USER14> I am looking for a systems programming language that would work well with battle prooven GUI toolkits such as Qt. Would Nim be a good choice?

<comment><USER3> Writing Qt-flavored C++ was a lot of fun last I tried it (some five years ago). And now there's QML. Have you tried writing your entire project in Qt? It may be a smoother experience than you think.

<comment><USER14> I have some experience using C++ with both Qt and wxWidgets, but I am looking for an excuse to learn something new. And a systems language with Python inspired syntax and comparable expressiveness that is suitable for creating GUI apps is something that would definitely make me want to investigate Nim further.

<comment><USER15> You may be in luck: [https://github.com/filcuc/NimQml](https://github.com/filcuc/NimQml)

<comment><USER14> Thank you for the link, but I can reasonably expect that Qt bindings for Nim either exist or will become available soon, as most languages that get any traction get them. I still don't know, however, if it's a good choice to try and put a GUI app together in Nim. Will it surprise me with a huge binary, like Rust does? [1] Will it be easy to develop in it on multiple platforms? I can find all those things myself, but in the spirit of what the OP describes as putting the best foot forward, I'd like to be able to quickly assess whether the language is potentially a better choice to solve my next problem than those I already know or am planning to learn.

[1] - [http://endl.ch/content/cxx2rust-pains-wrapping-c-rust- exampl...](http://endl.ch/content/cxx2rust-pains-wrapping-c-rust-example-qt5)

<comment><USER15> Having seen multiple young open source projects, like Nim, play themselves out over the course of my many years in the Bitcoin space, I expect the savviest developers at this point to benefit most from Nim by establishing themselves as core contributors.

The devs who got into Bitcoin on the ground floor in 2011 and 2012 are three years later on the most sought after venture backed teams, with the whole world hanging onto their every last word. Young and profoundly promising languages like Nim offer a similar opportunity if you're willing to risk it.

It could very well be better to use the myriad of other, more mature and stable languages to solve truly pressing issues. Nim is very young, and the risk:reward ratio is skewed more towards contributors than users who can't risk bugs.

</comment></comment></comment></comment></comment></comment><comment><USER16> I find it disturbing that languages are now adopting promotion strategies because not enough people are using them.

For fuck's sake, focus on improving the language and this will come organically. This isn't a competition as to who has the most popular language, and the most popular languages are always going to be total shit.

Just write good code, marketing departments are the bane of the entire IT sphere, you shouldn't be trying to create one for a niche language.

<comment><USER17> That's not true. Being the first few people means that the tools are not developed, common questions aren't answered, books aren't written, getting help is difficult, etc.

There are only 18 questions on StackOverFlow, for example:

[http://stackoverflow.com/questions/tagged/nim](http://stackoverflow.com/questions/tagged/nim)

People don't want to burn too many cycles discovering how the language works.

<comment><USER5> The fact that there are only 18 questions on StackOverflow could also mean that the language is that good that rarely you find something requiring further explanation. Or maybe Nim users use the Nim forum instead for questions ([http://forum.nim-lang.org](http://forum.nim-lang.org)), where finding a C forum or Python forum is not that easy/direct.</comment></comment></comment>