Textmate2 Goes Open Source - <AUTHOR> https://github.com/textmate/textmate <comment><USER2> I don't want to pick on Allan Odgaard, but I think the way he's handled the TM2 project is pretty bad.

Allen is a great guy and I _love_ TM. However, here are some facts. TM2 has taken SIX YEARS. It was "90% done" 2009.

This is a living, breathing case study in why quick customer-driven releases are better than "big upfront plans" and "giant system rewrites." Anyone who has developed a major application knows what I'm talking about. These "big rewrites" almost always take much longer than expected, as has clearly happened here.

I have learned to listen to what your customers want, and just build it. Develop it in a few weeks, release it, and then ask again what your customers want. Some people call it "customer-driven development" and I think that's a good way of phrasing it.

<comment><USER3> It's a famous Joel Spolsky essay: Things You Should Never Do, Part 1 [1]. Never do a full system rewrite. Pain and sadness follow.

[1]: <http://www.joelonsoftware.com/articles/fog0000000069.html>

<comment><USER4> When I joined thredup.com as the very first employee and the 2nd programmer, we decided to do a complete rewrite. It ended up being a great decision.

I think that the real lesson to be learned is that you should beware of rules that start with "always" and "never".

</comment></comment></comment><comment><USER5> It doesn't appear to be a code-dump to GitHub, which would be indicative of fading interest on behalf of the original developer.

The ReadMe, build process, and licensing instructions all point towards this being a well planned Open Sourcing of a product.

I'm pleased with this, and hopefully it will spur development of TM2, allowing it to truly compete with the up-and-coming Sublime Text 2.

<comment><USER6> > _the up-and-coming Sublime Text 2_

Sublime Text 2 has been released since later this June[0]

[0] [http://www.sublimetext.com/blog/articles/sublime- text-2-0-re...](http://www.sublimetext.com/blog/articles/sublime- text-2-0-released)

<comment><USER7> You're confusing _up-and-coming_ , meaning "showing promise", with _upcoming_ , meaning "forthcoming".

<comment><USER6> Thanks, I'm not a native english speaker so I misunderstood the idiomatic expression's true meaning.

<comment><USER8> To be fair, that is a particularly idiotic idiom.

</comment></comment></comment></comment></comment><comment><USER9> I don't understand the negativity being shown towards TextMate in this thread. This thread is about an extremely popular editor going open source, something people have been asking for for a long time and something we should all be THRILLED about, regardless of whether we use it.

Can we save the editor wars for another thread and maybe, just maybe, actually talk about the code?

<comment><USER10> I'm personally _super_ excited. Ever since reading this[1] blog post I've wanted to cobble together an editor with some level of libclang integration, but the amount of wheel reinvention involved kept it an idle idea. As a TM user already, having the codebase available is way more than I could've asked for.

[1] <http://etoileos.com/news/archive/2010/10/15/1401/>

</comment></comment><comment><USER11> I've been using Textmate 2 for the past week or so and I still prefer 1 + missing drawer. For me, the only thing Textmate 1 is missing are split views (something 2 is missing as well).

I wonder if there is still enough interest in the app where people will contribute all of the community's desired changes -- I hope there is.

Am I one of the only Textmate users who feel that Sublime isn't the right "upgrade"? I much prefer Chocolat or Vico as they feel more like native OS X applications.

<comment><USER12> ST2 is native enough for me and cross-platform should be on the list of every programmer's requirements for a text editor.

I use Vim on OS X right now and I like it but used ST2 just yesterday (odd timing). I agree with others that it is basically the TM2 we were waiting for.

<comment><USER8> If you haven't heard of it, ST2 has something called "Vintage Mode" which adopts a fairly large subset of the Vi/m commands. It will completely change the command set of ST2 and gets close enough to Vi for most of my needs. Notably absent is the search/replace syntax of vim, but ST2's works well enough I suppose.

<comment><USER13> Vintage mode is admittedly getting more complete every day, but as a regular Vim user the absences that exist are often a jolting distraction — attempting to use some command that doesn't exist jerks you out of your flow and it feels so inefficient to then have to remember the Sublime way of doing things.

I find it more annoying than the lack of, say, Cmd + T in MacVim, so I've found myself spending more and more time using that — even after a concerted effort to use SublimeText for a while.

<comment><USER14> This is the biggest problem with learning vim. I was an Emacs user for ages, and I could use other text editors fine. The "Emacs mode" in them usually just meant readline hotkeys: C-p and C-n for previous/next line, C-a and C-e for beginning and end of line, and C-k for delete line. These are the only real movement commands I used in Emacs, and they work in almost anything. Even nano has most of these.

But now my hands are wrecked. My fingers have minds of their own. My muscles have learned a subset of vim that does not seem to be a subset of any vim plugin maintainer's usage. I'll be in Vimperator or Eclipse vi-mode or Vintage and I'll hit cf" and nothing will happen, and I'll realize I'm completely lost. I lose my chain of thought and it takes me a non-negligible amount of time to realize that I'm going to have to hold down shift and use the arrow keys like some sort of Notepad user. I become irrationally angry and wish horrible torture upon the poor soul who wrote the (altogether pretty good) vim plugin I'm using, because who are they to not include EVERY SINGLE VERB AND NOUN that my fingers have painstakingly learned over the past few years. I slam my keyboard against the table, throw the mouse through the monitor, then roundhouse-kick the tower off the desk before I set fire to my desk every time a vim plugin is incomplete.

Vim has reduced me to a pathetic teary wreck every time I have to use a text editor that isn't vim.

I wrote this in vim.

</comment></comment></comment></comment></comment><comment><USER15> Blog post explaining why: <http://blog.macromates.com/2012/textmate-2-at- github/>

<comment><USER16> This doesn't really explain much.

The way TextMate2 was going, I'd bet it's something like, it wasn't getting done, and the author was too busy or uninterested to continue, so open sourcing it probably is a way that it may actually ever get finished.

<comment><USER15> You don't think it has anything to do with Sandboxing requirements in the App Store and Mountain Lion having an option not to allow non-App Store applications? I'm seriously asking, not trying to start a flame war here.

<comment><USER17> The original TextMate isn't on the App Store and I'd have to think the number of people who go into the Security settings and disallow running of non-App Store signed binaries is vanishingly small.

<comment><USER18> That setting only changes the ability to launch an app right after download by double clicking it. It doesn't actually prevent the program from running, you just have to take more steps to launch it the first time.

However TM is being kept out of the app store for other reasons, such as it's ability to prompt for an administrator password to allow writing to protected files.

<comment><USER19> Yeah, this is not true. There is a separate setting which controls a pop-up confirming whether or not you want to run an app that you downloaded, the first time you attempt to run it. The setting referred to here actually BLOCKS the running of new non-app-store (or, by default, non-signed) applications.

<http://imgur.com/7bduA>

Apparently, you can alt-click and select open, but my gut tells me that this was an oversight and will not last long. This feature was touted as a potential way to protect children, and if overriding it doesn't even require entering a password it fails at that pretty badly.

<comment><USER20> Control-Click. I don't think it is an oversight, because "Manual Override" is touted officially as a feature:

<http://www.apple.com/osx/whats-new/features.html#gatekeeper>

And even explicitly mentioned in a system dialogue:

[http://cdn.arstechnica.net/wp- content/uploads/2012/07/gateke...](http://cdn.arstechnica.net/wp- content/uploads/2012/07/gatekeeper-anywhere-warning.png)

I am still not on Mountain Lion, so I don't know if you need to enter a password (which children shouldn't have) or right clicking is enough?

</comment></comment></comment></comment></comment></comment></comment><comment><USER21> I created a binary from the current source, if anyone is interested:

[https://github.com/downloads/albertz/textmate/TextMate-2012-...](https://github.com/downloads/albertz/textmate/TextMate-2012-08-09.zip)

</comment><comment><USER22> Awesome! Can someone comment on how probable it would be to port it to linux (and windows)?

<comment><USER23> Well it's all in Obj-C, so either a non-OS X Obj-C compilation tool or rewrite it out all out of Obj-C.

Doable, but I don't think many people will be queuing up for the task.

<comment><USER24> Erm, Objective-C probably isn't the problem, even NeXt used gcc for that (not sure how much of the current feature bloat isn't in copyrighted libraries).

But GNUStep isn't exactly up to par with Cocoa, never mind not exactly the default desktop environment. Porting the whole editor from Obj-C++/Cocoa to C++/KDE or C/Gtk would be a pretty huge task, where you'd better off starting from scratch anyway (as with a lot of GUI apps, it's mostly about the ideas, not the implementation).

<comment><USER25> Cocotron[1] attempts to be up to part with Cocoa though. I'm not sure if it would cover everything needed by TM, but it would be a pretty decent start.

[1] <http://www.cocotron.org/>

<comment><USER24> Considering that the only reason GNUStep isn't considered a totally failed project is the existence of Hurd, I wouldn't bet my money on creating a reasonably up-to-date cross-platform Cocoa/OpenStep clone. Try to see if the base Objective-C++ is working alright and create a GUI-independent layer, at least that way you'll get your native look-and-feel for free. But looking at the code, I think that'll excisce a lot of the core TM tech, so whether it's worth the conversion effort instead of just starting brand new is a pretty good question.

</comment></comment></comment></comment></comment><comment><USER26> I am not sure how to read the statement by Allan Odgaard. It does not include any hint on the future of Textmate: Has Textmate become open source abandonware or does Allan Odgaard intend to lead the future development of Textmate as an open source?

The latter would of course be great, the former rather sad from a user perspective since most former closed source apps do not survive for long after a switch to open source.

<comment><USER27> It does, actually. He mentions asking about pull requests and such, so it's clear he (or they, as it's all pluralized) intend(s) on maintaining the repository.

</comment></comment><comment><USER28> I stopped using TM1 for one reason: undo (which is one. character. at. a. time.) TM had a lot of fans, but no-one liked its undo (some didn't dislike it that much).

TM2 alpha didn't fix it. Gave up.

<comment><USER29> I've been using it since the first alpha release back in December or whenever it was released and I can say that alpha absolutely fixes the one character at a time undo (and has since the first release).

</comment></comment><comment><USER30> This is going to be a mess initially. So far there has been a pull request changing the license, and issues such as "Improve syntax highlighting performance. It sucks much compared to Chocolat or Textmate 1 currently."

<comment><USER31> "Improve syntax highlighting performance"

To be fair… watching a highlighter crawl down a small file is a complete joke

</comment></comment><comment><USER32> TextMate 2 tarball for your downloading convenience: <https://dl.dropbox.com/u/10047/TextMate2.tbz> Binary uses latest source. [I compiled and tested, works on Lion at very least]

</comment><comment><USER33> I'm confused as to how he could ever sell Textmate2 if he's linking against ragel, a GPL library. Well of course he could sell it but he'd have to release the source.

<comment> X-Istence ragel isn't a library. ragel takes input and turns it into a state machine. The output from ragel isn't under any specific license other than your own, see the section on license on the homepage (copied below) <http://www.complang.org/ragel/>

      > Note: parts of Ragel output are copied from Ragel source covered by the GNU
      > GPL. As a special exception to the GPL, you may use the parts of Ragel output
      > copied from Ragel source without restriction. The remainder of Ragel output
      > is derived from the input and inherits the copyright status of the input
      > file. Use of Ragel makes no requirements about the license of generated code.

ragel is actually extremely cool and I've used it extensively in non-open source code for various things. I haven't even scratched the surface yet with how powerful it can be and I definitely want to spend more time with it.

<comment><USER33> Thanks that makes sense. I've been meaning to get to know ragel better.

</comment></comment></comment><comment><USER34> I think this was something a lot of people had seen coming. Really at this point it was only Hardcore Textmate users left. Most people in our office use either Sublime Text 2, BBEdit, Emacs or Vim. Sublime has made the change from Textmate really easy.

It's good to see that they are sharing their hard work with the community but it's sad to see a legendary text editor basically die!

</comment><comment><USER35> I am cloning it right now before it is too late!

</comment><comment><USER36> Am I the only one surprised by total lack of history in the repository?

<comment><USER18> Considering the age of the project it probably wasn't being managed in git to begin with. I've seen some estimates that Allan started development around 2006. Git only came out in 2005 and took a few years to gain traction.

<comment><USER37> Doesn't matter. Git has plugins to pull from other repositories.

<comment><USER38> I've never used the git transfer plugins but I've had some past headaches moving to different source control systems. I usually just start a fresh repository with the latest snapshot and keep the old system available if anyone has to really go back that far (in my experience after a few weeks no one really ever has had to do this).

<comment><USER39> The product I'm working on most of the time started 2004 as a CVS repository, then was converted to svn where it spent a lot of time in and hen in 2009 it was converted to git where it lives ever since.

I've had no issues during either conversion (aside of mistyping an author name, but filter-branch helped). All branches were successfully transferred.

Now I wonder what kind of issues you were having.

</comment></comment></comment></comment></comment><comment><USER40> The best part about TextMate is that it led us to Sublime.

<comment><USER41> I'm thankful that the stagnation of TextMate led me to VIM, on both my Mac and all my Linux boxes. Thank you TM, it's been a great ride!

</comment></comment><comment><USER42> Unexpected. Just awesome.

</comment><comment><USER43> That's great :) I mostly use vim on the command line, but Textmate is great for dealing with Rails applications. I know, there is the nerdtree plugin for [Mac]vim, but the Textmate folder view is much better imho. Also, it's a nice tool for beginners, because you have zero configuration, but a really powerful editor.

</comment><comment><USER44> I think this is the right move. In my opinion, TextMate is not that great to ST2 or some other editors. Hopefully, it being open-source can get it closer to being on par. All I can say is thank god I never bought previous versions of TextMate and went with great alternatives.

<comment><USER45> On the other hand, I am very happy to have given him my money and supported the development of (what I think is) a fantastic editor.

Even though he said he would make TM2 a free upgrade, my plan was always to buy another copy to show my support.

</comment></comment><comment><USER46> So awesome. Thank you Allan.

</comment><comment><USER47> where can i download the precompiled app?

<comment><USER48> His website still has the binary on it.

<comment><USER21> Where?

On <http://macromates.com/> I just see a download to a 30 day trial version of TextMate 1.5.11.

After googling, I found <http://blog.macromates.com/2011/textmate-2-0-alpha/> which is from 2011 and provides a TextMate 2.0 alpha (r8930) download which also requires a licence key.

</comment></comment></comment><comment><USER49> Uhm...is it strange that comments are no where to be found in the source?

</comment><comment><USER50> I can't imagine this was because Textmate isn't selling well enough. I could be wrong but I always looked at Textmate as an example of an app that must be making a killing.

</comment><comment><USER51> Textmate2 needs one thing for me to use it prime-time again: split screen in OS X full screen mode (just like Sublime Text 2).

Now I can code that myself (oh joy :)

</comment><comment><USER52> Maybe now I can add my own "replace" button in the Find Dialog of TextMate 2. The only button there for me is "replace all"

Does anyone else have that issue?

<comment><USER38> I haven't used TM2 since it was first released however if its like TM1 'Replace' only appears on the single file find (cmd+f) and is not available on the find in project (shift+cmd+f) dialog.

<comment><USER53> In the Find in Project (in TM1, I've not used 2), just cmd-click to select the found instances you want to replace, and the Replace All button because Replace Selected

</comment></comment></comment><comment><USER54> Anyone else keen to add vim keybindings? ...anyone?

</comment><comment><USER55> What is the motivation for this? Is the developer losing steam / is support for it not strong enough? Either way, this is pretty cool.

<comment><USER56> Whilst the site is down at the minute I can only offer conjecture. Basically, people waited so long for TextMate 2 to arrive, when the betas finally did land it was totally underwhelming. Sublime Text 2 had already started to replace TextMate for most Mac developers and finding out that the wait for TextMate 2 probably wasn't worth it probably just made people move over. I know a year ago everyone in my office used TextMate and now none do for pretty much these reasons.

</comment></comment><comment><USER57> Personally, if I could combine the TM modules into BBEdit (which uses Applescript for scripting), I'd be perfectly happy.

</comment><comment><USER58> Is it wrong of me to be a little miffed that I paid $50, admittedly a long time ago, for Textmate?

<comment><USER5> I don't know how long ago you paid for it, but I paid £35 for my license 3 years and 1 day ago (August 8th, 2009)

3 years of use from a product, plus we now have the almost-guaranteed support and speedier development because of it's Open Source nature. I think it was worth the money.

If you paid for it last week, I can understand frustration.

<comment><USER20> > plus we now have the almost-guaranteed support and speedier development > because of it's Open Source nature

I am more pessimistic. I think the project died and open sourcing will just reanimate it to undead Zombie status.

<comment><USER59> Because it's developement were happening at a rapid pace before the releasing of the code?

</comment></comment></comment></comment><comment><USER60> Just got a tweet from @macromates, they say they'll still charge for binaries

</comment><comment><USER61> a very unexpected news for today! certainly i hope allan won't abandon it but keep leading this open source project.

</comment><comment><USER62> Linux (QT/Gtk) port.. Anyone? Anyone?

</comment><comment><USER63> Can it be ported to run on GNUStep?

</comment><comment><USER64> Time to go bug hunting...

</comment><comment><USER65> Where can I donate???

</comment><comment><USER66> another victim of the second system effect?

</comment><comment><USER67> Alas, there is no linux port.

</comment><comment><USER68> If we are completely honest, Textmate was always a sub-par editor.

No Vim or Emacs style brilliantness, no BBEdit style tons of features and mature engine, no IntelliJ like, er, intelligence, no ST2 comprehensiveness, etc etc.

Plus, the Textmate 1.x text engine was probably a mess too -- I remember the very first versions being laggy (and that's coming from someone who doesn't find even Eclipse laggy). That he couldn't easily fix the one-character-undo is another pointer to that (and, for all I've seen, the 2.x engine is not that better).

It's main saving grace was the many extensions it had, and looking half-decent and native on OS X. Basically, it caught on because it appeared on the right time, and appealed to OS X users like web programmers etc, that wasn't old- time unix buffs, and wanted something native looking without forking for BBEdit (which itself was/is Carbon based and with a custom text display widget).

I don't think Textmate deserved all that success --it should have happened to a better editor.

<comment><USER69> > _I don't think Textmate deserved all that success --it should have happened > to a better editor._

This doesn't make much sense. You say it appeared at the right time and was pretty and native on OS X. What better editors deserved the success it had that were available when TextMate was popular? Which editors were available on OS X that weren't giant IDEs or VIM and Emacs style editors that have a ridiculously steep learning curve?

I've never understood when people say something or someone doesn't deserve something they've earned. TextMate is successful because lots of people bought it. Who are you to judge what deserves to be successful and what doesn't, especially when history seems to disagree with you?

<comment><USER68> > _This doesn't make much sense. You say it appeared at the right time and was > pretty and native on OS X. What better editors deserved the success it had > that were available when TextMate was popular?_

The fact that there wasn't another editor that fulfilled all those "needs" at the time (mainly: simple, native looking) does not mean that it deserved it's success. What I'm saying is, it got its success not because it was better than the competition but because there was no competition.

So, I just wish that something better was available at the time with the wanted features (mainly: simple to use and native looking). It would have been better for all in the long run.

> _I've never understood when people say something or someone doesn't deserve > something they've earned._

You can earn something for lots of reasons, not always the best of reasons. Nixon got the vote, doesn't mean he also deserved it.

> _TextMate is successful because lots of people bought it. Who are you to > judge what deserves to be successful and what doesn't, especially when > history seems to disagree with you?_

Actually history seems to agree with me. Did you miss the part when the promised 2 was worked on for 6 years, got nowhere, delivered a disappointing alpha and got dump half-heartedly (as in with GPLv3, with the intention to continue commercial development) on GitHub?

<comment><USER70> > The fact that there wasn't another editor that fulfilled all those "needs" > at the time (mainly: simple, native looking) does not mean that it deserved > it's success. What I'm saying is, it got its success not because it was > better than the competition but because there was no competition.

What kind of doublespeak is that? If there's a race, and you're the only one who shows up, you deserve to win, period.

<comment><USER71> Uhh, the same doublespeak as pointing out that if you're the only one to show up for a race, you're also the one that came in last?

</comment></comment></comment></comment></comment><comment><USER72> Found a build from today.

Not sure why this is supposedly so much better than ST2. I've got a Go(lang) bundle installed and a theme that I prefer over my current ST2 theme, but I like the file browser, tabs and menu layout better in ST2.

</comment><comment><USER73> Well TextMate was cool while it lasted. Good thing I switched to Sublime Text 2 a week ago.

<comment><USER74> So you think open-sourcing is a death sentence? Care to share your reasons? (genuine question)

<comment><USER75> I think it's a _likely_ death sentence, although certainly not a guaranteed one.

Clearly a lot of open source projects maintain active development for years/decades. But there aren't many success stories I can think of offhand when it comes to closed source commercial software converting to open source. The only big one that immediately comes to mind is OpenOffice, and it's a pretty unusual case -- it still had an actively-maintained proprietary commercial version developed alongside the open source version for years.

If you as an individual think having your editor be open source is critical, you weren't using TextMate anyway. If you _were_ using TextMate, you're probably willing to pay $50-60 for a "Mac-like experience" with a closed source but extensible editor. That mostly leaves diehard TextMate 1.x users and people who would try a free TextMate for beer rather than speech reasons. I'm dubious as to whether there's a sufficiently talented and motivated subset of that group to seriously further TM's development.

Lastly, I'm not sure there's _room_ for another "industrial strength" open source text editors. It's probably not much of an exaggeration to say that there's Emacs, Vim, jEdit for the weirdos, and Things Only Their Authors Use. And in all of the big cases, those editors are cross-platform. TM isn't and won't be for years, if ever.</comment></comment></comment>


Have Your MVP Running Publicly Within 15 Minutes with Serverless - <AUTHOR> http://www.theodo.fr/blog/2017/02/have-your-mvp-running-in-prod-within-15-minutes-with-serverless/ <comment> wc- I'd change the title to: "Have your MVP publicly facing within 15 minutes with Serverless." IMO takes a lot more care, planning, and implementation details before I'd consider something a truly prod env.

<comment><USER2> It's basically a wrapper around a AWS Lambda function. So, Amazon is the one doing the care, planning, and implementation.

<comment><USER3> That's not how it works.

<comment><USER4> How does it work?

<comment><USER5> The reality involves a bunch of Engineering by Poking[1]. Did you get all the IAM roles and permissions right? How about hooking up logging--more IAM poking. S3, database service, CloudFront? More poking. Certificates, CORS, domain aliases, throttling, etc?

The docs might be clear but you need to already be an expert in how everything fits together to do it right the first time. The problem is an embarrassment of riches in terms of configurability, which makes for longer learning and poking times.

1: [http://lambda-the-ultimate.org/node/5335](http://lambda-the- ultimate.org/node/5335)

</comment></comment></comment></comment></comment><comment><USER6> I'm not sure any product is viable if it takes just 15 minutes to develop and deploy.

Aside from that: Does serverless support proper staging, documentation of all deployments, role management (who is allowed to deploy to what stage)?

Because that would be actually useful for us.

<comment><USER7> You can manage staging with API Gateway, the AWS service that manages which HTTP requests map to which Lambdas.

All functions are documented in Lambda with a description if you wish, and of course because the serverless framework is basically a node app with hooks that knows how to turn it into a Lambda app, you can use any/all of the same tools as you would for documentation.

Role management is standard AWS IAM.

At my workplace we like the idea of Lambda, but we're building some Ansible playbooks for it and banning Serverless.io because part of its structure is that all devs have root access to CloudFormation and can do whatever they feel like. For smaller shops, it's probably ideal though.

</comment></comment><comment><USER8> This is such a good idea! I love the concept of an MVP to quickly test your business ideas. However it's usually painful to rapidly test an idea in the web world. The fact that I can run such a test so quickly is amazing. Keep it up!

</comment><comment><USER9> Cool idea. Anyone know of a python flavoured version of this?

<comment><USER10> Serverless[0] does support Python (the CLI tools are just written in JS). Zappa[1] is also a great option if you want to deploy a WSGI compatible app (flask/django) to Lambda.

I've used them both in production and have had a great experience.

[0] [https://serverlesscode.com/post/python-on-serverless- intro/](https://serverlesscode.com/post/python-on-serverless-intro/) [1] [https://github.com/Miserlou/Zappa](https://github.com/Miserlou/Zappa)</comment></comment>


The Known Universe Scientifically Rendered For All to See - <AUTHOR> http://www.amnh.org/news/2009/12/the-known-universe/

<comment><USER2> Celestia (<http://www.shatters.net/celestia/> ) is an interactive version of the known universe, similar to the video.

One thing I like about it is that it can show you how fast you're traveling, which really helps give a sense of scale.

It's the Total Perspective Vortex, for sure.

<comment><USER3> Thanks for the link. I've just watched a sunrise from the surface of Mercury. :)

<comment><USER4> Try Saturn, through the rings.

</comment></comment></comment><comment><USER5> Though the model may be accurate, the fly-through speed is impossible to comprehend, and compromises any feeling of scale or distance. You still have to read the captions and think. Anyone know of an animation that's a little more appreciable in that regard? I imagine it's terribly boring.

Edit: (8.8e26 meters) / (mach .8) = 1.02435103e17 years to traverse the known universe in a 767. Well that doesn't help.

<comment><USER6> yes, i was thinking about the same thing. as far as i can tell, the velocity is increasing exponentially throughout the video.

an alternative would be to use a constant velocity and, once that gets boring, go back to the start and use a velocity ten times faster. that way, you'll know when the velocity increases and you'll also see what was shown earlier flash by, giving a sense of scale.

unfortunately, i don't know of a film that does that.

</comment></comment><comment><USER7> An inaccuracy at 4:57 to 5:00 where it shows the Moon's orbit to be circular(ish) around the Earth. If you show the Moon's orbit without the Earth it isn't a set of spirals, it's almost indistinguishable from a circle(ish) around the Sun. The size of the orbit around the Earth is so small compared with the distance to the Sun that the Moon's orbit looks like a circle with a slight wobble.

It's even convex.

[http://en.wikipedia.org/wiki/Orbit_of_the_Moon#Path_of_Earth...](http://en.wikipedia.org/wiki/Orbit_of_the_Moon#Path_of_Earth_and_Moon_around_Sun)

<comment><USER8> Moon's orbit is usually depicted around the Earth. Even if the choice is arbitrary it's also customary to claim that an object orbits the closest large mass, or more precisely the object and the large mass orbit the center of mass of the combined system. It is also customary to depict several orbital systems in a single image using multiple frames of reference, the context can usually be deduced even bay laymen.

More generally, your comment is inconsistently heliocentric. Why not pick the center of the Milky Way as the reference for orbit instead.

<comment><USER7> I'm not sure if you've simply missed my point, think my point is wrong, or actually disagree. Let me address your points.

Of course the Moon's orbit is usually shown as being around the Earth (and more generally, the Earth around the Sun, and other moons around their nearest large mass, _etc._ )

What I'm saying is that they have shown what might be interpreted as the paths of the planets as those beautifully rendered trails. People will interpret them as trails, and when they do so, the "trail" of the Moon is thoroughly misleading.

And I'm not being arbitrary, I'm interpreting those lines as trails, and hence your comment about using other points of reference is irrelevant. The Sun in this rendering is not shown as having such a trail, and hence the image shown is heliocentric.

And when using this sort of image for laymen, they may correctly take from it that the Moon orbits the Earth, but in my direct personal experience, they then believe that the Moon makes a loopy orbit when seen from a heliocentric point of view.

Which is wrong. Hence my comment.

</comment></comment></comment><comment><USER9> The original is still the best: <http://www.youtube.com/watch?v=A2cmlhfdxuY> It's the Powers of 10 documentary made by Ray and Charles Eames.

</comment><comment><USER10> In the video, it's shown that there's a cone of galaxies and quasars that we have seen and mapped. What prevents us from mapping the dark areas? Is the the light from the other stars in the Milky Way?

<comment><USER9> Layman to layman: it's light that hasn't reached us yet. Light that left alpha centuri last week wont get into the cone of visibility for another 4 years. During the period of inflation after the big bang some parts of the universe got so far away so quickly the we will never see those parts.

<comment><USER11> I think you're confusing the boundary of the visible universe (the limit after which light cannot have reached us yet) and the unmapped volume the poster is talking about, which is inside this boundary. Alpha Centauri was emitting light just fine 4 years ago so of course it can be mapped.

</comment></comment></comment><comment><USER12> It's quite pretty, but to me still doesn't quite manage to convey the real scale of the universe. We zoom so rapidly from the solar system scale to the galaxy scale that we forget we've just gone up in scale by a factor of 10^9 or so. Then we do the same trick again in going from galaxy to universe scale.

<comment><USER4> I don't know.

What really hit home for me was the sphere of radio signals. To imagine that, from all that space we can see, that tiny little bubble is the frontier of where man is detectable.

For the rest of the universe, we are only a tiny speck of rock orbiting a very average and unremarkable star.

<comment><USER12> If it's any consolation, for anyone more than 4.5 billion light years out the rock and the star don't exist yet either.

<comment><USER4> Good point ;-)

</comment></comment></comment></comment><comment><USER13> Direct link to the video on YouTube: [http://www.youtube.com/watch?v=17jymDn0W6U&hd=1](http://www.youtube.com/watch?v=17jymDn0W6U&hd=1)</comment>


Show HN: Ggraph – A graph visualization library for big messy data - <AUTHOR> https://gransk.com/ggraph.html <comment><USER2> It's hard to get a sense of how it'll look with 'big messy data', when it's sample dataset is so tiny, at 14 nodes and 21 edges. Is there a way to see how it'll look with a ton of data?

<comment><AUTHOR> Still not big, but a bit bigger: [https://gransk.com/ggraph- bigger.html](https://gransk.com/ggraph-bigger.html) (Nodes: 611 Edges: 2499)

Edit: It's an excerpt of the Enron dataset.

<comment><USER3> Thanks, that really helps.

It's getting noticeably laggy for me at 611 nodes, but below you say it shouldn't be a problem to do 5k. Have you actually tested that kind of data?

<comment><AUTHOR> Yes. Animating node placement has a significant performance penalty, so in larger graphs you would either hide edges or everything during the first couple of seconds.

</comment></comment></comment></comment><comment><USER4> Perhaps related, I came across this graph library comparison page recently. You can try out the different libraries with different graphs and sizes:

[https://anvaka.github.io/graph-drawing- libraries/#/all](https://anvaka.github.io/graph-drawing-libraries/#/all)

</comment><comment><USER5> How big can the graph get before taking a noticeable performance hit? A better example should be shown if advertising "Big ... Data"

<comment><AUTHOR> It's built on top D3 and has the same limits (5k nodes 20k edges should not be a big problem). The "big" here is the ability to combine nodes, for two reasons:

1 Reduce the number of nodes and edges, thus increasing capacity

2 Combine nodes that should be seen together (e.g. alternative spellings and typos), to better deal with the variation-aspect of big data

I see your point, but the goal here was to illustrate the functionality.

<comment><USER6> I don't have the time to try it right now, but can it visualize 20M-100M nodes?

<comment><AUTHOR> At some point you run out of pixels on your screen to visualize it all at once. My suggestion in your case is to not use a graph at the beginning, but rather grouping nodes based on metrics. Like degree (group all contacts of a node that only has a single contact), communities, or value (e.g. IP's in a subnet).

</comment></comment></comment></comment><comment><USER7> Congratulations! This seems to be a great tool to effectively display large graphs.

Maybe you can submit it to Neo4j team to be showcased in their "Graph Visualization for Neo4j" section: [https://neo4j.com/developer/guide-data- visualization/](https://neo4j.com/developer/guide-data-visualization/)

</comment><comment><USER8> We used [https://www.graphistry.com](https://www.graphistry.com) to visualize hundreds of thousands of edges for [https://www.yesgraph.com/twinmaps/](https://www.yesgraph.com/twinmaps/)

The minimal eng required to get good performance was wonderful

<comment><USER9> For the yesgraph link I had to disable ad-blocking and uBlock Origin and still only got a static image with a Twitter advertisement pasted over the image.

The graphistry link is similarly useless just showing a few sentences on a few pages ending in a "request demo" button.

<comment><USER10> Hi Ivan, always cool to see what Graphistry users are doing!

Phil, sad to hear you weren't able to see your Twin Graph. Many of us use ad blockers and this is the first report we've gotten like yours, so we'll dig in. Meanwhile, you may be able to try a direct link to my own YesGraph TwinMap: [https://labs.graphistry.com/graph/graph.html?dataset=lmeyero...](https://labs.graphistry.com/graph/graph.html?dataset=lmeyerov_twitter_graph) . (Note: best on laptops, and we recently relaunched with Falcor/React, so currently porting all our page load optimizations.)

For more information about graphistry, we have users piloting the three below layers of our stack. Because we can load 10-100X more data at the visual tier than other systems here (so 100K-1M+ things), people have been exploring connections across events/entities for some fascinating reasons:

* Investigation & Response -- Connect to systems like Splunk and get rich, scalable visual graph views and easy workflow automation. Ex: build an investigation template that takes an indicator of compromise and runs queries that connect it to various users, devices, alerts, etc. Or, "here are our ssh trails and anomalies around them."

* Exploration: Data scientists and data analysts will explore connections in their events or samples, e.g., for week-over-week model tuning, security research & forensics, & even now loan analysis. They'll load in a bunch of events or samples where each may have a lot of attributes (IPs, times, amounts, ...), and then they can see correlations. Ex: most false positives are from events with 3 particular combinations of characteristics, or an outage involved 4 distinct phases of behavior and entities.

* Developers: folks building internal apps for scenarios like the above.

For the latter two use cases, a good place to get started is our API: [https://github.com/graphistry/pygraphistry](https://github.com/graphistry/pygraphistry) . Feel free to contact us at info@ if this may solve a problem for you. (And.. we're hiring! Help us build web-based visual tools with GPUs acceleration to solve real data problems!)

</comment></comment></comment><comment><USER11> What are typical use cases for this?

<comment><AUTHOR> To better understand complex relations in the data you are analyzing, e.g. friendship in social networks.

<comment><USER11> Do you know of any good sources for example data like that? Would be fun to explore.

<comment><AUTHOR> Stanford has a good collection of network datasets: [https://snap.stanford.edu/data/](https://snap.stanford.edu/data/)

</comment></comment></comment></comment><comment><USER12> The lasso seems to "give up" if too many nodes are selected - stops updating the selection although it keeps updating the green shaded area. Perhaps it should only give up on the text labels which are probably the expensive part.

</comment><comment><USER13> I really like the grouping features and the labels work pretty well too (though it's slow with large groups). Thanks for sharing your work!

<comment><AUTHOR> Thank you! I know, there are plenty of things yet to be done to make this thing run smoothly.

</comment></comment><comment><USER14> "Hold right click and drag" doesn't work on laptops with trackpads.</comment>


Fintech Firm Plaid Raises $44M - <AUTHOR> http://www.wsj.com/articles/fintech-firm-plaid-raises-44-million-1466377808 <comment><USER2> Note that there are _two_ major security flaws in Plaid when it comes to authentication:

Since banks don't provide secure mechanisms for third-party authentication and authorization, e.g. OAuth, Plaid receives you credentials in plain text and will then use them to communicate with the bank. So you really have to _trust_ Plaid.

The second weakness is even more dangerous: Apps implementing the Plaid authentication flow will show the Plaid "login page" with bank selection in an overlay on their own sites. Since this is _not_ a redirect again, you don't even see whether your credentials are transferred to Plaid or the third-party app. That is, you have to trust your bank (sure!), Plaid (okay!) _and_ the app using the auth flow (dangerous!).

You should fix this!

<comment><USER3> The lack of secure auth mechanisms is exactly why companies like Plaid (and Yodlee, and Dwolla, and Intuit) exist. Take away that constraint, and this is easy enough to package as a library and not a product.

Many "disruptive" industries like this "API on top of legacy systems" segment are merely arbitrage schemes; they profit from entrenched players' greed and apathy. Luckily, banks are starting to wake up.

As such, it's not really Plaid's responsibility to "fix" this problem, it's the banks'.

\- [http://www.americanbanker.com/bankthink/a-neobanks- prognosis...](http://www.americanbanker.com/bankthink/a-neobanks-prognosis- for-neolithic-banks-1081347-1.html) \- [http://www.americanbanker.com/news/bank-technology/wells- far...](http://www.americanbanker.com/news/bank-technology/wells-fargos-bid- to-vanquish-screen-scraping-1081367-1.html)

<comment><USER2> But this isn't helping matters in any way, is it?

Just because this is the reason for Plaid's existence doesn't mean you should make a product where security cannot be guaranteed for the user. Some things just shouldn't be done, because they're not possible yet. Not possible because support from the banks is lacking.

<comment><USER3> You're asking Plaid to leave $44 million on the table and walk away in the name of best practices. This is noble but unrealistic.

The security flaws boil down to the requirement that the end user must place their trust in Plaid. Plaid considers themselves trustworthy and competent enough to act on the customer's behalf with their credentials. If Plaid were to suffer a breach, their customers would not purchase their services. It is in everyone's best interest to avoid security breaches.

The web product is indeed worrisome, but it's also in Plaid's best interest to avoid dealing with fraudsters.

The unfortunate part here is that the banks have zero liability in the case that their customers lose money due to a breach of their online banking credentials. This could be solved via legislation, and I'm willing to bet that would light a fire under the industry to start embracing options such as OAuth and restricted-access credentials.

</comment></comment></comment></comment><comment><USER4> This is really good news. Plaid has an amazing API, it makes it very easy to get your own financial data. I'm trying to analyze my own spending habits / make a budget-allocator using my own patterns, so it's been insanely helpful. My big fear with all small SaaS's if they just suddenly shutter, so a new round of fundraising is always good news :)

<comment><USER5> Also good news because they are currently being sued by Yodlee for patent infringement. Shameful anti-competitive bullshit on the part of Yodlee, who let their product get so bad it opened up the door for Plaid. Now Yodlee are trying to litigate instead of compete.

<comment><USER6> it's good to see Yodlee have some competition, hopefully they can choose to compete as well.

</comment></comment></comment><comment><USER7> I'm going to be really rude here (forgive me) but I feel like every time a security question comes up you dodge the question really hard.

I want to know one thing: If I log into your service with my bank credentials. Do you store these as plaintext files (or "encrypted" files of which you have the encryption key)? Yes/No.

Furthermore, congratulations! I've been trying to start something up like this in Europe but I feel like there are way more restrictions in Europe on banking data and this kind of third-party aggregation. Sorry for being so rude.

</comment><comment><USER8> For those interested in a European perspective, the Revised Payment Services Directive (aka PSD2) will in a similar fashion to Plaid's API, force banks to offer APIs for not only client information but payment. If implemented it will probably create radical change and opportunity in FinTech across the EU.

<comment><USER9> If anyone is interested in working on a PSD2-based project drop me a line: email@example.com.

</comment></comment><comment><USER10> Congrats Plaid! Opening up banking data via API is a great enabler for fintech startups to create valuable apps. I interviewed them a couple months back: [https://medium.com/get-put-post/how-plaid-s-api-brings- finan...](https://medium.com/get-put-post/how-plaid-s-api-brings-finance-into- the-21st-century-efc174028f09#.si7lqyoik)

</comment><comment><USER11> Was just looking at Plaid this weekend, seems really slick. The only thing that gave me brief pause was no public pricing (or indication of order of magnitude).

<comment><USER12> Hey Matt - Working on getting that up on the website! In the meantime, feel free to email me (charley@plaid) and I can send over details.

<comment><USER13> From my experience so far, sending an email to Charley is essentially the same as finding the info online since he answers so quickly! Great onboarding, I was really impressed!

<comment><USER12> Thanks Matt!! As always, happy to answer any and all questions on Plaid :D

</comment></comment></comment></comment><comment><USER14> Badass team and product - I don't think people realize how difficult what they're doing is. Super pumped for them!

<comment><USER15> What do they do? The article doesn't make it clear. It just discusses them finding alternatives to screen-scraping customers bank accounts after being given the credentials.

Seems like a startup-y Mint.

<comment><USER16> Co-founder of Plaid[1] here. We build an API for developers to connect to their users bank accounts. All the 'startup-y Mints' are our clients.

[1] - [https://plaid.com/](https://plaid.com/)

<comment><USER17> This API looks really shiny and well documented, kudos!

Do you only screen scrape or have backend/backoffice/negotiated integrations with various banks? How do you deal with enduser bank credential storage (both technically and legally when dealing with bank ToS)?

Also, in your experience, have any standards like OFX actually achieved critical mass for adoption amongst banks, and has that made your team's lives any easier?

<comment><USER16> Thanks for the kind words :)

For the top 14 banks we work closely with the banks to build connections - however for the smaller and mid-size banks we work and connect with a variety of vendors that serve those banks.

I personally sit on the OFX consortium (and a couple other financial standards committees) and I'm not overly bullish. I'll just leave this link here.... [https://xkcd.com/927/](https://xkcd.com/927/)

<comment><USER17> That XKCD strip is very true for financial standards. :(

I think you missed a question (unless it was intentional :), but how do you deal with enduser bank credential storage (both technically and legally when dealing with bank ToS)?

For example, on the technical side, do you store the credentials themselves or just session tokens/cookies?

</comment></comment></comment></comment></comment></comment><comment><USER18> Played around with the API a little bit.

Cool discovery: if you search for a financial institution, they return logos as Base64.

Super rad.

</comment><comment><USER19> Seems like the auth flow doesn't redirect to the bank's website. Does that mean that my bank credentials are sent to Plaid?

<comment><USER20> Yes. That's how Mint, Digit, and all the other "connect your bank account" apps tend to do it. I've yet to find a major US bank that has anything resembling an OAuth style flow - the closest I've seen is Wells Fargo offering generation of read-only credentials.

<comment><USER19> Then how is Plaid more secure than Mint, Digit etc?

<comment><USER21> From this [1], they are a _provider_ for these.

1: [https://news.ycombinator.com/item?id=11939425](https://news.ycombinator.com/item?id=11939425)

<comment><USER19> So, they're just punting the liability to another 3rd party (Plaid)? Given all the excitement in this thread, I was hoping that Plaid had miraculously cajoled the banks into an OAuth integration.

But I guess banks will be banks :-/

<comment><USER22> I'm sure something of the sort is in the works; it's hard to build momentum with a ton of huge legacy systems to bring along.

</comment></comment></comment></comment></comment></comment><comment><USER23> Awesome news! Does anyone have a detailed and unbiased pros/cons of Plaid vs. Yodlee - thinking about integrating with one for my startup. I think the one area I'm most interested to learn about is the data quality / depth / breadth - do they offer the same? Which one is better? Why?

<comment><USER24> regarding data quality, i've run into a few issues using Plaid for pulling my own financial data. I've had instances where the API returns duplicate charges that I confirmed were not duplicate on my bank/credit card accounts. I've also had issues with their unique id's too. When a pending charge is returned by the API they give it a unique id, but when the charge is approved they generate a new unique id for the approved charge rather than updating the existing id. They provide a field that lists the pending charge id for you to match up but i've had instances where those id's don't match up. I also run into issues where the access token for some of the institutions i have connected expire or suddenly don't work anymore so I need to re-authenticate.

I've emailed them about these issues and they've been pretty helpful in getting them resolved, but it made me reconsider building a consumer facing product on their API. I still use the API for personal use however

<comment><USER16> Totally understand on the duplicate issue - its a hard problem to solve with the inconstancies from the end financial institution, and it definitely caused issues for us in the past. That being said - we've spent the past couple months making some huge improvements - I definitely encourage you to give it another shot.

Though I'll admit, I'm quite biased!

</comment></comment></comment><comment><USER25> Hey Plaid, interested user here! I'm curious why you don't have any pricing on your web page. I'm just trying to do a back of the envelope calculation on how much it would cost to use your service.

<comment><USER12> Hi Sunny - Feel free to email me (charley@plaid) and I can get you set up and share more deets! We're working on getting pricing up on the website soon :).

</comment></comment><comment><USER26> Does Plaid have billing data? Ostensibly, that was the motivation for Intuit's Check acquisition.

It's also interesting that Intuit runs their own massive aggregation effort that they haven't attempted to product-ize...

<comment><USER27> You've heard of Mint ([https://www.mint.com/](https://www.mint.com/)) right?

</comment></comment><comment><USER28> As a future consumer of your services, I'm excited to hear this, congrats!</comment>


Flask-SQLAlchemy Caching - <AUTHOR> http://www.debrice.com/flask-sqlalchemy-caching/ <comment><USER2> I think people often too quickly turn to caching when their problem is in the ORM/db.

I generally like having models, but I'm not a fan of ORMs (ie, OM without the R).

Where this often gets you is lazily loading attributes in loops -- it kills performance and it's non-obvious that it's happening when just looking at code. Caching helps this problem by either putting the related attribute in- process memory or in memcached which is relatively faster than hitting the database. But, to me, this seems like an anti-pattern -- caching instead of optimizing the underlying ORM/db access patterns or preferring a style in which database accesses become apparent in application logic instead of being masked by the ORM.

That being said, I think this is a very good article showing how to do caching when you really need it - I'm just trying to say that you probably don't for generic database accesses.

And if you're going to do it, I'd always just start out with redis/memcached to begin with - even on a single box, you're probably going to want a process per core, and having every process keep a copy gets expensive fast.

<comment><AUTHOR> I agree, with the exception in the case of a single web process, I wouldn't use memcached but just rely on dogpile memory backend, which is about the same (unless I'm proven wrong :p). I'd use memcached otherwise to share the benefit of caching poll. Caching in redis, I'm not fan, since redis will write to your drive at one point (a large cache would impact your drive space and slow down redis start dramatically).

I share your opinion on lazy loading, whenever possible you should know in advance if you're gonna need that related information before you pull your data: it's much faster to chain 2 select in one call (even with a connection pool).

That being said, I love redis and wish I could use it for all my objects (I could, since all my objects have a pk) but it lacks some fondamental real world features (relationships and transactions) that make business query/analytic a nightmare (who connect to your website, at what rate...) while Relational database does an amazing job at it.</comment></comment>


Obama administration defends massive phone record collection - <AUTHOR> http://www.reuters.com/article/2013/06/06/us-usa-wiretaps-verizon-idUSBRE95502920130606

<comment><USER2> So if metadata is somehow distinct and separate from what it describes, does this mean that the shutdown of sites pointing to copyright-infringing content will stop?

</comment><comment><USER3> Defending illegally spying on Americans and trashing the US Constitution...yup that sounds like Obama. We are the enemy. Pay your taxes citizen.

<comment><USER4> First, you claim it's illegal, but it has not been found to be illegal in a court of law.

Second, you claim it's trashing the US Constitution, but it was a power granted by Legislative branch, performed by the Executive, after a Judicial order. All the checks and balances you could ever ask for.

Third, he's properly referred to as, "President Obama," while he is in office. Respect the title, if not the officeholder.

Fourth, it's the Administration, not President Obama himself.

Fifth, grammatically it should be, "Pay your taxes, citizen."

Sixth, the law was written by President Bush's Administration and the Republicans in Congress. So blaming "Obama" is particularly stupid.</comment></comment>


Will Android Tablets Overtake the iPad? - <AUTHOR> http://www.thefastertimes.com/venturecapital/2012/10/06/will-android-tablets-overtake-the-ipad/

<comment><USER3> Doubt it. If as rumored, Apple releases an iPad 'mini', I think this is going to be an iPod situation, where the iPad grabs the majority of the market, with a lot of smaller manufacturers picking up the bits.

The thing is, Android phones were pushed by carriers. For tablets, this hardly happens, although I've seen "free tablet" promotions when getting a new internet connection, but that has hardly done anything as far as I know.

The Kindle fire might be a contender, but I don't really see that as a real competitor, and that's hardly Android. The software quality also seems to be very low to me.

And then you have Microsoft, who is making the same mistakes they made the first time they attempted a tablet. They will maybe sell a few, but if the sales of the 'ultrabooks' are a good indication, expensive tablets running Windows 8 won't be the success they're hoping for.

I know dozens of people having an iPad, including complete non-techies - <USER2> they love it. I know 1 guy with an Asus transformer, and 1 with a Galaxy tab 10", both IT guys. The Asus's build quality surprised me in the positive way, the galaxy tab in a negative way. In both cases, I thought the software was useless, and web-browsing slow and sluggish compared to my iPad 2.

That said, the moment I can order a Nexus 7 here in Belgium, I'm getting one.

<comment><USER4> Not only is iPad popular, it also has an incredible amount of market penetration in a large variety of industries. You have airlines talking about giving their pilots iPads, you have schools giving their teachers and even students iPads for classroom learning, you have car manufacturers designing iPad docks for their cars with integrated controls... it is extremely unlikely that a competitor is going to be able to dislodge Apple from their leadership position anytime soon.

If I were an Apple competitor, I'd forget about tablets and smartphones and invest all my resources into developing the next "big thing". The reason is simple: Apple cannot be beat in its own game. So the smart thing to do is to innovate a new revolutionary device, something that will create its own category.

I'd say that the only tech company who has the potential for a feat of this magnitude is Google. However, even they seem to lack focus. That's one huge advantage that Apple had: Steve Jobs' extraordinary focus and vision. If you look at Google, you see that they are working on cool projects like self- driving cars and Google Glass, but what they need is a technology that converges these inventions in one physical device. And that's possible only with a visionary who will set a clear and ambitious direction for all the various product teams at Google.

<comment><USER5> I think there's a ton of room at the bottom. There was a time when DVD players and home computers were out of the reach of the poor or lower middle class. Most (responsible) people can't afford a $500+ tablet, but the promise of tablet computing still applies to them. It's a lie to say that the promise of tablet computing can only be accomplished in a $500+ package. There's tons of room for tablets that are $250 or less - see all the "low end" phone providers that now have Android offerings.

<comment><USER4> There's room at the bottom in terms of market share, but not in terms of profits. Android has captured a bigger chunk of the market, but Apple is reaping most of the profits (since their devices are high margin) and is therefore in a much better position to innovate or leap on opportunities that come up. I mean, who wouldn't want to sit on a $120+ billion warchest?

What's going to be really interesting is when (if) the "iPad Mini" comes out. There was an article on Ars Technica last month where a reputable analyst claimed Apple found a way to manufacture iPad Minis with the last generation iPad's components. What that means is that even at $250, the device would be extremely high margin. Not only would it allow Apple to lock down the lower market segment, but also result in a huge increase in profits. From a business standpoint it seems like a no-brainer, because a cheap and yet high margin device is not something that other manufacturers can imitate.

<comment><USER6> _Apple is reaping most of the profits (since their devices are high margin)_

Why would anyone except Apple and its shareholders care about that? Why would _you_ care about that?

All that should tell you is that you (as an Apple-customer) is getting ripped off.

<comment><USER4> Not really. As a consumer, I am perfectly fine with Apple making a lot of money as long as I get a high quality product with superior user experience. The way I look at it is that I am paying a premium for a much better product.

</comment></comment></comment></comment></comment></comment><comment><USER7> Sure it will. Everyone who says no is (in my oppinion) trapped in the famous Apple Reality Distortion Field(tm) ;)

It's the same situation as with the iPhone. People said "noooo, the iPhone will be king ever!!". Nope.

Honestly, how could this even happen? With one tablet? Or 2? Where is the diversity? Where is the low-cost option? Where is my tablet with keyboard attached? Etc. etc.

Android is about choice, it will win this, undoubtly. As long as Apple doesn't produce much more different hardware and delivers at much more price points, they will not win the whole market. But well. There is always the high margin top niche ;)

<comment><USER8> No, I can't think of a single product range where apple held on to a 70%+ market share despite lower priced and more feature rich competitors _cough_ ipod _cough_

<comment><USER9> The iPod has 4 form factors.

<comment><USER8> The iPod had one form factor for the first three years of its existence. The iPad already has two distinct models and is likely to have a third this month.

<comment><USER9> Are you calling the iPad 2 and iPad 3 "distinct models"? I think you are misunderstanding what the grandparent means when they talk about choice.

</comment></comment></comment></comment></comment><comment><USER10> If the iPad mini (or whatever it is actually going to be called) comes out at $199, that takes a lot of room out of the bottom of the market. If it is $249 or $299, then the Fire and other have some space to play in.

<comment><USER11> I think Apple's strategy here is to keep the price pressure on for the bottom end of the market. Since their margins are better than everyone else's, they can give up some of that to deliver a 7" tablet that keeps the competitors in the low price ghetto, struggling to compete, while Apple makes $$ in the high end.

<comment><USER12> I absolutely agree but I am guessing a price tag of $300 which wouldn't quite achieve keeping the competition at bay.

<comment><USER10> If they repeat their iPod strategy, then $199 is the 7" price point. I agree, $299 is not going to put the pricing pressure on the low-end.

<comment><USER11> So, the Shuffle was to put pressure on the low end of the market?

If they can sell a tablet for $199 and still make $30 profit, they can put a heck of a lot of price pressure on competitors.

</comment></comment></comment></comment></comment><comment><USER13> All I know is that after having the Nexus 7, the iPad (3, which I also have) feels like a burden to use. Slower, double the weight, not nearly as portable and iOS feels byzantine next to Jelly Bean. My friends universally share my sentiments.

Apple market share will fall over time to 20%-30%. You can't cater to all users with a single product, and they don't bring enough innovation to make their product a must-have.

</comment><comment><USER5> My other take: no. Highly customized Android tablets will (Kindle Fires and Nook tablets). Most users don't root them; they use them as the walled gardens they're designed to be. Pure Android devices will continue to be a minority. (OP's article suggests this, with Fire's at 50% of "Android" tablet market)

To me, a Kindle Fire is as much an "Android" tablet as a Nexus 7 is a "Linux" tablet.

<comment><USER9> I don't think Fred Wilson's point was that they all use a common interface. Being a VC, I'm guessing the app compatibility is the thing he cares about.

</comment></comment><comment><USER14> Android may overtake iPad or not - what are the real implications of this? I think it'll stabilize at some point where either iPad has a little more market share or little less.

Practically it doesn't really matter much. The iOS ecosystem will still thrive, and I personally I think a bit of healthy competition is exactly what we need to push the tablet world forward.

</comment><comment><USER15> How come people don't consider ebooks as competitors to the tablets?

I had a first generation iPad, and I did use it a lot. Then I got a notebook computer to replace my desktop, and started using that more.

I pretty much only used my iPad while commuting, and then made an impulse purchase and got a Kindle. I find I use my Kindle all the time, and haven't touched the iPad for over a year now.

As a device that sits between a phone and a notebook/desktop, I think ebook readers are viable competition to tablets

</comment><comment><USER16> Likely, so long as the definition of an Android tablet stays stretchy and flexible: does it need Google's seal of approval? does it need to be upgradeable to subsequent official Android releases? does it just need to have started with Android? There's no solid definition, so we end up trying to compare one product from one company against a spectrum of design and build approaches (and quality levels) and boil it down to a coherent horse race. No wonder the numbers from different tracking points are out of whack.

</comment><comment><USER17> Due to subsidies, there's a lot more room at the bottom in tablets. You can get an iPhone for $99, if you sign a contract, so to be much "cheaper" than that the difference is a subtle one based on contract terms. If the next nexus is $100 as rumored that puts it into impulse buy territory for most westerners.

</comment><comment><USER18> I could see it happen, especially if you count things like the Nook and Kindle Touch as Android tablets.

</comment><comment><USER19> Yes, they may have already done so. Recent reports show Android with 48% of market share WITHOUT counting Nexus 7 or Kindle Fire HD/Nook Tablet HD sales. I'd say those will easily cover the missing 2%.

</comment><comment><USER20> Almost certainly, if by "overtake" one means "sells more units" and Android means "is built on a fork of Android". Apple will likely continue to capture the overwhelming share of profit.

</comment><comment><USER21> I believe these figures were pre-Nexus 7 (and pre Kindle Fire HD line), so all else held equal, I'd imagine these figures are skewed slightly downwards.

</comment><comment><USER22> The question is too coarse.

In which markets will Android tablets overtake iPads?

In which markets do Android tablets already outsell iPads?

<comment><USER22> This article has 3 graphs (for mobile phone market) that are very relevant to tablets as well:

[http://www.asymco.com/2012/05/03/the-phone-market- in-2012-a-...](http://www.asymco.com/2012/05/03/the-phone-market- in-2012-a-tale-of-two-disruptions/)

Does "crossing the chasm" apply to the tablet market? Massive sales of cheap tablets/phablets in countries where iPad's are grossly unaffordable (price of a Rolex is US equivalent) will generate companies that can eat Apple's pie. It will take years, but it has already happened (Korea and China), and is happening in non-US markets.

<comment><USER22> Oops correction - Koreans buy iPads (70% to 80% market share in Q2).

</comment></comment></comment><comment><USER23> Why not? It happened with the I phone, and the PC when it took over the market (although Apple wasn't as dominant back then).

</comment><comment><USER24> Volume? Almost certainly. Profit? No chance.

</comment><comment><USER5> Sure. Most responsible lower-middle class or poor families can't afford an iPad, but something in the $99-300 range is doable. This is the trajectory that smart phones went: I see tons of people with Android Crickets.

As others have pointed out, this should have no bearing on Apple's success: I've never met someone who wanted a MacBook Pro who changed their mind because of a $299 laptop at WalMart.

<comment><USER6> _Most responsible lower-middle class or poor families can't afford an iPad_

That statement makes no sense. Everyone and their grandma can have an iPad. They are being given away for "free", bundled with 3G subscriptions.

Android tablets you actually have to buy.

Where I live, Android-tablets are mostly found among wealthier, informed people.

</comment></comment><comment><USER25> No. (as per Betteridge's law)

</comment><comment><USER26> probably</comment>


Snowden considers returning to the US - <AUTHOR> http://rt.com/news/161396-snowden-conditions-return-usa/

<comment><USER2> Here's Glenn Greenwald's take on the story: [https://firstlook.org/theintercept/2015/03/04/snowden- wants-...](https://firstlook.org/theintercept/2015/03/04/snowden-wants-come- home-stories-case-study-media-deceit/)

</comment><comment><USER3> That story is from May 2014.</comment>


IBM's "Watson" finally ready for prime-time Jeopardy - <AUTHOR> http://www.hpcwire.com/features/Must-See-TV-IBM-Watson-Heads-for-Jeopardy-Showdown-115684499.html <comment><USER2> This is going to be, for lack of better words, epic. On a personal level however, it's a reminder of how truly wondrous the future is.

When Deep Blue beat Kasparov I was a kid in an Italian high school, trying to explain to disinterested fellow students why that was a big deal.

Not even in my wildest dreams I would have imagined to be working for the company that made that and this event possible, in a different country than my own.

Sorry for the slightly off topic comment, but I just wanted to remind everyone that the future can be a beautiful and surprising time if you stick around long enough to witness it.

<comment><USER3> Watson's defeat of top Jeopardy! players, while an impressive feat of natural language processing, is not as impressive as a defeat of a grand master, for one reason.

The reason is that when multiple contestants know the correct response in Jeopardy!, it all comes down to reflexive timing. It's no surprise that a computer could buzz in faster than a human being, and there's no evidence that Watson knows _more_ correct responses than the human competitors.

<comment><USER4> I don't think you fully understand what is going on here. Watson will Process the slang in the sentence, figure out how the words are being used, understand the context of the question and then look through its internal databases to see if it can piece different pieces of knowledge that it has to come up with an answer.

THATS FUCKING HUGE!!!

<comment><USER3> What led you to believe that I don't understand what's going on? I definitely understand and appreciate IBM's accomplishment. I'm not minimizing the coolness of Watson being able to come up with so many correct responses, I'm minimizing the importance of its actual Jeopardy! gameplay. I'm not sure you understand the real game mechanics of Jeopardy! With highly proficient contestants, it comes down to being able to buzz in as soon as the light turns on indicating that the clue has been delivered by the host. All three contestants know the correct responses to most clues, and it really does come down to the buzzer reflexes, something Watson clearly has a huge advantage in.

Again, I fully understand and appreciate what an advance in natural language processing and sheer computation Watson represents. Even being able to match a below-average human Jeopardy! contestant would be impressive. However, the true game-winning technique in Jeopardy! (when all three contestants are highly skilled) is buzzer reflexes.

<comment><USER5> I think you are exactly right. Other commenters seem to be misunderstanding you.

It's clear that this is a huge achievement. However, it is different than a computer beating the human chess champion. All this will prove is that a computer is about the same as the best humans, not strictly better than them.

(And, that it is strictly better at timing its buzzer response, which is completely not impressive for a machine to excel at.)

<comment><USER6> I'd be curious in knowing the mean response time of Watson. That would shed some light on the topic.

</comment></comment></comment></comment></comment></comment><comment><USER7> I've been working on the periphery of this project for about a year now (my company created the on screen "avatar" that's shown at Watson's podium on stage) and it's been amazing to watch as Watson has progressed.

Some of the early matches I got data from were downright funny. Lots of nonsense answers and weird correlations that kinda made sense but made it obvious Watson didn't really understand the problem space.

In case you're interested the avatar gets realtime data from Watson and visualizes both Watson's internal state and the game state. The core bit is a collection of "threads" that swarm around the surface of a sphere. The speed, variability, color, and length of the threads are all tied to the data we get from Watson. The colors roughly correspond to confidence, and when the threads bunch up it has to do with what Watson is "doing" (i.e. if he gets an answer wrong the threads will go slow and gather to the bottom of the sphere, if one of the other contestants is answering the threads will gather on that side of of the sphere, etc).

The designer and my team agreed early on that there would be exactly 42 threads. ;-)

Oh, and when Watson speaks the threads push off the surface of the sphere to the intensity of the audio. It also makes a subtle glow in the center of the sphere brighten in an homage to Hal.

</comment><comment><USER8> Humanity is closing in on building machines to pass the Turing Test. Watson beat both champs in the demo round a few weeks ago and I fully expect him - yes, him - to beat them in the match coming up. I can't wait to watch this live. What an epic moment in the process of moving slowly from weak AI -> strong AI.

Watson can do amazing NLP (and presumably Machine Learning), which is something that the general public perceives as straight up "AI". NLP has been lagging far behind expectations for decades, but with Google's new Translate apps and Watson competing on Jeopardy, it seems like NLP is pretty close to being fully solved.

Very exciting. Very, very exciting.

Edit:

> Eventually the machine will prevail.

This sends chills down my spine.

<comment><USER9> Watson is a simulation, a clever puppet designed to mimic human behavior. We have no more to fear from it, than from a punch and judy show.

Cleverer and cleverer puppets will not become intelligent machines. A dedicated effort to build a massive neural cluster simulation might become intelligent, but it will think such incomprehensible thoughts as to be fairly useless to us (think intelligent rhododendon; what would you have to talk about?)

We will know when Watson is dangerous when it feels fear, angst, want. Not just an algorithm to sort facts and simulate speech.

<comment><USER10> A simulation of what?

It can actually answer Jeopardy questions. As good or better than the best Jeopardy player on the planet. (We will see.) Another task, previously only conquered by evolution, now conquered by algorithms written by humans. Watson doesn't pretend to be good at Jeopardy, it just is.

<comment><USER11> > It can actually answer Jeopardy questions.

I think you mean, it can actually question Jeopardy answers.

Sigh... I always hated that about the show.

<comment><USER10> That sounds like a fun diversion!

No, I most definitely mean that it can actually answer Jeopardy questions. Formulating sentences oddly doesn't turn questions into answers and answers into questions. You could define "question" and "answer" as sentences with a certain syntax but that would be an extremely stupid definition.

</comment></comment></comment></comment></comment><comment><USER12> One interesting observation, which I don't intend in any way to diminish the significance of this achievement, which I think will be seen as one of the most important milestones in the development of computer science, is that today there is so much information on the web that even Google with it's non- NLP algorithms can "almost" answer Jeopardy questions.

There is an archive of past Jeopardy questions here :

<http://www.j-archive.com/>

Try choosing a question and typing the category + the clue verbatim into Google. I've tried this a few times and in most cases the correct answer was in the top couple of sites (usually in the summary text on the Google search page).

Of course there's still the problem of actually extracting the answer from the page and presenting it in the proper form.

</comment><comment><USER13> A lot of geeks' girlfriends are going to be disappointed on 14th (aka Valentine's Day, in case you're wondering ;) ), as they huddle around a TV like schoolgirls watching a Bieber concert.

</comment><comment><AUTHOR> The crazy thing is that Watson is < 5 years in the making.

On PBS last night - <http://video.pbs.org/video/1786674622>

The "Making Stuff" series on Nova that's been airing the last week or so is very interesting: Making Suff Smarter / Stronger / Cleaner / Smaller. Watson was first mentioned during the episode of "Making Stuff Smarter" but I guess it merits its own segment.

</comment><comment><USER14> > Watson is comprised of 90 Power 750 servers, 16 TB of memory and 4 TB of > disk storage

4TB disk? something is wrong here.

<comment><USER15> [https://www.ibm.com/developerworks/mydeveloperworks/blogs/In...](https://www.ibm.com/developerworks/mydeveloperworks/blogs/InsideSystemStorage/entry/watson_what_is_the_smartest_machine_on_earth7?lang=en)

"Several readers of my blog have asked for details on the storage aspects of Watson. Basically, it is a modified version of IBM Scale-Out NAS [SONAS] that IBM offers commercially, but running Linux on POWER instead of Linux-x86. System p expansion drawers of SAS 15K RPM 450GB drives, 12 drives each, are dual-connected to two storage nodes, for a total of 21.6TB of raw disk capacity. The storage nodes use IBM's General Parallel File System (GPFS) to provide clustered NFS access to the rest of the system. Each Power 750 has minimal internal storage mostly to hold the Linux operating system and programs.

When Watson is booted up, the 15TB of total RAM are loaded up, and thereafter the DeepQA processing is all done from memory. According to IBM Research, "The actual size of the data (analyzed and indexed text, knowledge bases, etc.) used for candidate answer generation and evidence evaluation is under 1TB." For performance reasons, various subsets of the data are replicated in RAM on different functional groups of cluster nodes. The entire system is self- contained, Watson is NOT going to the internet searching for answers. "

</comment></comment><comment><USER16> _Eventually the machine will prevail_

What a great ending line. The NOVA special last night was great!

[http://www.pbs.org/wgbh/nova/tech/smartest-machine-on- earth....](http://www.pbs.org/wgbh/nova/tech/smartest-machine-on-earth.html)

</comment><comment><USER17> Affordable tablets, intelligent computers, commercial space travel... it's so great to live in the future.

Now, won't somebody please design a viable successor to the Concorde?

<comment><USER18> First we need a viable airline industry any where in the world.

</comment></comment><comment><USER19> engadget reported that Watson "destroyed" its human competition while others reported that humans were "taken down."

In reality, Ken Jennings was beaten on the final answer.

I'm interested in seeing whether Watson has an "aggressiveness" algorithm that allows it to respond before the answer is fully spoken. Humans have an advantage in this regard because it goes right to the heart of intelligence. If the game boils down to reaction time, Watson will probably win.

<comment><USER20> "I'm interested in seeing whether Watson has an "aggressiveness" algorithm that allows it to respond before the answer is fully spoken.

You cannot buzz in until the question is completely read out.

<http://en.wikipedia.org/wiki/Jeopardy!#cite_note-9>

<comment><USER21> True. But along the same lines...

I wonder if it's able to say to itself "I think I'm in the right ballpark, so I'll buzz now" and then take the 3-4 seconds it takes the host to recognize the buzz and ask for an answer, to finish its processing to get to the answer it's most confident of.

In other words, it doesn't need to actually have the answer to buzz. It just needs some confidence that the answer is close at hand.

<comment><USER22> I'd wager that humans do the same thing (e.g., when they have an answer and it's on the tip of their tongue)

<comment><USER23> It wouldn't surprise me, when watching (and playing along) I often start saying the "What is" or "Who was" before I have the answer.

</comment></comment></comment></comment></comment><comment><USER24> NYTimes had 'Play against IBM Watson' interactive feature few months back.

Here's the link: [http://www.nytimes.com/interactive/2010/06/16/magazine/watso...](http://www.nytimes.com/interactive/2010/06/16/magazine/watson- trivia-game.html?ref=magazine)

<comment><USER25> IIRC that does not actually let you play against Watson.

</comment></comment><comment><USER26> I've looked around a bit but couldn't find any information on how the game state and clues are input to Watson. Are they typed in on the fly? Pre-loaded and just revealed as clues are selected? Is it parsing speech and reading the board visually?

<comment><USER27> I've heard (from Engadget) that it is given text on the fly. It does not do speech or visual recognition. But it gets the question the same time the contestants see them.

</comment></comment><comment><USER28> Watson's voice isn't menacing enough.

<comment><USER29> I'd settle for a decent Sean Connery impression.

</comment></comment><comment><USER30> I am intrigued by Watson. NLP is definitely something that has failed to live up to its billing so far. I will be very interested to see what kinds of questions Watson is good at and which he tends to miss, to see if there are any patterns there.

Going forward, the other real questions will be: is Watson overfitted to the problem of solving Jeopardy questions, and how practical is the technology? The former is a real risk to the general applicability of Watson's technologies, the latter is a question of who can afford it. The article mentioned on commodity hardware, Watson takes about 2 hours per clue. They only achieve reasonable response times by using about 3000 cores. That limits the potential audience.

Either way, I'm very interested to see what happens next week. I watched the demo videos on YouTube and it was quite cool.

</comment><comment><USER31> Maybe they should give it the answer "The specifications for a machine to beat you at Jeopardy."

</comment><comment><USER32> Won't Watson have a significant advantage on the final Jeopardy question where time is not nearly as much of a factor? If it can just keep pace for most of the game and then bet it all on that last question, it should be no contest.

<comment><USER33> I wouldn't bet on it. In the "preview" round, Watson seemed to come up with its answers more or less instantaneously. (Or at least, before the host finished reading the question.) My guess is the limiting factor is the quality of the underlying dataset and cleverness of the algorithms, not CPU time.

<comment><USER34> Human contestants are also able to see the question typed out (much like it's displayed on the television), so people are able to do this as well.

I'm usually able to read any given question in about a second. Granted, I used to practice speed-reading trivia questions for an hour or so per day (one of the leagues we competed in projected questions via powerpoint), but there's at least one reason most Jeopardy! contestants are able to buzz in almost immediately on most questions.

</comment></comment></comment><comment><USER35> Seems unfair for the humans.

Wouldn't a more fair match be a series of individual 1-on-1 matches with Watson and Jennings / Rutter?

The current configuration means the two humans will both share the questions that are naturally difficult for computers, but Watson will dominate all the questions naturally hard for humans.

Alternatively, to make it fair, we would need a 2nd copy of Watson competing, and if the two Watson's buzz at the same time, randomly pick one to answer.

</comment><comment><USER27> How much of the Watson code is Java?

<comment><USER36> I don't know the percentages but Watson is built on UIAM <http://uima.apache.org/> which has bits in C++ and bits in Java. From there I am sure different teams used language bindings to use the language that best suited their particular needs to accomplish their tasks. Not that I know it to be true, but I would not be at all surprised to find R, Haskel and even some lisp in their somewhere.

<comment><USER27> Thanks for the link. That looks really impressive. For IBM this is presumably PR for their products/services. I do hope they disclose more of the architecture as I'd love to build out a system like this, even on a smaller scale (takes 6 hours to come back with an answer rather than three seconds).

<comment><USER36> My understanding that this is the beginning of their DeepQA product line, my assumption is that the research and portions of the technology will be used for business insight and analytical to answer what if type questions.

,Just to be clear I don't work for IBM, and I do not know their intentions for the project, but I do take projects from IBM and have taken projects related to Watson, but I do not know their plans to monetize Watson, the former is just pure speculation on my part.

</comment></comment></comment></comment><comment><USER37> I think it would be cool to see IBM's creation versus a creation from Google versus (a person or another machine), although I doubt another company would want to make such a risky move. If Watson wins, huge PR win; if Watson loses, still a pretty big PR win.

</comment><comment><USER38> > When the software was run on a lone 2.6 GHz CPU, it took around 2 hours to > process a typical Jeopardy clue -- not a very practical implementation. But > when they parallelized the algorithms across the 2,880-core Watson, they > were able to cut the processing time from a couple of hours to between 2 and > 6 seconds.

That is an impressive amount of parallelism! This is very back of the napkin (and I realise I'm comparing apples and oranges), but a rough estimate for the time taken if the problem was parallelised with 100% efficiency would be:

(2 hours) / ((2880 * 3.55) / 2.6) = 1.83098592 seconds

<comment><USER39> What's 3.55?

<comment><USER38> Oops! The cores in Watson are 3.55 Ghz, sorry. I realise that ghz is not necessarily a measure of speed, this estimate could easily be out by a factor of 2 or 3.

</comment></comment></comment><comment><USER40> <http://www-943.ibm.com/innovation/us/watson/>

<comment><USER40> A classic use of watson that IBM is saying is in the field of medical science - Imagine if we can feed the entire set of medical books into watson (somebody estimated that would just take about a week for watson to process and make the connections) and then you have the watson physician's assistant which can listen into the symptoms and spit out the first five most probably causes - that would be so damn amazing!

<comment><USER41> There are already experimental clinical decision support applications which, given a list of symptoms, can produce the differential diagnosis. But they generally aren't very useful since in most cases the physician can figure out the same diagnosis just as quickly. Real medicine isn't like House, MD.

IBM is more likely to apply Watson technology toward analytics and data mining. There are huge amounts of clinical data locked up in unstructured text reports. If they can analyze that data in a useful way to draw correlations between symptoms, patient demographics, medications, treatments, and outcomes then that could add a lot of value for medical researchers.

</comment></comment></comment><comment><USER42> Here's a fascinating overview of how Watson (DeepQA) works:

<http://www.youtube.com/watch?v=3G2H3DZ8rNc>

The video goes into some detail, and looks at how Watson analyzes particular questions.

It feels like AI is starting to become what people once thought it could be.

</comment><comment><USER43> There's a problem with statistical significance given that the match consists only of two games:

[http://www.quora.com/Does-IBM-Watson-Jeopardy-match- contribu...](http://www.quora.com/Does-IBM-Watson-Jeopardy-match-contribute- to-ignorance-about-statistics)

<comment><USER8> I highly doubt that IBM will dismantle the machine when they're done (yes, I know that's what they did with Deep Blue, but this is different). I'd bet that far more than two games will be played with Watson, over the next few years.

Also, as pointed out in that thread, this is no different than a regular Jeopardy game. One game consists of dozens and dozens of questions. If Watson answers them _all_ correctly and the humans get zero points, that _is_ indeed statistically significant.

If the score is close, then repetition in games will be necessary. The problem with the assumption in that thread is that one Jeopardy game is vastly different than the next. In reality, you could string together a bunch of games and call it one game. Or take one game and split it up into 50 "games". The point is, inside one game, there are enough different questions to definitely count as statistically significant.

<comment><USER43> That depends on what "better" means. If better means actually winning in Jeopardy, then two games is not enough. Final Jeopardy plays a significant role in who wins a game, so it's not just like combining multiple games into one.

Even on the question level, a disclaimer should be given if Watson doesn't answer enough questions for statistical significance.

Finally, the issue here is with contributing to the statistical ignorance of people watching the two game match. Sure, Watson has played many games already before the match but that's not what's being shown on TV.

</comment></comment></comment><comment><USER44> I wonder whether Watson 'reads' the question or if the questions are fed to it directly as parameters, or if it has to 'listen' to the spoken words and start thinking about its response after that.

</comment><comment><USER45> We don't need to know math anymore because of calculators, now we don't need to know facts anymore because of watson.

I guess from their shift to broadcasting opinions that the "news" outlets saw this coming...

</comment><comment><USER46> When deep blue beat Kasparov, IBM quickly moved to shut down the program. Let's hope it doesn't happen in this case, if Watson wins.

</comment><comment><USER47> That article is sort of bullshit. They already taped the Watson Jeopardy episodes last month, they just haven't aired them.

Watson won.

<comment><USER48> That was a practice round: <http://www.wired.com/epicenter/2011/01/ibm-watson- jeopardy/>

</comment></comment><comment><USER49> When the humans playing against Watson get slapped around it'll be a weird moment for humanity.

</comment><comment><USER50> Did they give him HAL 9000's voice or am I crazy?

</comment><comment><USER51> This is the right way to test AI.

"Alan Turing, meet Alex Trebeck."</comment>


New solar installs beat wind and coal two years in a row - <AUTHOR> http://www.computerworld.com/article/2895013/new-solar-installs-beat-wind-and-coal-two-years-in-a-row.html

<comment><USER2> As usual, this compares nameplate capacity instead of delivered generation.

Falling solar install prices are a good thing, but they don't close down other generation types. Coal plants still have to burn even if there is a glut of electricity because of favourable solar conditions.

Solar boosters like the article author love to prove how baseline producers have to accept negative rates means that they are somehow wrong in the market, but having baseline producers run at loss or worse, close down permanently is not something that any of us want - especially in the middle of a harsh winter.

Some care is needed to have grid stability designed in - which also should give price stability - because a highly volatile generation market is not something that is good for anyone.

<comment><USER3> > Falling solar install prices are a good thing, but they don't close down > other generation types. Coal plants still have to burn even if there is a > glut of electricity because of favourable solar conditions.

And as such, coal plants should be deprecated for combined cycle natural gas plants that can be throttled more granularly for demand (and can also meet air regulations coal plants cannot without great expense).

> Solar boosters like the article author love to prove how baseline producers > have to accept negative rates means that they are somehow wrong in the > market, but having baseline producers run at loss or worse, close down > permanently is not something that any of us want - especially in the middle > of a harsh winter.

Negative rates are a signal, and effect all base load (coal, nuclear, hydro, etc) equally. I don't see how sending proper market signals are a bad thing.

<comment><USER4> "Negative rates are a signal, and effect all base load (coal, nuclear, hydro, etc) equally. I don't see how sending proper market signals are a bad thing."

But those market signals are not sent to the solar installations. i.e. if you produce a lot of power when it is sunny and there is a glut, the owner of the panel typically still get the same rebate.

If we sent proper market signals to renewable energy providers it would indeed be a good thing.

<comment><USER3> > But those market signals are not sent to the solar installations. i.e. if > you produce a lot of power when it is sunny and there is a glut, the owner > of the panel typically still get the same rebate. > If we sent proper market > signals to renewable energy providers it would indeed be a good thing.

Of course those signals aren't sent to solar installations; we value their output over that of coal plants, and their output is clean regardless of when its produced. We _do_ send those signals to wind farms, which must curtail their output when insufficient demand exists.

In time, as more renewables come online, stationary utility storage (battery cargo containers), and fossil fuels are deprecated, the problem solves itself.</comment></comment></comment></comment>