Stinger.next is the second large effort Hortonworks has spearheaded in order to improve the performance of Hive. The company recently completed its goals for the initial Stinger project, on which it began work in 2012 and which the company claims improved the performance of Hive by 100 times while also improving its functionality.
The company explains the details of how it, along with the Hadoop community, plans to pull this off in a blog post on Wednesday. It also gives a rough timeline of when the three-phase plan will be complete: ACID transactions by the end of this year; sub-second queries and Spark integration in the first half of next year; and full SQL queries along with geographically distributed queries by the end of 2015. There are a few other deliverables tied to each of those timeframes, as well.
A successful Stinger.next initiative could be a major annoyance, to say the least, for the slew of companies that have already committed untold man-hours and financial resources building out their own SQL-on-Hadoop engines based on the premise that Hive — even running on Spark — would never be fast enough. Commercially available products include Cloudera Impala, IBM Big SQL, Pivotal Greenplum and startup Splice Machine’s eponymous database technology. Open source projects and others still under development include the Facebook-built Presto, the Salesforce.com-built Apache Phoenix and the MapR-led Apache Drill.
The Apache Spark community is also working on its own interactive SQL engines called Spark SQL and BlinkDB.
That might continue to be true, and it might be true that every technology mentioned will continue to be better than a new-and-improved Hive in one way or another. But it’s also true that a more-capable Hive is going to look very appealing to a lot of users that have been using Hive for years and don’t want to incorporate an entirely new technology (although Stinger.next does involve some major architectural changes), or that prefer their Hadoop components to be as open as possible.
I’m not sure running SQL jobs on Hadoop is the ultimate use case for what many claim will be a revolutionary data platform, but given the size of the database market, it’s potentially a lucrative one and it’s helping highlight some big differences between each vendor’s approach to selling Hadoop.
I would recommend updating your piece to reflect that Impala is (and has been) an open source project, freely available on github . The fact that MapR and Amazon have both adopted it and provide support for it should also be noted, to that effect. The only thing Cloudera ‘charges’ for Impala is for support.