From hive-ohampton@example.com Thu Jan 01 01:29:10 2009 Return-Path: <hive-ohampton@example.com> Delivered-To: ohampton@example.com Received: (qmail 35120 invoked from network); 1 Jan 2009 01:29:08 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jan 2009 01:29:08 -0000 Received: (qmail 3863 invoked by uid 500); 1 Jan 2009 01:29:08 -0000 Delivered-To: ohampton@example.com Received: (qmail 3816 invoked by uid 500); 1 Jan 2009 01:29:08 -0000 Mailing-List: contact ohampton@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:ohampton@example.com> List-Unsubscribe: <mailto:ohampton@example.com> List-Post: <mailto:ohampton@example.com> List-Id: <hive-dev.hadoop.apache.org> Reply-To: ohampton@example.com Delivered-To: mailing list ohampton@example.com Received: (qmail 3805 invoked by uid 99); 1 Jan 2009 01:29:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 31 Dec 2008 17:29:07 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Jan 2009 01:29:05 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 5E38D234C4A8 for <ohampton@example.com>; Wed, 31 Dec 2008 17:28:44 -0800 (PST) Message-ID: <1619136000.1230773324384.JavaMail.jira@brutus> Date: Wed, 31 Dec 2008 17:28:44 -0800 (PST) From: "Prasad Chakka (JIRA)" <ohampton@example.com> To: ohampton@example.com Subject: [jira] Assigned: (HIVE-84) MetaStore Client is not thread safe In-Reply-To: <776488778.1227783944227.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HIVE-84?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Chakka reassigned HIVE-84: --------------------------------- Assignee: Prasad Chakka > MetaStore Client is not thread safe > ----------------------------------- > > Key: HIVE-84 > URL: https://issues.apache.org/jira/browse/HIVE-84 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore > Environment: with patch for hive-77 - run: > ant -lib ./testlibs -Dtestcase=TestMTQueries test > Reporter: Joydeep Sen Sarma > Assignee: Prasad Chakka > > when running DDL Tasks in concurrent threads - the following exception trace is observed: > java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a duplicate ke\ y value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:207) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:209) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:174) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:185) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:210) > at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:390) > at org.apache.hadoop.hive.ql.QTestUtil$QTRunner.run(QTestUtil.java:681) > at java.lang.Thread.run(Thread.java:619) > Caused by: javax.jdo.JDODataStoreException: Insert of object "org.apache.hadoop.hive.metastore.model.MTable@3bc8d400" us\ ing statement "INSERT INTO TBLS (TBL_ID,CREATE_TIME,DB_ID,RETENTION,TBL_NAME,SD_ID,OWNER,LAST_ACCESS_TIME) VALUES (?,?,?\ ,?,?,?,?,?)" failed : The statement was aborted because it would have caused a duplicate key value in a unique or primar\ y key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > NestedThrowables: > java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a duplicate ke\ y value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.jpox.jdo.JPOXJDOHelper.getJDOExceptionForJPOXException(JPOXJDOHelper.java:291) > at org.jpox.jdo.AbstractPersistenceManager.jdoMakePersistent(AbstractPersistenceManager.java:671) > at org.jpox.jdo.AbstractPersistenceManager.makePersistent(AbstractPersistenceManager.java:691) > at org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:479) > at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table(HiveMetaStore.java:292) > at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:252) > at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:205) > ... 7 more > Caused by: java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a d\ uplicate key value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source) > at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) > at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) > at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) > at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) > at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) > at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) > at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) > at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) > at org.jpox.store.rdbms.SQLController.executeStatementUpdate(SQLController.java:396) > at org.jpox.store.rdbms.request.InsertRequest.execute(InsertRequest.java:370) > at org.jpox.store.rdbms.RDBMSPersistenceHandler.insertTable(RDBMSPersistenceHandler.java:157) > at org.jpox.store.rdbms.RDBMSPersistenceHandler.insertObject(RDBMSPersistenceHandler.java:136) > at org.jpox.state.JDOStateManagerImpl.internalMakePersistent(JDOStateManagerImpl.java:3082) > at org.jpox.state.JDOStateManagerImpl.makePersistent(JDOStateManagerImpl.java:3062) > at org.jpox.ObjectManagerImpl.persistObjectInternal(ObjectManagerImpl.java:1231) > at org.jpox.ObjectManagerImpl.persistObject(ObjectManagerImpl.java:1077) > at org.jpox.jdo.AbstractPersistenceManager.jdoMakePersistent(AbstractPersistenceManager.java:666) > ... 12 more > Caused by: java.sql.SQLException: The statement was aborted because it would have caused a duplicate key value in a uniq\ ue or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) > at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source) > ... 30 more > Caused by: ERROR 23505: The statement was aborted because it would have caused a duplicate key value in a unique or prim\ ary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) > at org.apache.derby.impl.sql.execute.IndexChanger.insertAndCheckDups(Unknown Source) > at org.apache.derby.impl.sql.execute.IndexChanger.doInsert(Unknown Source) > at org.apache.derby.impl.sql.execute.IndexChanger.insert(Unknown Source) > at org.apache.derby.impl.sql.execute.IndexSetChanger.insert(Unknown Source) > at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown Source) > at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source) > at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source) > when running normal select queries as well - one hits exception, stack trace: > 2008-11-27 01:54:00,216 ERROR metadata.Hive (Hive.java:getTable(275)) - NoSuchObjectException(message:default.dummySrc t\ > able not found) > at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:347) > at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:433) > at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:472) > at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:272) > at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:254) > at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:544) > at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:3192) > at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:73) > at org.apache.hadoop.hive.ql.QTestUtil.analyzeAST(QTestUtil.java:672) > at org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_table1(TestParseNegative.java:231) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at junit.framework.TestCase.runTest(TestCase.java:154) > at junit.framework.TestCase.runBare(TestCase.java:127) > at junit.framework.TestResult$1.protect(TestResult.java:106) > at junit.framework.TestResult.runProtected(TestResult.java:124) > at junit.framework.TestResult.run(TestResult.java:109) > at junit.framework.TestCase.run(TestCase.java:118) > at junit.framework.TestSuite.runTest(TestSuite.java:208) > at junit.framework.TestSuite.run(TestSuite.java:203) > at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:297) > at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:672) > at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:567) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From hive-ohampton@example.com Thu Jan 01 01:31:05 2009 Return-Path: <hive-ohampton@example.com> Delivered-To: ohampton@example.com Received: (qmail 36417 invoked from network); 1 Jan 2009 01:31:05 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jan 2009 01:31:05 -0000 Received: (qmail 4169 invoked by uid 500); 1 Jan 2009 01:31:05 -0000 Delivered-To: ohampton@example.com Received: (qmail 4149 invoked by uid 500); 1 Jan 2009 01:31:05 -0000 Mailing-List: contact ohampton@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:ohampton@example.com> List-Unsubscribe: <mailto:ohampton@example.com> List-Post: <mailto:ohampton@example.com> List-Id: <hive-dev.hadoop.apache.org> Reply-To: ohampton@example.com Delivered-To: mailing list ohampton@example.com Received: (qmail 4138 invoked by uid 99); 1 Jan 2009 01:31:05 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 31 Dec 2008 17:31:05 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Jan 2009 01:31:04 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 4DD9B234C4A8 for <ohampton@example.com>; Wed, 31 Dec 2008 17:30:44 -0800 (PST) Message-ID: <975628929.1230773444317.JavaMail.jira@brutus> Date: Wed, 31 Dec 2008 17:30:44 -0800 (PST) From: "Prasad Chakka (JIRA)" <ohampton@example.com> To: ohampton@example.com Subject: [jira] Updated: (HIVE-84) MetaStore Client is not thread safe In-Reply-To: <776488778.1227783944227.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HIVE-84?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Chakka updated HIVE-84: ------------------------------ Attachment: hive-84.patch > MetaStore Client is not thread safe > ----------------------------------- > > Key: HIVE-84 > URL: https://issues.apache.org/jira/browse/HIVE-84 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore > Environment: with patch for hive-77 - run: > ant -lib ./testlibs -Dtestcase=TestMTQueries test > Reporter: Joydeep Sen Sarma > Assignee: Prasad Chakka > Attachments: hive-84.patch > > > when running DDL Tasks in concurrent threads - the following exception trace is observed: > java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a duplicate ke\ y value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:207) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:209) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:174) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:185) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:210) > at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:390) > at org.apache.hadoop.hive.ql.QTestUtil$QTRunner.run(QTestUtil.java:681) > at java.lang.Thread.run(Thread.java:619) > Caused by: javax.jdo.JDODataStoreException: Insert of object "org.apache.hadoop.hive.metastore.model.MTable@3bc8d400" us\ ing statement "INSERT INTO TBLS (TBL_ID,CREATE_TIME,DB_ID,RETENTION,TBL_NAME,SD_ID,OWNER,LAST_ACCESS_TIME) VALUES (?,?,?\ ,?,?,?,?,?)" failed : The statement was aborted because it would have caused a duplicate key value in a unique or primar\ y key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > NestedThrowables: > java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a duplicate ke\ y value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.jpox.jdo.JPOXJDOHelper.getJDOExceptionForJPOXException(JPOXJDOHelper.java:291) > at org.jpox.jdo.AbstractPersistenceManager.jdoMakePersistent(AbstractPersistenceManager.java:671) > at org.jpox.jdo.AbstractPersistenceManager.makePersistent(AbstractPersistenceManager.java:691) > at org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:479) > at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table(HiveMetaStore.java:292) > at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:252) > at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:205) > ... 7 more > Caused by: java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a d\ uplicate key value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source) > at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) > at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) > at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) > at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) > at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) > at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) > at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) > at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) > at org.jpox.store.rdbms.SQLController.executeStatementUpdate(SQLController.java:396) > at org.jpox.store.rdbms.request.InsertRequest.execute(InsertRequest.java:370) > at org.jpox.store.rdbms.RDBMSPersistenceHandler.insertTable(RDBMSPersistenceHandler.java:157) > at org.jpox.store.rdbms.RDBMSPersistenceHandler.insertObject(RDBMSPersistenceHandler.java:136) > at org.jpox.state.JDOStateManagerImpl.internalMakePersistent(JDOStateManagerImpl.java:3082) > at org.jpox.state.JDOStateManagerImpl.makePersistent(JDOStateManagerImpl.java:3062) > at org.jpox.ObjectManagerImpl.persistObjectInternal(ObjectManagerImpl.java:1231) > at org.jpox.ObjectManagerImpl.persistObject(ObjectManagerImpl.java:1077) > at org.jpox.jdo.AbstractPersistenceManager.jdoMakePersistent(AbstractPersistenceManager.java:666) > ... 12 more > Caused by: java.sql.SQLException: The statement was aborted because it would have caused a duplicate key value in a uniq\ ue or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) > at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source) > ... 30 more > Caused by: ERROR 23505: The statement was aborted because it would have caused a duplicate key value in a unique or prim\ ary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) > at org.apache.derby.impl.sql.execute.IndexChanger.insertAndCheckDups(Unknown Source) > at org.apache.derby.impl.sql.execute.IndexChanger.doInsert(Unknown Source) > at org.apache.derby.impl.sql.execute.IndexChanger.insert(Unknown Source) > at org.apache.derby.impl.sql.execute.IndexSetChanger.insert(Unknown Source) > at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown Source) > at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source) > at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source) > when running normal select queries as well - one hits exception, stack trace: > 2008-11-27 01:54:00,216 ERROR metadata.Hive (Hive.java:getTable(275)) - NoSuchObjectException(message:default.dummySrc t\ > able not found) > at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:347) > at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:433) > at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:472) > at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:272) > at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:254) > at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:544) > at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:3192) > at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:73) > at org.apache.hadoop.hive.ql.QTestUtil.analyzeAST(QTestUtil.java:672) > at org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_table1(TestParseNegative.java:231) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at junit.framework.TestCase.runTest(TestCase.java:154) > at junit.framework.TestCase.runBare(TestCase.java:127) > at junit.framework.TestResult$1.protect(TestResult.java:106) > at junit.framework.TestResult.runProtected(TestResult.java:124) > at junit.framework.TestResult.run(TestResult.java:109) > at junit.framework.TestCase.run(TestCase.java:118) > at junit.framework.TestSuite.runTest(TestSuite.java:208) > at junit.framework.TestSuite.run(TestSuite.java:203) > at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:297) > at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:672) > at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:567) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From hive-ohampton@example.com Thu Jan 01 01:31:05 2009 Return-Path: <hive-ohampton@example.com> Delivered-To: ohampton@example.com Received: (qmail 36441 invoked from network); 1 Jan 2009 01:31:05 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jan 2009 01:31:05 -0000 Received: (qmail 4225 invoked by uid 500); 1 Jan 2009 01:31:05 -0000 Delivered-To: ohampton@example.com Received: (qmail 4209 invoked by uid 500); 1 Jan 2009 01:31:05 -0000 Mailing-List: contact ohampton@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:ohampton@example.com> List-Unsubscribe: <mailto:ohampton@example.com> List-Post: <mailto:ohampton@example.com> List-Id: <hive-dev.hadoop.apache.org> Reply-To: ohampton@example.com Delivered-To: mailing list ohampton@example.com Received: (qmail 4194 invoked by uid 99); 1 Jan 2009 01:31:05 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 31 Dec 2008 17:31:05 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Jan 2009 01:31:04 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 532D0234C4AB for <ohampton@example.com>; Wed, 31 Dec 2008 17:30:44 -0800 (PST) Message-ID: <1404725430.1230773444339.JavaMail.jira@brutus> Date: Wed, 31 Dec 2008 17:30:44 -0800 (PST) From: "Prasad Chakka (JIRA)" <ohampton@example.com> To: ohampton@example.com Subject: [jira] Commented: (HIVE-84) MetaStore Client is not thread safe In-Reply-To: <776488778.1227783944227.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HIVE-84?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660169#action_12660169 ] Prasad Chakka commented on HIVE-84: ----------------------------------- in the patch, Hive.java maintains metastore client in tss which makes it thread safe. Also it provides a new function to refresh the HiveConf. > MetaStore Client is not thread safe > ----------------------------------- > > Key: HIVE-84 > URL: https://issues.apache.org/jira/browse/HIVE-84 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore > Environment: with patch for hive-77 - run: > ant -lib ./testlibs -Dtestcase=TestMTQueries test > Reporter: Joydeep Sen Sarma > Assignee: Prasad Chakka > Attachments: hive-84.patch > > > when running DDL Tasks in concurrent threads - the following exception trace is observed: > java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a duplicate ke\ y value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:207) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:209) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:174) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:185) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:210) > at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:390) > at org.apache.hadoop.hive.ql.QTestUtil$QTRunner.run(QTestUtil.java:681) > at java.lang.Thread.run(Thread.java:619) > Caused by: javax.jdo.JDODataStoreException: Insert of object "org.apache.hadoop.hive.metastore.model.MTable@3bc8d400" us\ ing statement "INSERT INTO TBLS (TBL_ID,CREATE_TIME,DB_ID,RETENTION,TBL_NAME,SD_ID,OWNER,LAST_ACCESS_TIME) VALUES (?,?,?\ ,?,?,?,?,?)" failed : The statement was aborted because it would have caused a duplicate key value in a unique or primar\ y key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > NestedThrowables: > java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a duplicate ke\ y value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.jpox.jdo.JPOXJDOHelper.getJDOExceptionForJPOXException(JPOXJDOHelper.java:291) > at org.jpox.jdo.AbstractPersistenceManager.jdoMakePersistent(AbstractPersistenceManager.java:671) > at org.jpox.jdo.AbstractPersistenceManager.makePersistent(AbstractPersistenceManager.java:691) > at org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:479) > at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table(HiveMetaStore.java:292) > at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:252) > at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:205) > ... 7 more > Caused by: java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a d\ uplicate key value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source) > at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) > at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) > at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) > at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) > at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) > at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) > at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) > at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) > at org.jpox.store.rdbms.SQLController.executeStatementUpdate(SQLController.java:396) > at org.jpox.store.rdbms.request.InsertRequest.execute(InsertRequest.java:370) > at org.jpox.store.rdbms.RDBMSPersistenceHandler.insertTable(RDBMSPersistenceHandler.java:157) > at org.jpox.store.rdbms.RDBMSPersistenceHandler.insertObject(RDBMSPersistenceHandler.java:136) > at org.jpox.state.JDOStateManagerImpl.internalMakePersistent(JDOStateManagerImpl.java:3082) > at org.jpox.state.JDOStateManagerImpl.makePersistent(JDOStateManagerImpl.java:3062) > at org.jpox.ObjectManagerImpl.persistObjectInternal(ObjectManagerImpl.java:1231) > at org.jpox.ObjectManagerImpl.persistObject(ObjectManagerImpl.java:1077) > at org.jpox.jdo.AbstractPersistenceManager.jdoMakePersistent(AbstractPersistenceManager.java:666) > ... 12 more > Caused by: java.sql.SQLException: The statement was aborted because it would have caused a duplicate key value in a uniq\ ue or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) > at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source) > ... 30 more > Caused by: ERROR 23505: The statement was aborted because it would have caused a duplicate key value in a unique or prim\ ary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) > at org.apache.derby.impl.sql.execute.IndexChanger.insertAndCheckDups(Unknown Source) > at org.apache.derby.impl.sql.execute.IndexChanger.doInsert(Unknown Source) > at org.apache.derby.impl.sql.execute.IndexChanger.insert(Unknown Source) > at org.apache.derby.impl.sql.execute.IndexSetChanger.insert(Unknown Source) > at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown Source) > at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source) > at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source) > when running normal select queries as well - one hits exception, stack trace: > 2008-11-27 01:54:00,216 ERROR metadata.Hive (Hive.java:getTable(275)) - NoSuchObjectException(message:default.dummySrc t\ > able not found) > at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:347) > at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:433) > at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:472) > at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:272) > at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:254) > at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:544) > at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:3192) > at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:73) > at org.apache.hadoop.hive.ql.QTestUtil.analyzeAST(QTestUtil.java:672) > at org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_table1(TestParseNegative.java:231) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at junit.framework.TestCase.runTest(TestCase.java:154) > at junit.framework.TestCase.runBare(TestCase.java:127) > at junit.framework.TestResult$1.protect(TestResult.java:106) > at junit.framework.TestResult.runProtected(TestResult.java:124) > at junit.framework.TestResult.run(TestResult.java:109) > at junit.framework.TestCase.run(TestCase.java:118) > at junit.framework.TestSuite.runTest(TestSuite.java:208) > at junit.framework.TestSuite.run(TestSuite.java:203) > at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:297) > at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:672) > at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:567) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From hive-ohampton@example.com Thu Jan 01 01:33:07 2009 Return-Path: <hive-ohampton@example.com> Delivered-To: ohampton@example.com Received: (qmail 37573 invoked from network); 1 Jan 2009 01:33:07 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jan 2009 01:33:07 -0000 Received: (qmail 4440 invoked by uid 500); 1 Jan 2009 01:33:07 -0000 Delivered-To: ohampton@example.com Received: (qmail 4417 invoked by uid 500); 1 Jan 2009 01:33:07 -0000 Mailing-List: contact ohampton@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:ohampton@example.com> List-Unsubscribe: <mailto:ohampton@example.com> List-Post: <mailto:ohampton@example.com> List-Id: <hive-dev.hadoop.apache.org> Reply-To: ohampton@example.com Delivered-To: mailing list ohampton@example.com Received: (qmail 4406 invoked by uid 99); 1 Jan 2009 01:33:07 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 31 Dec 2008 17:33:07 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Jan 2009 01:33:06 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 88679234C4A8 for <ohampton@example.com>; Wed, 31 Dec 2008 17:32:46 -0800 (PST) Message-ID: <900284394.1230773566557.JavaMail.jira@brutus> Date: Wed, 31 Dec 2008 17:32:46 -0800 (PST) From: "Prasad Chakka (JIRA)" <ohampton@example.com> To: ohampton@example.com Subject: [jira] Updated: (HIVE-84) MetaStore Client is not thread safe In-Reply-To: <776488778.1227783944227.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HIVE-84?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Chakka updated HIVE-84: ------------------------------ Fix Version/s: 0.2.0 Affects Version/s: 0.2.0 Status: Patch Available (was: Open) > MetaStore Client is not thread safe > ----------------------------------- > > Key: HIVE-84 > URL: https://issues.apache.org/jira/browse/HIVE-84 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore > Affects Versions: 0.2.0 > Environment: with patch for hive-77 - run: > ant -lib ./testlibs -Dtestcase=TestMTQueries test > Reporter: Joydeep Sen Sarma > Assignee: Prasad Chakka > Fix For: 0.2.0 > > Attachments: hive-84.patch > > > when running DDL Tasks in concurrent threads - the following exception trace is observed: > java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a duplicate ke\ y value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:207) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:209) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:174) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:185) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:210) > at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:390) > at org.apache.hadoop.hive.ql.QTestUtil$QTRunner.run(QTestUtil.java:681) > at java.lang.Thread.run(Thread.java:619) > Caused by: javax.jdo.JDODataStoreException: Insert of object "org.apache.hadoop.hive.metastore.model.MTable@3bc8d400" us\ ing statement "INSERT INTO TBLS (TBL_ID,CREATE_TIME,DB_ID,RETENTION,TBL_NAME,SD_ID,OWNER,LAST_ACCESS_TIME) VALUES (?,?,?\ ,?,?,?,?,?)" failed : The statement was aborted because it would have caused a duplicate key value in a unique or primar\ y key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > NestedThrowables: > java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a duplicate ke\ y value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.jpox.jdo.JPOXJDOHelper.getJDOExceptionForJPOXException(JPOXJDOHelper.java:291) > at org.jpox.jdo.AbstractPersistenceManager.jdoMakePersistent(AbstractPersistenceManager.java:671) > at org.jpox.jdo.AbstractPersistenceManager.makePersistent(AbstractPersistenceManager.java:691) > at org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:479) > at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table(HiveMetaStore.java:292) > at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:252) > at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:205) > ... 7 more > Caused by: java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a d\ uplicate key value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source) > at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) > at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) > at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) > at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) > at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) > at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) > at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) > at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) > at org.jpox.store.rdbms.SQLController.executeStatementUpdate(SQLController.java:396) > at org.jpox.store.rdbms.request.InsertRequest.execute(InsertRequest.java:370) > at org.jpox.store.rdbms.RDBMSPersistenceHandler.insertTable(RDBMSPersistenceHandler.java:157) > at org.jpox.store.rdbms.RDBMSPersistenceHandler.insertObject(RDBMSPersistenceHandler.java:136) > at org.jpox.state.JDOStateManagerImpl.internalMakePersistent(JDOStateManagerImpl.java:3082) > at org.jpox.state.JDOStateManagerImpl.makePersistent(JDOStateManagerImpl.java:3062) > at org.jpox.ObjectManagerImpl.persistObjectInternal(ObjectManagerImpl.java:1231) > at org.jpox.ObjectManagerImpl.persistObject(ObjectManagerImpl.java:1077) > at org.jpox.jdo.AbstractPersistenceManager.jdoMakePersistent(AbstractPersistenceManager.java:666) > ... 12 more > Caused by: java.sql.SQLException: The statement was aborted because it would have caused a duplicate key value in a uniq\ ue or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) > at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source) > ... 30 more > Caused by: ERROR 23505: The statement was aborted because it would have caused a duplicate key value in a unique or prim\ ary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) > at org.apache.derby.impl.sql.execute.IndexChanger.insertAndCheckDups(Unknown Source) > at org.apache.derby.impl.sql.execute.IndexChanger.doInsert(Unknown Source) > at org.apache.derby.impl.sql.execute.IndexChanger.insert(Unknown Source) > at org.apache.derby.impl.sql.execute.IndexSetChanger.insert(Unknown Source) > at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown Source) > at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source) > at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source) > when running normal select queries as well - one hits exception, stack trace: > 2008-11-27 01:54:00,216 ERROR metadata.Hive (Hive.java:getTable(275)) - NoSuchObjectException(message:default.dummySrc t\ > able not found) > at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:347) > at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:433) > at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:472) > at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:272) > at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:254) > at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:544) > at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:3192) > at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:73) > at org.apache.hadoop.hive.ql.QTestUtil.analyzeAST(QTestUtil.java:672) > at org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_table1(TestParseNegative.java:231) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at junit.framework.TestCase.runTest(TestCase.java:154) > at junit.framework.TestCase.runBare(TestCase.java:127) > at junit.framework.TestResult$1.protect(TestResult.java:106) > at junit.framework.TestResult.runProtected(TestResult.java:124) > at junit.framework.TestResult.run(TestResult.java:109) > at junit.framework.TestCase.run(TestCase.java:118) > at junit.framework.TestSuite.runTest(TestSuite.java:208) > at junit.framework.TestSuite.run(TestSuite.java:203) > at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:297) > at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:672) > at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:567) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From hive-ohampton@example.com Fri Jan 02 07:36:07 2009 Return-Path: <hive-ohampton@example.com> Delivered-To: ohampton@example.com Received: (qmail 62961 invoked from network); 2 Jan 2009 07:36:07 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Jan 2009 07:36:07 -0000 Received: (qmail 60557 invoked by uid 500); 2 Jan 2009 07:36:07 -0000 Delivered-To: ohampton@example.com Received: (qmail 60542 invoked by uid 500); 2 Jan 2009 07:36:07 -0000 Mailing-List: contact ohampton@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:ohampton@example.com> List-Unsubscribe: <mailto:ohampton@example.com> List-Post: <mailto:ohampton@example.com> List-Id: <hive-dev.hadoop.apache.org> Reply-To: ohampton@example.com Delivered-To: mailing list ohampton@example.com Received: (qmail 60531 invoked by uid 99); 2 Jan 2009 07:36:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Jan 2009 23:36:07 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Jan 2009 07:36:05 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 535A6234C4A8 for <ohampton@example.com>; Thu, 1 Jan 2009 23:35:44 -0800 (PST) Message-ID: <289-223-5279.1230881744340.JavaMail.jira@brutus> Date: Thu, 1 Jan 2009 23:35:44 -0800 (PST) From: "Joydeep Sen Sarma (JIRA)" <ohampton@example.com> To: ohampton@example.com Subject: [jira] Commented: (HIVE-204) Provide option to run hadoop via TestMiniMR In-Reply-To: <1020240712.1230755984270.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HIVE-204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660268#action_12660268 ] Joydeep Sen Sarma commented on HIVE-204: ---------------------------------------- we do have the option of submitting the job from the same vm (which is what we do in non-local mode). in local mode - we ran into issues with multiple tasks running in same jvm (we generally assume one task per jvm). is this asking for running tests in non-local mode against a test cluster (miniMR?) (same as hive-117?) > Provide option to run hadoop via TestMiniMR > ------------------------------------------- > > Key: HIVE-204 > URL: https://issues.apache.org/jira/browse/HIVE-204 > Project: Hadoop Hive > Issue Type: Improvement > Components: Testing Infrastructure > Reporter: Raghotham Murthy > > Right now MapRedTask does an exec on the hadoop command line. This prevents us from stepping through query execution code. If there were an option to run hadoop via MiniMR, this will create tasks within the same VM and so allows stepping into the execution code. See, src/test/org/apache/hadoop/mapred/TestMiniMRWithDFS.java in the hadoop source code for an example. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From hive-ohampton@example.com Fri Jan 02 09:26:06 2009 Return-Path: <hive-ohampton@example.com> Delivered-To: ohampton@example.com Received: (qmail 94873 invoked from network); 2 Jan 2009 09:26:06 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Jan 2009 09:26:06 -0000 Received: (qmail 94647 invoked by uid 500); 2 Jan 2009 09:26:06 -0000 Delivered-To: ohampton@example.com Received: (qmail 94629 invoked by uid 500); 2 Jan 2009 09:26:05 -0000 Mailing-List: contact ohampton@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:ohampton@example.com> List-Unsubscribe: <mailto:ohampton@example.com> List-Post: <mailto:ohampton@example.com> List-Id: <hive-dev.hadoop.apache.org> Reply-To: ohampton@example.com Delivered-To: mailing list ohampton@example.com Received: (qmail 94618 invoked by uid 99); 2 Jan 2009 09:26:05 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Jan 2009 01:26:05 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Jan 2009 09:26:05 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id D0E3E234C4AE for <ohampton@example.com>; Fri, 2 Jan 2009 01:25:44 -0800 (PST) Message-ID: <1683744036.1230888344854.JavaMail.jira@brutus> Date: Fri, 2 Jan 2009 01:25:44 -0800 (PST) From: "Raghotham Murthy (JIRA)" <ohampton@example.com> To: ohampton@example.com Subject: [jira] Updated: (HIVE-48) Support JDBC connections for interoperability between Hive and RDBMS MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HIVE-48?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghotham Murthy updated HIVE-48: --------------------------------- Attachment: hive-48.6.patch I have added functionality to getByte, getBoolean, getDouble, getFloat, getInt, getLong, getObject, getShort, and changed getString to do a toString. Also added a single test to test getInt. Right now, anything other than select * uses MetaDataTypedColumnSetSerDe - so, the result schema has all columns as strings. Will add more tests to jdbc once we start using DynamicSerDe for all queries. > Support JDBC connections for interoperability between Hive and RDBMS > -------------------------------------------------------------------- > > Key: HIVE-48 > URL: https://issues.apache.org/jira/browse/HIVE-48 > Project: Hadoop Hive > Issue Type: Bug > Components: Clients > Reporter: YoungWoo Kim > Assignee: Michi Mutsuzaki > Priority: Minor > Attachments: hadoop-4101.1.patch, hadoop-4101.2.patch, hadoop-4101.3.patch, hadoop-4101.4.patch, hive-48.5.patch, hive-48.6.patch > > > In many DW and BI systems, the data are stored in RDBMS for now such as oracle, mysql, postgresql ... for reporting, charting and etc. > It would be useful to be able to import data from RDBMS and export data to RDBMS using JDBC connections. > If Hive support JDBC connections, It wll be much easier to use 3rd party DW/BI tools. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From hive-ohampton@example.com Fri Jan 02 10:08:06 2009 Return-Path: <hive-ohampton@example.com> Delivered-To: ohampton@example.com Received: (qmail 19711 invoked from network); 2 Jan 2009 10:08:06 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Jan 2009 10:08:06 -0000 Received: (qmail 8139 invoked by uid 500); 2 Jan 2009 10:08:06 -0000 Delivered-To: ohampton@example.com Received: (qmail 8117 invoked by uid 500); 2 Jan 2009 10:08:06 -0000 Mailing-List: contact ohampton@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:ohampton@example.com> List-Unsubscribe: <mailto:ohampton@example.com> List-Post: <mailto:ohampton@example.com> List-Id: <hive-dev.hadoop.apache.org> Reply-To: ohampton@example.com Delivered-To: mailing list ohampton@example.com Received: (qmail 8106 invoked by uid 99); 2 Jan 2009 10:08:06 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Jan 2009 02:08:06 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Jan 2009 10:08:05 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 3402D234C4A8 for <ohampton@example.com>; Fri, 2 Jan 2009 02:07:45 -0800 (PST) Message-ID: <282675596.1230890865212.JavaMail.jira@brutus> Date: Fri, 2 Jan 2009 02:07:45 -0800 (PST) From: "Raghotham Murthy (JIRA)" <ohampton@example.com> To: ohampton@example.com Subject: [jira] Commented: (HIVE-204) Provide option to run hadoop via TestMiniMR In-Reply-To: <1020240712.1230755984270.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HIVE-204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660288#action_12660288 ] Raghotham Murthy commented on HIVE-204: --------------------------------------- Ashish mentioned to me that MapRedTask(local mode, different vm) and ExecDriver(non-local mode, same vm) existed. This jira is to be able to run local mode, same vm for junit tests only for stepping through map and reduce tasks. This jira is asking for the ability to submit jobs using the MiniMRCluster class. According to javadocs - "This class creates a single-process Map-Reduce cluster for junit testing. One thread is created for each server." > Provide option to run hadoop via TestMiniMR > ------------------------------------------- > > Key: HIVE-204 > URL: https://issues.apache.org/jira/browse/HIVE-204 > Project: Hadoop Hive > Issue Type: Improvement > Components: Testing Infrastructure > Reporter: Raghotham Murthy > > Right now MapRedTask does an exec on the hadoop command line. This prevents us from stepping through query execution code. If there were an option to run hadoop via MiniMR, this will create tasks within the same VM and so allows stepping into the execution code. See, src/test/org/apache/hadoop/mapred/TestMiniMRWithDFS.java in the hadoop source code for an example. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From hive-ohampton@example.com Fri Jan 02 12:54:06 2009 Return-Path: <hive-ohampton@example.com> Delivered-To: ohampton@example.com Received: (qmail 80084 invoked from network); 2 Jan 2009 12:54:06 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Jan 2009 12:54:06 -0000 Received: (qmail 28551 invoked by uid 500); 2 Jan 2009 12:54:06 -0000 Delivered-To: ohampton@example.com Received: (qmail 28521 invoked by uid 500); 2 Jan 2009 12:54:05 -0000 Mailing-List: contact ohampton@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:ohampton@example.com> List-Unsubscribe: <mailto:ohampton@example.com> List-Post: <mailto:ohampton@example.com> List-Id: <hive-dev.hadoop.apache.org> Reply-To: ohampton@example.com Delivered-To: mailing list ohampton@example.com Received: (qmail 28509 invoked by uid 99); 2 Jan 2009 12:54:05 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Jan 2009 04:54:05 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Jan 2009 12:54:04 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 912D8234C4A9 for <ohampton@example.com>; Fri, 2 Jan 2009 04:53:44 -0800 (PST) Message-ID: <1987680338.1230900824593.JavaMail.jira@brutus> Date: Fri, 2 Jan 2009 04:53:44 -0800 (PST) From: "Raghotham Murthy (JIRA)" <ohampton@example.com> To: ohampton@example.com Subject: [jira] Updated: (HIVE-132) Show table and describe results should be read via FetchTask In-Reply-To: <1316186952.1228739684585.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HIVE-132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghotham Murthy updated HIVE-132: ---------------------------------- Attachment: hive-132.2.patch In this patch: 1. added support for show partitions 2. describe extended is also supported via FetchTask - the detailed information becomes an additional row in the output. 3. added comments 4. captured new outputs dor > Show table and describe results should be read via FetchTask > ------------------------------------------------------------ > > Key: HIVE-132 > URL: https://issues.apache.org/jira/browse/HIVE-132 > Project: Hadoop Hive > Issue Type: New Feature > Components: Query Processor > Affects Versions: 0.2.0 > Reporter: Raghotham Murthy > Assignee: Raghotham Murthy > Priority: Minor > Fix For: 0.2.0 > > Attachments: hive-132.1.patch, hive-132.2.patch > > > Right now there is a different code path in the Driver for show tables/describe etc. By adding the FetchTask the code paths will be merged. In addition, the results of these statements will be readable via JDBC. > Going forward, we should provide SQL access to our metastore similar to access provided by Oracle or mysql to their catalog tables. In this jira, I will just add schema information to show tables and describe queries. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From hive-ohampton@example.com Sat Jan 03 00:00:25 2009 Return-Path: <hive-ohampton@example.com> Delivered-To: ohampton@example.com Received: (qmail 67620 invoked from network); 3 Jan 2009 00:00:24 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 3 Jan 2009 00:00:24 -0000 Received: (qmail 96379 invoked by uid 500); 3 Jan 2009 00:00:24 -0000 Delivered-To: ohampton@example.com Received: (qmail 96361 invoked by uid 500); 3 Jan 2009 00:00:24 -0000 Mailing-List: contact ohampton@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:ohampton@example.com> List-Unsubscribe: <mailto:ohampton@example.com> List-Post: <mailto:ohampton@example.com> List-Id: <hive-dev.hadoop.apache.org> Reply-To: ohampton@example.com Delivered-To: mailing list ohampton@example.com Received: (qmail 96350 invoked by uid 99); 3 Jan 2009 00:00:24 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Jan 2009 16:00:24 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 03 Jan 2009 00:00:23 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 5081B234C4A8 for <ohampton@example.com>; Fri, 2 Jan 2009 15:59:44 -0800 (PST) Message-ID: <774449354.1230940784328.JavaMail.jira@brutus> Date: Fri, 2 Jan 2009 15:59:44 -0800 (PST) From: "Joydeep Sen Sarma (JIRA)" <ohampton@example.com> To: ohampton@example.com Subject: [jira] Commented: (HIVE-204) Provide option to run hadoop via TestMiniMR In-Reply-To: <1020240712.1230755984270.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HIVE-204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660427#action_12660427 ] Joydeep Sen Sarma commented on HIVE-204: ---------------------------------------- ok. i am not sure why exactly - but we had trouble trying to do this (local mode same vm). if the localmode, same vm combination can be made to work - the minimr stuff is not relevant - is it? also - for the minimr cluster - will it run in a separate jvm? or will it be running in the same jvm (as cli/driver etc.)? > Provide option to run hadoop via TestMiniMR > ------------------------------------------- > > Key: HIVE-204 > URL: https://issues.apache.org/jira/browse/HIVE-204 > Project: Hadoop Hive > Issue Type: Improvement > Components: Testing Infrastructure > Reporter: Raghotham Murthy > > Right now MapRedTask does an exec on the hadoop command line. This prevents us from stepping through query execution code. If there were an option to run hadoop via MiniMR, this will create tasks within the same VM and so allows stepping into the execution code. See, src/test/org/apache/hadoop/mapred/TestMiniMRWithDFS.java in the hadoop source code for an example. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From hive-ohampton@example.com Mon Jan 05 08:08:08 2009 Return-Path: <hive-ohampton@example.com> Delivered-To: ohampton@example.com Received: (qmail 16642 invoked from network); 5 Jan 2009 08:08:07 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 5 Jan 2009 08:08:07 -0000 Received: (qmail 15996 invoked by uid 500); 5 Jan 2009 08:08:07 -0000 Delivered-To: ohampton@example.com Received: (qmail 15970 invoked by uid 500); 5 Jan 2009 08:08:07 -0000 Mailing-List: contact ohampton@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:ohampton@example.com> List-Unsubscribe: <mailto:ohampton@example.com> List-Post: <mailto:ohampton@example.com> List-Id: <hive-dev.hadoop.apache.org> Reply-To: ohampton@example.com Delivered-To: mailing list ohampton@example.com Received: (qmail 15959 invoked by uid 99); 5 Jan 2009 08:08:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 00:08:07 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 08:08:05 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id ABD73234C485 for <ohampton@example.com>; Mon, 5 Jan 2009 00:07:44 -0800 (PST) Message-ID: <1685439223.1231142864702.JavaMail.jira@brutus> Date: Mon, 5 Jan 2009 00:07:44 -0800 (PST) From: "Zheng Shao (JIRA)" <ohampton@example.com> To: ohampton@example.com Subject: [jira] Created: (HIVE-205) file sink operator needs unique file names MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org file sink operator needs unique file names ------------------------------------------ Key: HIVE-205 URL: https://issues.apache.org/jira/browse/HIVE-205 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Reporter: Zheng Shao a query like "SELECT * FROM (SELECT 'a' from table UNION ALL SELECT 'b' from table) c" will fail, because the 2 sub queries of "UNION" will be executed inside the same mapper (because the input is the same) towards the same destination table, and the 2 file sink operators will have the same output file name. We need to append the file sink operator id to the file name to make it unique. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From hive-ohampton@example.com Mon Jan 05 08:24:07 2009 Return-Path: <hive-ohampton@example.com> Delivered-To: ohampton@example.com Received: (qmail 22940 invoked from network); 5 Jan 2009 08:24:07 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 5 Jan 2009 08:24:07 -0000 Received: (qmail 25455 invoked by uid 500); 5 Jan 2009 08:24:07 -0000 Delivered-To: ohampton@example.com Received: (qmail 25432 invoked by uid 500); 5 Jan 2009 08:24:07 -0000 Mailing-List: contact ohampton@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:ohampton@example.com> List-Unsubscribe: <mailto:ohampton@example.com> List-Post: <mailto:ohampton@example.com> List-Id: <hive-dev.hadoop.apache.org> Reply-To: ohampton@example.com Delivered-To: mailing list ohampton@example.com Received: (qmail 25421 invoked by uid 99); 5 Jan 2009 08:24:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 00:24:07 -0800 X-ASF-Spam-Status: No, hits=-1999.6 required=10.0 tests=ALL_TRUSTED,SUBJECT_FUZZY_TION X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 08:24:05 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 5E6B9234C485 for <ohampton@example.com>; Mon, 5 Jan 2009 00:23:44 -0800 (PST) Message-ID: <1422848058.1231143824384.JavaMail.jira@brutus> Date: Mon, 5 Jan 2009 00:23:44 -0800 (PST) From: "Zheng Shao (JIRA)" <ohampton@example.com> To: ohampton@example.com Subject: [jira] Created: (HIVE-206) partition pruning check fails for uppercase table names MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org partition pruning check fails for uppercase table names ------------------------------------------------------- Key: HIVE-206 URL: https://issues.apache.org/jira/browse/HIVE-206 Project: Hadoop Hive Issue Type: Bug Reporter: Zheng Shao The failed query is like: SELECT "a" from table_TAG where table_TAG.ds = '2008-12-31' The error message is: FAILED: Error in semantic analysis: line 1:227 No Partition Predicate Found "2008-12-31": for Alias table_tag Table table_tag -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From hive-ohampton@example.com Mon Jan 05 14:04:07 2009 Return-Path: <hive-ohampton@example.com> Delivered-To: ohampton@example.com Received: (qmail 78596 invoked from network); 5 Jan 2009 14:04:07 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 5 Jan 2009 14:04:07 -0000 Received: (qmail 51237 invoked by uid 500); 5 Jan 2009 14:04:07 -0000 Delivered-To: ohampton@example.com Received: (qmail 51216 invoked by uid 500); 5 Jan 2009 14:04:07 -0000 Mailing-List: contact ohampton@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:ohampton@example.com> List-Unsubscribe: <mailto:ohampton@example.com> List-Post: <mailto:ohampton@example.com> List-Id: <hive-dev.hadoop.apache.org> Reply-To: ohampton@example.com Delivered-To: mailing list ohampton@example.com Received: (qmail 51205 invoked by uid 99); 5 Jan 2009 14:04:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 06:04:07 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 14:04:05 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 6DBE9234C48D for <ohampton@example.com>; Mon, 5 Jan 2009 06:03:44 -0800 (PST) Message-ID: <1208601007.1231164224445.JavaMail.jira@brutus> Date: Mon, 5 Jan 2009 06:03:44 -0800 (PST) From: "Ashish Thusoo (JIRA)" <ohampton@example.com> To: ohampton@example.com Subject: [jira] Commented: (HIVE-203) fix eclipse templates to get junit tests working In-Reply-To: <1003272416.1230689204414.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HIVE-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660763#action_12660763 ] Ashish Thusoo commented on HIVE-203: ------------------------------------ I was able to import the project using this patch (this is very useful), however I got the following errors: 1. There is a duplicate entry for serde/src/gen-java in the .classpath files - this gives a build error in eclipse 2. The build is looking for hadoopcore/hadoop-83.161.120.155 - in the checked in files we should probably have this as 0.19. (I was able to get rid of the build issues once I did ant package -Dhadoop.version="83.161.120.155" I think without fixing 1 this will not build in eclipse. Also for the hadoop version, we can potentially do the following in the .classpath file so that it brings in the correct version. Using @HADOOPVER@ in .classpath similar to how we use @PROJECT@ in .project and then add the following in <root>/build.xml <filter token="HADOOPVER" value="${hadoop.version}"/> after the token for PROJECT I do see the test configurations... This looks great.. However, when I try to run say the TestCliDriver configuration it seems to give me an error "Input type of the launch configuration does not exist". Also while running TestHive I get the following: "The archive: /hive_local_ws3/build/metastore/metastore_model.jar which is referenced by the classpath, does not exist." even though the jar is there (I checked it). > fix eclipse templates to get junit tests working > ------------------------------------------------ > > Key: HIVE-203 > URL: https://issues.apache.org/jira/browse/HIVE-203 > Project: Hadoop Hive > Issue Type: Bug > Components: Build Infrastructure > Affects Versions: 0.2.0 > Reporter: Raghotham Murthy > Assignee: Raghotham Murthy > Fix For: 0.2.0 > > Attachments: hive-203.1.patch, hive-203.2.patch > > > right now eclipse templates are not complete. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From hive-ohampton@example.com Mon Jan 05 14:34:05 2009 Return-Path: <hive-ohampton@example.com> Delivered-To: ohampton@example.com Received: (qmail 95863 invoked from network); 5 Jan 2009 14:34:05 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 5 Jan 2009 14:34:05 -0000 Received: (qmail 68774 invoked by uid 500); 5 Jan 2009 14:34:05 -0000 Delivered-To: ohampton@example.com Received: (qmail 68756 invoked by uid 500); 5 Jan 2009 14:34:05 -0000 Mailing-List: contact ohampton@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:ohampton@example.com> List-Unsubscribe: <mailto:ohampton@example.com> List-Post: <mailto:ohampton@example.com> List-Id: <hive-dev.hadoop.apache.org> Reply-To: ohampton@example.com Delivered-To: mailing list ohampton@example.com Received: (qmail 68745 invoked by uid 99); 5 Jan 2009 14:34:05 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 06:34:05 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 14:34:04 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 80932234C48D for <ohampton@example.com>; Mon, 5 Jan 2009 06:33:44 -0800 (PST) Message-ID: <1298374454.1231166024525.JavaMail.jira@brutus> Date: Mon, 5 Jan 2009 06:33:44 -0800 (PST) From: "Ashish Thusoo (JIRA)" <ohampton@example.com> To: ohampton@example.com Subject: [jira] Commented: (HIVE-132) Show table and describe results should be read via FetchTask In-Reply-To: <1316186952.1228739684585.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HIVE-132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660770#action_12660770 ] Ashish Thusoo commented on HIVE-132: ------------------------------------ A few nitpicks: 1. Why did we get rid of : in the explain plan output "Table Information:" has been replaced by "Table Information" without a new line. Also there seems to be extra space/tab after the column metadata is printed. Thanks for fixing dailed to failed :).. that has been there for so long... :)) For the general code: 1. In DDLSemanticAnalyzer.java in the function createFetchTask setSerializationNullFormat is "" - should this be a null string instead or a "" Otherwise it looks good to me. > Show table and describe results should be read via FetchTask > ------------------------------------------------------------ > > Key: HIVE-132 > URL: https://issues.apache.org/jira/browse/HIVE-132 > Project: Hadoop Hive > Issue Type: New Feature > Components: Query Processor > Affects Versions: 0.2.0 > Reporter: Raghotham Murthy > Assignee: Raghotham Murthy > Priority: Minor > Fix For: 0.2.0 > > Attachments: hive-132.1.patch, hive-132.2.patch > > > Right now there is a different code path in the Driver for show tables/describe etc. By adding the FetchTask the code paths will be merged. In addition, the results of these statements will be readable via JDBC. > Going forward, we should provide SQL access to our metastore similar to access provided by Oracle or mysql to their catalog tables. In this jira, I will just add schema information to show tables and describe queries. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From hive-ohampton@example.com Mon Jan 05 14:50:05 2009 Return-Path: <hive-ohampton@example.com> Delivered-To: ohampton@example.com Received: (qmail 2849 invoked from network); 5 Jan 2009 14:50:05 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 5 Jan 2009 14:50:05 -0000 Received: (qmail 85749 invoked by uid 500); 5 Jan 2009 14:50:05 -0000 Delivered-To: ohampton@example.com Received: (qmail 85726 invoked by uid 500); 5 Jan 2009 14:50:05 -0000 Mailing-List: contact ohampton@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:ohampton@example.com> List-Unsubscribe: <mailto:ohampton@example.com> List-Post: <mailto:ohampton@example.com> List-Id: <hive-dev.hadoop.apache.org> Reply-To: ohampton@example.com Delivered-To: mailing list ohampton@example.com Received: (qmail 85715 invoked by uid 99); 5 Jan 2009 14:50:05 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 06:50:05 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 14:50:04 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 6175E234C4A9 for <ohampton@example.com>; Mon, 5 Jan 2009 06:49:44 -0800 (PST) Message-ID: <318559105.1231166984398.JavaMail.jira@brutus> Date: Mon, 5 Jan 2009 06:49:44 -0800 (PST) From: "Ashish Thusoo (JIRA)" <ohampton@example.com> To: ohampton@example.com Subject: [jira] Commented: (HIVE-48) Support JDBC connections for interoperability between Hive and RDBMS MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HIVE-48?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660774#action_12660774 ] Ashish Thusoo commented on HIVE-48: ----------------------------------- A few questions: 1. What is the motivation for Driver.java:87 change? 2. In JdbcSessionState.java what are execString and fileName variables used for? 3. Shouldn't HiveResultSetMetadata.java be doing something? 4. In HiveResultSet.java:436 the getDouble function is missing TODO comment? Same is true for getFloat and getInt. 5. For the non implemented functions should we through a non implemented run time exception instead of just returning 0. > Support JDBC connections for interoperability between Hive and RDBMS > -------------------------------------------------------------------- > > Key: HIVE-48 > URL: https://issues.apache.org/jira/browse/HIVE-48 > Project: Hadoop Hive > Issue Type: Bug > Components: Clients > Reporter: YoungWoo Kim > Assignee: Michi Mutsuzaki > Priority: Minor > Attachments: hadoop-4101.1.patch, hadoop-4101.2.patch, hadoop-4101.3.patch, hadoop-4101.4.patch, hive-48.5.patch, hive-48.6.patch > > > In many DW and BI systems, the data are stored in RDBMS for now such as oracle, mysql, postgresql ... for reporting, charting and etc. > It would be useful to be able to import data from RDBMS and export data to RDBMS using JDBC connections. > If Hive support JDBC connections, It wll be much easier to use 3rd party DW/BI tools. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From hive-ohampton@example.com Mon Jan 05 15:10:08 2009 Return-Path: <hive-ohampton@example.com> Delivered-To: ohampton@example.com Received: (qmail 33557 invoked from network); 5 Jan 2009 15:10:08 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 5 Jan 2009 15:10:08 -0000 Received: (qmail 12118 invoked by uid 500); 5 Jan 2009 15:10:08 -0000 Delivered-To: ohampton@example.com Received: (qmail 12087 invoked by uid 500); 5 Jan 2009 15:10:07 -0000 Mailing-List: contact ohampton@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:ohampton@example.com> List-Unsubscribe: <mailto:ohampton@example.com> List-Post: <mailto:ohampton@example.com> List-Id: <hive-dev.hadoop.apache.org> Reply-To: ohampton@example.com Delivered-To: mailing list ohampton@example.com Received: (qmail 11863 invoked by uid 99); 5 Jan 2009 15:10:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 07:10:07 -0800 X-ASF-Spam-Status: No, hits=-1999.6 required=10.0 tests=ALL_TRUSTED,SUBJECT_FUZZY_TION X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 15:10:05 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id A795C234C48D for <ohampton@example.com>; Mon, 5 Jan 2009 07:09:44 -0800 (PST) Message-ID: <647585433.1231168184685.JavaMail.jira@brutus> Date: Mon, 5 Jan 2009 07:09:44 -0800 (PST) From: "Ashish Thusoo (JIRA)" <ohampton@example.com> To: ohampton@example.com Subject: [jira] Assigned: (HIVE-206) partition pruning check fails for uppercase table names In-Reply-To: <1422848058.1231143824384.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HIVE-206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Thusoo reassigned HIVE-206: ---------------------------------- Assignee: Ashish Thusoo > partition pruning check fails for uppercase table names > ------------------------------------------------------- > > Key: HIVE-206 > URL: https://issues.apache.org/jira/browse/HIVE-206 > Project: Hadoop Hive > Issue Type: Bug > Reporter: Zheng Shao > Assignee: Ashish Thusoo > > The failed query is like: > SELECT "a" from table_TAG where table_TAG.ds = '2008-12-31' > The error message is: > FAILED: Error in semantic analysis: line 1:227 No Partition Predicate Found "2008-12-31": for Alias table_tag Table table_tag -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From hive-ohampton@example.com Mon Jan 05 15:12:05 2009 Return-Path: <hive-ohampton@example.com> Delivered-To: ohampton@example.com Received: (qmail 34350 invoked from network); 5 Jan 2009 15:12:05 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 5 Jan 2009 15:12:05 -0000 Received: (qmail 14294 invoked by uid 500); 5 Jan 2009 15:12:05 -0000 Delivered-To: ohampton@example.com Received: (qmail 14274 invoked by uid 500); 5 Jan 2009 15:12:05 -0000 Mailing-List: contact ohampton@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:ohampton@example.com> List-Unsubscribe: <mailto:ohampton@example.com> List-Post: <mailto:ohampton@example.com> List-Id: <hive-dev.hadoop.apache.org> Reply-To: ohampton@example.com Delivered-To: mailing list ohampton@example.com Received: (qmail 14263 invoked by uid 99); 5 Jan 2009 15:12:05 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 07:12:05 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 15:12:04 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 371E0234C485 for <ohampton@example.com>; Mon, 5 Jan 2009 07:11:44 -0800 (PST) Message-ID: <2007661186.1231168304224.JavaMail.jira@brutus> Date: Mon, 5 Jan 2009 07:11:44 -0800 (PST) From: "Ashish Thusoo (JIRA)" <ohampton@example.com> To: ohampton@example.com Subject: [jira] Commented: (HIVE-205) file sink operator needs unique file names In-Reply-To: <1685439223.1231142864702.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HIVE-205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660781#action_12660781 ] Ashish Thusoo commented on HIVE-205: ------------------------------------ what happens in the test ./ql/src/test/queries/clientpositive/union.q the seems to have the same general pattern as the reported query... > file sink operator needs unique file names > ------------------------------------------ > > Key: HIVE-205 > URL: https://issues.apache.org/jira/browse/HIVE-205 > Project: Hadoop Hive > Issue Type: Bug > Components: Query Processor > Reporter: Zheng Shao > > a query like "SELECT * FROM (SELECT 'a' from table UNION ALL SELECT 'b' from table) c" will fail, because the 2 sub queries of "UNION" will be executed inside the same mapper (because the input is the same) towards the same destination table, and the 2 file sink operators will have the same output file name. > We need to append the file sink operator id to the file name to make it unique. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From hive-ohampton@example.com Mon Jan 05 15:22:08 2009 Return-Path: <hive-ohampton@example.com> Delivered-To: ohampton@example.com Received: (qmail 40026 invoked from network); 5 Jan 2009 15:22:08 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 5 Jan 2009 15:22:08 -0000 Received: (qmail 25002 invoked by uid 500); 5 Jan 2009 15:22:08 -0000 Delivered-To: ohampton@example.com Received: (qmail 24985 invoked by uid 500); 5 Jan 2009 15:22:08 -0000 Mailing-List: contact ohampton@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:ohampton@example.com> List-Unsubscribe: <mailto:ohampton@example.com> List-Post: <mailto:ohampton@example.com> List-Id: <hive-dev.hadoop.apache.org> Reply-To: ohampton@example.com Delivered-To: mailing list ohampton@example.com Received: (qmail 24974 invoked by uid 99); 5 Jan 2009 15:22:08 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 07:22:08 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 15:22:05 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id A5605234C48B for <ohampton@example.com>; Mon, 5 Jan 2009 07:21:44 -0800 (PST) Message-ID: <2130261344.1231168904676.JavaMail.jira@brutus> Date: Mon, 5 Jan 2009 07:21:44 -0800 (PST) From: "Ashish Thusoo (JIRA)" <ohampton@example.com> To: ohampton@example.com Subject: [jira] Commented: (HIVE-84) MetaStore Client is not thread safe In-Reply-To: <776488778.1227783944227.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HIVE-84?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660785#action_12660785 ] Ashish Thusoo commented on HIVE-84: ----------------------------------- Do you still need the needsRefresh code in SessionState.java Also why just check for the refresh stuff only for MetaVars and why not do it for all the vars? Why the special case? > MetaStore Client is not thread safe > ----------------------------------- > > Key: HIVE-84 > URL: https://issues.apache.org/jira/browse/HIVE-84 > Project: Hadoop Hive > Issue Type: Bug > Components: Metastore > Affects Versions: 0.2.0 > Environment: with patch for hive-77 - run: > ant -lib ./testlibs -Dtestcase=TestMTQueries test > Reporter: Joydeep Sen Sarma > Assignee: Prasad Chakka > Fix For: 0.2.0 > > Attachments: hive-84.patch > > > when running DDL Tasks in concurrent threads - the following exception trace is observed: > java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a duplicate ke\ y value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'. > at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:207) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:209) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:174) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:185) > at
{'source': 'Apache Software Foundation Public Mail Archives'}