From pig-dev-heatherfrank@example.org Fri Feb 01 00:47:35 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 26248 invoked from network); 1 Feb 2008 00:47:35 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Feb 2008 00:47:35 -0000 Received: (qmail 93330 invoked by uid 500); 1 Feb 2008 00:47:26 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 93289 invoked by uid 500); 1 Feb 2008 00:47:26 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 93280 invoked by uid 99); 1 Feb 2008 00:47:26 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 31 Jan 2008 16:47:26 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 00:47:19 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id A49DF71403C for <heatherfrank@example.org>; Thu, 31 Jan 2008 16:47:11 -0800 (PST) Message-ID: <18012606.1201826831626.JavaMail.jira@brutus> Date: Thu, 31 Jan 2008 16:47:11 -0800 (PST) From: "Xu Zhang (JIRA)" <heatherfrank@example.org> To: heatherfrank@example.org Subject: [jira] Updated: (PIG-72) Porting Pig unit tests to use MiniDFSCluster and MiniMRCluster on the local machine In-Reply-To: <22145126.1201130914821.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/PIG-72?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Zhang updated PIG-72: ------------------------ Attachment: (was: PortPigUnitTestToMiniClusters.patch) > Porting Pig unit tests to use MiniDFSCluster and MiniMRCluster on the local machine > ----------------------------------------------------------------------------------- > > Key: PIG-72 > URL: https://issues.apache.org/jira/browse/PIG-72 > Project: Pig > Issue Type: Test > Components: tools > Reporter: Xu Zhang > > We have the need to port the Pig unit tests to use MiniDFSCluster and MiniMRCluster, so that tests can be executed with the DFS and MR threads on the local machine. This feature will eliminate the need to set up a real distributed hadoop cluster before running the unit tests, as everything will now be carried out with the (mini) cluster on the user's local machine. > One prerequisite for using this feature is a hadoop jar that has the class files for MiniDFSCluster, MiniMRCluster and other supporting components. I have been able to generate such a jar file with a special target added by myself to hadoop's build.xml and have also logged a hadoop jira to request this target be a permanent part of that build file. If possible, we can just replace hadoop15.jar with this jar file on the SVN source tree and then the users will never need to worry about the availability of this jar file. Please find such a hadoop jar file in the attachment. > To use the feature in unit tests, the user just need to call MiniClusterBuilder.buildCluster() before a PigServer instance is created with the string "mapreduce" as the parameter to its constructor. Here is an example of how the MiniClusterBuilder is used in a test case class: > public class TestWhatEver extends TestCase { > private String initString = "mapreduce"; > private MiniClusterBuilder cluster = MiniClusterBuilder.buildCluster(); > > @Test > public void testGroupCountWithMultipleFields() throws Exception { > PigServer pig = new PigServer(initString); > // Do something with the pig server, such as registering and executing Pig > // queries. The queries will executed with the local cluster. > } > > // More test cases if needed > } > To run the unit tests with the local cluster, under the top directory of the source tree, issue the command "ant test". Notice that you do not need to specify the location of the hadoop-site.xml file with the command line option "-Djunit.hadoop.conf=<dir>" anymore. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pig-dev-heatherfrank@example.org Fri Feb 01 00:49:30 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 26458 invoked from network); 1 Feb 2008 00:49:30 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Feb 2008 00:49:30 -0000 Received: (qmail 96136 invoked by uid 500); 1 Feb 2008 00:49:21 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 96076 invoked by uid 500); 1 Feb 2008 00:49:21 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 96067 invoked by uid 99); 1 Feb 2008 00:49:21 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 31 Jan 2008 16:49:21 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 00:49:01 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 04BB471403C for <heatherfrank@example.org>; Thu, 31 Jan 2008 16:49:09 -0800 (PST) Message-ID: <17603238.1201826949006.JavaMail.jira@brutus> Date: Thu, 31 Jan 2008 16:49:09 -0800 (PST) From: "Xu Zhang (JIRA)" <heatherfrank@example.org> To: heatherfrank@example.org Subject: [jira] Updated: (PIG-72) Porting Pig unit tests to use MiniDFSCluster and MiniMRCluster on the local machine In-Reply-To: <22145126.1201130914821.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/PIG-72?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Zhang updated PIG-72: ------------------------ Attachment: PortPigUnitTestToMiniClusters.patch > Porting Pig unit tests to use MiniDFSCluster and MiniMRCluster on the local machine > ----------------------------------------------------------------------------------- > > Key: PIG-72 > URL: https://issues.apache.org/jira/browse/PIG-72 > Project: Pig > Issue Type: Test > Components: tools > Reporter: Xu Zhang > Attachments: PortPigUnitTestToMiniClusters.patch > > > We have the need to port the Pig unit tests to use MiniDFSCluster and MiniMRCluster, so that tests can be executed with the DFS and MR threads on the local machine. This feature will eliminate the need to set up a real distributed hadoop cluster before running the unit tests, as everything will now be carried out with the (mini) cluster on the user's local machine. > One prerequisite for using this feature is a hadoop jar that has the class files for MiniDFSCluster, MiniMRCluster and other supporting components. I have been able to generate such a jar file with a special target added by myself to hadoop's build.xml and have also logged a hadoop jira to request this target be a permanent part of that build file. If possible, we can just replace hadoop15.jar with this jar file on the SVN source tree and then the users will never need to worry about the availability of this jar file. Please find such a hadoop jar file in the attachment. > To use the feature in unit tests, the user just need to call MiniClusterBuilder.buildCluster() before a PigServer instance is created with the string "mapreduce" as the parameter to its constructor. Here is an example of how the MiniClusterBuilder is used in a test case class: > public class TestWhatEver extends TestCase { > private String initString = "mapreduce"; > private MiniClusterBuilder cluster = MiniClusterBuilder.buildCluster(); > > @Test > public void testGroupCountWithMultipleFields() throws Exception { > PigServer pig = new PigServer(initString); > // Do something with the pig server, such as registering and executing Pig > // queries. The queries will executed with the local cluster. > } > > // More test cases if needed > } > To run the unit tests with the local cluster, under the top directory of the source tree, issue the command "ant test". Notice that you do not need to specify the location of the hadoop-site.xml file with the command line option "-Djunit.hadoop.conf=<dir>" anymore. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pig-dev-heatherfrank@example.org Fri Feb 01 00:49:30 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 26489 invoked from network); 1 Feb 2008 00:49:30 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Feb 2008 00:49:30 -0000 Received: (qmail 96185 invoked by uid 500); 1 Feb 2008 00:49:21 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 96163 invoked by uid 500); 1 Feb 2008 00:49:21 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 96154 invoked by uid 99); 1 Feb 2008 00:49:21 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 31 Jan 2008 16:49:21 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 00:49:02 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 8618D714047 for <heatherfrank@example.org>; Thu, 31 Jan 2008 16:49:09 -0800 (PST) Message-ID: <22881602.1201826949545.JavaMail.jira@brutus> Date: Thu, 31 Jan 2008 16:49:09 -0800 (PST) From: "Xu Zhang (JIRA)" <heatherfrank@example.org> To: heatherfrank@example.org Subject: [jira] Commented: (PIG-72) Porting Pig unit tests to use MiniDFSCluster and MiniMRCluster on the local machine In-Reply-To: <22145126.1201130914821.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/PIG-72?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12564573#action_12564573 ] Xu Zhang commented on PIG-72: ----------------------------- This patch contains an overhaul from the 1st version of the patch on the architecture of how the local mini clusters are set up and torn down. Now we start the local mini clusters in the setUp() method and shut them down in the teardown() method. Any Pig unit test case class that wants to make use of the local mini clusters needs to extend the PigTestCase abstract class, and then calls super.setUp() from its own setUp() method and super.tearDown() from its own teardown() method. Also, make sure PigServer is instantiated after super.setUp() has been called. If you need to, you can specify the number of datanodes and tasktrackers through the corresponding PigTestCase constructor. The default values are both 2. Note that if you do not want to use the local mini clusters (such as when you just want to run Pig in local mode) you can override the setUp() and teardown() methods and DO NOT call super.setUp() and super.tearDown() in those methods if your class happens to extend PigTestCase. Alternatively, you can just simply extends from TestCase. There is no change in the way of invoking the Pig unit tests from the 1st version of the patch; i.e., you still issue the "ant test" command at the top directory of your Pig project. On a related note, the total amount of time used to execute all the existing Pig test cases is 19 - 20 minutes on my computer. > Porting Pig unit tests to use MiniDFSCluster and MiniMRCluster on the local machine > ----------------------------------------------------------------------------------- > > Key: PIG-72 > URL: https://issues.apache.org/jira/browse/PIG-72 > Project: Pig > Issue Type: Test > Components: tools > Reporter: Xu Zhang > Attachments: PortPigUnitTestToMiniClusters.patch > > > We have the need to port the Pig unit tests to use MiniDFSCluster and MiniMRCluster, so that tests can be executed with the DFS and MR threads on the local machine. This feature will eliminate the need to set up a real distributed hadoop cluster before running the unit tests, as everything will now be carried out with the (mini) cluster on the user's local machine. > One prerequisite for using this feature is a hadoop jar that has the class files for MiniDFSCluster, MiniMRCluster and other supporting components. I have been able to generate such a jar file with a special target added by myself to hadoop's build.xml and have also logged a hadoop jira to request this target be a permanent part of that build file. If possible, we can just replace hadoop15.jar with this jar file on the SVN source tree and then the users will never need to worry about the availability of this jar file. Please find such a hadoop jar file in the attachment. > To use the feature in unit tests, the user just need to call MiniClusterBuilder.buildCluster() before a PigServer instance is created with the string "mapreduce" as the parameter to its constructor. Here is an example of how the MiniClusterBuilder is used in a test case class: > public class TestWhatEver extends TestCase { > private String initString = "mapreduce"; > private MiniClusterBuilder cluster = MiniClusterBuilder.buildCluster(); > > @Test > public void testGroupCountWithMultipleFields() throws Exception { > PigServer pig = new PigServer(initString); > // Do something with the pig server, such as registering and executing Pig > // queries. The queries will executed with the local cluster. > } > > // More test cases if needed > } > To run the unit tests with the local cluster, under the top directory of the source tree, issue the command "ant test". Notice that you do not need to specify the location of the hadoop-site.xml file with the command line option "-Djunit.hadoop.conf=<dir>" anymore. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pig-dev-heatherfrank@example.org Fri Feb 01 02:49:29 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 57252 invoked from network); 1 Feb 2008 02:49:29 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Feb 2008 02:49:29 -0000 Received: (qmail 90465 invoked by uid 500); 1 Feb 2008 02:49:20 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 90441 invoked by uid 500); 1 Feb 2008 02:49:20 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 90432 invoked by uid 99); 1 Feb 2008 02:49:20 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 31 Jan 2008 18:49:20 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 02:49:01 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 3B97D714043 for <heatherfrank@example.org>; Thu, 31 Jan 2008 18:49:08 -0800 (PST) Message-ID: <13830914.1201834148240.JavaMail.jira@brutus> Date: Thu, 31 Jan 2008 18:49:08 -0800 (PST) From: "Olga Natkovich (JIRA)" <heatherfrank@example.org> To: heatherfrank@example.org Subject: [jira] Commented: (PIG-32) Abstraction Layer to decouple Pig from Back-End In-Reply-To: <7651636.1196198743011.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/PIG-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12564590#action_12564590 ] Olga Natkovich commented on PIG-32: ----------------------------------- I verified that with this patch the unit tests take the same time. In the process of committing the patch ... > Abstraction Layer to decouple Pig from Back-End > ----------------------------------------------- > > Key: PIG-32 > URL: https://issues.apache.org/jira/browse/PIG-32 > Project: Pig > Issue Type: New Feature > Components: impl > Reporter: Antonio Magnaghi > Assignee: Antonio Magnaghi > Attachments: 2008.01.29.patch, DataStorage.diff, DataStorage20071212.diff, patch.2008.01.16.merge_w_trunk, patch.2008.01.23.diff, PATCH.2008.01.31, patch2007_12_26.diff, patch2007_12_26_II.diff, patch2007_12_27.diff, pig.jar, pig.jar.2008.01.16, TEST.LOG, TEST.LOGS > > > I'm opening a new issue to track the development work to support an abstraction layer for Pig as defined at http://wiki.apache.org/pig/PigAbstractionLayer -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pig-dev-heatherfrank@example.org Fri Feb 01 03:19:33 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 63937 invoked from network); 1 Feb 2008 03:19:33 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Feb 2008 03:19:33 -0000 Received: (qmail 8014 invoked by uid 500); 1 Feb 2008 03:19:24 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 7987 invoked by uid 500); 1 Feb 2008 03:19:24 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 7978 invoked by uid 99); 1 Feb 2008 03:19:24 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 31 Jan 2008 19:19:24 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 03:19:04 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id BAE3771403C for <heatherfrank@example.org>; Thu, 31 Jan 2008 19:19:11 -0800 (PST) Message-ID: <25259740.1201835951744.JavaMail.jira@brutus> Date: Thu, 31 Jan 2008 19:19:11 -0800 (PST) From: "Olga Natkovich (JIRA)" <heatherfrank@example.org> To: heatherfrank@example.org Subject: [jira] Resolved: (PIG-32) Abstraction Layer to decouple Pig from Back-End In-Reply-To: <7651636.1196198743011.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/PIG-32?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich resolved PIG-32. ------------------------------- Resolution: Fixed changes committed > Abstraction Layer to decouple Pig from Back-End > ----------------------------------------------- > > Key: PIG-32 > URL: https://issues.apache.org/jira/browse/PIG-32 > Project: Pig > Issue Type: New Feature > Components: impl > Reporter: Antonio Magnaghi > Assignee: Antonio Magnaghi > Attachments: 2008.01.29.patch, DataStorage.diff, DataStorage20071212.diff, patch.2008.01.16.merge_w_trunk, patch.2008.01.23.diff, PATCH.2008.01.31, patch2007_12_26.diff, patch2007_12_26_II.diff, patch2007_12_27.diff, pig.jar, pig.jar.2008.01.16, TEST.LOG, TEST.LOGS > > > I'm opening a new issue to track the development work to support an abstraction layer for Pig as defined at http://wiki.apache.org/pig/PigAbstractionLayer -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pig-dev-heatherfrank@example.org Fri Feb 01 11:15:41 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 71548 invoked from network); 1 Feb 2008 11:15:41 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Feb 2008 11:15:41 -0000 Received: (qmail 27556 invoked by uid 500); 1 Feb 2008 11:15:32 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 27530 invoked by uid 500); 1 Feb 2008 11:15:31 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 27521 invoked by uid 99); 1 Feb 2008 11:15:31 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 03:15:31 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 11:15:01 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 7BD2B714043 for <heatherfrank@example.org>; Fri, 1 Feb 2008 03:15:08 -0800 (PST) Message-ID: <21225698.1201864508476.JavaMail.jira@brutus> Date: Fri, 1 Feb 2008 03:15:08 -0800 (PST) From: "Benjamin Francisoud (JIRA)" <heatherfrank@example.org> To: heatherfrank@example.org Subject: [jira] Commented: (PIG-32) Abstraction Layer to decouple Pig from Back-End In-Reply-To: <7651636.1196198743011.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/PIG-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12564725#action_12564725 ] Benjamin Francisoud commented on PIG-32: ---------------------------------------- I think this commit broke compatibility with java 5 :( For example in PigServer line 281: {code:java} catch (ExecException e) { throw new IOException("Unable to open iterator for alias: " + id, e); } {code} This constructor is only available in java 6: * [java 5 IOException api|http://java.sun.com/j2se/1.5.0/docs/api/java/io/IOException.html#IOException()] * [java 6 IOException api|http://java.sun.com/javase/6/docs/api/java/io/IOException.html#IOException(java.lang.String,%20java.lang.Throwable)] I made the same mistake in PIG-80 ;) I made my local eclipse use a 1.5 jdk now to avoid this kind of mistakes... See [pig-user mailing list|http://www.mail-archive.com/pig-user%40incubator.apache.org/msg00052.html] and [Getting started wiki page|http://wiki.apache.org/pig/GettingStarted] for details. > Abstraction Layer to decouple Pig from Back-End > ----------------------------------------------- > > Key: PIG-32 > URL: https://issues.apache.org/jira/browse/PIG-32 > Project: Pig > Issue Type: New Feature > Components: impl > Reporter: Antonio Magnaghi > Assignee: Antonio Magnaghi > Attachments: 2008.01.29.patch, DataStorage.diff, DataStorage20071212.diff, patch.2008.01.16.merge_w_trunk, patch.2008.01.23.diff, PATCH.2008.01.31, patch2007_12_26.diff, patch2007_12_26_II.diff, patch2007_12_27.diff, pig.jar, pig.jar.2008.01.16, TEST.LOG, TEST.LOGS > > > I'm opening a new issue to track the development work to support an abstraction layer for Pig as defined at http://wiki.apache.org/pig/PigAbstractionLayer -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pig-dev-heatherfrank@example.org Fri Feb 01 16:24:33 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 49761 invoked from network); 1 Feb 2008 16:24:33 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Feb 2008 16:24:33 -0000 Received: (qmail 88747 invoked by uid 500); 1 Feb 2008 16:23:38 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 88708 invoked by uid 500); 1 Feb 2008 16:23:38 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 88696 invoked by uid 99); 1 Feb 2008 16:23:38 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 08:23:38 -0800 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO mrout3.yahoo.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 16:23:20 +0000 Received: from BUR-EXBH01.ds.corp.yahoo.com (bur-exbh01.ds.corp.yahoo.com [83.161.120.155]) by mrout3.yahoo.com (8.13.6/8.13.6/y.out) with ESMTP id m11GMhLW026593 for <heatherfrank@example.org>; Fri, 1 Feb 2008 08:22:43 -0800 (PST) DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns; h=received:x-mimeole:content-class:mime-version: content-type:content-transfer-encoding:subject:date:message-id: in-reply-to:x-ms-has-attach:x-ms-tnef-correlator:thread-topic: thread-index:references:from:to:return-path:x-originalarrivaltime; b=DjWDScKQuOBX0l+uJQgDXn/JxJuKLiwDfyRWM4BbOhvYq7zbxjx+c8uvs6YVCmd0 Received: from BUR-EXVS02.ds.corp.yahoo.com ([83.161.120.155]) by BUR-EXBH01.ds.corp.yahoo.com with Microsoft SMTPSVC442-873-5642); Fri, 1 Feb 2008 08:22:42 -0800 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [jira] Commented: (PIG-32) Abstraction Layer to decouple Pig from Back-End Date: Fri, 1 Feb 2008 08:22:42 -0800 Message-ID: <heatherfrank@example.org> In-Reply-To: <21225698.1201864508476.JavaMail.jira@brutus> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [jira] Commented: (PIG-32) Abstraction Layer to decouple Pig from Back-End Thread-Index: Achkw9y47kIwAP7PT2OIs3VNY+YG6gAKpoSw References: <7651636.1196198743011.JavaMail.jira@brutus> <21225698.1201864508476.JavaMail.jira@brutus> From: "Antonio Magnaghi" <heatherfrank@example.org> To: <heatherfrank@example.org> X-OriginalArrivalTime: 01 Feb 2008 16:22:42.0837 (UTC) FILETIME=[ABDBEC50:01C864EE] X-Virus-Checked: Checked by ClamAV on apache.org Will prepare a patch for it. Thanks, Ben, for finding this problem. -a. -----Original Message----- From: Benjamin Francisoud (JIRA) [mailto:heatherfrank@example.org]=20 Sent: Friday, February 01, 2008 3:15 AM To: heatherfrank@example.org Subject: [jira] Commented: (PIG-32) Abstraction Layer to decouple Pig from Back-End [ https://issues.apache.org/jira/browse/PIG-32?page=3Dcom.atlassian.jira.pl= u gin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D12564725#ac= t ion_12564725 ]=20 Benjamin Francisoud commented on PIG-32: ---------------------------------------- I think this commit broke compatibility with java 5 :( For example in PigServer line 281: {code:java} catch (ExecException e) { throw new IOException("Unable to open iterator for alias: " + id, e); } {code} This constructor is only available in java 6: * [java 5 IOException api|http://java.sun.com/j2se/1.5.0/docs/api/java/io/IOException.html#IOE xception()] * [java 6 IOException api|http://java.sun.com/javase/6/docs/api/java/io/IOException.html#IOExc eption(java.lang.String,%20java.lang.Throwable)] I made the same mistake in PIG-80 ;)=20 I made my local eclipse use a 1.5 jdk now to avoid this kind of mistakes... See [pig-user mailing list|http://www.mail-archive.com/pig-user%40incubator.apache.org/msg0005 2.html] and [Getting started wiki page|http://wiki.apache.org/pig/GettingStarted] for details. > Abstraction Layer to decouple Pig from Back-End > ----------------------------------------------- > > Key: PIG-32 > URL: https://issues.apache.org/jira/browse/PIG-32 > Project: Pig > Issue Type: New Feature > Components: impl > Reporter: Antonio Magnaghi > Assignee: Antonio Magnaghi > Attachments: 2008.01.29.patch, DataStorage.diff, DataStorage20071212.diff, patch.2008.01.16.merge_w_trunk, patch.2008.01.23.diff, PATCH.2008.01.31, patch2007_12_26.diff, patch2007_12_26_II.diff, patch2007_12_27.diff, pig.jar, pig.jar.2008.01.16, TEST.LOG, TEST.LOGS > > > I'm opening a new issue to track the development work to support an abstraction layer for Pig as defined at http://wiki.apache.org/pig/PigAbstractionLayer --=20 This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pig-dev-heatherfrank@example.org Fri Feb 01 18:45:28 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 11454 invoked from network); 1 Feb 2008 18:45:28 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Feb 2008 18:45:28 -0000 Received: (qmail 39918 invoked by uid 500); 1 Feb 2008 18:45:20 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 39886 invoked by uid 500); 1 Feb 2008 18:45:20 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 39875 invoked by uid 99); 1 Feb 2008 18:45:20 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 10:45:20 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 18:45:00 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id CF201714044 for <heatherfrank@example.org>; Fri, 1 Feb 2008 10:45:07 -0800 (PST) Message-ID: <17103555.1201891507844.JavaMail.jira@brutus> Date: Fri, 1 Feb 2008 10:45:07 -0800 (PST) From: "Stefan Groschupf (JIRA)" <heatherfrank@example.org> To: heatherfrank@example.org Subject: [jira] Created: (PIG-86) load and store function should be file independent MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org load and store function should be file independent -------------------------------------------------- Key: PIG-86 URL: https://issues.apache.org/jira/browse/PIG-86 Project: Pig Issue Type: Improvement Reporter: Stefan Groschupf Priority: Critical Currently pig is very file system centric for input LoadFunc and output StoreFunc. However in many buzz usecases the data are not in the dfs directly but come from a remote service. Also it is common to store the results into a database for a webbased ui. There for I suggest to make filepath elements in the gramar optional. For example: A = load 'mydata' using PigStorage() // " 'myData' " would be optional STORE A into 'output.bz2' USING myStore(); // " into 'output.bz2' " would be optional. Current workaround for implementing Load and Store functions that work without file system access is to create temporary dummy files since pig is checking for this files, even if the load and store functions do not need those file. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pig-dev-heatherfrank@example.org Fri Feb 01 18:58:49 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 17325 invoked from network); 1 Feb 2008 18:58:49 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Feb 2008 18:58:49 -0000 Received: (qmail 68857 invoked by uid 500); 1 Feb 2008 18:58:40 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 68833 invoked by uid 500); 1 Feb 2008 18:58:40 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 68824 invoked by uid 99); 1 Feb 2008 18:58:40 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 10:58:40 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO mslinux.101tec.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 18:58:10 +0000 Received: from localhost (localhost [83.161.120.155]) by mslinux.101tec.com (Postfix) with ESMTP id 424D8294105 for <heatherfrank@example.org>; Fri, 1 Feb 2008 19:58:13 +0100 (CET) X-Virus-Scanned: by amavisd-new-2.4.2 (20060627) at 101tec.com Received: from [83.161.120.155] (71-6-42-130.static-ip.telepacific.net [83.161.120.155]) by mslinux.101tec.com (Postfix) with ESMTP id 8FD8029411F for <heatherfrank@example.org>; Fri, 1 Feb 2008 19:58:00 +0100 (CET) Message-Id: <heatherfrank@example.org> From: Stefan Groschupf <heatherfrank@example.org> To: heatherfrank@example.org Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v915) Subject: ExecException extends Throwable Date: Fri, 1 Feb 2008 10:57:58 -0800 X-Mailer: Apple Mail (2.915) X-Virus-Checked: Checked by ClamAV on apache.org Hi there, what is the reason behind ExecException extends Throwable and not Exception? Thanks for clarification. Cheers, Stefan ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 101tec Inc. Menlo Park, California, USA http://www.101tec.com From pig-dev-heatherfrank@example.org Fri Feb 01 19:49:15 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 35470 invoked from network); 1 Feb 2008 19:49:14 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Feb 2008 19:49:14 -0000 Received: (qmail 49111 invoked by uid 500); 1 Feb 2008 19:49:06 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 49080 invoked by uid 500); 1 Feb 2008 19:49:06 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 49071 invoked by uid 99); 1 Feb 2008 19:49:06 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 11:49:06 -0800 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO mrout2.yahoo.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 19:48:48 +0000 Received: from BUR-EXPF01.ds.corp.yahoo.com (bur-expf01.ds.corp.yahoo.com [83.161.120.155]) by mrout2.yahoo.com (8.13.6/8.13.6/y.out) with ESMTP id m11JlFFJ008220 for <heatherfrank@example.org>; Fri, 1 Feb 2008 11:47:15 -0800 (PST) DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns; h=received:x-mimeole:content-class:mime-version: content-type:content-transfer-encoding:subject:date:message-id: in-reply-to:x-ms-has-attach:x-ms-tnef-correlator:thread-topic: thread-index:references:from:to:return-path:x-originalarrivaltime; b=SwBrON3hvB243IAErygQ0b7i818saiGe0k1XatS1CfU9cC5QxKkIQN8g5pIfoWJr Received: from BUR-EXVS02.ds.corp.yahoo.com ([83.161.120.155]) by BUR-EXPF01.ds.corp.yahoo.com with Microsoft SMTPSVC442-873-5642); Fri, 1 Feb 2008 11:47:15 -0800 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: ExecException extends Throwable Date: Fri, 1 Feb 2008 11:47:14 -0800 Message-ID: <heatherfrank@example.org> In-Reply-To: <heatherfrank@example.org> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: ExecException extends Throwable Thread-Index: AchlBKgDGhcMbI0mTX6MaH7wNVkl6wABlCvw References: <heatherfrank@example.org> From: "Antonio Magnaghi" <heatherfrank@example.org> To: <heatherfrank@example.org> X-OriginalArrivalTime: 01 Feb 2008 19:47:15.0438 (UTC) FILETIME=[3EE638E0:01C8650B] X-Virus-Checked: Checked by ClamAV on apache.org I think, actually both DataStorageExceptiona and ExecException should be sub classes of Exception. -a. -----Original Message----- From: Stefan Groschupf [mailto:heatherfrank@example.org]=20 Sent: Friday, February 01, 2008 10:58 AM To: heatherfrank@example.org Subject: ExecException extends Throwable Hi there, what is the reason behind ExecException extends Throwable and not =20 Exception? Thanks for clarification. Cheers, Stefan ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 101tec Inc. Menlo Park, California, USA http://www.101tec.com From pig-dev-heatherfrank@example.org Fri Feb 01 19:55:37 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 38550 invoked from network); 1 Feb 2008 19:55:37 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Feb 2008 19:55:37 -0000 Received: (qmail 54418 invoked by uid 500); 1 Feb 2008 19:55:28 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 54279 invoked by uid 500); 1 Feb 2008 19:55:28 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 54147 invoked by uid 99); 1 Feb 2008 19:55:27 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 11:55:27 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 19:55:08 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 8D3C4714064 for <heatherfrank@example.org>; Fri, 1 Feb 2008 11:55:15 -0800 (PST) Message-ID: <11520213.1201895715575.JavaMail.jira@brutus> Date: Fri, 1 Feb 2008 11:55:15 -0800 (PST) From: "Antonio Magnaghi (JIRA)" <heatherfrank@example.org> To: heatherfrank@example.org Subject: [jira] Updated: (PIG-32) Abstraction Layer to decouple Pig from Back-End In-Reply-To: <7651636.1196198743011.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/PIG-32?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antonio Magnaghi updated PIG-32: -------------------------------- Attachment: PIG-32.jdk1.5.patch attaching patch: - to make the changes in the abstraction layer backword compatible with jdk1.5 - DataStorageException and ExecException subclass Exception as initially intended > Abstraction Layer to decouple Pig from Back-End > ----------------------------------------------- > > Key: PIG-32 > URL: https://issues.apache.org/jira/browse/PIG-32 > Project: Pig > Issue Type: New Feature > Components: impl > Reporter: Antonio Magnaghi > Assignee: Antonio Magnaghi > Attachments: 2008.01.29.patch, DataStorage.diff, DataStorage20071212.diff, patch.2008.01.16.merge_w_trunk, patch.2008.01.23.diff, PATCH.2008.01.31, patch2007_12_26.diff, patch2007_12_26_II.diff, patch2007_12_27.diff, PIG-32.jdk1.5.patch, pig.jar, pig.jar.2008.01.16, TEST.LOG, TEST.LOGS > > > I'm opening a new issue to track the development work to support an abstraction layer for Pig as defined at http://wiki.apache.org/pig/PigAbstractionLayer -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pig-dev-heatherfrank@example.org Fri Feb 01 21:25:40 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 80442 invoked from network); 1 Feb 2008 21:25:40 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Feb 2008 21:25:40 -0000 Received: (qmail 43320 invoked by uid 500); 1 Feb 2008 21:25:31 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 43294 invoked by uid 500); 1 Feb 2008 21:25:31 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 43285 invoked by uid 99); 1 Feb 2008 21:25:31 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 13:25:31 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 21:25:16 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 46A66714035 for <heatherfrank@example.org>; Fri, 1 Feb 2008 13:25:08 -0800 (PST) Message-ID: <11459279.1201901108276.JavaMail.jira@brutus> Date: Fri, 1 Feb 2008 13:25:08 -0800 (PST) From: "Craig Macdonald (JIRA)" <heatherfrank@example.org> To: heatherfrank@example.org Subject: [jira] Created: (PIG-87) Improvements to pig.pl: make pigclient.conf optional; check JAVA_HOME MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org Improvements to pig.pl: make pigclient.conf optional; check JAVA_HOME ---------------------------------------------------------------------- Key: PIG-87 URL: https://issues.apache.org/jira/browse/PIG-87 Project: Pig Issue Type: Improvement Reporter: Craig Macdonald Brief notes about the pig.pl, and a patch to resolve some of these 1. Is conf/pigclient.conf really required? pig.pl dies straight away if $ROOT/conf/pigclient.conf does not exist. This is a shame, for a couple of reasons: * the only really necessary detail in pigclient.conf is $pigJarRoot. * $pigJarRoot, $hodRoot and $defaultCluster can be set using pigclient.conf - why cant they also be: (a) worked out from defaults ? - eg $pigJarRoot my $JAR = $0;#/scripts/pig.pl $JAR =~ s/pig\.pl/..\/pig.jar/ (b) $hodRoot - seem an obvious example to be configurable using the command line arguments? (c) $defaultCluster - ditto? * if conf/pigclient.conf doesnt exist, pig.pl dies before the --help options can displayed (big shame) -> means that scripts/pig.pl -h doesnt work out the box as well as most of http://wiki.apache.org/pig/GettingStarted * As far as I can see minimum setup for a new Pig user: cd pig (ant) mkdir conf echo "\$pigJarRoot = \"$PWD\"" > conf/pigclient.conf mkdir -p libexec/pig//released/ cp pig.jar libexec/pig//released/ ROOT=$PWD scripts/pig.pl or specify the class path manually. 2. Java binary is looked for in a special Yahoo place or in PATH, but JAVA_HOME is not checked, as per other common startup scripts (eg Tomcat). 3. looking for java in the path `which java 2>&1 > /dev/null`; if ($? != 0) { I cant help thinking that this would be better/quicker in Perl: sub inpath { my $bin = shift; foreach my $dir (split /:/, $ENV{PATH}) { return 1 if -e $dir/$bin; } return 0; } If this is deemed desirable, I can update the patch for this sub-issue too. Please find attached a patch for pig.pl that resolves issues 1 & 2. This will allow the GettingStarted documentation to perform as expected without all the rigmarole associated with pigclient.conf -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pig-dev-heatherfrank@example.org Fri Feb 01 21:37:42 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 86888 invoked from network); 1 Feb 2008 21:37:41 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Feb 2008 21:37:41 -0000 Received: (qmail 56875 invoked by uid 500); 1 Feb 2008 21:37:33 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 56843 invoked by uid 500); 1 Feb 2008 21:37:33 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 56834 invoked by uid 99); 1 Feb 2008 21:37:33 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 13:37:33 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 21:37:03 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 096E2714058 for <heatherfrank@example.org>; Fri, 1 Feb 2008 13:37:11 -0800 (PST) Message-ID: <11198791.1201901831031.JavaMail.jira@brutus> Date: Fri, 1 Feb 2008 13:37:11 -0800 (PST) From: "Craig Macdonald (JIRA)" <heatherfrank@example.org> To: heatherfrank@example.org Subject: [jira] Updated: (PIG-87) Improvements to pig.pl: make pigclient.conf optional; check JAVA_HOME In-Reply-To: <11459279.1201901108276.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/PIG-87?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Macdonald updated PIG-87: ------------------------------- Attachment: pig.pl.patch Patch attached. Additional perl module used: Fil888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4Basename, included in all modern Perls (since at least 5.00404). > Improvements to pig.pl: make pigclient.conf optional; check JAVA_HOME > ---------------------------------------------------------------------- > > Key: PIG-87 > URL: https://issues.apache.org/jira/browse/PIG-87 > Project: Pig > Issue Type: Improvement > Reporter: Craig Macdonald > Attachments: pig.pl.patch > > > Brief notes about the pig.pl, and a patch to resolve some of these > 1. Is conf/pigclient.conf really required? > pig.pl dies straight away if $ROOT/conf/pigclient.conf does not exist. > This is a shame, for a couple of reasons: > * the only really necessary detail in pigclient.conf is $pigJarRoot. > * $pigJarRoot, $hodRoot and $defaultCluster can be set using > pigclient.conf - why cant they also be: > (a) worked out from defaults ? - eg $pigJarRoot > my $JAR = $0;#/scripts/pig.pl > $JAR =~ s/pig\.pl/..\/pig.jar/ > (b) $hodRoot - seem an obvious example to be configurable using the > command line arguments? > (c) $defaultCluster - ditto? > * if conf/pigclient.conf doesnt exist, pig.pl dies before the --help options can displayed (big shame) > -> means that scripts/pig.pl -h doesnt work out the box as well as most of > http://wiki.apache.org/pig/GettingStarted > * As far as I can see minimum setup for a new Pig user: > cd pig > (ant) > mkdir conf > echo "\$pigJarRoot = \"$PWD\"" > conf/pigclient.conf > mkdir -p libexec/pig//released/ > cp pig.jar libexec/pig//released/ > ROOT=$PWD scripts/pig.pl > or specify the class path manually. > 2. Java binary is looked for in a special Yahoo place or in PATH, but > JAVA_HOME is not checked, as per other common startup scripts (eg Tomcat). > 3. looking for java in the path > `which java 2>&1 > /dev/null`; > if ($? != 0) { > I cant help thinking that this would be better/quicker in Perl: > sub inpath > { > my $bin = shift; > foreach my $dir (split /:/, $ENV{PATH}) > { > return 1 if -e $dir/$bin; > } > return 0; > } > If this is deemed desirable, I can update the patch for this sub-issue too. > Please find attached a patch for pig.pl that resolves issues 1 & 2. This > will allow the GettingStarted documentation to perform as expected > without all the rigmarole associated with pigclient.conf -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pig-dev-heatherfrank@example.org Sat Feb 02 00:21:31 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 7416 invoked from network); 2 Feb 2008 00:21:31 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Feb 2008 00:21:31 -0000 Received: (qmail 64464 invoked by uid 500); 2 Feb 2008 00:21:21 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 64380 invoked by uid 500); 2 Feb 2008 00:21:21 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 64339 invoked by uid 99); 2 Feb 2008 00:21:21 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2008 16:21:21 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 Feb 2008 00:21:01 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id C034C71404A for <heatherfrank@example.org>; Fri, 1 Feb 2008 16:21:08 -0800 (PST) Message-ID: <1608983.1201911668783.JavaMail.jira@brutus> Date: Fri, 1 Feb 2008 16:21:08 -0800 (PST) From: "Olga Natkovich (JIRA)" <heatherfrank@example.org> To: heatherfrank@example.org Subject: [jira] Commented: (PIG-32) Abstraction Layer to decouple Pig from Back-End In-Reply-To: <7651636.1196198743011.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/PIG-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12564987#action_12564987 ] Olga Natkovich commented on PIG-32: ----------------------------------- Antonio, For IOException, you want to use the other constructor and setup the causing exception via initCause. > Abstraction Layer to decouple Pig from Back-End > ----------------------------------------------- > > Key: PIG-32 > URL: https://issues.apache.org/jira/browse/PIG-32 > Project: Pig > Issue Type: New Feature > Components: impl > Reporter: Antonio Magnaghi > Assignee: Antonio Magnaghi > Attachments: 2008.01.29.patch, DataStorage.diff, DataStorage20071212.diff, patch.2008.01.16.merge_w_trunk, patch.2008.01.23.diff, PATCH.2008.01.31, patch2007_12_26.diff, patch2007_12_26_II.diff, patch2007_12_27.diff, PIG-32.jdk1.5.patch, pig.jar, pig.jar.2008.01.16, TEST.LOG, TEST.LOGS > > > I'm opening a new issue to track the development work to support an abstraction layer for Pig as defined at http://wiki.apache.org/pig/PigAbstractionLayer -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pig-dev-heatherfrank@example.org Sat Feb 02 08:23:28 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 88561 invoked from network); 2 Feb 2008 08:23:28 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Feb 2008 08:23:28 -0000 Received: (qmail 68402 invoked by uid 500); 2 Feb 2008 08:23:20 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 68375 invoked by uid 500); 2 Feb 2008 08:23:20 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 68354 invoked by uid 99); 2 Feb 2008 08:23:20 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 Feb 2008 00:23:20 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 Feb 2008 08:23:00 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id AF0BD714047 for <heatherfrank@example.org>; Sat, 2 Feb 2008 00:23:07 -0800 (PST) Message-ID: <10347833.1201940587699.JavaMail.jira@brutus> Date: Sat, 2 Feb 2008 00:23:07 -0800 (PST) From: "Pi Song (JIRA)" <heatherfrank@example.org> To: heatherfrank@example.org Subject: [jira] Created: (PIG-88) the project does not compile because of reference to HadoopExe class in Main.java MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org the project does not compile because of reference to HadoopExe class in Main.java --------------------------------------------------------------------------------- Key: PIG-88 URL: https://issues.apache.org/jira/browse/PIG-88 Project: Pig Issue Type: Bug Affects Versions: 0.1.0 Environment: Win XP Reporter: Pi Song Priority: Minor The project does not compile because of this line in Main.java import org.apache.hadoop.util.HadoopExe; >From HADOOP-435, the patch to introduce this class has been canceled plus the class itself is not being used at all in Main.java. The simple patch removes that particular line and now the project compiles successfully. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pig-dev-heatherfrank@example.org Sat Feb 02 08:43:29 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 94085 invoked from network); 2 Feb 2008 08:43:29 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Feb 2008 08:43:29 -0000 Received: (qmail 73395 invoked by uid 500); 2 Feb 2008 08:43:20 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 73365 invoked by uid 500); 2 Feb 2008 08:43:20 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 73356 invoked by uid 99); 2 Feb 2008 08:43:20 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 Feb 2008 00:43:20 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 Feb 2008 08:43:00 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 07D8B714042 for <heatherfrank@example.org>; Sat, 2 Feb 2008 00:43:08 -0800 (PST) Message-ID: <19642297.1201941788020.JavaMail.jira@brutus> Date: Sat, 2 Feb 2008 00:43:08 -0800 (PST) From: "Pi Song (JIRA)" <heatherfrank@example.org> To: heatherfrank@example.org Subject: [jira] Updated: (PIG-88) the project does not compile because of reference to HadoopExe class in Main.java In-Reply-To: <10347833.1201940587699.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/PIG-88?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pi Song updated PIG-88: ----------------------- Attachment: PIG-88.HadoopExe.patch A very simple patch to remove reference to HadoopExe > the project does not compile because of reference to HadoopExe class in Main.java > --------------------------------------------------------------------------------- > > Key: PIG-88 > URL: https://issues.apache.org/jira/browse/PIG-88 > Project: Pig > Issue Type: Bug > Affects Versions: 0.1.0 > Environment: Win XP > Reporter: Pi Song > Priority: Minor > Attachments: PIG-88.HadoopExe.patch > > Original Estimate: 0.08h > Remaining Estimate: 0.08h > > The project does not compile because of this line in Main.java > import org.apache.hadoop.util.HadoopExe; > From HADOOP-435, the patch to introduce this class has been canceled plus the class itself is not being used at all in Main.java. > The simple patch removes that particular line and now the project compiles successfully. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pig-dev-heatherfrank@example.org Sat Feb 02 11:26:31 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 36134 invoked from network); 2 Feb 2008 11:26:31 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Feb 2008 11:26:31 -0000 Received: (qmail 46065 invoked by uid 500); 2 Feb 2008 11:26:23 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 46001 invoked by uid 500); 2 Feb 2008 11:26:22 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 45992 invoked by uid 99); 2 Feb 2008 11:26:22 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 Feb 2008 03:26:22 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 Feb 2008 11:26:15 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 319EB714045 for <heatherfrank@example.org>; Sat, 2 Feb 2008 03:26:08 -0800 (PST) Message-ID: <26651299.1201951568177.JavaMail.jira@brutus> Date: Sat, 2 Feb 2008 03:26:08 -0800 (PST) From: "Pi Song (JIRA)" <heatherfrank@example.org> To: heatherfrank@example.org Subject: [jira] Commented: (PIG-83) logging abstraction In-Reply-To: <29248600.1201735774410.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/PIG-83?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12565054#action_12565054 ] Pi Song commented on PIG-83: ---------------------------- I also vote for commons-logging. > logging abstraction > ------------------- > > Key: PIG-83 > URL: https://issues.apache.org/jira/browse/PIG-83 > Project: Pig > Issue Type: Wish > Reporter: Stefan Groschupf > Attachments: log4j.properties, PIG-83-v01.patch > > > Pig is logging quite a lot into System.out or System.err. Using a embedded pig in a production environment requires a logging abstraction like log4j, commons logging, slf4j or something like that. > I would be happy to work on a patch if we decide what would be the best choice. Hadoop uses log4j. > Thanks. > Stefan -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pig-dev-heatherfrank@example.org Sat Feb 02 11:42:29 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 39678 invoked from network); 2 Feb 2008 11:42:29 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Feb 2008 11:42:29 -0000 Received: (qmail 49534 invoked by uid 500); 2 Feb 2008 11:42:20 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 49497 invoked by uid 500); 2 Feb 2008 11:42:20 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 49488 invoked by uid 99); 2 Feb 2008 11:42:20 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 Feb 2008 03:42:20 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 Feb 2008 11:42:01 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 87B5E714045 for <heatherfrank@example.org>; Sat, 2 Feb 2008 03:42:08 -0800 (PST) Message-ID: <26527106.1201952528543.JavaMail.jira@brutus> Date: Sat, 2 Feb 2008 03:42:08 -0800 (PST) From: "Pi Song (JIRA)" <heatherfrank@example.org> To: heatherfrank@example.org Subject: [jira] Issue Comment Edited: (PIG-83) logging abstraction In-Reply-To: <29248600.1201735774410.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/PIG-83?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12565054#action_12565054 ] pi_song edited comment on PIG-83 at 2/2/08 3:40 AM: ---------------------------------------------------- As a user, I vote for commons-logging/log4j. Different systems have different requirements. You should keep it generic. By adding dependencies on external logging framework which is used by almost everyone, I don't see that's an issue. It's a very common case that when someone is trying to turn on logging for an app, the first thing he'll do (before going for the app's user manual) is looking for log4j.xml!!! was (Author: pi_song): I also vote for commons-logging. > logging abstraction > ------------------- > > Key: PIG-83 > URL: https://issues.apache.org/jira/browse/PIG-83 > Project: Pig > Issue Type: Wish > Reporter: Stefan Groschupf > Attachments: log4j.properties, PIG-83-v01.patch > > > Pig is logging quite a lot into System.out or System.err. Using a embedded pig in a production environment requires a logging abstraction like log4j, commons logging, slf4j or something like that. > I would be happy to work on a patch if we decide what would be the best choice. Hadoop uses log4j. > Thanks. > Stefan -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pig-dev-heatherfrank@example.org Mon Feb 04 10:53:32 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received: (qmail 60074 invoked from network); 4 Feb 2008 10:53:32 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 4 Feb 2008 10:53:32 -0000 Received: (qmail 3379 invoked by uid 500); 4 Feb 2008 10:53:24 -0000 Delivered-To: heatherfrank@example.org Received: (qmail 3310 invoked by uid 500); 4 Feb 2008 10:53:23 -0000 Mailing-List: contact heatherfrank@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:heatherfrank@example.org> List-Unsubscribe: <mailto:heatherfrank@example.org> List-Post: <mailto:heatherfrank@example.org> List-Id: <pig-dev.incubator.apache.org> Reply-To: heatherfrank@example.org Delivered-To: mailing list heatherfrank@example.org Received: (qmail 3301 invoked by uid 99); 4 Feb 2008 10:53:23 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 Feb 2008 02:53:23 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 Feb 2008 10:53:16 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id A3AA071406F for <heatherfrank@example.org>; Mon, 4 Feb 2008 02:53:08 -0800 (PST) Message-ID: <26612872.1202122388667.JavaMail.jira@brutus> Date: Mon, 4 Feb 2008 02:53:08 -0800 (PST) From: "Craig Macdonald (JIRA)" <heatherfrank@example.org> To: heatherfrank@example.org Subject: [jira] Commented: (PIG-32) Abstraction Layer to decouple Pig from Back-End In-Reply-To: <7651636.1196198743011.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/PIG-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12565303#action_12565303 ] Craig Macdonald commented on PIG-32: ------------------------------------ Hello, the PIG-32 commit (r617338) broke local execution: {{{ java.lang.ClassCastException: org.apache.pig.backend.local.executionengine.LocalExecutionEngine cannot be cast to heatherfrank@example.org(GruntParser.java:93) at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:42) at org.apache.pig.Main.main(Main.java:245) }}} The relevant code from GruntParser is: {{{ // TODO: this violates the abstraction layer decoupling between // front end and back end and needs to be changed. // Right now I am not clear on how the Job Id comes from to tell // the back end to kill a given job (mJobClient is used only in // processKill) // mJobClient = ((HExecutionEngine)(mPigServer.getPigContext().getExecutionEngine())).getJobClient(); }}} Thanks C > Abstraction Layer to decouple Pig from Back-End > ----------------------------------------------- > > Key: PIG-32 > URL: https://issues.apache.org/jira/browse/PIG-32 > Project: Pig > Issue Type: New Feature > Components: impl > Reporter: Antonio Magnaghi > Assignee: Antonio Magnaghi > Attachments: 2008.01.29.patch, DataStorage.diff, DataStorage20071212.diff, patch.2008.01.16.merge_w_trunk, patch.2008.01.23.diff, PATCH.2008.01.31, patch2007_12_26.diff, patch2007_12_26_II.diff, patch2007_12_27.diff, PIG-32.jdk1.5.patch, pig.jar, pig.jar.2008.01.16, TEST.LOG, TEST.LOGS > > > I'm opening a new issue to track the development work to support an abstraction layer for Pig as defined at http://wiki.apache.org/pig/PigAbstractionLayer -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pig-dev-heatherfrank@example.org Mon Feb 04 15:01:32 2008 Return-Path: <pig-dev-heatherfrank@example.org> Delivered-To: heatherfrank@example.org Received:
{'source': 'Apache Software Foundation Public Mail Archives'}