From common-issues-returneric91@example.com Mon Aug 02 05:20:46 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 69853 invoked from network); 2 Aug 2010 05:20:46 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 05:20:46 -0000 Received: (qmail 10838 invoked by uid 500); 2 Aug 2010 05:20:45 -0000 Delivered-To: eric91@example.com Received: (qmail 9742 invoked by uid 500); 2 Aug 2010 05:20:42 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 9620 invoked by uid 99); 2 Aug 2010 05:20:41 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 05:20:41 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 05:20:39 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o725KHK3027142 for <eric91@example.com>; Mon, 2 Aug 2010 05:20:17 GMT Message-ID: <14318368.112951280726417561.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 01:20:17 -0400 (EDT) From: "Owen O'Malley (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Resolved: (HADOOP-4851) Change filesystem permissions to use JAAS rather than UGI MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-4851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HADOOP-4851. ----------------------------------- Resolution: Won't Fix I don't think this is a good idea. Having implemented on top of the JAAS stuff, the impedance mismatch is pretty annoying. Can I close this as won't fix? > Change filesystem permissions to use JAAS rather than UGI > --------------------------------------------------------- > > Key: HADOOP-4851 > URL: https://issues.apache.org/jira/browse/HADOOP-4851 > Project: Hadoop Common > Issue Type: Bug > Components: fs > Affects Versions: 0.20.0 > Reporter: Arun C Murthy > Assignee: Arun C Murthy > > Hadoop embraced JAAS via HADOOP-4348. > We need to fix HDFS to use JAAS concepts such as Subject, Principal, Permission etc. rather than UserGroupInformation for file-system permissions. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 05:28:42 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 71843 invoked from network); 2 Aug 2010 05:28:42 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 05:28:42 -0000 Received: (qmail 11737 invoked by uid 500); 2 Aug 2010 05:28:42 -0000 Delivered-To: eric91@example.com Received: (qmail 11672 invoked by uid 500); 2 Aug 2010 05:28:40 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 11655 invoked by uid 99); 2 Aug 2010 05:28:40 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 05:28:40 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 05:28:39 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o725SJxK027211 for <eric91@example.com>; Mon, 2 Aug 2010 05:28:19 GMT Message-ID: <12314272.113111280726899551.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 01:28:19 -0400 (EDT) From: "Owen O'Malley (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Created: (HADOOP-6893) FileContext needs a method to test for existence of a file or directory. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 FileContext needs a method to test for existence of a file or directory. ------------------------------------------------------------------------ Key: HADOOP-6893 URL: https://issues.apache.org/jira/browse/HADOOP-6893 Project: Hadoop Common Issue Type: New Feature Components: fs Reporter: Owen O'Malley FileContext is missing a method to ask if a file or directory exists. Currently the only mechanism is to use getFileStatus, which throws an exception if the file doesn't exist, which I don't think is an appropriate use of that method. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 05:38:44 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 73186 invoked from network); 2 Aug 2010 05:38:43 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 05:38:43 -0000 Received: (qmail 18577 invoked by uid 500); 2 Aug 2010 05:38:43 -0000 Delivered-To: eric91@example.com Received: (qmail 18441 invoked by uid 500); 2 Aug 2010 05:38:41 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 18433 invoked by uid 99); 2 Aug 2010 05:38:40 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 05:38:40 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 05:38:38 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o725cGeJ027247 for <eric91@example.com>; Mon, 2 Aug 2010 05:38:16 GMT Message-ID: <15708894.113141280727496179.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 01:38:16 -0400 (EDT) From: "Owen O'Malley (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Commented: (HADOOP-6893) FileContext needs a method to test for existence of a file or directory. In-Reply-To: <12314272.113111280726899551.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-6893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894465#action_12894465 ] Owen O'Malley commented on HADOOP-6893: --------------------------------------- I see now that there is a Util method that checks exist, but it still uses getFileStatus and catches the exception. There should be some way of asking an AbstractFileSystem whether a name exists that doesn't throw an exception. > FileContext needs a method to test for existence of a file or directory. > ------------------------------------------------------------------------ > > Key: HADOOP-6893 > URL: https://issues.apache.org/jira/browse/HADOOP-6893 > Project: Hadoop Common > Issue Type: New Feature > Components: fs > Reporter: Owen O'Malley > > FileContext is missing a method to ask if a file or directory exists. Currently the only mechanism is to use getFileStatus, which throws an exception if the file doesn't exist, which I don't think is an appropriate use of that method. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 05:41:40 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 73295 invoked from network); 2 Aug 2010 05:41:40 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 05:41:40 -0000 Received: (qmail 18828 invoked by uid 500); 2 Aug 2010 05:41:40 -0000 Delivered-To: eric91@example.com Received: (qmail 18701 invoked by uid 500); 2 Aug 2010 05:41:38 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 18693 invoked by uid 99); 2 Aug 2010 05:41:38 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 05:41:38 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 05:41:37 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o725fH2j027273 for <eric91@example.com>; Mon, 2 Aug 2010 05:41:17 GMT Message-ID: <10064811.113191280727677005.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 01:41:17 -0400 (EDT) From: "Amareshwari Sriramadasu (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Updated: (HADOOP-5759) IllegalArgumentException when CombineFileInputFormat is used as job InputFormat MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-5759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amareshwari Sriramadasu updated HADOOP-5759: -------------------------------------------- Attachment: patch-5759-ydist.txt Patch for Yahoo! distribution > IllegalArgumentException when CombineFileInputFormat is used as job InputFormat > ------------------------------------------------------------------------------- > > Key: HADOOP-5759 > URL: https://issues.apache.org/jira/browse/HADOOP-5759 > Project: Hadoop Common > Issue Type: Bug > Affects Versions: 0.20.1 > Reporter: Amareshwari Sriramadasu > Assignee: Amareshwari Sriramadasu > Fix For: 0.20.2 > > Attachments: patch-5759-1.txt, patch-5759-2.txt, patch-5759-ydist.txt, patch-5759.txt > > > As per my understanding, CombineFileInputFormat is creating splits with rackname as split location. > When I use CombineFileInputFormat as the InputFormat for job, job initialization fails with following exception : > 2009-04-28 14:10:40,162 ERROR mapred.EagerTaskInitializationListener (EagerTaskInitializationListener.java:run(83)) - Job initialization failed: > java.lang.IllegalArgumentException: Network location name contains /: /default-rack > at org.apache.hadoop.net.NodeBase.set(NodeBase.java:76) > at org.apache.hadoop.net.NodeBase.<init>(NodeBase.java:57) > at org.apache.hadoop.mapred.JobTracker.addHostToNodeMapping(JobTracker.java:2342) > at org.apache.hadoop.mapred.JobTracker.resolveAndAddToTopology(JobTracker.java:2336) > at org.apache.hadoop.mapred.JobInProgress.createCache(JobInProgress.java:344) > at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:441) > at org.apache.hadoop.mapred.EagerTaskInitializationListener$InitJob.run(EagerTaskInitializationListener.java:81) > at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) > at java.lang.Thread.run(Thread.java:619) > When I changed CombineFileInputFormat to pass just rackname (without '/'), JT wrongly resolves the node as /default-rack/<rack-name>. > Solution is to pass hostnames holding the block(on the rack), instead of rackname. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 07:24:41 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 6337 invoked from network); 2 Aug 2010 07:24:41 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 07:24:41 -0000 Received: (qmail 65705 invoked by uid 500); 2 Aug 2010 07:24:41 -0000 Delivered-To: eric91@example.com Received: (qmail 65573 invoked by uid 500); 2 Aug 2010 07:24:38 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 65565 invoked by uid 99); 2 Aug 2010 07:24:37 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 07:24:37 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 07:24:37 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o727OGgS027953 for <eric91@example.com>; Mon, 2 Aug 2010 07:24:16 GMT Message-ID: <10134473.113661280733856346.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 03:24:16 -0400 (EDT) From: "Alejandro Abdelnur (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Created: (HADOOP-6894) Common foundation for Hadoop client tools MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Common foundation for Hadoop client tools ----------------------------------------- Key: HADOOP-6894 URL: https://issues.apache.org/jira/browse/HADOOP-6894 Project: Hadoop Common Issue Type: New Feature Reporter: Alejandro Abdelnur As Hadoop widespreads and matures the number of tools and utilities for users keeps growing. Some of them are bundled with Hadoop core, some with Hadoop contrib, some on their own, some are full fledged servers on their own. For example, just to name a few: distcp, streaming, pipes, har, pig, hive, oozie. Today there is no standard mechanism for making these tools available to users. Neither there is a standard mechanism for these tools to integrate and distributed them with each other. The lack of a common foundation creates issues for developers and users. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 07:27:40 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 7353 invoked from network); 2 Aug 2010 07:27:40 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 07:27:40 -0000 Received: (qmail 67567 invoked by uid 500); 2 Aug 2010 07:27:40 -0000 Delivered-To: eric91@example.com Received: (qmail 67248 invoked by uid 500); 2 Aug 2010 07:27:38 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 67240 invoked by uid 99); 2 Aug 2010 07:27:37 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 07:27:37 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 07:27:36 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o727RGsK027968 for <eric91@example.com>; Mon, 2 Aug 2010 07:27:16 GMT Message-ID: <23338976.113691280734036299.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 03:27:16 -0400 (EDT) From: "Alejandro Abdelnur (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Updated: (HADOOP-6894) Common foundation for Hadoop client tools In-Reply-To: <10134473.113661280733856346.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur updated HADOOP-6894: --------------------------------------- Attachment: HadoopClientTools.pdf This proposal defines simple and well understood guidelines for: * Development (optional) * Directory layout (mandatory) * Packaging (mandatory) * Platform packaging (optional) * Stack release (optional) * Execution (mandatory) The Hadoop hadoop command line tool is out of scope. Server tools (like start/stop Hadoop or Oozie) are out of scope. > Common foundation for Hadoop client tools > ----------------------------------------- > > Key: HADOOP-6894 > URL: https://issues.apache.org/jira/browse/HADOOP-6894 > Project: Hadoop Common > Issue Type: New Feature > Reporter: Alejandro Abdelnur > Attachments: HadoopClientTools.pdf > > > As Hadoop widespreads and matures the number of tools and utilities for users keeps growing. > Some of them are bundled with Hadoop core, some with Hadoop contrib, some on their own, some are full fledged servers on their own. For example, just to name a few: distcp, streaming, pipes, har, pig, hive, oozie. > Today there is no standard mechanism for making these tools available to users. Neither there is a standard mechanism for these tools to integrate and distributed them with each other. > The lack of a common foundation creates issues for developers and users. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 17:26:41 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 45964 invoked from network); 2 Aug 2010 17:26:41 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 17:26:41 -0000 Received: (qmail 96848 invoked by uid 500); 2 Aug 2010 17:26:41 -0000 Delivered-To: eric91@example.com Received: (qmail 96757 invoked by uid 500); 2 Aug 2010 17:26:40 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 96743 invoked by uid 99); 2 Aug 2010 17:26:39 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:26:39 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:26:39 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o72HQIdi002728 for <eric91@example.com>; Mon, 2 Aug 2010 17:26:19 GMT Message-ID: <26672875.119141280769978779.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 13:26:18 -0400 (EDT) From: "Hairong Kuang (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Updated: (HADOOP-6890) Improve listFiles API introduced by HADOOP-6870 In-Reply-To: <6227281.77781280442376526.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HADOOP-6890: ---------------------------------- Attachment: improveListFiles.patch > Improve listFiles API introduced by HADOOP-6870 > ----------------------------------------------- > > Key: HADOOP-6890 > URL: https://issues.apache.org/jira/browse/HADOOP-6890 > Project: Hadoop Common > Issue Type: Improvement > Components: fs > Affects Versions: 0.22.0 > Reporter: Hairong Kuang > Assignee: Hairong Kuang > Fix For: 0.22.0 > > Attachments: improveListFiles.patch, listFilesInFS.patch > > > This jira is mainly for addressing Suresh's review comments for HADOOP-6870: > 1. General comment: I have concerns about recursive listing. This could be abused by the applications, creating a lot of requests into HDFS. > 2. Any deletion of files/directories while reursing through directories results in RuntimeException and application has a partial result. Should we ignore if a directory was in stack and was not found later when iterating through it? > 3. FileSystem.java > * listFile() - method javadoc could be better organized - first write about if path is directory and two cases recursive=true and false. Then if path is file and two cases recursive=true or false. > * listFile() - document throwing RuntimeException, UnsupportedOperationException and the possible cause. IOException is no longer thrown. > 4. TestListFiles.java > * testDirectory() - comments test empty directory and test directory with 1 file should be moved up to relevant sections of the test. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 17:26:42 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 46016 invoked from network); 2 Aug 2010 17:26:41 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 17:26:41 -0000 Received: (qmail 96959 invoked by uid 500); 2 Aug 2010 17:26:41 -0000 Delivered-To: eric91@example.com Received: (qmail 96775 invoked by uid 500); 2 Aug 2010 17:26:41 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 96758 invoked by uid 99); 2 Aug 2010 17:26:40 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:26:40 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:26:39 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o72HQJP0002743 for <eric91@example.com>; Mon, 2 Aug 2010 17:26:19 GMT Message-ID: <26370499.119191280769979782.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 13:26:19 -0400 (EDT) From: "Hairong Kuang (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Updated: (HADOOP-6890) Improve listFiles API introduced by HADOOP-6870 In-Reply-To: <6227281.77781280442376526.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HADOOP-6890: ---------------------------------- Attachment: (was: improveListFiles.patch) > Improve listFiles API introduced by HADOOP-6870 > ----------------------------------------------- > > Key: HADOOP-6890 > URL: https://issues.apache.org/jira/browse/HADOOP-6890 > Project: Hadoop Common > Issue Type: Improvement > Components: fs > Affects Versions: 0.22.0 > Reporter: Hairong Kuang > Assignee: Hairong Kuang > Fix For: 0.22.0 > > Attachments: improveListFiles.patch, listFilesInFS.patch > > > This jira is mainly for addressing Suresh's review comments for HADOOP-6870: > 1. General comment: I have concerns about recursive listing. This could be abused by the applications, creating a lot of requests into HDFS. > 2. Any deletion of files/directories while reursing through directories results in RuntimeException and application has a partial result. Should we ignore if a directory was in stack and was not found later when iterating through it? > 3. FileSystem.java > * listFile() - method javadoc could be better organized - first write about if path is directory and two cases recursive=true and false. Then if path is file and two cases recursive=true or false. > * listFile() - document throwing RuntimeException, UnsupportedOperationException and the possible cause. IOException is no longer thrown. > 4. TestListFiles.java > * testDirectory() - comments test empty directory and test directory with 1 file should be moved up to relevant sections of the test. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 17:26:42 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 46031 invoked from network); 2 Aug 2010 17:26:41 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 17:26:41 -0000 Received: (qmail 97126 invoked by uid 500); 2 Aug 2010 17:26:41 -0000 Delivered-To: eric91@example.com Received: (qmail 97093 invoked by uid 500); 2 Aug 2010 17:26:41 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 96988 invoked by uid 99); 2 Aug 2010 17:26:41 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:26:41 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:26:41 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o72HQLZ5002773 for <eric91@example.com>; Mon, 2 Aug 2010 17:26:21 GMT Message-ID: <6656466.119291280769981054.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 13:26:21 -0400 (EDT) From: "Hairong Kuang (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Updated: (HADOOP-6890) Improve listFiles API introduced by HADOOP-6870 In-Reply-To: <6227281.77781280442376526.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HADOOP-6890: ---------------------------------- Attachment: improveListFiles.patch > Improve listFiles API introduced by HADOOP-6870 > ----------------------------------------------- > > Key: HADOOP-6890 > URL: https://issues.apache.org/jira/browse/HADOOP-6890 > Project: Hadoop Common > Issue Type: Improvement > Components: fs > Affects Versions: 0.22.0 > Reporter: Hairong Kuang > Assignee: Hairong Kuang > Fix For: 0.22.0 > > Attachments: improveListFiles.patch, listFilesInFS.patch > > > This jira is mainly for addressing Suresh's review comments for HADOOP-6870: > 1. General comment: I have concerns about recursive listing. This could be abused by the applications, creating a lot of requests into HDFS. > 2. Any deletion of files/directories while reursing through directories results in RuntimeException and application has a partial result. Should we ignore if a directory was in stack and was not found later when iterating through it? > 3. FileSystem.java > * listFile() - method javadoc could be better organized - first write about if path is directory and two cases recursive=true and false. Then if path is file and two cases recursive=true or false. > * listFile() - document throwing RuntimeException, UnsupportedOperationException and the possible cause. IOException is no longer thrown. > 4. TestListFiles.java > * testDirectory() - comments test empty directory and test directory with 1 file should be moved up to relevant sections of the test. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 17:26:43 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 46089 invoked from network); 2 Aug 2010 17:26:42 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 17:26:42 -0000 Received: (qmail 97293 invoked by uid 500); 2 Aug 2010 17:26:42 -0000 Delivered-To: eric91@example.com Received: (qmail 97261 invoked by uid 500); 2 Aug 2010 17:26:42 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 97253 invoked by uid 99); 2 Aug 2010 17:26:42 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:26:42 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:26:41 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o72HQLeK002788 for <eric91@example.com>; Mon, 2 Aug 2010 17:26:21 GMT Message-ID: <30841605.119341280769981723.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 13:26:21 -0400 (EDT) From: "Hairong Kuang (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Reopened: (HADOOP-6890) Improve listFiles API introduced by HADOOP-6870 In-Reply-To: <6227281.77781280442376526.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang reopened HADOOP-6890: ----------------------------------- > Improve listFiles API introduced by HADOOP-6870 > ----------------------------------------------- > > Key: HADOOP-6890 > URL: https://issues.apache.org/jira/browse/HADOOP-6890 > Project: Hadoop Common > Issue Type: Improvement > Components: fs > Affects Versions: 0.22.0 > Reporter: Hairong Kuang > Assignee: Hairong Kuang > Fix For: 0.22.0 > > Attachments: improveListFiles.patch, listFilesInFS.patch > > > This jira is mainly for addressing Suresh's review comments for HADOOP-6870: > 1. General comment: I have concerns about recursive listing. This could be abused by the applications, creating a lot of requests into HDFS. > 2. Any deletion of files/directories while reursing through directories results in RuntimeException and application has a partial result. Should we ignore if a directory was in stack and was not found later when iterating through it? > 3. FileSystem.java > * listFile() - method javadoc could be better organized - first write about if path is directory and two cases recursive=true and false. Then if path is file and two cases recursive=true or false. > * listFile() - document throwing RuntimeException, UnsupportedOperationException and the possible cause. IOException is no longer thrown. > 4. TestListFiles.java > * testDirectory() - comments test empty directory and test directory with 1 file should be moved up to relevant sections of the test. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 17:26:45 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 46209 invoked from network); 2 Aug 2010 17:26:44 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 17:26:44 -0000 Received: (qmail 98091 invoked by uid 500); 2 Aug 2010 17:26:44 -0000 Delivered-To: eric91@example.com Received: (qmail 98044 invoked by uid 500); 2 Aug 2010 17:26:43 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 97997 invoked by uid 99); 2 Aug 2010 17:26:43 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:26:43 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:26:41 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o72HQKed002758 for <eric91@example.com>; Mon, 2 Aug 2010 17:26:20 GMT Message-ID: <4922825.119241280769980426.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 13:26:20 -0400 (EDT) From: "Hairong Kuang (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Updated: (HADOOP-6890) Improve listFiles API introduced by HADOOP-6870 In-Reply-To: <6227281.77781280442376526.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HADOOP-6890: ---------------------------------- Attachment: (was: improveListFiles.patch) > Improve listFiles API introduced by HADOOP-6870 > ----------------------------------------------- > > Key: HADOOP-6890 > URL: https://issues.apache.org/jira/browse/HADOOP-6890 > Project: Hadoop Common > Issue Type: Improvement > Components: fs > Affects Versions: 0.22.0 > Reporter: Hairong Kuang > Assignee: Hairong Kuang > Fix For: 0.22.0 > > Attachments: improveListFiles.patch, listFilesInFS.patch > > > This jira is mainly for addressing Suresh's review comments for HADOOP-6870: > 1. General comment: I have concerns about recursive listing. This could be abused by the applications, creating a lot of requests into HDFS. > 2. Any deletion of files/directories while reursing through directories results in RuntimeException and application has a partial result. Should we ignore if a directory was in stack and was not found later when iterating through it? > 3. FileSystem.java > * listFile() - method javadoc could be better organized - first write about if path is directory and two cases recursive=true and false. Then if path is file and two cases recursive=true or false. > * listFile() - document throwing RuntimeException, UnsupportedOperationException and the possible cause. IOException is no longer thrown. > 4. TestListFiles.java > * testDirectory() - comments test empty directory and test directory with 1 file should be moved up to relevant sections of the test. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 17:26:46 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 46321 invoked from network); 2 Aug 2010 17:26:45 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 17:26:45 -0000 Received: (qmail 98734 invoked by uid 500); 2 Aug 2010 17:26:45 -0000 Delivered-To: eric91@example.com Received: (qmail 98709 invoked by uid 500); 2 Aug 2010 17:26:45 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 98701 invoked by uid 99); 2 Aug 2010 17:26:45 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:26:45 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:26:43 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o72HQMm2002806 for <eric91@example.com>; Mon, 2 Aug 2010 17:26:22 GMT Message-ID: <3833802.119401280769982504.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 13:26:22 -0400 (EDT) From: "Hairong Kuang (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Updated: (HADOOP-6890) Improve listFiles API introduced by HADOOP-6870 In-Reply-To: <6227281.77781280442376526.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HADOOP-6890: ---------------------------------- Status: Patch Available (was: Reopened) > Improve listFiles API introduced by HADOOP-6870 > ----------------------------------------------- > > Key: HADOOP-6890 > URL: https://issues.apache.org/jira/browse/HADOOP-6890 > Project: Hadoop Common > Issue Type: Improvement > Components: fs > Affects Versions: 0.22.0 > Reporter: Hairong Kuang > Assignee: Hairong Kuang > Fix For: 0.22.0 > > Attachments: improveListFiles.patch, listFilesInFS.patch > > > This jira is mainly for addressing Suresh's review comments for HADOOP-6870: > 1. General comment: I have concerns about recursive listing. This could be abused by the applications, creating a lot of requests into HDFS. > 2. Any deletion of files/directories while reursing through directories results in RuntimeException and application has a partial result. Should we ignore if a directory was in stack and was not found later when iterating through it? > 3. FileSystem.java > * listFile() - method javadoc could be better organized - first write about if path is directory and two cases recursive=true and false. Then if path is file and two cases recursive=true or false. > * listFile() - document throwing RuntimeException, UnsupportedOperationException and the possible cause. IOException is no longer thrown. > 4. TestListFiles.java > * testDirectory() - comments test empty directory and test directory with 1 file should be moved up to relevant sections of the test. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 17:31:39 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 47322 invoked from network); 2 Aug 2010 17:31:39 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 17:31:39 -0000 Received: (qmail 3361 invoked by uid 500); 2 Aug 2010 17:31:39 -0000 Delivered-To: eric91@example.com Received: (qmail 3311 invoked by uid 500); 2 Aug 2010 17:31:38 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 3303 invoked by uid 99); 2 Aug 2010 17:31:38 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:31:38 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:31:37 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o72HVHxA002855 for <eric91@example.com>; Mon, 2 Aug 2010 17:31:17 GMT Message-ID: <31203020.119511280770277353.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 13:31:17 -0400 (EDT) From: "Hairong Kuang (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Updated: (HADOOP-6890) Improve listFiles API introduced by HADOOP-6870 In-Reply-To: <6227281.77781280442376526.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HADOOP-6890: ---------------------------------- Attachment: (was: improveListFiles.patch) > Improve listFiles API introduced by HADOOP-6870 > ----------------------------------------------- > > Key: HADOOP-6890 > URL: https://issues.apache.org/jira/browse/HADOOP-6890 > Project: Hadoop Common > Issue Type: Improvement > Components: fs > Affects Versions: 0.22.0 > Reporter: Hairong Kuang > Assignee: Hairong Kuang > Fix For: 0.22.0 > > Attachments: improveListFiles.patch, listFilesInFS.patch > > > This jira is mainly for addressing Suresh's review comments for HADOOP-6870: > 1. General comment: I have concerns about recursive listing. This could be abused by the applications, creating a lot of requests into HDFS. > 2. Any deletion of files/directories while reursing through directories results in RuntimeException and application has a partial result. Should we ignore if a directory was in stack and was not found later when iterating through it? > 3. FileSystem.java > * listFile() - method javadoc could be better organized - first write about if path is directory and two cases recursive=true and false. Then if path is file and two cases recursive=true or false. > * listFile() - document throwing RuntimeException, UnsupportedOperationException and the possible cause. IOException is no longer thrown. > 4. TestListFiles.java > * testDirectory() - comments test empty directory and test directory with 1 file should be moved up to relevant sections of the test. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 17:31:43 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 47383 invoked from network); 2 Aug 2010 17:31:43 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 17:31:43 -0000 Received: (qmail 3530 invoked by uid 500); 2 Aug 2010 17:31:43 -0000 Delivered-To: eric91@example.com Received: (qmail 3485 invoked by uid 500); 2 Aug 2010 17:31:42 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 3477 invoked by uid 99); 2 Aug 2010 17:31:42 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:31:42 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:31:40 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o72HVI2C002874 for <eric91@example.com>; Mon, 2 Aug 2010 17:31:18 GMT Message-ID: <7308996.119571280770278607.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 13:31:18 -0400 (EDT) From: "Hairong Kuang (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Updated: (HADOOP-6890) Improve listFiles API introduced by HADOOP-6870 In-Reply-To: <6227281.77781280442376526.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HADOOP-6890: ---------------------------------- Attachment: improveListFiles.patch > Improve listFiles API introduced by HADOOP-6870 > ----------------------------------------------- > > Key: HADOOP-6890 > URL: https://issues.apache.org/jira/browse/HADOOP-6890 > Project: Hadoop Common > Issue Type: Improvement > Components: fs > Affects Versions: 0.22.0 > Reporter: Hairong Kuang > Assignee: Hairong Kuang > Fix For: 0.22.0 > > Attachments: improveListFiles.patch, listFilesInFS.patch > > > This jira is mainly for addressing Suresh's review comments for HADOOP-6870: > 1. General comment: I have concerns about recursive listing. This could be abused by the applications, creating a lot of requests into HDFS. > 2. Any deletion of files/directories while reursing through directories results in RuntimeException and application has a partial result. Should we ignore if a directory was in stack and was not found later when iterating through it? > 3. FileSystem.java > * listFile() - method javadoc could be better organized - first write about if path is directory and two cases recursive=true and false. Then if path is file and two cases recursive=true or false. > * listFile() - document throwing RuntimeException, UnsupportedOperationException and the possible cause. IOException is no longer thrown. > 4. TestListFiles.java > * testDirectory() - comments test empty directory and test directory with 1 file should be moved up to relevant sections of the test. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 17:54:42 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 57905 invoked from network); 2 Aug 2010 17:54:42 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 17:54:42 -0000 Received: (qmail 45840 invoked by uid 500); 2 Aug 2010 17:54:42 -0000 Delivered-To: eric91@example.com Received: (qmail 45792 invoked by uid 500); 2 Aug 2010 17:54:41 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 45784 invoked by uid 99); 2 Aug 2010 17:54:41 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:54:41 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:54:38 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o72HsGUP003142 for <eric91@example.com>; Mon, 2 Aug 2010 17:54:17 GMT Message-ID: <26345646.120051280771656885.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 13:54:16 -0400 (EDT) From: "Hadoop QA (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Commented: (HADOOP-6890) Improve listFiles API introduced by HADOOP-6870 In-Reply-To: <6227281.77781280442376526.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894621#action_12894621 ] Hadoop QA commented on HADOOP-6890: ----------------------------------- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12451056/improveListFiles.patch against trunk revision 980953. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/655/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/655/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/655/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/655/console This message is automatically generated. > Improve listFiles API introduced by HADOOP-6870 > ----------------------------------------------- > > Key: HADOOP-6890 > URL: https://issues.apache.org/jira/browse/HADOOP-6890 > Project: Hadoop Common > Issue Type: Improvement > Components: fs > Affects Versions: 0.22.0 > Reporter: Hairong Kuang > Assignee: Hairong Kuang > Fix For: 0.22.0 > > Attachments: improveListFiles.patch, listFilesInFS.patch > > > This jira is mainly for addressing Suresh's review comments for HADOOP-6870: > 1. General comment: I have concerns about recursive listing. This could be abused by the applications, creating a lot of requests into HDFS. > 2. Any deletion of files/directories while reursing through directories results in RuntimeException and application has a partial result. Should we ignore if a directory was in stack and was not found later when iterating through it? > 3. FileSystem.java > * listFile() - method javadoc could be better organized - first write about if path is directory and two cases recursive=true and false. Then if path is file and two cases recursive=true or false. > * listFile() - document throwing RuntimeException, UnsupportedOperationException and the possible cause. IOException is no longer thrown. > 4. TestListFiles.java > * testDirectory() - comments test empty directory and test directory with 1 file should be moved up to relevant sections of the test. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 17:54:43 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 57951 invoked from network); 2 Aug 2010 17:54:43 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 17:54:43 -0000 Received: (qmail 46001 invoked by uid 500); 2 Aug 2010 17:54:43 -0000 Delivered-To: eric91@example.com Received: (qmail 45958 invoked by uid 500); 2 Aug 2010 17:54:43 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 45950 invoked by uid 99); 2 Aug 2010 17:54:43 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:54:42 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 17:54:40 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o72HsJPS003167 for <eric91@example.com>; Mon, 2 Aug 2010 17:54:19 GMT Message-ID: <27297706.120131280771659082.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 13:54:19 -0400 (EDT) From: "Owen O'Malley (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Commented: (HADOOP-6894) Common foundation for Hadoop client tools In-Reply-To: <10134473.113661280733856346.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894622#action_12894622 ] Owen O'Malley commented on HADOOP-6894: --------------------------------------- Comments: 1. We desperately need a packaging and deployment strategy in Hadoop. The current tarballs based deployment is insufficient. 2. A lot of discussion of potential layouts for deployment already happened on HADOOP-6255, which you've completely ignored. Installing each tool is a separate directory with the tool and version name is how it was done before we had package managers. Package managers have taken over because they are far more convenient than tarballs with install instructions. I don't think this proposal is moving the project in the correct direction. Granted, we probably need to produce tarballs for operating systems without package managers, but certainly I'd expect most installations to use packages. 3. Any packaging and/or deployment strategy that ignores cluster deployment won't work. 4. Naming versions based on the Hadoop version makes sense, but should only be to the branch. (ie. pig-0.7-0.20 not pig-0.7-0.20.2). Naturally this Jira isn't the place to discuss changes in other sub-projects (ie Pig). > Common foundation for Hadoop client tools > ----------------------------------------- > > Key: HADOOP-6894 > URL: https://issues.apache.org/jira/browse/HADOOP-6894 > Project: Hadoop Common > Issue Type: New Feature > Reporter: Alejandro Abdelnur > Attachments: HadoopClientTools.pdf > > > As Hadoop widespreads and matures the number of tools and utilities for users keeps growing. > Some of them are bundled with Hadoop core, some with Hadoop contrib, some on their own, some are full fledged servers on their own. For example, just to name a few: distcp, streaming, pipes, har, pig, hive, oozie. > Today there is no standard mechanism for making these tools available to users. Neither there is a standard mechanism for these tools to integrate and distributed them with each other. > The lack of a common foundation creates issues for developers and users. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 18:17:42 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 69390 invoked from network); 2 Aug 2010 18:17:40 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 18:17:40 -0000 Received: (qmail 86781 invoked by uid 500); 2 Aug 2010 18:17:40 -0000 Delivered-To: eric91@example.com Received: (qmail 86700 invoked by uid 500); 2 Aug 2010 18:17:39 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 86623 invoked by uid 99); 2 Aug 2010 18:17:39 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 18:17:39 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 18:17:39 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o72IHJBQ003547 for <eric91@example.com>; Mon, 2 Aug 2010 18:17:19 GMT Message-ID: <16462933.120781280773039051.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 14:17:19 -0400 (EDT) From: "Hairong Kuang (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Updated: (HADOOP-6890) Improve listFiles API introduced by HADOOP-6870 In-Reply-To: <6227281.77781280442376526.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HADOOP-6890: ---------------------------------- Status: Open (was: Patch Available) > Improve listFiles API introduced by HADOOP-6870 > ----------------------------------------------- > > Key: HADOOP-6890 > URL: https://issues.apache.org/jira/browse/HADOOP-6890 > Project: Hadoop Common > Issue Type: Improvement > Components: fs > Affects Versions: 0.22.0 > Reporter: Hairong Kuang > Assignee: Hairong Kuang > Fix For: 0.22.0 > > Attachments: improveListFiles.patch, improveListFiles1.patch, listFilesInFS.patch > > > This jira is mainly for addressing Suresh's review comments for HADOOP-6870: > 1. General comment: I have concerns about recursive listing. This could be abused by the applications, creating a lot of requests into HDFS. > 2. Any deletion of files/directories while reursing through directories results in RuntimeException and application has a partial result. Should we ignore if a directory was in stack and was not found later when iterating through it? > 3. FileSystem.java > * listFile() - method javadoc could be better organized - first write about if path is directory and two cases recursive=true and false. Then if path is file and two cases recursive=true or false. > * listFile() - document throwing RuntimeException, UnsupportedOperationException and the possible cause. IOException is no longer thrown. > 4. TestListFiles.java > * testDirectory() - comments test empty directory and test directory with 1 file should be moved up to relevant sections of the test. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 18:17:42 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 69422 invoked from network); 2 Aug 2010 18:17:41 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 18:17:41 -0000 Received: (qmail 86919 invoked by uid 500); 2 Aug 2010 18:17:41 -0000 Delivered-To: eric91@example.com Received: (qmail 86874 invoked by uid 500); 2 Aug 2010 18:17:40 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 86774 invoked by uid 99); 2 Aug 2010 18:17:40 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 18:17:40 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 18:17:40 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o72IHKDg003566 for <eric91@example.com>; Mon, 2 Aug 2010 18:17:20 GMT Message-ID: <6600695.120841280773039995.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 14:17:19 -0400 (EDT) From: "Hairong Kuang (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Updated: (HADOOP-6890) Improve listFiles API introduced by HADOOP-6870 In-Reply-To: <6227281.77781280442376526.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HADOOP-6890: ---------------------------------- Status: Patch Available (was: Open) > Improve listFiles API introduced by HADOOP-6870 > ----------------------------------------------- > > Key: HADOOP-6890 > URL: https://issues.apache.org/jira/browse/HADOOP-6890 > Project: Hadoop Common > Issue Type: Improvement > Components: fs > Affects Versions: 0.22.0 > Reporter: Hairong Kuang > Assignee: Hairong Kuang > Fix For: 0.22.0 > > Attachments: improveListFiles.patch, improveListFiles1.patch, listFilesInFS.patch > > > This jira is mainly for addressing Suresh's review comments for HADOOP-6870: > 1. General comment: I have concerns about recursive listing. This could be abused by the applications, creating a lot of requests into HDFS. > 2. Any deletion of files/directories while reursing through directories results in RuntimeException and application has a partial result. Should we ignore if a directory was in stack and was not found later when iterating through it? > 3. FileSystem.java > * listFile() - method javadoc could be better organized - first write about if path is directory and two cases recursive=true and false. Then if path is file and two cases recursive=true or false. > * listFile() - document throwing RuntimeException, UnsupportedOperationException and the possible cause. IOException is no longer thrown. > 4. TestListFiles.java > * testDirectory() - comments test empty directory and test directory with 1 file should be moved up to relevant sections of the test. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 18:17:42 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 69441 invoked from network); 2 Aug 2010 18:17:41 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 18:17:41 -0000 Received: (qmail 87079 invoked by uid 500); 2 Aug 2010 18:17:41 -0000 Delivered-To: eric91@example.com Received: (qmail 87024 invoked by uid 500); 2 Aug 2010 18:17:41 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 86880 invoked by uid 99); 2 Aug 2010 18:17:41 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 18:17:41 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 18:17:39 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o72IHHWx003521 for <eric91@example.com>; Mon, 2 Aug 2010 18:17:17 GMT Message-ID: <1067010.120701280773037491.JavaMail.jira@thor> Date: Mon, 2 Aug 2010 14:17:17 -0400 (EDT) From: "Hairong Kuang (JIRA)" <eric91@example.com> To: eric91@example.com Subject: [jira] Updated: (HADOOP-6890) Improve listFiles API introduced by HADOOP-6870 In-Reply-To: <6227281.77781280442376526.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HADOOP-6890: ---------------------------------- Attachment: improveListFiles1.patch Fixed the failed test. > Improve listFiles API introduced by HADOOP-6870 > ----------------------------------------------- > > Key: HADOOP-6890 > URL: https://issues.apache.org/jira/browse/HADOOP-6890 > Project: Hadoop Common > Issue Type: Improvement > Components: fs > Affects Versions: 0.22.0 > Reporter: Hairong Kuang > Assignee: Hairong Kuang > Fix For: 0.22.0 > > Attachments: improveListFiles.patch, improveListFiles1.patch, listFilesInFS.patch > > > This jira is mainly for addressing Suresh's review comments for HADOOP-6870: > 1. General comment: I have concerns about recursive listing. This could be abused by the applications, creating a lot of requests into HDFS. > 2. Any deletion of files/directories while reursing through directories results in RuntimeException and application has a partial result. Should we ignore if a directory was in stack and was not found later when iterating through it? > 3. FileSystem.java > * listFile() - method javadoc could be better organized - first write about if path is directory and two cases recursive=true and false. Then if path is file and two cases recursive=true or false. > * listFile() - document throwing RuntimeException, UnsupportedOperationException and the possible cause. IOException is no longer thrown. > 4. TestListFiles.java > * testDirectory() - comments test empty directory and test directory with 1 file should be moved up to relevant sections of the test. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-issues-returneric91@example.com Mon Aug 02 18:35:39 2010 Return-Path: <common-issues-returneric91@example.com> Delivered-To: eric91@example.com Received: (qmail 82635 invoked from network); 2 Aug 2010 18:35:39 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Aug 2010 18:35:39 -0000 Received: (qmail 14501 invoked by uid 500); 2 Aug 2010 18:35:39 -0000 Delivered-To: eric91@example.com Received: (qmail 14476 invoked by uid 500); 2 Aug 2010 18:35:38 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <common-issues.hadoop.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 14468 invoked by uid 99); 2 Aug 2010 18:35:38 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 18:35:38 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Aug 2010 18:35:37 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP
{'source': 'Apache Software Foundation Public Mail Archives'}