From common-devthomasstokes@example.com Wed Dec 01 01:01:35 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 69618 invoked from network); 1 Dec 2010 01:01:35 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 1 Dec 2010 01:01:35 -0000 Received: (qmail 5250 invoked by uid 500); 1 Dec 2010 01:01:32 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 5209 invoked by uid 500); 1 Dec 2010 01:01:32 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 5201 invoked by uid 99); 1 Dec 2010 01:01:31 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Dec 2010 01:01:31 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Dec 2010 01:01:31 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id oB111Bqb025328 for <thomasstokes@example.com>; Wed, 1 Dec 2010 01:01:11 GMT Message-ID: <23144942.38911291165271337.JavaMail.jira@thor> Date: Tue, 30 Nov 2010 20:01:11 -0500 (EST) From: "Konstantin Boudnik (JIRA)" <thomasstokes@example.com> To: thomasstokes@example.com Subject: [jira] Created: (HADOOP-7057) IOUtils.readFully has typo in exception creation's message MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 IOUtils.readFully has typo in exception creation's message ---------------------------------------------------------- Key: HADOOP-7057 URL: https://issues.apache.org/jira/browse/HADOOP-7057 Project: Hadoop Common Issue Type: Bug Components: util Reporter: Konstantin Boudnik Priority: Minor {noformat} throw new IOException( "Premeture EOF from inputStream"); {noformat} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-devthomasstokes@example.com Wed Dec 01 02:39:37 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 10056 invoked from network); 1 Dec 2010 02:39:36 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 1 Dec 2010 02:39:36 -0000 Received: (qmail 76450 invoked by uid 500); 1 Dec 2010 02:39:35 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 76245 invoked by uid 500); 1 Dec 2010 02:39:35 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 76236 invoked by uid 99); 1 Dec 2010 02:39:35 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Dec 2010 02:39:35 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=10.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of thomasstokes@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-qy0-f169.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Dec 2010 02:39:28 +0000 Received: by qyk7 with SMTP id 7so1950358qyk.14 for <thomasstokes@example.com>; Tue, 30 Nov 2010 18:39:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=ohN7ThvXD2utNB4j3c+T20pdZMBqti1By6ALe/lB/c8=; b=ekXShCVmobj5JY7ACSpGQcELz67KPpPK7RiiiGzP0X4GXIFcuexXTEKiV5i2M4uHXp WryNRGOFQHVIXJnF1DNpqcE0PNVN9sRGqgyv/FJKNUwinavbXmcuL9sJQkQFK2AS3325 ONcUHbqQ7uPYbhU6ygIuSWmW7mSxdSJfaOWCs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=Stzlb4K3XkC1eIRXSXHWz5Khs397In+0/r6iCuu3v9YAPRRAJ8WITAZz505KIIJppJ VzpqyWvDIlBp18szPXebpMss5u8ziIHQV1+lUgAUwcoYG74G8NOJ0E4gV/x7mR65HvPI xvBgiyC2MzGYFYNyYlVjVGsDwze0cgitNQ0lA= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id a5mr4858261qab.19.1291171147884; Tue, 30 Nov 2010 18:39:07 -0800 (PST) Received: by 83.161.120.155 with HTTP; Tue, 30 Nov 2010 18:39:07 -0800 (PST) In-Reply-To: <thomasstokes@example.com> References: <thomasstokes@example.com> Date: Wed, 1 Dec 2010 08:09:07 +0530 Message-ID: <thomasstokes@example.com> Subject: Re: Configuring Heap Size for Name Node From: Hemanth Yamijala <thomasstokes@example.com> To: thomasstokes@example.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org Hi, On Wed, Dec 1, 2010 at 1:38 AM, Dave Shine <thomasstokes@example.com> wrote: > How do I change the heap size for the Name Node (and Secondary Name Node)= without changing the heap size for the data nodes, job tracker, and task t= rackers? Which version are you using. I am pretty certain that 0.21 and above have HADOOP_NAMENODE_OPTS which can be used to set the heap size only for NameNode. You can check if this option is defined in hadoop-env.sh. Thanks Hemanth > > Thanks, > Dave Shine > Sr. Software Engineer > > 1180 Celebration Blvd Suite 101 > Celebration FL 34747 > Office +1-415-678-4794 > Mobile +1-415-678-4794 > Fax +1-415-678-4794 > thomasstokes@example.com<mailto:thomasstokes@example.com> > http://www.channelintelligence.com<http://www.channelintelligence.com/> > > > > ________________________________ > The information contained in this email message is considered confidentia= l and proprietary to the sender and is intended solely for review and use b= y the named recipient. Any unauthorized review, use or distribution is stri= ctly prohibited. If you have received this message in error, please advise = the sender by reply email and delete the message. > From common-devthomasstokes@example.com Wed Dec 01 09:53:33 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 5062 invoked from network); 1 Dec 2010 09:53:32 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 1 Dec 2010 09:53:32 -0000 Received: (qmail 65305 invoked by uid 500); 1 Dec 2010 09:53:31 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 64987 invoked by uid 500); 1 Dec 2010 09:53:31 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 64959 invoked by uid 99); 1 Dec 2010 09:53:29 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Dec 2010 09:53:29 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO aegis.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Dec 2010 09:53:27 +0000 Received: from aegis (localhost [83.161.120.155]) by aegis.apache.org (Postfix) with ESMTP id C8700C0164 for <thomasstokes@example.com>; Wed, 1 Dec 2010 09:53:05 +0000 (UTC) Date: Wed, 1 Dec 2010 09:53:05 +0000 (UTC) From: Apache Hudson Server <thomasstokes@example.com> To: thomasstokes@example.com Message-ID: <1227724934.13291291197185727.JavaMail.hudson@aegis> Subject: Hadoop-Common-22-branch - Build # 5 - Failure MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org See https://hudson.apache.org/hudson/job/Hadoop-Common-22-branch/5/ ################################################################################### ########################## LAST 60 LINES OF THE CONSOLE ########################### [...truncated 1590 lines...] [exec] checking for zconf.h... yes [exec] checking Checking for the 'actual' dynamic-library for '-lz'... (cached) "libz.so.1" [exec] checking fcntl.h usability... yes [exec] checking fcntl.h presence... yes [exec] checking for fcntl.h... yes [exec] checking for stdlib.h... (cached) yes [exec] checking for string.h... (cached) yes [exec] checking for unistd.h... (cached) yes [exec] checking for an ANSI C-conforming const... yes [exec] checking for memset... yes [exec] checking whether strerror_r is declared... yes [exec] checking for strerror_r... yes [exec] checking whether strerror_r returns char *... no [exec] configure: creating ./config.status [exec] config.status: creating Makefile [exec] config.status: creating config.h [exec] config.status: executing depfiles commands [exec] make all-am [exec] make[1]: Entering directory `/grid/0/hudson/hudson-slave/workspace/Hadoop-Common-22-branch/trunk/build/native/Linux-i386-32' [exec] if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I/grid/0/hudson/hudson-slave/workspace/Hadoop-Common-22-branch/trunk/src/native -I. -I/homes/hudson/tools/java/latest1.6/include -I/homes/hudson/tools/java/latest1.6/include/linux -I/grid/0/hudson/hudson-slave/workspace/Hadoop-Common-22-branch/trunk/src/native/src -Isrc/org/apache/hadoop/io/compress/zlib -Isrc/org/apache/hadoop/security -Isrc/org/apache/hadoop/io/nativeio/ -g -Wall -fPIC -O2 -m32 -g -O2 -MT ZlibCompressor.lo -MD -MP -MF ".deps/ZlibCompressor.Tpo" -c -o ZlibCompressor.lo `test -f 'src/org/apache/hadoop/io/compress/zlib/ZlibCompressor.c' || echo '/grid/0/hudson/hudson-slave/workspace/Hadoop-Common-22-branch/trunk/src/native/'`src/org/apache/hadoop/io/compress/zlib/ZlibCompressor.c; \ [exec] then mv -f ".deps/ZlibCompressor.Tpo" ".deps/ZlibCompressor.Plo"; else rm -f ".deps/ZlibCompressor.Tpo"; exit 1; fi [exec] mkdir .libs [exec] gcc -DHAVE_CONFIG_H -I. -I/grid/0/hudson/hudson-slave/workspace/Hadoop-Common-22-branch/trunk/src/native -I. -I/homes/hudson/tools/java/latest1.6/include -I/homes/hudson/tools/java/latest1.6/include/linux -I/grid/0/hudson/hudson-slave/workspace/Hadoop-Common-22-branch/trunk/src/native/src -Isrc/org/apache/hadoop/io/compress/zlib -Isrc/org/apache/hadoop/security -Isrc/org/apache/hadoop/io/nativeio/ -g -Wall -fPIC -O2 -m32 -g -O2 -MT ZlibCompressor.lo -MD -MP -MF .deps/ZlibCompressor.Tpo -c /grid/0/hudson/hudson-slave/workspace/Hadoop-Common-22-branch/trunk/src/native/src/org/apache/hadoop/io/compress/zlib/ZlibCompressor.c -fPIC -DPIC -o .libs/ZlibCompressor.o [exec] In file included from /usr/include/features.h:354, [exec] from /usr/include/stdio.h:28, [exec] from /grid/0/hudson/hudson-slave/workspace/Hadoop-Common-22-branch/trunk/src/native/src/org/apache/hadoop/io/compress/zlib/ZlibCompressor.c:24: [exec] /usr/include/gnu/stubs.h:7:27: gnu/stubs-32.h: No such file or directory [exec] make[1]: Leaving directory `/grid/0/hudson/hudson-slave/workspace/Hadoop-Common-22-branch/trunk/build/native/Linux-i386-32' [exec] make[1]: *** [ZlibCompressor.lo] Error 1 [exec] make: *** [all] Error 2 BUILD FAILED /grid/0/hudson/hudson-slave/workspace/Hadoop-Common-22-branch/trunk/build.xml:407: exec returned: 2 Total time: 21 minutes 39 seconds ====================================================================== ====================================================================== STORE: saving artifacts ====================================================================== ====================================================================== mv: cannot stat `build/*.tar.gz': No such file or directory mv: cannot stat `build/*.jar': No such file or directory mv: cannot stat `build/test/findbugs': No such file or directory mv: cannot stat `build/docs/api': No such file or directory Build Failed [FINDBUGS] Skipping publisher since build result is FAILURE [WARNINGS] Skipping publisher since build result is FAILURE Publishing Javadoc Archiving artifacts Recording test results Recording fingerprints Publishing Clover coverage report... No Clover report will be published due to a Build Failure Email was triggered for: Failure Sending email for trigger: Failure ################################################################################### ############################## FAILED TESTS (if any) ############################## No tests ran. From common-devthomasstokes@example.com Wed Dec 01 12:27:42 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 75425 invoked from network); 1 Dec 2010 12:27:42 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 1 Dec 2010 12:27:42 -0000 Received: (qmail 33517 invoked by uid 500); 1 Dec 2010 12:27:41 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 33173 invoked by uid 500); 1 Dec 2010 12:27:40 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 33165 invoked by uid 99); 1 Dec 2010 12:27:39 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Dec 2010 12:27:39 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO smtp.channelintelligence.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Dec 2010 12:27:35 +0000 From: Dave Shine <thomasstokes@example.com> To: "thomasstokes@example.com" <thomasstokes@example.com> Date: Wed, 1 Dec 2010 07:27:13 -0500 Subject: RE: Configuring Heap Size for Name Node Thread-Topic: Configuring Heap Size for Name Node Thread-Index: AcuRAP88y9P3AANNSdi/ucGbIugx1AAUeEuQ Message-ID: <thomasstokes@example.com> References: <thomasstokes@example.com> <thomasstokes@example.com> In-Reply-To: <thomasstokes@example.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 I'm using 0.20.2. I've seen references to HADOOP_NAMENODE_OPTS, but nothing that clearly expl= ains how to set them. Any help would be greatly appreciated. Dave Shine -----Original Message----- From: Hemanth Yamijala [mailto:thomasstokes@example.com]=20 Sent: Tuesday, November 30, 2010 9:39 PM To: thomasstokes@example.com Subject: Re: Configuring Heap Size for Name Node Hi, On Wed, Dec 1, 2010 at 1:38 AM, Dave Shine <thomasstokes@example.com> wrote: > How do I change the heap size for the Name Node (and Secondary Name Node)= without changing the heap size for the data nodes, job tracker, and task t= rackers? Which version are you using. I am pretty certain that 0.21 and above have HADOOP_NAMENODE_OPTS which can be used to set the heap size only for NameNode. You can check if this option is defined in hadoop-env.sh. Thanks Hemanth > > Thanks, > Dave Shine > Sr. Software Engineer > > 1180 Celebration Blvd Suite 101 > Celebration FL 34747 > Office +1-415-678-4794 > Mobile +1-415-678-4794 > Fax +1-415-678-4794 > thomasstokes@example.com<mailto:thomasstokes@example.com> > http://www.channelintelligence.com<http://www.channelintelligence.com/> > > > > ________________________________ > The information contained in this email message is considered confidentia= l and proprietary to the sender and is intended solely for review and use b= y the named recipient. Any unauthorized review, use or distribution is stri= ctly prohibited. If you have received this message in error, please advise = the sender by reply email and delete the message. > From common-devthomasstokes@example.com Wed Dec 01 20:58:01 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 11719 invoked from network); 1 Dec 2010 20:58:00 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 1 Dec 2010 20:58:00 -0000 Received: (qmail 3989 invoked by uid 500); 1 Dec 2010 20:57:59 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 3941 invoked by uid 500); 1 Dec 2010 20:57:59 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 3933 invoked by uid 99); 1 Dec 2010 20:57:58 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Dec 2010 20:57:58 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO smtp.channelintelligence.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Dec 2010 20:57:53 +0000 From: Dave Shine <thomasstokes@example.com> To: "thomasstokes@example.com" <thomasstokes@example.com> Date: Wed, 1 Dec 2010 15:57:32 -0500 Subject: hadoop.tmp.dir Thread-Topic: hadoop.tmp.dir Thread-Index: AcuRml9RFxnWAy96TvaPXU4dgUYkFA== Message-ID: <thomasstokes@example.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_D4D97B30C4253D46B5C0D29A30EB95B5111599462Fhqmail02chann_" MIME-Version: 1.0 --_000_D4D97B30C4253D46B5C0D29A30EB95B5111599462Fhqmail02chann_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable On a production cluster, is it acceptable to have the following properties = set to subfolders of hadoop.tmp.dir? (hadoop.tmp.dir=3D/tmp/hadoop-hadoop) fs.checkpoint.dir mapred.local.dir mapred.temp.dir fs.checkpoint.dir seems like it might be a problem, but I'm not sure I full= y understand how the Secondary Name Node and the checkpoint stuff actually = works. Thanks, Dave Shine Sr. Software Engineer 1180 Celebration Blvd Suite 101 Celebration FL 34747 Office +1-415-678-4794 Mobile +1-415-678-4794 Fax +1-415-678-4794 thomasstokes@example.com<mailto:thomasstokes@example.com> http://www.channelintelligence.com<http://www.channelintelligence.com/> ________________________________ The information contained in this email message is considered confidential = and proprietary to the sender and is intended solely for review and use by = the named recipient. Any unauthorized review, use or distribution is strict= ly prohibited. If you have received this message in error, please advise th= e sender by reply email and delete the message. --_000_D4D97B30C4253D46B5C0D29A30EB95B5111599462Fhqmail02chann_-- From common-devthomasstokes@example.com Thu Dec 02 06:12:40 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 63176 invoked from network); 2 Dec 2010 06:12:40 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Dec 2010 06:12:40 -0000 Received: (qmail 84325 invoked by uid 500); 2 Dec 2010 06:12:39 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 83973 invoked by uid 500); 2 Dec 2010 06:12:37 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 83965 invoked by uid 99); 2 Dec 2010 06:12:37 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 02 Dec 2010 06:12:37 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO mail-qy0-f176.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 02 Dec 2010 06:12:31 +0000 Received: by qyk10 with SMTP id 10so8310982qyk.14 for <thomasstokes@example.com>; Wed, 01 Dec 2010 22:12:09 -0800 (PST) Received: by 83.161.120.155 with SMTP id f10mr8221334qcm.212.1291270329374; Wed, 01 Dec 2010 22:12:09 -0800 (PST) MIME-Version: 1.0 Received: by 83.161.120.155 with HTTP; Wed, 1 Dec 2010 22:11:49 -0800 (PST) In-Reply-To: <thomasstokes@example.com> References: <thomasstokes@example.com> <thomasstokes@example.com> <thomasstokes@example.com> From: Philip Zeyliger <thomasstokes@example.com> Date: Wed, 1 Dec 2010 22:11:49 -0800 Message-ID: <thomasstokes@example.com> Subject: Re: Configuring Heap Size for Name Node To: thomasstokes@example.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org HADOOP_NAMENODE_OPTS=3D"-Xmx500m" will set it to 500MB. The "OPTS" here refers to JVM options. -Xmx is a common JVM option to set the maximum heap. -- Philip On Wed, Dec 1, 2010 at 4:27 AM, Dave Shine <thomasstokes@example.com> wrote: > I'm using 0.20.2. > > I've seen references to HADOOP_NAMENODE_OPTS, but nothing that clearly ex= plains how to set them. > > Any help would be greatly appreciated. > > Dave Shine > > -----Original Message----- > From: Hemanth Yamijala [mailto:thomasstokes@example.com] > Sent: Tuesday, November 30, 2010 9:39 PM > To: thomasstokes@example.com > Subject: Re: Configuring Heap Size for Name Node > > Hi, > > On Wed, Dec 1, 2010 at 1:38 AM, Dave Shine > <thomasstokes@example.com> wrote: >> How do I change the heap size for the Name Node (and Secondary Name Node= ) without changing the heap size for the data nodes, job tracker, and task = trackers? > > Which version are you using. I am pretty certain that 0.21 and above > have HADOOP_NAMENODE_OPTS which can be used to set the heap size only > for NameNode. You can check if this option is defined in > hadoop-env.sh. > > Thanks > Hemanth >> >> Thanks, >> Dave Shine >> Sr. Software Engineer >> >> 1180 Celebration Blvd Suite 101 >> Celebration FL 34747 >> Office +1-415-678-4794 >> Mobile +1-415-678-4794 >> Fax +1-415-678-4794 >> thomasstokes@example.com<mailto:thomasstokes@example.com= > >> http://www.channelintelligence.com<http://www.channelintelligence.com/> >> >> >> >> ________________________________ >> The information contained in this email message is considered confidenti= al and proprietary to the sender and is intended solely for review and use = by the named recipient. Any unauthorized review, use or distribution is str= ictly prohibited. If you have received this message in error, please advise= the sender by reply email and delete the message. >> > From common-devthomasstokes@example.com Fri Dec 03 13:37:44 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 99437 invoked from network); 3 Dec 2010 13:37:43 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 3 Dec 2010 13:37:43 -0000 Received: (qmail 53436 invoked by uid 500); 3 Dec 2010 13:37:43 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 53021 invoked by uid 500); 3 Dec 2010 13:37:40 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 53013 invoked by uid 99); 3 Dec 2010 13:37:40 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Dec 2010 13:37:40 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=10.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of thomasstokes@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-qy0-f176.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Dec 2010 13:37:32 +0000 Received: by qyk10 with SMTP id 10so9973839qyk.14 for <thomasstokes@example.com>; Fri, 03 Dec 2010 05:37:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from :user-agent:mime-version:to:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=ENHdTgUrFeLuJrIjv3xAtBItnZRa2Qlj3lKT+HhuPZc=; b=xy7VZHEGG955gKkMIFPpMcj18xmfsxO+fVXsqryXn44Vi/M07CRO57WFS+wCfPnnYV s8TL3cmJexLWyej2NBHvD3d7kl6Ksvz1UtBqDc80nLbPoKgmC+TQKz3195h2sMaf8ZPZ IRJ6PSF47URqa0BzSxhyMjdBG4KVl5NZhitog= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; b=N9CfEtv0asHMkfvq/RZabARcpTioCpjDSWdsZG1BRJJHEZE/TS3Zv/JL+SgC+L3ZjT m34V+u6WF7wbhcgC/G/U0zKDgYgUPX+FqVsr3blIkbR2VbTPmVFQfV7jGS/rR7t8gNII vdkPRlXWyUhq+NmGGaZb5S9ZtjbsWdyXWc4co= Received: by 83.161.120.155 with SMTP id gv16mr1260675qcb.110.1291383431733; Fri, 03 Dec 2010 05:37:11 -0800 (PST) Received: from [83.161.120.155] (pool-71-179-231-202.bltmmd.fios.verizon.net [83.161.120.155]) by mx.google.com with ESMTPS id k15sm1226931qcu.23.283.161.120.155.37.10 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 03 Dec 2010 05:37:10 -0800 (PST) Message-ID: <thomasstokes@example.com> Date: Fri, 03 Dec 2010 08:37:04 -0500 From: "David B. Ritch" <thomasstokes@example.com> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:83.161.120.155) Gecko/20101027 Thunderbird/3.1.6 MIME-Version: 1.0 To: thomasstokes@example.com Subject: Re: Configuring Heap Size for Name Node References: <thomasstokes@example.com> <thomasstokes@example.com> <thomasstokes@example.com> <thomasstokes@example.com> In-Reply-To: <thomasstokes@example.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org JVM command-line parameters such as this are generally set in the hadoop-env.sh file. dbr On 12/2/2010 1:11 AM, Philip Zeyliger wrote: > HADOOP_NAMENODE_OPTS="-Xmx500m" will set it to 500MB. The "OPTS" here > refers to JVM options. -Xmx is a common JVM option to set the maximum > heap. > > -- Philip > > On Wed, Dec 1, 2010 at 4:27 AM, Dave Shine > <thomasstokes@example.com> wrote: >> I'm using 0.20.2. >> >> I've seen references to HADOOP_NAMENODE_OPTS, but nothing that clearly explains how to set them. >> >> Any help would be greatly appreciated. >> >> Dave Shine >> >> -----Original Message----- >> From: Hemanth Yamijala [mailto:thomasstokes@example.com] >> Sent: Tuesday, November 30, 2010 9:39 PM >> To: thomasstokes@example.com >> Subject: Re: Configuring Heap Size for Name Node >> >> Hi, >> >> On Wed, Dec 1, 2010 at 1:38 AM, Dave Shine >> <thomasstokes@example.com> wrote: >>> How do I change the heap size for the Name Node (and Secondary Name Node) without changing the heap size for the data nodes, job tracker, and task trackers? >> Which version are you using. I am pretty certain that 0.21 and above >> have HADOOP_NAMENODE_OPTS which can be used to set the heap size only >> for NameNode. You can check if this option is defined in >> hadoop-env.sh. >> >> Thanks >> Hemanth >>> Thanks, >>> Dave Shine >>> Sr. Software Engineer >>> >>> 1180 Celebration Blvd Suite 101 >>> Celebration FL 34747 >>> Office +1-415-678-4794 >>> Mobile +1-415-678-4794 >>> Fax +1-415-678-4794 >>> thomasstokes@example.com<mailto:thomasstokes@example.com> >>> http://www.channelintelligence.com<http://www.channelintelligence.com/> >>> >>> >>> >>> ________________________________ >>> The information contained in this email message is considered confidential and proprietary to the sender and is intended solely for review and use by the named recipient. Any unauthorized review, use or distribution is strictly prohibited. If you have received this message in error, please advise the sender by reply email and delete the message. >>> From common-devthomasstokes@example.com Fri Dec 03 13:43:01 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 2196 invoked from network); 3 Dec 2010 13:43:01 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 3 Dec 2010 13:43:01 -0000 Received: (qmail 61837 invoked by uid 500); 3 Dec 2010 13:42:59 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 61515 invoked by uid 500); 3 Dec 2010 13:42:58 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 61140 invoked by uid 99); 3 Dec 2010 13:42:57 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Dec 2010 13:42:57 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO smtp.channelintelligence.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Dec 2010 13:42:52 +0000 From: Dave Shine <thomasstokes@example.com> To: "thomasstokes@example.com" <thomasstokes@example.com> Date: Fri, 3 Dec 2010 08:42:30 -0500 Subject: RE: Configuring Heap Size for Name Node Thread-Topic: Configuring Heap Size for Name Node Thread-Index: AcuS70RmDLhht89qQVaS2PZWFA9bPAAAIjuw Message-ID: <thomasstokes@example.com> References: <thomasstokes@example.com> <thomasstokes@example.com> <thomasstokes@example.com> <thomasstokes@example.com> <thomasstokes@example.com> In-Reply-To: <thomasstokes@example.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Phillip and David,=20 I got it to work. Thanks for the feedback. Dave -----Original Message----- From: David B. Ritch [mailto:thomasstokes@example.com]=20 Sent: Friday, December 03, 2010 8:37 AM To: thomasstokes@example.com Subject: Re: Configuring Heap Size for Name Node JVM command-line parameters such as this are generally set in the hadoop-env.sh file. dbr On 12/2/2010 1:11 AM, Philip Zeyliger wrote: > HADOOP_NAMENODE_OPTS=3D"-Xmx500m" will set it to 500MB. The "OPTS" here > refers to JVM options. -Xmx is a common JVM option to set the maximum > heap. > > -- Philip > > On Wed, Dec 1, 2010 at 4:27 AM, Dave Shine > <thomasstokes@example.com> wrote: >> I'm using 0.20.2. >> >> I've seen references to HADOOP_NAMENODE_OPTS, but nothing that clearly e= xplains how to set them. >> >> Any help would be greatly appreciated. >> >> Dave Shine >> >> -----Original Message----- >> From: Hemanth Yamijala [mailto:thomasstokes@example.com] >> Sent: Tuesday, November 30, 2010 9:39 PM >> To: thomasstokes@example.com >> Subject: Re: Configuring Heap Size for Name Node >> >> Hi, >> >> On Wed, Dec 1, 2010 at 1:38 AM, Dave Shine >> <thomasstokes@example.com> wrote: >>> How do I change the heap size for the Name Node (and Secondary Name Nod= e) without changing the heap size for the data nodes, job tracker, and task= trackers? >> Which version are you using. I am pretty certain that 0.21 and above >> have HADOOP_NAMENODE_OPTS which can be used to set the heap size only >> for NameNode. You can check if this option is defined in >> hadoop-env.sh. >> >> Thanks >> Hemanth >>> Thanks, >>> Dave Shine >>> Sr. Software Engineer >>> >>> 1180 Celebration Blvd Suite 101 >>> Celebration FL 34747 >>> Office +1-415-678-4794 >>> Mobile +1-415-678-4794 >>> Fax +1-415-678-4794 >>> thomasstokes@example.com<mailto:thomasstokes@example.com= m> >>> http://www.channelintelligence.com<http://www.channelintelligence.com/> >>> >>> >>> >>> ________________________________ >>> The information contained in this email message is considered confident= ial and proprietary to the sender and is intended solely for review and use= by the named recipient. Any unauthorized review, use or distribution is st= rictly prohibited. If you have received this message in error, please advis= e the sender by reply email and delete the message. >>> From common-devthomasstokes@example.com Sat Dec 04 14:28:36 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 19032 invoked from network); 4 Dec 2010 14:28:35 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 4 Dec 2010 14:28:35 -0000 Received: (qmail 65847 invoked by uid 500); 4 Dec 2010 14:28:35 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 65414 invoked by uid 500); 4 Dec 2010 14:28:32 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 65402 invoked by uid 99); 4 Dec 2010 14:28:31 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 04 Dec 2010 14:28:31 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,RFC_ABUSE_POST,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of thomasstokes@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO snt0-omc4-s19.snt0.hotmail.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 04 Dec 2010 14:28:23 +0000 Received: from SNT119-W60 ([83.161.120.155]) by snt0-omc4-s19.snt0.hotmail.com with Microsoft SMTPSVC+1-415-678-4794); Sat, 4 Dec 2010 06:28:02 -0800 Message-ID: <thomasstokes@example.com> Content-Type: multipart/alternative; boundary="_ee080b89-34d8-4a39-940c-da5c64dce5a5_" X-Originating-IP: [83.161.120.155] From: Rawan AlSaad <thomasstokes@example.com> To: <thomasstokes@example.com> Subject: FileInputFormat.setInputPaths Problem Date: Sat, 4 Dec 2010 22:28:02 +0800 Importance: Normal MIME-Version: 1.0 X-OriginalArrivalTime: 04 Dec 2010 14:28:02.0150 (UTC) FILETIME=[75052860:01CB93BF] --_ee080b89-34d8-4a39-940c-da5c64dce5a5_ Content-Type: text/plain; charset="windows-1256" Content-Transfer-Encoding: 8bit Dear all, I would very much appreciate your advise on this problem [I am using Hadoop 20.0]: I need to know how to pass the input folder path to the java class throught the function( FileInputFormat.setInputPaths(conf, new Path("input")) I have tried many alternatives to change the second argument of this method but could not solve the problem. The input folder is loaded into the HDFS using the command line from the VM and it contains two text files I am sure that the folder is there in this directory /user/guest/input And here is the error message I get: Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.mapred.FileInputFormat.setInputPaths(Lorg/apache/hadoop/mapred/JobConf;[Lorg/apache/hadoop/fs/Path;)thomasstokes@example.com(WordCount.java:68) Thanks in advance, Rawan --_ee080b89-34d8-4a39-940c-da5c64dce5a5_-- From common-devthomasstokes@example.com Sat Dec 04 15:51:15 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 44613 invoked from network); 4 Dec 2010 15:51:15 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 4 Dec 2010 15:51:15 -0000 Received: (qmail 6754 invoked by uid 500); 4 Dec 2010 15:51:14 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 6423 invoked by uid 500); 4 Dec 2010 15:51:12 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 6349 invoked by uid 99); 4 Dec 2010 15:51:11 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 04 Dec 2010 15:51:11 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=10.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of thomasstokes@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-fx0-f48.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 04 Dec 2010 15:51:06 +0000 Received: by fxm2 with SMTP id 2so8795885fxm.35 for <thomasstokes@example.com>; Sat, 04 Dec 2010 07:50:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:content-type; bh=c2R4orRfSaMEaH6DEN98E5WQh531gZfkKs4jnFnR+H0=; b=IjkTDfiXj2guGWVaa7hhLJlKiBkszU+CX7natX7+3VCqBuVVo8swQpOcHXTYitSY8O TfDrPTKvSuxVtdUJXo6gse4OXIs0X/2glViAyOCHjYfoORJ1ohLMNnEeYwzidwLoxeZS hpmq0CEMfY2xXMYXIsmAT9oFmq+ih3lMSeSa8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=K98LvCe3xj2IZFRUHf/m/thbRqKFoiwiCpyYltw+HMne2kAZ/IKNEP29uMDylcyL1g h9GWK8Ab0XsgI4UsvxUdbaG/taJR6rq0UizalZD/gKbpX1ptGmbghUhBzPpYblbgukh+ 2PvlS6YBMwXFEPc58tPS1i6qtHIyxoiMdurhw= Received: by 83.161.120.155 with SMTP id g7mr1425886fao.29.1291477844977; Sat, 04 Dec 2010 07:50:44 -0800 (PST) MIME-Version: 1.0 Received: by 83.161.120.155 with HTTP; Sat, 4 Dec 2010 07:50:24 -0800 (PST) In-Reply-To: <thomasstokes@example.com> References: <thomasstokes@example.com> From: Harsh J <thomasstokes@example.com> Date: Sat, 4 Dec 2010 21:20:24 +0530 Message-ID: <thomasstokes@example.com> Subject: Re: FileInputFormat.setInputPaths Problem To: thomasstokes@example.com Content-Type: text/plain; charset=ISO-8859-1 Hi, 2010/12/4 Rawan AlSaad <thomasstokes@example.com>: > I need to know how to pass the input folder path to the java class throught the function( FileInputFormat.setInputPaths(conf, new Path("input")) Try FileInputFormat.addInputPath(...) for a single path entry at a time perhaps? I'm not sure what's going wrong here though. -- Harsh J www.harshj.com From common-devthomasstokes@example.com Sat Dec 04 16:28:07 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 59443 invoked from network); 4 Dec 2010 16:28:07 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 4 Dec 2010 16:28:07 -0000 Received: (qmail 25792 invoked by uid 500); 4 Dec 2010 16:28:06 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 25638 invoked by uid 500); 4 Dec 2010 16:28:04 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 25629 invoked by uid 99); 4 Dec 2010 16:28:02 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 04 Dec 2010 16:28:02 +0000 X-ASF-Spam-Status: No, hits=4.7 required=10.0 tests=FREEMAIL_FROM,FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,RFC_ABUSE_POST,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of thomasstokes@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO snt0-omc4-s36.snt0.hotmail.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 04 Dec 2010 16:27:54 +0000 Received: from SNT119-W58 ([83.161.120.155]) by snt0-omc4-s36.snt0.hotmail.com with Microsoft SMTPSVC+1-415-678-4794); Sat, 4 Dec 2010 08:27:32 -0800 Message-ID: <thomasstokes@example.com> Content-Type: multipart/alternative; boundary="_69bc33f8-574f-4033-8623-bbce3e247dbd_" X-Originating-IP: [83.161.120.155] From: Rawan AlSaad <thomasstokes@example.com> To: <thomasstokes@example.com> Subject: RE: FileInputFormat.setInputPaths Problem Date: Sun, 5 Dec 2010 00:27:32 +0800 Importance: Normal In-Reply-To: <thomasstokes@example.com> References: <thomasstokes@example.com>,<thomasstokes@example.com> MIME-Version: 1.0 X-OriginalArrivalTime: 04 Dec 2010 16:27:32.0442 (UTC) FILETIME=[26D8E3A0:01CB93D0] X-Virus-Checked: Checked by ClamAV on apache.org --_69bc33f8-574f-4033-8623-bbce3e247dbd_ Content-Type: text/plain; charset="windows-1256" Content-Transfer-Encoding: 8bit Thank you for your suggestion, but it will not work since the only available method is addInputPaths and no method for addInputPath! I am using only one path at a time, currently trying to use these two lines and getting the below error: FileInputFormat.setInputPaths(conf, new Path(args[0])); FileOutputFormat.setOutputPath(conf, new Path(args[1])); Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: thomasstokes@example.com(WordCount.java:53) Any more suggestions??? Thanks, Rawan > From: thomasstokes@example.com > Date: Sat, 4 Dec 2010 21:20:24 +0530 > Subject: Re: FileInputFormat.setInputPaths Problem > To: thomasstokes@example.com > > Hi, > > 2010/12/4 Rawan AlSaad <thomasstokes@example.com>: > > I need to know how to pass the input folder path to the java class throught the function( FileInputFormat.setInputPaths(conf, new Path("input")) > > Try FileInputFormat.addInputPath(...) for a single path entry at a > time perhaps? I'm not sure what's going wrong here though. > > -- > Harsh J > www.harshj.com --_69bc33f8-574f-4033-8623-bbce3e247dbd_-- From common-devthomasstokes@example.com Sat Dec 04 23:44:06 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 12878 invoked from network); 4 Dec 2010 23:44:05 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 4 Dec 2010 23:44:05 -0000 Received: (qmail 50280 invoked by uid 500); 4 Dec 2010 23:44:04 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 50197 invoked by uid 500); 4 Dec 2010 23:44:03 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 50189 invoked by uid 99); 4 Dec 2010 23:44:03 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 04 Dec 2010 23:44:03 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of thomasstokes@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-01.cse.ucsc.edu) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 04 Dec 2010 23:43:54 +0000 Received: from [83.161.120.155] (soenat3.cse.ucsc.edu [83.161.120.155]) by mail-01.cse.ucsc.edu (Postfix) with ESMTP id 062EE7744003 for <thomasstokes@example.com>; Sat, 4 Dec 2010 15:43:34 -0800 (PST) From: Noah Watkins <thomasstokes@example.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Native code and autoconf generated files Date: Sat, 4 Dec 2010 15:43:33 -0800 Message-Id: <thomasstokes@example.com> To: thomasstokes@example.com Mime-Version: 1.0 (Apple Message framework v1081) X-Mailer: Apple Mail (2.1081) X-Virus-Checked: Checked by ClamAV on apache.org Hi, This question pertains to the 0.21 release. When adding code to the Hadoop native code library what is the preferred = method for regenerating Makefile.in? I've tried to use autoreconf, but = it seems as though my newer version of autotools is causing compilation = problems for the other native code libraries. Thanks, Noah= From common-devthomasstokes@example.com Sun Dec 05 01:41:47 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 52381 invoked from network); 5 Dec 2010 01:41:46 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 5 Dec 2010 01:41:46 -0000 Received: (qmail 98248 invoked by uid 500); 5 Dec 2010 01:41:46 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 98020 invoked by uid 500); 5 Dec 2010 01:41:45 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 98012 invoked by uid 99); 5 Dec 2010 01:41:45 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 05 Dec 2010 01:41:45 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO mail-qw0-f48.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 05 Dec 2010 01:41:39 +0000 Received: by qwi4 with SMTP id 4so60794qwi.35 for <thomasstokes@example.com>; Sat, 04 Dec 2010 17:41:17 -0800 (PST) MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id q7mr2795276qcn.286.1291513277509; Sat, 04 Dec 2010 17:41:17 -0800 (PST) Received: by 83.161.120.155 with HTTP; Sat, 4 Dec 2010 17:41:17 -0800 (PST) In-Reply-To: <thomasstokes@example.com> References: <thomasstokes@example.com> Date: Sat, 4 Dec 2010 17:41:17 -0800 Message-ID: <thomasstokes@example.com> Subject: Re: Native code and autoconf generated files From: Eli Collins <thomasstokes@example.com> To: thomasstokes@example.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org Hey Noah, The short answer is use older versions (RHEL/Centos 5) of autotools. Longer term we need to stop checking in auto-generated files (HADOOP-6436). This pertains to trunk and all releases btw. Thanks, Eli On Sat, Dec 4, 2010 at 3:43 PM, Noah Watkins <thomasstokes@example.com> wrote: > Hi, > > This question pertains to the 0.21 release. > > When adding code to the Hadoop native code library what is the preferred = method for regenerating Makefile.in? I've tried to use autoreconf, but it s= eems as though my newer version of autotools is causing compilation problem= s for the other native code libraries. > > Thanks, > Noah From common-devthomasstokes@example.com Mon Dec 06 15:30:04 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 69114 invoked from network); 6 Dec 2010 15:30:04 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 6 Dec 2010 15:30:04 -0000 Received: (qmail 867 invoked by uid 500); 6 Dec 2010 15:30:03 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 349 invoked by uid 500); 6 Dec 2010 15:30:01 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 333 invoked by uid 99); 6 Dec 2010 15:30:00 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Dec 2010 15:30:00 +0000 X-ASF-Spam-Status: No, hits=1.5 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of thomasstokes@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-yx0-f176.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Dec 2010 15:29:54 +0000 Received: by yxm8 with SMTP id 8so7040802yxm.35 for <thomasstokes@example.com>; Mon, 06 Dec 2010 07:29:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=pXEg08PDU1HDyhA+wfXaw+Kaa7EEeL1at2hNqaha5G8=; b=YLyJBvQD1qcIalyWUnr2ol1vII72BtJA8bBK4ni/61VMWutofSxTaw6F++T2LdB5Bl po2wrbLCYJLZdHzUulygk4c7Rj8HiYSHdfZi5bsow4DaQZhykaqQrX5f4U82yr73VUaB cXiyJ8eKq+QiLuyqeC8xuyUR6Rcx8NzKJk4zQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=ZqZgUMAr7zdv6F/qWSfmZijk7waL+Gq6O62SSSTA1rrPF9YSeUmLa/KhicX3Y8hY9D cv3aVj+SQq7vyBADxKD1mYO0afdH9m/vQvv1P8JsQLA1I6c5ivxkUCuc3ioWiCl1Ula6 TGnjSXBxgPAnZXQ1mDQmcWci+SXQnho4c5Xm4= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id t2mr9511311ybf.205.1291649373335; Mon, 06 Dec 2010 07:29:33 -0800 (PST) Received: by 83.161.120.155 with HTTP; Mon, 6 Dec 2010 07:29:33 -0800 (PST) Date: Mon, 6 Dec 2010 23:29:33 +0800 Message-ID: <thomasstokes@example.com> Subject: Explicit Data Placement From: yipeng <thomasstokes@example.com> To: thomasstokes@example.com Content-Type: multipart/alternative; boundary=000e0cd47b901272bc0496bf8e98 --000e0cd47b901272bc0496bf8e98 Content-Type: text/plain; charset=ISO-8859-1 Dear all, I understand that specifying data placement may not be in the spirit of Hadoop, however I would like to explicitly locate data input on various nodes to assert the value of data locality and cost of network transfer. Does anyone have any advice about how I can go about the data placement? Thanks, Yipeng --000e0cd47b901272bc0496bf8e98-- From common-devthomasstokes@example.com Mon Dec 06 15:53:55 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 78001 invoked from network); 6 Dec 2010 15:53:55 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 6 Dec 2010 15:53:55 -0000 Received: (qmail 36086 invoked by uid 500); 6 Dec 2010 15:53:54 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 35649 invoked by uid 500); 6 Dec 2010 15:53:52 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 35641 invoked by uid 99); 6 Dec 2010 15:53:51 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Dec 2010 15:53:51 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=10.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of thomasstokes@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-yw0-f51.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Dec 2010 15:53:44 +0000 Received: by ywe10 with SMTP id 10so44908ywe.38 for <thomasstokes@example.com>; Mon, 06 Dec 2010 07:53:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=oaiA5bqNmtNOfh3iSZl6wD/OB+JnDZ2vyhkoWR3mpho=; b=HMV5NnRA6WVEu3JqMdFPa3RRStvRIW3nsWhLONYu+BCUzUZ9g9tBWqPj2WwDuyIPWf iFGfjw/nFxHgLd+hhaS06TwTHZ2nSfOtJNBs30mkMDs6sRiJ96C2cOUI59iolkBalRcT ihP6wXU1eLxdU5ONm9UXmV+RJDogPs91/9xRg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=q+uZz8dHLA06HRN4bJEPmbTEOcCtkNv5nAn15nwkDI2Z42ny6QeGbW9xyhixc/HuDI qodSejLEPg/l3Qq7GswolQ7VLMQalR3NmMs7hFVP7oCLKgVORneZZfxwHdMTCEQOo/2N x5Qpb+UnwMkm5e0md8XDtWS+6oAk4yxJ6ojXY= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id c9mr5318548far.50.1291650803210; Mon, 06 Dec 2010 07:53:23 -0800 (PST) Received: by 83.161.120.155 with HTTP; Mon, 6 Dec 2010 07:53:23 -0800 (PST) In-Reply-To: <thomasstokes@example.com> References: <thomasstokes@example.com> Date: Mon, 6 Dec 2010 10:53:23 -0500 Message-ID: <thomasstokes@example.com> Subject: Re: Explicit Data Placement From: Faraz Ahmad <thomasstokes@example.com> To: thomasstokes@example.com Content-Type: text/plain; charset=ISO-8859-1 X-Virus-Checked: Checked by ClamAV on apache.org This paper might help you in this regard. I believe they have some public source code available to do that. http://www.eng.auburn.edu/~xqin/pubs/hcw10.pdf -Faraz On Mon, Dec 6, 2010 at 10:29 AM, yipeng <thomasstokes@example.com> wrote: > Dear all, > > I understand that specifying data placement may not be in the spirit of > Hadoop, however I would like to explicitly locate data input on various > nodes to assert the value of data locality and cost of network transfer. > > Does anyone have any advice about how I can go about the data placement? > > Thanks, > > Yipeng > From common-devthomasstokes@example.com Mon Dec 06 17:51:13 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 31352 invoked from network); 6 Dec 2010 17:51:12 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 6 Dec 2010 17:51:12 -0000 Received: (qmail 44480 invoked by uid 500); 6 Dec 2010 17:51:11 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 44422 invoked by uid 500); 6 Dec 2010 17:51:10 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Delivered-To: moderator for thomasstokes@example.com Received: (qmail 53174 invoked by uid 99); 6 Dec 2010 14:54:17 -0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of thomasstokes@example.com designates 83.161.120.155 as permitted sender) From: =?utf-8?Q?Alex=E2=80=8E_Nelson?= <thomasstokes@example.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Granting contributor status Date: Mon, 6 Dec 2010 06:53:46 -0800 Message-Id: <thomasstokes@example.com> To: thomasstokes@example.com Mime-Version: 1.0 (Apple Message framework v1082) X-Mailer: Apple Mail (2.1082) X-Virus-Checked: Checked by ClamAV on apache.org Hello all, A colleague of mine is doing some work on a patch while I am unavailable = to work on it, so I'd like to assign the JIRA item to him. To whom do I = address requests to make him a Contributor within the JIRA system? --Alex= From common-devthomasstokes@example.com Mon Dec 06 17:56:57 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 33814 invoked from network); 6 Dec 2010 17:56:57 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 6 Dec 2010 17:56:57 -0000 Received: (qmail 53364 invoked by uid 500); 6 Dec 2010 17:56:54 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 53279 invoked by uid 500); 6 Dec 2010 17:56:54 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 53052 invoked by uid 99); 6 Dec 2010 17:56:54 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Dec 2010 17:56:54 +0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO minotaur.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with SMTP; Mon, 06 Dec 2010 17:56:53 +0000 Received: (qmail 33575 invoked by uid 99); 6 Dec 2010 17:56:33 -0000 Received: from localhost.apache.org (HELO [83.161.120.155]) (83.161.120.155) (smtp-auth username cutting, mechanism plain) by minotaur.apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Dec 2010 17:56:33 +0000 Message-ID: <thomasstokes@example.com> Date: Mon, 06 Dec 2010 09:56:32 -0800 From: Doug Cutting <thomasstokes@example.com> User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:83.161.120.155) Gecko/20101027 Thunderbird/3.1.6 MIME-Version: 1.0 To: thomasstokes@example.com Subject: Re: Granting contributor status References: <thomasstokes@example.com> In-Reply-To: <thomasstokes@example.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Alex, Generally when someone contributes a patch and a committer reviews it, then the committer assigns the issue to the contributor. So when your colleague contributes the next version of the patch then it should be assigned to him. Committers add new Jira contributors as needed. Cheers, Doug On 12/06/2010 06:53 AM, Alex‎ Nelson wrote: > Hello all, > > A colleague of mine is doing some work on a patch while I am unavailable to work on it, so I'd like to assign the JIRA item to him. To whom do I address requests to make him a Contributor within the JIRA system? > > --Alex From common-devthomasstokes@example.com Tue Dec 07 00:46:36 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 50594 invoked from network); 7 Dec 2010 00:46:35 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 7 Dec 2010 00:46:35 -0000 Received: (qmail 75379 invoked by uid 500); 7 Dec 2010 00:46:34 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 75333 invoked by uid 500); 7 Dec 2010 00:46:34 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 75325 invoked by uid 99); 7 Dec 2010 00:46:34 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 07 Dec 2010 00:46:34 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 07 Dec 2010 00:46:32 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id oB70kBGu010124 for <thomasstokes@example.com>; Tue, 7 Dec 2010 00:46:11 GMT Message-ID: <17138789.12301291682771007.JavaMail.jira@thor> Date: Mon, 6 Dec 2010 19:46:11 -0500 (EST) From: "Milind Bhandarkar (JIRA)" <thomasstokes@example.com> To: thomasstokes@example.com Subject: [jira] Resolved: (HADOOP-2663) Buffer class' toString method should accept a code name for "true" utf-8 codeName MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-2663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Milind Bhandarkar resolved HADOOP-2663. --------------------------------------- Resolution: Won't Fix I am no longer developing Hadoop record I/O anymore. Also, there is a jira that proposes to deprecate Hadoop recordI/O, so marking this Jira as won't fix. > Buffer class' toString method should accept a code name for "true" utf-8 codeName > ---------------------------------------------------------------------------------- > > Key: HADOOP-2663 > URL: https://issues.apache.org/jira/browse/HADOOP-2663 > Project: Hadoop Common > Issue Type: Improvement > Components: record > Reporter: Runping Qi > Assignee: Milind Bhandarkar > > Currently, if one call toString("UTF-8"), a String object is created using Java's converion code. > That does not work properly for some rare but still true utf-8 bytes. Hadoop has its own utf-8 conversion > code for string serialization/deserialization. The same code should be used here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-devthomasstokes@example.com Tue Dec 07 03:53:34 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 18747 invoked from network); 7 Dec 2010 03:53:33 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 7 Dec 2010 03:53:33 -0000 Received: (qmail 22828 invoked by uid 500); 7 Dec 2010 03:53:32 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 22613 invoked by uid 500); 7 Dec 2010 03:53:31 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 22557 invoked by uid 99); 7 Dec 2010 03:53:30 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 07 Dec 2010 03:53:30 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 07 Dec 2010 03:53:30 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id oB73rAM3012947 for <thomasstokes@example.com>; Tue, 7 Dec 2010 03:53:10 GMT Message-ID: <7761586.16171291693990205.JavaMail.jira@thor> Date: Mon, 6 Dec 2010 22:53:10 -0500 (EST) From: "Todd Lipcon (JIRA)" <thomasstokes@example.com> To: thomasstokes@example.com Subject: [jira] Created: (HADOOP-7058) Expose number of bytes in FSOutputSummer buffer to implementatins MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Expose number of bytes in FSOutputSummer buffer to implementatins ----------------------------------------------------------------- Key: HADOOP-7058 URL: https://issues.apache.org/jira/browse/HADOOP-7058 Project: Hadoop Common Issue Type: Improvement Affects Versions: 0.22.0, 0.23.0 Reporter: Todd Lipcon Priority: Trivial Attachments: hadoop-7058.txt For HDFS-1497 it would be useful for an FSOutputSummer implementation to know how many bytes are in the FSOutputSummer buffer. This trivial patch adds a protected call to return this. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-devthomasstokes@example.com Tue Dec 07 21:51:36 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 23654 invoked from network); 7 Dec 2010 21:51:35 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 7 Dec 2010 21:51:35 -0000 Received: (qmail 35402 invoked by uid 500); 7 Dec 2010 21:51:34 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 35285 invoked by uid 500); 7 Dec 2010 21:51:33 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 35141 invoked by uid 99); 7 Dec 2010 21:51:33 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 07 Dec 2010 21:51:33 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO thor.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 07 Dec 2010 21:51:31 +0000 Received: from thor (localhost [83.161.120.155]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id oB7LpAnt003319 for <thomasstokes@example.com>; Tue, 7 Dec 2010 21:51:10 GMT Message-ID: <31393326.37221291758669990.JavaMail.jira@thor> Date: Tue, 7 Dec 2010 16:51:09 -0500 (EST) From: "Noah Watkins (JIRA)" <thomasstokes@example.com> To: thomasstokes@example.com Subject: [jira] Created: (HADOOP-7059) Remove "unused" warning in native code MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org Remove "unused" warning in native code -------------------------------------- Key: HADOOP-7059 URL: https://issues.apache.org/jira/browse/HADOOP-7059 Project: Hadoop Common Issue Type: Improvement Components: native Affects Versions: 0.21.0 Reporter: Noah Watkins The file: src/native/src/org_apache_hadoop.h declares the static function "do_dlsym" in the header as non-inline. Files including the header (e.g. for the THROW macro) receive a "defined but unused" warning during compilation. This function should either a) be inlined or b) use GCC "unused" attribute. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From common-devthomasstokes@example.com Wed Dec 08 00:10:02 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 85131 invoked from network); 8 Dec 2010 00:10:01 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 8 Dec 2010 00:10:01 -0000 Received: (qmail 3578 invoked by uid 500); 8 Dec 2010 00:10:00 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 3511 invoked by uid 500); 8 Dec 2010 00:10:00 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 3503 invoked by uid 99); 8 Dec 2010 00:10:00 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Dec 2010 00:10:00 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of thomasstokes@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO services.cse.ucsc.edu) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Dec 2010 00:09:51 +0000 Received: from [83.161.120.155] (soenat3.cse.ucsc.edu [83.161.120.155]) (authenticated bits=0) by services.cse.ucsc.edu (8.13.6/8.13.6) with ESMTP id oB809Tsq023201 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 7 Dec 2010 16:09:30 -0800 (PST) From: =?utf-8?Q?Alex=E2=80=8E_Nelson?= <thomasstokes@example.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Overloading file URI scheme Date: Tue, 7 Dec 2010 16:09:29 -0800 Message-Id: <thomasstokes@example.com> Cc: Noah Watkins <thomasstokes@example.com> To: thomasstokes@example.com Mime-Version: 1.0 (Apple Message framework v1082) X-Mailer: Apple Mail (2.1082) X-Virus-Checked: Checked by ClamAV on apache.org Hello all, We are looking for some expertise with Hadoop's handling of URI schemes. = Our problem is that we want to subclass RawLocalFileSystem for the Ceph = file system [1], and get all of RawLocal's functionality, but we don't = want the file:// URI scheme. That is, we want to treat this location in = the file hierarchy, a point where we mount Ceph: /mnt/ceph/ as: ceph:/// instead of: file:///mnt/ceph/ Since RawLocalFileSystem hard-codes that URI scheme, what's the best way = to solve that problem? --Alex [1] https://issues.apache.org/jira/browse/HADOOP-6779= From common-devthomasstokes@example.com Wed Dec 08 09:48:31 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 22266 invoked from network); 8 Dec 2010 09:48:31 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 8 Dec 2010 09:48:31 -0000 Received: (qmail 52815 invoked by uid 500); 8 Dec 2010 09:48:30 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 52614 invoked by uid 500); 8 Dec 2010 09:48:29 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 52605 invoked by uid 99); 8 Dec 2010 09:48:28 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Dec 2010 09:48:28 +0000 X-ASF-Spam-Status: No, hits=1.5 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of thomasstokes@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-ew0-f43.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Dec 2010 09:48:22 +0000 Received: by ewy22 with SMTP id 22so773062ewy.2 for <thomasstokes@example.com>; Wed, 08 Dec 2010 01:48:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=q4gzwjl04rT+5dyFwmkeodhyCojUZo1jvrPSQVavo/s=; b=WVZVbBl3Qqbw4LroDc0+Gkvs4OWnNF4+8gxhf6lSeC3AAx+EGGNudza/YIc/8zqKpq 7uwf8l/Sr88oO3LijQ+IPCMjffHYIq3p7a0ZqRbq+9z9fEpWq/m9NvPdfvKpamv4nWjU ZituZbtczsbqzCpg4WLErmAlQbTG8QLxrFPI8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=FpOZB/H9tws5A8n6bYQh7G4RFj28APolrkt3eFsutp1/UU06SpgR5Ri28Fk5b4A5x/ latpggQNTDp3G3RqoilC6/L+pEjHJMhwNBKm5x3I5UNa61bmYVLUPfNB/EoGdD8l+AeC e7EIsiwTouds0bo0z1zCsW7Yn15PfQHBJ8HEA= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id u10mr7440432eeb.39.1291801681255; Wed, 08 Dec 2010 01:48:01 -0800 (PST) Received: by 83.161.120.155 with HTTP; Wed, 8 Dec 2010 01:48:01 -0800 (PST) Date: Wed, 8 Dec 2010 15:18:01 +0530 Message-ID: <thomasstokes@example.com> Subject: Process ID and Hadoop job ID From: radheshyam nanduri <thomasstokes@example.com> To: thomasstokes@example.com Content-Type: multipart/alternative; boundary=90e6ba6153ea54dcef0496e304f5 --90e6ba6153ea54dcef0496e304f5 Content-Type: text/plain; charset=ISO-8859-1 Hi, I want to know if there is any way to find out the process id (PID) of a task running on a TaskTracker corresponding to a particular Hadoop job ID. All the Hadoop tasks are launched as java processes. So, is there any way to differentiate among them to get the PID of a particular task of a particular Hadoop job. Thanks a lot in advance, Radheshyam Nanduri --90e6ba6153ea54dcef0496e304f5-- From common-devthomasstokes@example.com Wed Dec 08 16:02:27 2010 Return-Path: <common-devthomasstokes@example.com> Delivered-To: thomasstokes@example.com Received: (qmail 30040 invoked from network); 8 Dec 2010 16:02:26 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 8 Dec 2010 16:02:26 -0000 Received: (qmail 81255 invoked by uid 500); 8 Dec 2010 16:02:24 -0000 Delivered-To: thomasstokes@example.com Received: (qmail 81004 invoked by uid 500); 8 Dec 2010 16:02:24 -0000 Mailing-List: contact thomasstokes@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:thomasstokes@example.com> List-Unsubscribe: <mailto:thomasstokes@example.com> List-Post: <mailto:thomasstokes@example.com> List-Id: <common-dev.hadoop.apache.org> Reply-To: thomasstokes@example.com Delivered-To: mailing list thomasstokes@example.com Received: (qmail 80994 invoked by uid 99); 8 Dec 2010 16:02:23 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Dec 2010 16:02:23 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=10.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of thomasstokes@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-fx0-f47.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Dec 2010 16:02:18 +0000 Received: by fxm17 with SMTP id 17so1137968fxm.34 for <thomasstokes@example.com>; Wed, 08 Dec 2010 08:01:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:content-type; bh=jAX21EEPLxRk8ErJ7whQp/h70p7Ne6nXCyRsos6/vT4=; b=c6YKIVtcPnyGdl1lkcBHpaBHCVlULpeO50tEvO+Kn42Hb2vU53NQPvDV9DkmWCYolp dhSjCBT59vgjhSaVuXDa/jIi+YQYmSavKLcQ/n13jElYh4glbECGB16eyo0SF3raVt91 5dZSP373A9lNMmBpm5UBWmvbd06spsV48R3s4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=KLpn0CcWHt24PYZoSA24Lv8I1Cg2G6d0zneQVitnecSDfwrdi6/fsXfvKjEFGKS0LQ NWH/ABVR1xPXYHFr3dcZZk6INm2IeqZfQ3liAfISOvG+ChUjCiSGYWut63SWl4kTRkhj OxLQFFZpqoYfq6pGPnKlzn7gqZbZ/GJdAQuIo= Received: by 83.161.120.155 with SMTP id e14mr26891fam.143.1291824117572; Wed, 08 Dec 2010 08:01:57 -0800 (PST) MIME-Version: 1.0 Received: by 83.161.120.155 with HTTP; Wed, 8 Dec 2010 08:01:37 -0800 (PST) In-Reply-To: <thomasstokes@example.com> References: <thomasstokes@example.com> From: Harsh J <thomasstokes@example.com> Date: Wed, 8 Dec 2010 21:31:37 +0530 Message-ID:
{'source': 'Apache Software Foundation Public Mail Archives'}