From core-userwgentry@example.com Wed Oct 01 00:22:22 2008 Return-Path: <core-userwgentry@example.com> Delivered-To: wgentry@example.com Received: (qmail 98780 invoked from network); 1 Oct 2008 00:22:21 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Oct 2008 00:22:21 -0000 Received: (qmail 34054 invoked by uid 500); 1 Oct 2008 00:22:14 -0000 Delivered-To: wgentry@example.com Received: (qmail 33810 invoked by uid 500); 1 Oct 2008 00:22:13 -0000 Mailing-List: contact wgentry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:wgentry@example.com> List-Unsubscribe: <mailto:wgentry@example.com> List-Post: <mailto:wgentry@example.com> List-Id: <core-user.hadoop.apache.org> Reply-To: wgentry@example.com Delivered-To: mailing list wgentry@example.com Received: (qmail 33799 invoked by uid 99); 1 Oct 2008 00:22:13 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 30 Sep 2008 17:22:13 -0700 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO mx02.metaweb.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with SMTP; Wed, 01 Oct 2008 00:21:13 +0000 Received: from zimbra01.corp.sjc1.metaweb.com (localhost [83.161.120.155]) by mx02.metaweb.com (Spam Firewall) with ESMTP id D8ED86DD67 for <wgentry@example.com>; Tue, 30 Sep 2008 17:24:59 -0700 (PDT) Received: from zimbra01.corp.sjc1.metaweb.com (zimbra01.corp.sjc1.metaweb.com [83.161.120.155]) by mx02.metaweb.com with ESMTP id SffFzuO87NQsI4f0 for <wgentry@example.com>; Tue, 30 Sep 2008 17:24:59 -0700 (PDT) Received: from localhost (localhost.localdomain [83.161.120.155]) by zimbra01.corp.sjc1.metaweb.com (Postfix) with ESMTP id BB0E8CDC043 for <wgentry@example.com>; Tue, 30 Sep 2008 17:21:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at X-Spam-Score: -4.399 X-Spam-Level: Received: from zimbra01.corp.sjc1.metaweb.com ([83.161.120.155]) by localhost (zimbra01.corp.sjc1.metaweb.com [83.161.120.155]) (amavisd-new, port 10024) with ESMTP id t6ho68M93EA0 for <wgentry@example.com>; Tue, 30 Sep 2008 17:21:46 -0700 (PDT) Received: from dhcp-wired-209.corp.631h.metaweb.com (dhcp-wired-209.corp.631h.metaweb.com [83.161.120.155]) by zimbra01.corp.sjc1.metaweb.com (Postfix) with ESMTP id 30AF0CDC046 for <wgentry@example.com>; Tue, 30 Sep 2008 17:21:46 -0700 (PDT) Message-ID: <wgentry@example.com> Date: Tue, 30 Sep 2008 17:21:10 -0700 From: Colin Evans <wgentry@example.com> User-Agent: Thunderbird 83.161.120.155 (Macintosh/20080914) MIME-Version: 1.0 To: wgentry@example.com Subject: Re: LZO and native hadoop libraries References: <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> In-Reply-To: <wgentry@example.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org X-Old-Spam-Flag: NO X-Old-Spam-Status: No, score=-4.399 tagged_above=-10 required=6.6 tests=[ALL_TRUSTED=-1.8, BAYES_00=-2.599] Hi Nathan, This is defined in build/native/<architecture>/config.h. It is generated by autoconf during the build, and if it is missing or incorrect then you probably need to make sure that the LZO libraries and headers are in your search paths and then do a clean build. -Colin Nathan Marz wrote: > Unfortunately, setting those environment variables did not help my > issue. It appears that the "HADOOP_LZO_LIBRARY" variable is not > defined in both LzoCompressor.c and LzoDecompressor.c. Where is this > variable supposed to be set? > > > > On Sep 30, 2008, at 12:33 PM, Colin Evans wrote: > >> Hi Nathan, >> You probably need to add the Java headers to your build path as well >> - I don't know why the Mac doesn't ship with this as a default setting: >> >> export >> CPATH="/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home/include >> " >> export >> CPPFLAGS="-I/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home/include" >> >> >> >> >> >> Nathan Marz wrote: >>> Thanks for the help. I was able to get past my previous issue, but >>> the native build is still failing. Here is the end of the log output: >>> >>> [exec] then mv -f ".deps/LzoCompressor.Tpo" >>> ".deps/LzoCompressor.Plo"; else rm -f ".deps/LzoCompressor.Tpo"; >>> exit 1; fi >>> [exec] mkdir .libs >>> [exec] gcc -DHAVE_CONFIG_H -I. >>> -I/Users/nathan/Downloads/hadoop-0.18.1/src/native/src/org/apache/hadoop/io/compress/lzo >>> -I../../../../../../.. -I/Library/Java/Home//include >>> -I/Users/nathan/Downloads/hadoop-0.18.1/src/native/src -g -Wall >>> -fPIC -O2 -m32 -g -O2 -MT LzoCompressor.lo -MD -MP -MF >>> .deps/LzoCompressor.Tpo -c >>> /Users/nathan/Downloads/hadoop-0.18.1/src/native/src/org/apache/hadoop/io/compress/lzo/LzoCompressor.c >>> -fno-common -DPIC -o .libs/LzoCompressor.o >>> [exec] >>> /Users/nathan/Downloads/hadoop-0.18.1/src/native/src/org/apache/hadoop/io/compress/lzo/LzoCompressor.c: >>> In function >>> 'Java_org_apache_hadoop_io_compress_lzo_LzoCompressor_initIDs': >>> [exec] >>> /Users/nathan/Downloads/hadoop-0.18.1/src/native/src/org/apache/hadoop/io/compress/lzo/LzoCompressor.c:135: >>> error: syntax error before ',' token >>> [exec] make[2]: *** [LzoCompressor.lo] Error 1 >>> [exec] make[1]: *** [all-recursive] Error 1 >>> [exec] make: *** [all] Error 2 >>> >>> >>> Any ideas? >>> >>> >>> >>> On Sep 30, 2008, at 11:53 AM, Colin Evans wrote: >>> >>>> There's a patch to get the native targets to build on Mac OS X: >>>> >>>> http://issues.apache.org/jira/browse/HADOOP-3659 >>>> >>>> You probably will need to monkey with LDFLAGS as well to get it to >>>> work, but we've been able to build the native libs for the Mac >>>> without too much trouble. >>>> >>>> >>>> Doug Cutting wrote: >>>>> Arun C Murthy wrote: >>>>>> You need to add libhadoop.so to your java.library.patch. >>>>>> libhadoop.so is available in the corresponding release in the >>>>>> lib/native directory. >>>>> >>>>> I think he needs to first build libhadoop.so, since he appears to >>>>> be running on OS X and we only provide Linux builds of this in >>>>> releases. >>>>> >>>>> Doug >>>> >>> >> > From core-userwgentry@example.com Wed Oct 01 02:19:39 2008 Return-Path: <core-userwgentry@example.com> Delivered-To: wgentry@example.com Received: (qmail 12022 invoked from network); 1 Oct 2008 02:19:38 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Oct 2008 02:19:38 -0000 Received: (qmail 5667 invoked by uid 500); 1 Oct 2008 02:19:32 -0000 Delivered-To: wgentry@example.com Received: (qmail 5625 invoked by uid 500); 1 Oct 2008 02:19:32 -0000 Mailing-List: contact wgentry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:wgentry@example.com> List-Unsubscribe: <mailto:wgentry@example.com> List-Post: <mailto:wgentry@example.com> List-Id: <core-user.hadoop.apache.org> Reply-To: wgentry@example.com Delivered-To: mailing list wgentry@example.com Received: (qmail 5613 invoked by uid 99); 1 Oct 2008 02:19:31 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 30 Sep 2008 19:19:31 -0700 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO mrout2-b.corp.re1.yahoo.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 02:18:27 +0000 Received: from pineapple-lm.corp.yahoo.com (pineapple-lm.corp.yahoo.com [83.161.120.155]) by mrout2-b.corp.re1.yahoo.com (8.13.8/8.13.8/y.out) with ESMTP id m912IRbo046094 for <wgentry@example.com>; Tue, 30 Sep 2008 19:18:27 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns; h=message-id:from:to:in-reply-to:content-type: content-transfer-encoding:mime-version:subject:date:references:x-mailer; b=qjnsN86/NymFxJV3aSuznXXwa2MqVxrrRL+MO5pjrJKsLNHDKGtxw6MZfGTpABMC Message-Id: <wgentry@example.com> From: Chris Douglas <wgentry@example.com> To: wgentry@example.com In-Reply-To: <wgentry@example.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v929.2) Subject: Re: rename return values Date: Tue, 30 Sep 2008 19:18:26 -0700 References: <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> X-Mailer: Apple Mail (2.929.2) X-Virus-Checked: Checked by ClamAV on apache.org It's not ignored; it returns failure. Further, the point wasn't that the File API is good, but rather that the File API doesn't provide a cause for FileSystem to convert into a descriptive exception if the error originates from there (as it does in many FileSystem implementations). Finally, unlike File, the FileSystem API *permits* rename to throw IOExceptions (IIRC, it's possible to get quota and permission exceptions from rename in HDFS), but not all failures result in exceptions; application code that relies on a set of exceptions from a particular FileSystem assumes more than the interface claims. Ignoring the return value from rename causes silent failures. I'm sympathetic to the debugging burden, but if you were debugging this with LocalFileSystem, an exception would be no more descriptive than checking the return value. All that said, you can certainly emulate your preferred behavior with a FilterFileSystem that throws instead of returning false. -C On Sep 30, 2008, at 4:47 PM, Bryan Duxbury wrote: > It's very interesting that the Java File API doesn't return > exceptions, but that doesn't mean it's a good interface. The fact > that there IS further exceptional information somewhere in the > system but that it is currently ignored is sort of troubling. > Perhaps, at least, we could add an overload that WILL throw an > exception if there is one to report. > > -Bryan > > On Sep 30, 2008, at 1:53 PM, Chris Douglas wrote: > >> FileSystem888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4rename doesn't always have the cause, per >> java.io.Fil888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4renameTo: >> >> http://java.sun.com/javase/6/docs/api/java/io/File.html#renameTo(java.io.File) >> >> Even if it did, it's not clear to FileSystem that the failure to >> rename is fatal/exceptional to the application. -C >> >> On Sep 30, 2008, at 1:37 PM, Bryan Duxbury wrote: >> >>> Hey all, >>> >>> Why is it that FileSystem.rename returns true or false instead of >>> throwing an exception? It seems incredibly inconvenient to get a >>> false result and then have to go poring over the namenode logs >>> looking for the actual error message. I had this case recently >>> where I'd forgotten to create the parent directories, but I had no >>> idea it was failing since there were no exceptions. >>> >>> -Bryan >> > From core-userwgentry@example.com Wed Oct 01 03:52:57 2008 Return-Path: <core-userwgentry@example.com> Delivered-To: wgentry@example.com Received: (qmail 72177 invoked from network); 1 Oct 2008 03:52:57 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Oct 2008 03:52:57 -0000 Received: (qmail 52882 invoked by uid 500); 1 Oct 2008 03:52:50 -0000 Delivered-To: wgentry@example.com Received: (qmail 52851 invoked by uid 500); 1 Oct 2008 03:52:50 -0000 Mailing-List: contact wgentry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:wgentry@example.com> List-Unsubscribe: <mailto:wgentry@example.com> List-Post: <mailto:wgentry@example.com> List-Id: <core-user.hadoop.apache.org> Reply-To: wgentry@example.com Delivered-To: mailing list wgentry@example.com Received: (qmail 52840 invoked by uid 99); 1 Oct 2008 03:52:50 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 30 Sep 2008 20:52:50 -0700 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO mrout1-b.corp.re1.yahoo.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 03:51:46 +0000 Received: from [83.161.120.155] (ownstrong-lr.eglbp.corp.yahoo.com [83.161.120.155]) by mrout1-b.corp.re1.yahoo.com (8.13.8/8.13.8/y.out) with ESMTP id m913oQC2060986 for <wgentry@example.com>; Tue, 30 Sep 2008 20:50:27 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns; h=message-id:date:from:user-agent:mime-version:to:subject: references:in-reply-to:content-type:content-transfer-encoding; b=HpauyApP/YYMI2rs0ifZrSAGjZIpL4+9craKlLU3KKdIF2eX/lCMPOtPbl5TcgbJ Message-ID: <wgentry@example.com> Date: Wed, 01 Oct 2008 09:15:19 +0530 From: Amareshwari Sriramadasu <wgentry@example.com> User-Agent: Thunderbird 83.161.120.155 (X11/20060524) MIME-Version: 1.0 To: wgentry@example.com Subject: Re: LZO and native hadoop libraries References: <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> In-Reply-To: <wgentry@example.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org Are you seeing HADOOP-2009? Thanks Amareshwari Nathan Marz wrote: > Unfortunately, setting those environment variables did not help my > issue. It appears that the "HADOOP_LZO_LIBRARY" variable is not > defined in both LzoCompressor.c and LzoDecompressor.c. Where is this > variable supposed to be set? > > > > On Sep 30, 2008, at 12:33 PM, Colin Evans wrote: > >> Hi Nathan, >> You probably need to add the Java headers to your build path as well >> - I don't know why the Mac doesn't ship with this as a default setting: >> >> export >> CPATH="/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home/include >> " >> export >> CPPFLAGS="-I/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home/include" >> >> >> >> >> >> Nathan Marz wrote: >>> Thanks for the help. I was able to get past my previous issue, but >>> the native build is still failing. Here is the end of the log output: >>> >>> [exec] then mv -f ".deps/LzoCompressor.Tpo" >>> ".deps/LzoCompressor.Plo"; else rm -f ".deps/LzoCompressor.Tpo"; >>> exit 1; fi >>> [exec] mkdir .libs >>> [exec] gcc -DHAVE_CONFIG_H -I. >>> -I/Users/nathan/Downloads/hadoop-0.18.1/src/native/src/org/apache/hadoop/io/compress/lzo >>> -I../../../../../../.. -I/Library/Java/Home//include >>> -I/Users/nathan/Downloads/hadoop-0.18.1/src/native/src -g -Wall >>> -fPIC -O2 -m32 -g -O2 -MT LzoCompressor.lo -MD -MP -MF >>> .deps/LzoCompressor.Tpo -c >>> /Users/nathan/Downloads/hadoop-0.18.1/src/native/src/org/apache/hadoop/io/compress/lzo/LzoCompressor.c >>> -fno-common -DPIC -o .libs/LzoCompressor.o >>> [exec] >>> /Users/nathan/Downloads/hadoop-0.18.1/src/native/src/org/apache/hadoop/io/compress/lzo/LzoCompressor.c: >>> In function >>> 'Java_org_apache_hadoop_io_compress_lzo_LzoCompressor_initIDs': >>> [exec] >>> /Users/nathan/Downloads/hadoop-0.18.1/src/native/src/org/apache/hadoop/io/compress/lzo/LzoCompressor.c:135: >>> error: syntax error before ',' token >>> [exec] make[2]: *** [LzoCompressor.lo] Error 1 >>> [exec] make[1]: *** [all-recursive] Error 1 >>> [exec] make: *** [all] Error 2 >>> >>> >>> Any ideas? >>> >>> >>> >>> On Sep 30, 2008, at 11:53 AM, Colin Evans wrote: >>> >>>> There's a patch to get the native targets to build on Mac OS X: >>>> >>>> http://issues.apache.org/jira/browse/HADOOP-3659 >>>> >>>> You probably will need to monkey with LDFLAGS as well to get it to >>>> work, but we've been able to build the native libs for the Mac >>>> without too much trouble. >>>> >>>> >>>> Doug Cutting wrote: >>>>> Arun C Murthy wrote: >>>>>> You need to add libhadoop.so to your java.library.patch. >>>>>> libhadoop.so is available in the corresponding release in the >>>>>> lib/native directory. >>>>> >>>>> I think he needs to first build libhadoop.so, since he appears to >>>>> be running on OS X and we only provide Linux builds of this in >>>>> releases. >>>>> >>>>> Doug >>>> >>> >> > From core-userwgentry@example.com Wed Oct 01 06:37:21 2008 Return-Path: <core-userwgentry@example.com> Delivered-To: wgentry@example.com Received: (qmail 52752 invoked from network); 1 Oct 2008 06:37:21 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Oct 2008 06:37:21 -0000 Received: (qmail 32022 invoked by uid 500); 1 Oct 2008 06:37:15 -0000 Delivered-To: wgentry@example.com Received: (qmail 31982 invoked by uid 500); 1 Oct 2008 06:37:15 -0000 Mailing-List: contact wgentry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:wgentry@example.com> List-Unsubscribe: <mailto:wgentry@example.com> List-Post: <mailto:wgentry@example.com> List-Id: <core-user.hadoop.apache.org> Reply-To: wgentry@example.com Delivered-To: mailing list wgentry@example.com Received: (qmail 31971 invoked by uid 99); 1 Oct 2008 06:37:15 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 30 Sep 2008 23:37:15 -0700 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO mail6.cognizant.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 06:36:14 +0000 Received: from 83.161.120.155 ([83.161.120.155]) by CTSINCHNSXUC.cts.com ([83.161.120.155]) with Microsoft Exchange Server HTTP-DAV ; Wed, 1 Oct 2008 06:36:12 +0000 Received: from master by ctsinchnsxuc; 01 Oct 2008 12:07:14 +0530 Subject: too-many fetch failures From: chandra <wgentry@example.com> To: hadoop core user <wgentry@example.com> Content-Type: text/plain Content-Transfer-Encoding: 7bit Date: Wed, 01 Oct 2008 12:07:14 +0530 Message-Id: <1222843034.3092.17.camel@master> Mime-Version: 1.0 X-Mailer: Evolution 2.0.2 (2.0.2-8) X-Virus-Checked: Checked by ClamAV on apache.org hi all... i'm using hadoop-17.2 version.. i'm tryin to cluster 3 nodes. when i run applns, map tasks gets over and reduce tasks just halts.. And i get this Too-many fetch failures.. [root@master hadoop-83.161.120.155]# bin/hadoop jar word/word.jar org.myorg.WordCount input output2 08/10/01 10:56:26 INFO mapred.FileInputFormat: Total input paths to process : 1008/10/01 10:56:26 INFO mapred.JobClient: Running job: job_200810011054_0005 08/10/01 10:56:27 INFO mapred.JobClient: map 0% reduce 0% 08/10/01 10:56:31 INFO mapred.JobClient: map 40% reduce 0% 08/10/01 10:56:32 INFO mapred.JobClient: map 60% reduce 0% 08/10/01 10:56:33 INFO mapred.JobClient: map 100% reduce 0% 08/10/01 10:56:42 INFO mapred.JobClient: map 100% reduce 16% 08/10/01 10:56:47 INFO mapred.JobClient: map 100% reduce 20% 08/10/01 11:37:04 INFO mapred.JobClient: map 89% reduce 20% 08/10/01 11:37:04 INFO mapred.JobClient: Task Id : task_200810011054_0005_m_000005_0, Status : FAILED Too many fetch-failures 08/10/01 11:37:04 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog? plaintext=true&taskid=task_200810011054_0005_m_000005_0&filter=stdout 08/10/01 11:37:04 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog? plaintext=true&taskid=task_200810011054_0005_m_000005_0&filter=stderr 08/10/01 11:37:06 INFO mapred.JobClient: map 100% reduce 20% 08/10/01 11:37:14 INFO mapred.JobClient: map 100% reduce 23% 08/10/01 11:44:34 INFO mapred.JobClient: Task Id : task_200810011054_0005_m_000007_0, Status : FAILED Too many fetch-failures 08/10/01 11:44:34 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog? plaintext=true&taskid=task_200810011054_0005_m_000007_0&filter=stdout 08/10/01 11:44:34 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog? plaintext=true&taskid=task_200810011054_0005_m_000007_0&filter=stderr 08/10/01 11:44:46 INFO mapred.JobClient: map 100% reduce 26% 08/10/01 11:52:04 INFO mapred.JobClient: Task Id : task_200810011054_0005_m_000004_0, Status : FAILED Too many fetch-failures 08/10/01 11:52:04 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog? plaintext=true&taskid=task_200810011054_0005_m_000004_0&filter=stdout 08/10/01 11:52:04 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog? plaintext=true&taskid=task_200810011054_0005_m_000004_0&filter=stderr 08/10/01 11:52:06 INFO mapred.JobClient: map 89% reduce 26% 08/10/01 11:52:07 INFO mapred.JobClient: map 100% reduce 26% 08/10/01 11:52:22 INFO mapred.JobClient: map 100% reduce 30% i'm not sure where i went wrong.. kindly help in solving this.. -- Best Regards S.Chandravadana This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email or any action taken in reliance on this e-mail is strictly prohibited and may be unlawful. From core-userwgentry@example.com Wed Oct 01 17:17:45 2008 Return-Path: <core-userwgentry@example.com> Delivered-To: wgentry@example.com Received: (qmail 1974 invoked from network); 1 Oct 2008 17:17:45 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Oct 2008 17:17:45 -0000 Received: (qmail 63329 invoked by uid 500); 1 Oct 2008 17:17:39 -0000 Delivered-To: wgentry@example.com Received: (qmail 63295 invoked by uid 500); 1 Oct 2008 17:17:38 -0000 Mailing-List: contact wgentry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:wgentry@example.com> List-Unsubscribe: <mailto:wgentry@example.com> List-Post: <mailto:wgentry@example.com> List-Id: <core-user.hadoop.apache.org> Reply-To: wgentry@example.com Delivered-To: mailing list wgentry@example.com Received: (qmail 63284 invoked by uid 99); 1 Oct 2008 17:17:38 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 10:17:38 -0700 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO web35701.mail.mud.yahoo.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with SMTP; Wed, 01 Oct 2008 17:16:36 +0000 Received: (qmail 51148 invoked by uid 60001); 1 Oct 2008 17:17:09 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Message-ID; b=VYDIKDhrDTMHn/pKwYTBEN0Mb1r14N8zzmOks0HMGn92H+QQ52/V67s9+qeAsJ/N+iivEXV322VG0GWcewuLv2o2VhBXpZQdHcepsg+BOuQMHbq3JU9DK2N5sC83QN8dw9wU3amUhIgj3OUEJahhXvK3SK8jhH2ATe+px80m4xs=; X-YMail-OSG: BK2vx_MVM1kWIb0wvo.529oOKX_8ifm8pqRotCmT9tPWQhTHq1kifkD0SxFpyCuhxTXtefaMdAmOfQ5vGLyS2v4oO2k1o4tL3vq8HaBHhbhN8KOcYRNGQZV9R6ue6wGUxQSsv1D6wZCxUHZrYH_WdnbqVT5qlD8pTaIVJMZIQficTndMDmI- Received: from [83.161.120.155] by web35701.mail.mud.yahoo.com via HTTP; Wed, 01 Oct 2008 10:17:08 PDT X-Mailer: YahooMailWebService/83.161.120.155 Date: Wed, 1 Oct 2008 10:17:08 -0700 (PDT) From: "Terrence A. Pietrondi" <wgentry@example.com> Subject: architecture diagram To: wgentry@example.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Message-ID: <wgentry@example.com> X-Virus-Checked: Checked by ClamAV on apache.org I am trying to plan out my map-reduce implementation and I have some questions of where computation should be split in order to take advantage of the distributed nodes. Looking at the architecture diagram (http://hadoop.apache.org/core/images/architecture.gif), are the map boxes the major computation areas or is the reduce the major computation area? Thanks. Terrence A. Pietrondi From core-userwgentry@example.com Wed Oct 01 18:00:06 2008 Return-Path: <core-userwgentry@example.com> Delivered-To: wgentry@example.com Received: (qmail 29920 invoked from network); 1 Oct 2008 18:00:05 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Oct 2008 18:00:05 -0000 Received: (qmail 18720 invoked by uid 500); 1 Oct 2008 17:59:59 -0000 Delivered-To: wgentry@example.com Received: (qmail 18676 invoked by uid 500); 1 Oct 2008 17:59:59 -0000 Mailing-List: contact wgentry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:wgentry@example.com> List-Unsubscribe: <mailto:wgentry@example.com> List-Post: <mailto:wgentry@example.com> List-Id: <core-user.hadoop.apache.org> Reply-To: wgentry@example.com Delivered-To: mailing list wgentry@example.com Received: (qmail 18665 invoked by uid 99); 1 Oct 2008 17:59:59 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 10:59:59 -0700 X-ASF-Spam-Status: No, hits=2.0 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of wgentry@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO fk-out-0910.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 17:58:58 +0000 Received: by fk-out-0910.google.com with SMTP id 26so208302fkx.13 for <wgentry@example.com>; Wed, 01 Oct 2008 10:59:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:in-reply-to:mime-version:content-type:references; bh=MIcTpKDHx48REHvLh5v+QMaDiBwSlS1WY1bMtPgZ5Vo=; b=RRr00zHhPXAJzz6d6/w65OqnVCamEZ3z1Ib3XFYWB/xYAvSOA+OGQRqlAR+YbOJks9 kvUqMEZu/3TO33AgRCt5SDcVPOnFialoZsfOn2czwxi638zJQoL0FkaJLEfc3JFzWe68 WtT7KHtI5n+V3rED9dVIHjMGcIpjrsrBGnckE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version :content-type:references; b=tR+qZ4rs5oPHQR0nC3c2kZDydWwfL621tuwMQk5DnlBZX2kaQlgQxoeSDr+mzUSe43 bqr+ei+haitwgkGEj1GfOPwEGsHJiivwr4xk0cxhK4Z1w2yPshDzTPkJCtxwYyaL9t8g 1UgpSy2I7Ky3mTy67KVtziApFEqv7NYJwtOK0= Received: by 83.161.120.155 with SMTP id h8mr4678034bkg.10.1222883971307; Wed, 01 Oct 2008 10:59:31 -0700 (PDT) Received: by 83.161.120.155 with HTTP; Wed, 1 Oct 2008 10:59:31 -0700 (PDT) Message-ID: <wgentry@example.com> Date: Wed, 1 Oct 2008 10:59:31 -0700 From: "Alex Loddengaard" <wgentry@example.com> To: wgentry@example.com Subject: Re: architecture diagram In-Reply-To: <wgentry@example.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_14543_5559412.1222883971305" References: <wgentry@example.com> X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_14543_5559412.1222883971305 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi Terrence, It really depends on your job I think. Often reduce steps can be the bottleneck if you want a single output file (one reducer). Hope this helps. Alex On Wed, Oct 1, 2008 at 10:17 AM, Terrence A. Pietrondi < wgentry@example.com> wrote: > I am trying to plan out my map-reduce implementation and I have some > questions of where computation should be split in order to take advantage of > the distributed nodes. > > Looking at the architecture diagram ( > http://hadoop.apache.org/core/images/architecture.gif), are the map boxes > the major computation areas or is the reduce the major computation area? > > Thanks. > > Terrence A. Pietrondi > > > > ------=_Part_14543_5559412.1222883971305-- From core-userwgentry@example.com Wed Oct 01 18:06:40 2008 Return-Path: <core-userwgentry@example.com> Delivered-To: wgentry@example.com Received: (qmail 33227 invoked from network); 1 Oct 2008 18:06:39 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Oct 2008 18:06:39 -0000 Received: (qmail 27345 invoked by uid 500); 1 Oct 2008 18:06:32 -0000 Delivered-To: wgentry@example.com Received: (qmail 27316 invoked by uid 500); 1 Oct 2008 18:06:32 -0000 Mailing-List: contact wgentry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:wgentry@example.com> List-Unsubscribe: <mailto:wgentry@example.com> List-Post: <mailto:wgentry@example.com> List-Id: <core-user.hadoop.apache.org> Reply-To: wgentry@example.com Delivered-To: mailing list wgentry@example.com Received: (qmail 27305 invoked by uid 99); 1 Oct 2008 18:06:32 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 11:06:32 -0700 X-ASF-Spam-Status: No, hits=2.4 required=10.0 tests=RCVD_IN_BL_SPAMCOP_NET,SPF_HELO_FAIL,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO megamail.netsight.co.uk) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 18:05:30 +0000 Received: from nat76.mia.three.co.uk ([83.161.120.155] helo=[83.161.120.155]) by megamail.netsight.co.uk with esmtp (Exim 4.63 (FreeBSD)) (envelope-from <wgentry@example.com>) id 1Kl65K-000GOy-OQ for wgentry@example.com; Wed, 01 Oct 2008 19:06:03 +0100 Subject: Re: architecture diagram From: Tim Wintle <wgentry@example.com> Reply-To: wgentry@example.com To: wgentry@example.com In-Reply-To: <wgentry@example.com> References: <wgentry@example.com> <wgentry@example.com> Content-Type: text/plain Organization: Team Rubber Date: Wed, 01 Oct 2008 19:06:00 +0100 Message-Id: <1222884360.18156.6.camel@tim-laptop> Mime-Version: 1.0 X-Mailer: Evolution 83.161.120.155 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org I normally find the intermediate stage of copying data to the reducers from the mappers to be a significant step - but that's not over the best quality switches... The mappers and reducers work on the same boxes, close to the data. On Wed, 2008-10-01 at 10:59 -0700, Alex Loddengaard wrote: > > It really depends on your job I think. Often reduce steps can be the > bottleneck if you want a single output file (one reducer). > > Hope this helps. > > Alex From core-userwgentry@example.com Wed Oct 01 18:08:11 2008 Return-Path: <core-userwgentry@example.com> Delivered-To: wgentry@example.com Received: (qmail 33649 invoked from network); 1 Oct 2008 18:08:11 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Oct 2008 18:08:11 -0000 Received: (qmail 30137 invoked by uid 500); 1 Oct 2008 18:08:06 -0000 Delivered-To: wgentry@example.com Received: (qmail 29506 invoked by uid 500); 1 Oct 2008 18:08:04 -0000 Mailing-List: contact wgentry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:wgentry@example.com> List-Unsubscribe: <mailto:wgentry@example.com> List-Post: <mailto:wgentry@example.com> List-Id: <core-user.hadoop.apache.org> Reply-To: wgentry@example.com Delivered-To: mailing list wgentry@example.com Received: (qmail 29495 invoked by uid 99); 1 Oct 2008 18:08:04 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 11:08:04 -0700 X-ASF-Spam-Status: No, hits=3.2 required=10.0 tests=HTML_MESSAGE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO fg-out-1718.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 18:07:03 +0000 Received: by fg-out-1718.google.com with SMTP id l26so449120fgb.35 for <wgentry@example.com>; Wed, 01 Oct 2008 11:07:19 -0700 (PDT) Received: by 83.161.120.155 with SMTP id l8mr4664213bkj.28.1222884439214; Wed, 01 Oct 2008 11:07:19 -0700 (PDT) Received: by 83.161.120.155 with HTTP; Wed, 1 Oct 2008 11:07:19 -0700 (PDT) Message-ID: <wgentry@example.com> Date: Wed, 1 Oct 2008 11:07:19 -0700 From: "Per Jacobsson" <wgentry@example.com> To: wgentry@example.com Subject: Merging of the local FS files threw an exception MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_13956_31532283.1222884439205" X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_13956_31532283.1222884439205 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi everyone, (apologies if this gets posted on the list twice for some reason, my first attempt was denied as "suspected spam") I ran a job last night with Hadoop 0.18.0 on EC2, using the standard small AMI. The job was producing gzipped output, otherwise I haven't changed the configuration. The final reduce steps failed with this error that I haven't seem before: 2008-10-01 05:02:39,810 WARN org.apache.hadoop.mapred.ReduceTask: attempt_200809301822_0005_r_000001_0 Merging of the local FS files threw an exception: java.io.IOException: java.io.IOException: Rec# 289050: Negative value-length: -wgentry@example.com$Reader.next(IFile.java:331) at org.apache.hadoop.mapred.Merger$Segment.next(Merger.java:134) at org.apache.hadoop.mapred.Merger$MergeQueue.adjustPriorityQueue(Merger.java:225) at org.apache.hadoop.mapred.Merger$MergeQueue.next(Merger.java:242) at org.apache.hadoop.mapred.Merger.writeFile(Merger.java:83) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$LocalFSMerger.run(ReduceTask.java:2021) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$LocalFSMerger.run(ReduceTask.java:2025) 2008-10-01 05:02:44,131 WARN org.apache.hadoop.mapred.TaskTracker: Error running child java.io.IOException: attempt_200809301822_0005_r_000001_0The reduce copier wgentry@example.com(ReduceTask.java:255) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2209) When I try to download the data from HDFS I get a "Found checksum error" warning message. Any ideas what could be the cause? Would upgrading to 0.18.1 solve it? Thanks, / Per ------=_Part_13956_31532283.1222884439205-- From core-userwgentry@example.com Wed Oct 01 18:18:24 2008 Return-Path: <core-userwgentry@example.com> Delivered-To: wgentry@example.com Received: (qmail 37218 invoked from network); 1 Oct 2008 18:18:24 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Oct 2008 18:18:24 -0000 Received: (qmail 45167 invoked by uid 500); 1 Oct 2008 18:18:17 -0000 Delivered-To: wgentry@example.com Received: (qmail 45126 invoked by uid 500); 1 Oct 2008 18:18:17 -0000 Mailing-List: contact wgentry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:wgentry@example.com> List-Unsubscribe: <mailto:wgentry@example.com> List-Post: <mailto:wgentry@example.com> List-Id: <core-user.hadoop.apache.org> Reply-To: wgentry@example.com Delivered-To: mailing list wgentry@example.com Received: (qmail 45115 invoked by uid 99); 1 Oct 2008 18:18:17 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 11:18:17 -0700 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO mrout2.yahoo.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 18:17:12 +0000 Received: from walkduty-lm.corp.yahoo.com (walkduty-lm.corp.yahoo.com [83.161.120.155]) by mrout2.yahoo.com (8.13.6/8.13.6/y.out) with ESMTP id m91IGDWQ083127 for <wgentry@example.com>; Wed, 1 Oct 2008 11:16:13 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns; h=message-id:from:to:in-reply-to:content-type: content-transfer-encoding:mime-version:subject:date:references:x-mailer; b=uITYMw17qa8tl/PikSMdOzBJhB+hDAXPCujQ4DJA3tIc63BkFnuErZKCKqaxRtrg Message-Id: <wgentry@example.com> From: Arun C Murthy <wgentry@example.com> To: wgentry@example.com In-Reply-To: <wgentry@example.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v929.2) Subject: Re: architecture diagram Date: Wed, 1 Oct 2008 11:16:13 -0700 References: <wgentry@example.com> X-Mailer: Apple Mail (2.929.2) X-Virus-Checked: Checked by ClamAV on apache.org On Oct 1, 2008, at 10:17 AM, Terrence A. Pietrondi wrote: > I am trying to plan out my map-reduce implementation and I have some > questions of where computation should be split in order to take > advantage of the distributed nodes. > > Looking at the architecture diagram (http://hadoop.apache.org/core/images/architecture.gif > ), are the map boxes the major computation areas or is the reduce > the major computation area? > Usually the maps perform the 'embarrassingly parallel' computational steps where-in each map works independently on a 'split' on your input and the reduces perform the 'aggregate' computations. From http://hadoop.apache.org/core/ : Hadoop implements MapReduce, using the Hadoop Distributed File System (HDFS). MapReduce divides applications into many small blocks of work. HDFS creates multiple replicas of data blocks for reliability, placing them on compute nodes around the cluster. MapReduce can then process the data where it is located. The Hadoop Map-Reduce framework is quite good at scheduling your 'maps' on the actual data-nodes where the input-blocks are present, leading to i/o efficiencies... Arun > Thanks. > > Terrence A. Pietrondi > > > From core-userwgentry@example.com Wed Oct 01 18:24:50 2008 Return-Path: <core-userwgentry@example.com> Delivered-To: wgentry@example.com Received: (qmail 48391 invoked from network); 1 Oct 2008 18:24:49 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Oct 2008 18:24:49 -0000 Received: (qmail 52253 invoked by uid 500); 1 Oct 2008 18:24:42 -0000 Delivered-To: wgentry@example.com Received: (qmail 52223 invoked by uid 500); 1 Oct 2008 18:24:42 -0000 Mailing-List: contact wgentry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:wgentry@example.com> List-Unsubscribe: <mailto:wgentry@example.com> List-Post: <mailto:wgentry@example.com> List-Id: <core-user.hadoop.apache.org> Reply-To: wgentry@example.com Delivered-To: mailing list wgentry@example.com Received: (qmail 52212 invoked by uid 99); 1 Oct 2008 18:24:42 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 11:24:42 -0700 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO mrout2.yahoo.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 18:23:38 +0000 Received: from walkduty-lm.corp.yahoo.com (walkduty-lm.corp.yahoo.com [83.161.120.155]) by mrout2.yahoo.com (8.13.6/8.13.6/y.out) with ESMTP id m91INhkA086301 for <wgentry@example.com>; Wed, 1 Oct 2008 11:23:43 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns; h=message-id:from:to:in-reply-to:content-type: content-transfer-encoding:mime-version:subject:date:references:x-mailer; b=FxE/DnPgrJvUfCFyw91EwYlnHQAX9zkeO/PNxgUIwSzeJHlWdUtNfZUlS9ZjxfeI Message-Id: <wgentry@example.com> From: Arun C Murthy <wgentry@example.com> To: wgentry@example.com In-Reply-To: <wgentry@example.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v929.2) Subject: Re: Merging of the local FS files threw an exception Date: Wed, 1 Oct 2008 11:23:43 -0700 References: <wgentry@example.com> X-Mailer: Apple Mail (2.929.2) X-Virus-Checked: Checked by ClamAV on apache.org On Oct 1, 2008, at 11:07 AM, Per Jacobsson wrote: > I ran a job last night with Hadoop 0.18.0 on EC2, using the standard > small > AMI. The job was producing gzipped output, otherwise I haven't > changed the > configuration. > > The final reduce steps failed with this error that I haven't seem > before: > > 2008-10-01 05:02:39,810 WARN org.apache.hadoop.mapred.ReduceTask: > attempt_200809301822_0005_r_000001_0 Merging of the local FS files > threw an > exception: java.io.IOException: java.io.IOException: Rec# 289050: > Negative > value-length: -96 Do you still have the task logs for the reduce? I suspect are running into http://issues.apache.org/jira/browse/HADOOP-3647 which we never could reproduce reliably to pin it down or fix. However, in light of http://issues.apache.org/jira/browse/HADOOP-4277 we suspect this could be caused by a bug in the LocalFileSystem which could hide data-corruption on your local disk leading to errors on these nature. Could you try running your job with that patch once the release 0.18.2 is available? Any information you provide could greatly aid to confirm our above hypothesis, so it's much appreciated! Arun From core-userwgentry@example.com Wed Oct 01 19:05:26 2008 Return-Path: <core-userwgentry@example.com> Delivered-To: wgentry@example.com Received: (qmail 82177 invoked from network); 1 Oct 2008 19:05:26 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Oct 2008 19:05:26 -0000 Received: (qmail 4438 invoked by uid 500); 1 Oct 2008 19:05:20 -0000 Delivered-To: wgentry@example.com Received: (qmail 4274 invoked by uid 500); 1 Oct 2008 19:05:19 -0000 Mailing-List: contact wgentry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:wgentry@example.com> List-Unsubscribe: <mailto:wgentry@example.com> List-Post: <mailto:wgentry@example.com> List-Id: <core-user.hadoop.apache.org> Reply-To: wgentry@example.com Delivered-To: mailing list wgentry@example.com Received: (qmail 4263 invoked by uid 99); 1 Oct 2008 19:05:19 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 12:05:19 -0700 X-ASF-Spam-Status: No, hits=3.2 required=10.0 tests=HTML_MESSAGE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO fk-out-0910.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 19:04:18 +0000 Received: by fk-out-0910.google.com with SMTP id 26so229735fkx.13 for <wgentry@example.com>; Wed, 01 Oct 2008 12:04:40 -0700 (PDT) Received: by 83.161.120.155 with SMTP id a13mr4757982bki.42.1222887879966; Wed, 01 Oct 2008 12:04:39 -0700 (PDT) Received: by 83.161.120.155 with HTTP; Wed, 1 Oct 2008 12:04:39 -0700 (PDT) Message-ID: <wgentry@example.com> Date: Wed, 1 Oct 2008 12:04:39 -0700 From: "Per Jacobsson" <wgentry@example.com> To: wgentry@example.com Subject: Re: Merging of the local FS files threw an exception In-Reply-To: <wgentry@example.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_15092_6487383.1222887879955" References: <wgentry@example.com> <wgentry@example.com> X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_15092_6487383.1222887879955 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline I've collected the syslogs from the failed reduce jobs. What's the best way to get them to you? Let me know if you need anything else, I'll have to shut down these instances some time later today. Overall I've run this same job before with no problems. The only change is the added gzip of the output. Don't know if it's worth anything, but the four failures all happened on different machines. I'll be running this job plenty of times so if the problem keeps happening it will be obvious. / Per On Wed, Oct 1, 2008 at 11:23 AM, Arun C Murthy <wgentry@example.com> wrote: > > Do you still have the task logs for the reduce? > > I suspect are running into > http://issues.apache.org/jira/browse/HADOOP-3647 which we never could > reproduce reliably to pin it down or fix. > > However, in light of http://issues.apache.org/jira/browse/HADOOP-4277 we > suspect this could be caused by a bug in the LocalFileSystem which could > hide data-corruption on your local disk leading to errors on these nature. > Could you try running your job with that patch once the release 0.18.2 is > available? > > Any information you provide could greatly aid to confirm our above > hypothesis, so it's much appreciated! > > Arun > > ------=_Part_15092_6487383.1222887879955-- From core-userwgentry@example.com Wed Oct 01 19:55:23 2008 Return-Path: <core-userwgentry@example.com> Delivered-To: wgentry@example.com Received: (qmail 6918 invoked from network); 1 Oct 2008 19:55:23 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Oct 2008 19:55:23 -0000 Received: (qmail 77099 invoked by uid 500); 1 Oct 2008 19:55:18 -0000 Delivered-To: wgentry@example.com Received: (qmail 76140 invoked by uid 500); 1 Oct 2008 19:55:16 -0000 Mailing-List: contact wgentry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:wgentry@example.com> List-Unsubscribe: <mailto:wgentry@example.com> List-Post: <mailto:wgentry@example.com> List-Id: <core-user.hadoop.apache.org> Reply-To: wgentry@example.com Delivered-To: mailing list wgentry@example.com Received: (qmail 76129 invoked by uid 99); 1 Oct 2008 19:55:16 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 12:55:16 -0700 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of wgentry@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail.rapleaf.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 19:54:15 +0000 Received: from mail.rapleaf.com (localhost.localdomain [83.161.120.155]) by mail.rapleaf.com (Postfix) with ESMTP id 6C321125033F for <wgentry@example.com>; Wed, 1 Oct 2008 12:54:49 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=rapleaf.com; q=dns; s=m1; b=fh5YZ p3AXVGR/wdjRCz3ScyNIp72RQztH0hw/njsKVgfwM7DhMl7O9HAPz1h+drxVOk9Z o3HQ6or5593tvkVwTLVy+sjwH638TWtvWrRrQJG9mD58c/ZvbccoTX4J1vicaMtV nzRtmQ9NMgdmjqAfPJ+GRIwFTOa6VSIa6b/EDY= Received: from [83.161.120.155] (unknown [83.161.120.155]) by mail.rapleaf.com (Postfix) with ESMTP id 45E6F125017C for <wgentry@example.com>; Wed, 1 Oct 2008 12:54:49 -0700 (PDT) Message-Id: <wgentry@example.com> From: Nathan Marz <wgentry@example.com> To: wgentry@example.com In-Reply-To: <wgentry@example.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v926) Subject: Re: LZO and native hadoop libraries Date: Wed, 1 Oct 2008 12:54:19 -0700 References: <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> X-Mailer: Apple Mail (2.926) X-Virus-Checked: Checked by ClamAV on apache.org Yes, this is exactly what I'm seeing. To be honest, I don't know which LZO native library it should be looking for. The LZO install dropped "liblzo2.la" and "liblzo2.a" in my /usr/local/lib directory, but not a file with a ".so" extension. Hardcoding would be fine as a temporary solution, but I don't know what to hardcode. Thanks, Nathan On Sep 30, 2008, at 8:45 PM, Amareshwari Sriramadasu wrote: > Are you seeing HADOOP-2009? > > Thanks > Amareshwari > Nathan Marz wrote: >> Unfortunately, setting those environment variables did not help my >> issue. It appears that the "HADOOP_LZO_LIBRARY" variable is not >> defined in both LzoCompressor.c and LzoDecompressor.c. Where is >> this variable supposed to be set? >> >> >> >> On Sep 30, 2008, at 12:33 PM, Colin Evans wrote: >> >>> Hi Nathan, >>> You probably need to add the Java headers to your build path as >>> well - I don't know why the Mac doesn't ship with this as a >>> default setting: >>> >>> export CPATH="/System/Library/Frameworks/JavaVM.framework/Versions/ >>> CurrentJDK/Home/include " >>> export CPPFLAGS="-I/System/Library/Frameworks/JavaVM.framework/ >>> Versions/CurrentJDK/Home/include" >>> >>> >>> >>> >>> Nathan Marz wrote: >>>> Thanks for the help. I was able to get past my previous issue, >>>> but the native build is still failing. Here is the end of the log >>>> output: >>>> >>>> [exec] then mv -f ".deps/LzoCompressor.Tpo" ".deps/ >>>> LzoCompressor.Plo"; else rm -f ".deps/LzoCompressor.Tpo"; exit 1; >>>> fi >>>> [exec] mkdir .libs >>>> [exec] gcc -DHAVE_CONFIG_H -I. -I/Users/nathan/Downloads/ >>>> hadoop-0.18.1/src/native/src/org/apache/hadoop/io/compress/lzo - >>>> I../../../../../../.. -I/Library/Java/Home//include -I/Users/ >>>> nathan/Downloads/hadoop-0.18.1/src/native/src -g -Wall -fPIC -O2 - >>>> m32 -g -O2 -MT LzoCompressor.lo -MD -MP -MF .deps/ >>>> LzoCompressor.Tpo -c /Users/nathan/Downloads/hadoop-0.18.1/src/ >>>> native/src/org/apache/hadoop/io/compress/lzo/LzoCompressor.c - >>>> fno-common -DPIC -o .libs/LzoCompressor.o >>>> [exec] /Users/nathan/Downloads/hadoop-0.18.1/src/native/src/ >>>> org/apache/hadoop/io/compress/lzo/LzoCompressor.c: In function >>>> 'Java_org_apache_hadoop_io_compress_lzo_LzoCompressor_initIDs': >>>> [exec] /Users/nathan/Downloads/hadoop-0.18.1/src/native/src/ >>>> org/apache/hadoop/io/compress/lzo/LzoCompressor.c:135: error: >>>> syntax error before ',' token >>>> [exec] make[2]: *** [LzoCompressor.lo] Error 1 >>>> [exec] make[1]: *** [all-recursive] Error 1 >>>> [exec] make: *** [all] Error 2 >>>> >>>> >>>> Any ideas? >>>> >>>> >>>> >>>> On Sep 30, 2008, at 11:53 AM, Colin Evans wrote: >>>> >>>>> There's a patch to get the native targets to build on Mac OS X: >>>>> >>>>> http://issues.apache.org/jira/browse/HADOOP-3659 >>>>> >>>>> You probably will need to monkey with LDFLAGS as well to get it >>>>> to work, but we've been able to build the native libs for the >>>>> Mac without too much trouble. >>>>> >>>>> >>>>> Doug Cutting wrote: >>>>>> Arun C Murthy wrote: >>>>>>> You need to add libhadoop.so to your java.library.patch. >>>>>>> libhadoop.so is available in the corresponding release in the >>>>>>> lib/native directory. >>>>>> >>>>>> I think he needs to first build libhadoop.so, since he appears >>>>>> to be running on OS X and we only provide Linux builds of this >>>>>> in releases. >>>>>> >>>>>> Doug >>>>> >>>> >>> >> > From core-userwgentry@example.com Wed Oct 01 20:12:56 2008 Return-Path: <core-userwgentry@example.com> Delivered-To: wgentry@example.com Received: (qmail 16987 invoked from network); 1 Oct 2008 20:12:55 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Oct 2008 20:12:55 -0000 Received: (qmail 98309 invoked by uid 500); 1 Oct 2008 20:12:49 -0000 Delivered-To: wgentry@example.com Received: (qmail 98026 invoked by uid 500); 1 Oct 2008 20:12:48 -0000 Mailing-List: contact wgentry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:wgentry@example.com> List-Unsubscribe: <mailto:wgentry@example.com> List-Post: <mailto:wgentry@example.com> List-Id: <core-user.hadoop.apache.org> Reply-To: wgentry@example.com Delivered-To: mailing list wgentry@example.com Received: (qmail 98015 invoked by uid 99); 1 Oct 2008 20:12:48 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 13:12:48 -0700 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO web35705.mail.mud.yahoo.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with SMTP; Wed, 01 Oct 2008 20:11:44 +0000 Received: (qmail 97934 invoked by uid 60001); 1 Oct 2008 20:11:16 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Message-ID; b=oslOXHRpFEbZ/UzFPMOdhAw1hTLhedsjGU7lpJNzzd8qnnPHueY7WP3bIq64Iov5h8gWuS/FtfegiJRL0BsxJJW7ixt9FRr904VdPPL7MSuK3PDILFQonNXrJ3fsv2t6t9p7L5SeAca+o3M2DVs29KRypwJKaopM4nG1fow0E2U=; X-YMail-OSG: FRmuLsgVM1mia0j37x.5WDoqLsGdz4Ov25_lsDGD7YT2r5ygvLZVwTrxDMQfrN56BQ-- Received: from [83.161.120.155] by web35705.mail.mud.yahoo.com via HTTP; Wed, 01 Oct 2008 13:11:16 PDT X-Mailer: YahooMailWebService/83.161.120.155 Date: Wed, 1 Oct 2008 13:11:16 -0700 (PDT) From: "Terrence A. Pietrondi" <wgentry@example.com> Subject: Re: architecture diagram To: wgentry@example.com In-Reply-To: <wgentry@example.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Message-ID: <wgentry@example.com> X-Virus-Checked: Checked by ClamAV on apache.org So to be "distributed" in a sense, you would want to do your computation on the disconnected parts of data in the map phase I would guess? Terrence A. Pietrondi http://del.icio.us/tepietrondi --- On Wed, 10/1/08, Arun C Murthy <wgentry@example.com> wrote: > From: Arun C Murthy <wgentry@example.com> > Subject: Re: architecture diagram > To: wgentry@example.com > Date: Wednesday, October 1, 2008, 2:16 PM > On Oct 1, 2008, at 10:17 AM, Terrence A. Pietrondi wrote: > > > I am trying to plan out my map-reduce implementation > and I have some > > questions of where computation should be split in > order to take > > advantage of the distributed nodes. > > > > Looking at the architecture diagram > (http://hadoop.apache.org/core/images/architecture.gif > > ), are the map boxes the major computation areas or is > the reduce > > the major computation area? > > > > Usually the maps perform the 'embarrassingly > parallel' computational > steps where-in each map works independently on a > 'split' on your input > and the reduces perform the 'aggregate' > computations. > > From http://hadoop.apache.org/core/ : > > Hadoop implements MapReduce, using the Hadoop Distributed > File System > (HDFS). MapReduce divides applications into many small > blocks of work. > HDFS creates multiple replicas of data blocks for > reliability, placing > them on compute nodes around the cluster. MapReduce can > then process > the data where it is located. > > The Hadoop Map-Reduce framework is quite good at scheduling > your > 'maps' on the actual data-nodes where the > input-blocks are present, > leading to i/o efficiencies... > > Arun > > > Thanks. > > > > Terrence A. Pietrondi > > > > > > From core-userwgentry@example.com Wed Oct 01 20:36:21 2008 Return-Path: <core-userwgentry@example.com> Delivered-To: wgentry@example.com Received: (qmail 29652 invoked from network); 1 Oct 2008 20:36:20 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Oct 2008 20:36:20 -0000 Received: (qmail 17682 invoked by uid 500); 1 Oct 2008 20:36:14 -0000 Delivered-To: wgentry@example.com Received: (qmail 17654 invoked by uid 500); 1 Oct 2008 20:36:14 -0000 Mailing-List: contact wgentry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:wgentry@example.com> List-Unsubscribe: <mailto:wgentry@example.com> List-Post: <mailto:wgentry@example.com> List-Id: <core-user.hadoop.apache.org> Reply-To: wgentry@example.com Delivered-To: mailing list wgentry@example.com Received: (qmail 17643 invoked by uid 99); 1 Oct 2008 20:36:14 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 13:36:14 -0700 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO mrout2-b.corp.re1.yahoo.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 20:35:11 +0000 Received: from walkduty-lm.corp.yahoo.com (walkduty-lm.corp.yahoo.com [83.161.120.155]) by mrout2-b.corp.re1.yahoo.com (8.13.8/8.13.8/y.out) with ESMTP id m91KYjC7086342 for <wgentry@example.com>; Wed, 1 Oct 2008 13:34:45 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns; h=message-id:from:to:in-reply-to:content-type: content-transfer-encoding:mime-version:subject:date:references:x-mailer; b=QtKxwgFBVS1AkE/gJ+lO6wgEzXxVas/pRxo4mYPJBsTmTSb2GEC9sgLpz8FzUo2q Message-Id: <wgentry@example.com> From: Arun C Murthy <wgentry@example.com> To: wgentry@example.com In-Reply-To: <wgentry@example.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v929.2) Subject: Re: LZO and native hadoop libraries Date: Wed, 1 Oct 2008 13:34:44 -0700 References: <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> X-Mailer: Apple Mail (2.929.2) X-Virus-Checked: Checked by ClamAV on apache.org On Oct 1, 2008, at 12:54 PM, Nathan Marz wrote: > Yes, this is exactly what I'm seeing. To be honest, I don't know > which LZO native library it should be looking for. The LZO install > dropped "liblzo2.la" and "liblzo2.a" in my /usr/local/lib directory, > but not a file with a ".so" extension. Hardcoding would be fine as a > temporary solution, but I don't know what to hardcode. > You do need liblzo2.so for this to work. The hardcoded value has to be liblzo2.so too ... Arun > Thanks, > Nathan > > > On Sep 30, 2008, at 8:45 PM, Amareshwari Sriramadasu wrote: > >> Are you seeing HADOOP-2009? >> >> Thanks >> Amareshwari >> Nathan Marz wrote: >>> Unfortunately, setting those environment variables did not help my >>> issue. It appears that the "HADOOP_LZO_LIBRARY" variable is not >>> defined in both LzoCompressor.c and LzoDecompressor.c. Where is >>> this variable supposed to be set? >>> >>> >>> >>> On Sep 30, 2008, at 12:33 PM, Colin Evans wrote: >>> >>>> Hi Nathan, >>>> You probably need to add the Java headers to your build path as >>>> well - I don't know why the Mac doesn't ship with this as a >>>> default setting: >>>> >>>> export CPATH="/System/Library/Frameworks/JavaVM.framework/ >>>> Versions/CurrentJDK/Home/include " >>>> export CPPFLAGS="-I/System/Library/Frameworks/JavaVM.framework/ >>>> Versions/CurrentJDK/Home/include" >>>> >>>> >>>> >>>> >>>> Nathan Marz wrote: >>>>> Thanks for the help. I was able to get past my previous issue, >>>>> but the native build is still failing. Here is the end of the >>>>> log output: >>>>> >>>>> [exec] then mv -f ".deps/LzoCompressor.Tpo" ".deps/ >>>>> LzoCompressor.Plo"; else rm -f ".deps/LzoCompressor.Tpo"; exit >>>>> 1; fi >>>>> [exec] mkdir .libs >>>>> [exec] gcc -DHAVE_CONFIG_H -I. -I/Users/nathan/Downloads/ >>>>> hadoop-0.18.1/src/native/src/org/apache/hadoop/io/compress/lzo - >>>>> I../../../../../../.. -I/Library/Java/Home//include -I/Users/ >>>>> nathan/Downloads/hadoop-0.18.1/src/native/src -g -Wall -fPIC -O2 >>>>> -m32 -g -O2 -MT LzoCompressor.lo -MD -MP -MF .deps/ >>>>> LzoCompressor.Tpo -c /Users/nathan/Downloads/hadoop-0.18.1/src/ >>>>> native/src/org/apache/hadoop/io/compress/lzo/LzoCompressor.c - >>>>> fno-common -DPIC -o .libs/LzoCompressor.o >>>>> [exec] /Users/nathan/Downloads/hadoop-0.18.1/src/native/src/ >>>>> org/apache/hadoop/io/compress/lzo/LzoCompressor.c: In function >>>>> 'Java_org_apache_hadoop_io_compress_lzo_LzoCompressor_initIDs': >>>>> [exec] /Users/nathan/Downloads/hadoop-0.18.1/src/native/src/ >>>>> org/apache/hadoop/io/compress/lzo/LzoCompressor.c:135: error: >>>>> syntax error before ',' token >>>>> [exec] make[2]: *** [LzoCompressor.lo] Error 1 >>>>> [exec] make[1]: *** [all-recursive] Error 1 >>>>> [exec] make: *** [all] Error 2 >>>>> >>>>> >>>>> Any ideas? >>>>> >>>>> >>>>> >>>>> On Sep 30, 2008, at 11:53 AM, Colin Evans wrote: >>>>> >>>>>> There's a patch to get the native targets to build on Mac OS X: >>>>>> >>>>>> http://issues.apache.org/jira/browse/HADOOP-3659 >>>>>> >>>>>> You probably will need to monkey with LDFLAGS as well to get it >>>>>> to work, but we've been able to build the native libs for the >>>>>> Mac without too much trouble. >>>>>> >>>>>> >>>>>> Doug Cutting wrote: >>>>>>> Arun C Murthy wrote: >>>>>>>> You need to add libhadoop.so to your java.library.patch. >>>>>>>> libhadoop.so is available in the corresponding release in the >>>>>>>> lib/native directory. >>>>>>> >>>>>>> I think he needs to first build libhadoop.so, since he appears >>>>>>> to be running on OS X and we only provide Linux builds of this >>>>>>> in releases. >>>>>>> >>>>>>> Doug >>>>>> >>>>> >>>> >>> >> > From core-userwgentry@example.com Wed Oct 01 20:36:50 2008 Return-Path: <core-userwgentry@example.com> Delivered-To: wgentry@example.com Received: (qmail 29979 invoked from network); 1 Oct 2008 20:36:50 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Oct 2008 20:36:50 -0000 Received: (qmail 20069 invoked by uid 500); 1 Oct 2008 20:36:43 -0000 Delivered-To: wgentry@example.com Received: (qmail 20039 invoked by uid 500); 1 Oct 2008 20:36:43 -0000 Mailing-List: contact wgentry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:wgentry@example.com> List-Unsubscribe: <mailto:wgentry@example.com> List-Post: <mailto:wgentry@example.com> List-Id: <core-user.hadoop.apache.org> Reply-To: wgentry@example.com Delivered-To: mailing list wgentry@example.com Received: (qmail 20028 invoked by uid 99); 1 Oct 2008 20:36:43 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 13:36:43 -0700 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO mrout2-b.corp.re1.yahoo.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 20:35:39 +0000 Received: from walkduty-lm.corp.yahoo.com (walkduty-lm.corp.yahoo.com [83.161.120.155]) by mrout2-b.corp.re1.yahoo.com (8.13.8/8.13.8/y.out) with ESMTP id m91KXZn7085846 for <wgentry@example.com>; Wed, 1 Oct 2008 13:33:36 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns; h=message-id:from:to:in-reply-to:content-type: content-transfer-encoding:mime-version:subject:date:references:x-mailer; b=T2vHbCfq+XGC+kzM4TBi2ztMCi1bnQc3JUTLjAheDoZDL75gTnGeHV/q2TnNhLTp Message-Id: <wgentry@example.com> From: Arun C Murthy <wgentry@example.com> To: wgentry@example.com In-Reply-To: <wgentry@example.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v929.2) Subject: Re: Merging of the local FS files threw an exception Date: Wed, 1 Oct 2008 13:33:35 -0700 References: <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> X-Mailer: Apple Mail (2.929.2) X-Virus-Checked: Checked by ClamAV on apache.org On Oct 1, 2008, at 12:04 PM, Per Jacobsson wrote: > I've collected the syslogs from the failed reduce jobs. What's the > best way > to get them to you? Let me know if you need anything else, I'll have > to shut > down these instances some time later today. > Could you please attach them to the jira: http://issues.apache.org/jira/browse/HADOOP-3647? Thanks! Arun > Overall I've run this same job before with no problems. The only > change is > the added gzip of the output. Don't know if it's worth anything, but > the > four failures all happened on different machines. I'll be running > this job > plenty of times so if the problem keeps happening it will be obvious. > / Per > With 0.18 we rewrote the path from the output of the map, shuffle and the merge on the reducer. So, that could be a bug - again, we hope http://issues.apache.org/jira/browse/HADOOP-4277 will fix this. Arun > On Wed, Oct 1, 2008 at 11:23 AM, Arun C Murthy <wgentry@example.com> > wrote: > >> >> Do you still have the task logs for the reduce? >> >> I suspect are running into >> http://issues.apache.org/jira/browse/HADOOP-3647 which we never could >> reproduce reliably to pin it down or fix. >> >> However, in light of http://issues.apache.org/jira/browse/ >> HADOOP-4277 we >> suspect this could be caused by a bug in the LocalFileSystem which >> could >> hide data-corruption on your local disk leading to errors on these >> nature. >> Could you try running your job with that patch once the release >> 0.18.2 is >> available? >> >> Any information you provide could greatly aid to confirm our above >> hypothesis, so it's much appreciated! >> >> Arun >> >> From core-userwgentry@example.com Wed Oct 01 20:45:27 2008 Return-Path: <core-userwgentry@example.com> Delivered-To: wgentry@example.com Received: (qmail 35034 invoked from network); 1 Oct 2008 20:45:27 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Oct 2008 20:45:27 -0000 Received: (qmail 34971 invoked by uid 500); 1 Oct 2008 20:45:20 -0000 Delivered-To: wgentry@example.com Received: (qmail 34938 invoked by uid 500); 1 Oct 2008 20:45:20 -0000 Mailing-List: contact wgentry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:wgentry@example.com> List-Unsubscribe: <mailto:wgentry@example.com> List-Post: <mailto:wgentry@example.com> List-Id: <core-user.hadoop.apache.org> Reply-To: wgentry@example.com Delivered-To: mailing list wgentry@example.com Received: (qmail 34927 invoked by uid 99); 1 Oct 2008 20:45:20 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 13:45:20 -0700 X-ASF-Spam-Status: No, hits=3.2 required=10.0 tests=HTML_MESSAGE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO fg-out-1718.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 20:44:19 +0000 Received: by fg-out-1718.google.com with SMTP id l26so485201fgb.35 for <wgentry@example.com>; Wed, 01 Oct 2008 13:44:52 -0700 (PDT) Received: by 83.161.120.155 with SMTP id d2mr4931684bkg.26.1222893892177; Wed, 01 Oct 2008 13:44:52 -0700 (PDT) Received: by 83.161.120.155 with HTTP; Wed, 1 Oct 2008 13:44:52 -0700 (PDT) Message-ID: <wgentry@example.com> Date: Wed, 1 Oct 2008 13:44:52 -0700 From: "Per Jacobsson" <wgentry@example.com> To: wgentry@example.com Subject: Re: Merging of the local FS files threw an exception In-Reply-To: <wgentry@example.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_16396_32959183.1222893892191" References: <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> <wgentry@example.com> X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_16396_32959183.1222893892191 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Attached to the ticket. Hope this helps. / Per On Wed, Oct 1, 2008 at 1:33 PM, Arun C Murthy <wgentry@example.com> wrote: > > On Oct 1, 2008, at 12:04 PM, Per Jacobsson wrote: > > I've collected the syslogs from the failed reduce jobs. What's the best >> way >> to get them to you? Let me know if you need anything else, I'll have to >> shut >> down these instances some time later today. >> >> > Could you please attach them to the jira: > http://issues.apache.org/jira/browse/HADOOP-3647? Thanks! > > Arun > ------=_Part_16396_32959183.1222893892191-- From core-userwgentry@example.com Wed Oct 01 23:44:59 2008 Return-Path: <core-userwgentry@example.com> Delivered-To: wgentry@example.com Received: (qmail 62597 invoked from network); 1 Oct 2008 23:44:58 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Oct 2008 23:44:58 -0000 Received: (qmail 50585 invoked by uid 500); 1 Oct 2008 23:44:52 -0000 Delivered-To: wgentry@example.com Received: (qmail 50554 invoked by uid 500); 1 Oct 2008 23:44:52 -0000 Mailing-List: contact wgentry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:wgentry@example.com> List-Unsubscribe: <mailto:wgentry@example.com> List-Post: <mailto:wgentry@example.com> List-Id: <core-user.hadoop.apache.org> Reply-To: wgentry@example.com Delivered-To: mailing list wgentry@example.com Received: (qmail 50541 invoked by uid 99); 1 Oct 2008 23:44:52 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 16:44:52 -0700 X-ASF-Spam-Status: No, hits=2.0 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of wgentry@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO fk-out-0910.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Oct 2008 23:43:47 +0000 Received: by fk-out-0910.google.com with SMTP id 26so329564fkx.13 for <wgentry@example.com>; Wed, 01 Oct 2008 16:44:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:in-reply-to:mime-version:content-type:references; bh=kbbyfOfwgCvkDVnmc1p7SFmSlq+OYBRXw1G+x+jikLk=; b=iC5nNf0paDdbhtu8UNu4sqESycTU+0HXivWEl9pARYPtdhpjQ3hDU368x/6+Z4yhiR ge5JnYqSOSGa9+lqLa4mEPZ70InCkE393NxXFBg/e5Ejo23l+awMGFAYZwGWoHmlGeIh RXMbgbzd1ATp9Lz1Jb3uaCRd7hRCYdwd8ClM8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version :content-type:references; b=X0gokMZiHU8rK20tzwYYO8cdF9A6XAmJf69muo7d62xTdLfm+rkRjGRh1NGNWVejfH H3SSNd5TEHr9xkpUyGEuceB2hh6+X53ArQzOaieELEfNpdXQutaJofrThpV01OVn9Bik 4bcKkvqoniOeqEMfJASttHtHPHlCaXaty91a4= Received: by 83.161.120.155 with SMTP id p11mr5103291bkj.9.1222904650652; Wed, 01 Oct 2008 16:44:10 -0700 (PDT) Received: by 83.161.120.155 with HTTP; Wed, 1 Oct 2008 16:44:10 -0700 (PDT) Message-ID: <wgentry@example.com> Date: Wed, 1 Oct 2008 16:44:10 -0700 From: "Alex Loddengaard" <wgentry@example.com> To: wgentry@example.com Subject: Re: architecture diagram In-Reply-To: <wgentry@example.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_2722_25406080.1222904650635" References: <wgentry@example.com> <wgentry@example.com> X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_2722_25406080.1222904650635 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline I'm not sure what you mean by "disconnected parts of data," but Hadoop is implemented to try and perform map tasks on machines that have input data. This is to lower the amount of network traffic,
{'source': 'Apache Software Foundation Public Mail Archives'}