From hadoop-userrmartinez@example.com Thu Mar 01 08:10:04 2007 Return-Path: <hadoop-userrmartinez@example.com> Delivered-To: rmartinez@example.com Received: (qmail 66169 invoked from network); 1 Mar 2007 08:10:03 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Mar 2007 08:10:03 -0000 Received: (qmail 67014 invoked by uid 500); 1 Mar 2007 08:10:10 -0000 Delivered-To: rmartinez@example.com Received: (qmail 66981 invoked by uid 500); 1 Mar 2007 08:10:09 -0000 Mailing-List: contact rmartinez@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:rmartinez@example.com> List-Unsubscribe: <mailto:rmartinez@example.com> List-Post: <mailto:rmartinez@example.com> List-Id: <hadoop-user.lucene.apache.org> Reply-To: rmartinez@example.com Delivered-To: mailing list rmartinez@example.com Received: (qmail 66972 invoked by uid 99); 1 Mar 2007 08:10:09 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 00:10:09 -0800 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: pass (herse.apache.org: local policy) Received: from [83.161.120.155] (HELO cyrus.richardyang.net) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 00:09:58 -0800 Received: from ACER ([83.161.120.155]) by cyrus.richardyang.net with Microsoft SMTPSVC(5.0.2195.6713); Thu, 1 Mar 2007 00:12:32 -0800 Reply-To: <rmartinez@example.com> From: "Richard Yang" <rmartinez@example.com> To: <rmartinez@example.com> Subject: RE: Problem running sample Grep Date: Thu, 1 Mar 2007 00:08:49 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 Thread-Index: Acdbylkdoeoa8fYlTUyzBL8wqIerZAADOOYg X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3028 In-Reply-To: <rmartinez@example.com> Message-ID: <rmartinez@example.com> X-OriginalArrivalTime: 01 Mar 2007 08:12:32.0554 (UTC) FILETIME=[5CC120A0:01C75BD9] X-Virus-Checked: Checked by ClamAV on apache.org Hello David, Thank you for responding to my help. I have only been using version 0.11.2. I have to run grep in the single node as illustrated in the manual online and it looks successful. Doesn't the 'bin/hadoop jar hadoop-0.11.3-dev-examples.jar grep /bar/ac /tmp/ac foo' test the single node mode only? Because toward the end of the manual the distributed version of sample Grep is mentioned as "bin/hadoop org.apache.hadoop.samples.Grep myinput myoutput 'dfs'" I am trying to make sure it runs on 2 nodes before I add more machine to it. Thank you. Best Regards Richard Yang rmartinez@example.com rmartinez@example.com -----Original Message----- From: David Bowen [mailto:rmartinez@example.com] Sent: Wednesday, February 28, 2007 10:21 PM To: rmartinez@example.com Subject: Re: Problem running sample Grep Here's how I run it: bin/hadoop jar hadoop-0.11.3-dev-examples.jar grep /bar/ac /tmp/ac foo - David Richard Yang wrote: > Hello, > > > > I am new to Hadoop. After setting my testing environment (which I am not > sure if it is correct), I tried to run the Grep sample by > > Bin/hadoop org.apache.hadoop.samples.Grep myinput myoutput 'dfs' > > However, it always says 'Exception in thread "main" > java.lang.NoClassDefFoundError: org/apache/hadoop/examples/Grep' > > I didn't know what went wrong. > > Any help is appreciated. Thanks. > > > > Best Regards > > > > Richard Yang > > <mailto:rmartinez@example.com> rmartinez@example.com > > <mailto:rmartinez@example.com> rmartinez@example.com > > > > > > > > > From hadoop-userrmartinez@example.com Thu Mar 01 09:32:59 2007 Return-Path: <hadoop-userrmartinez@example.com> Delivered-To: rmartinez@example.com Received: (qmail 4428 invoked from network); 1 Mar 2007 09:32:58 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Mar 2007 09:32:58 -0000 Received: (qmail 1888 invoked by uid 500); 1 Mar 2007 09:33:05 -0000 Delivered-To: rmartinez@example.com Received: (qmail 1868 invoked by uid 500); 1 Mar 2007 09:33:05 -0000 Mailing-List: contact rmartinez@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:rmartinez@example.com> List-Unsubscribe: <mailto:rmartinez@example.com> List-Post: <mailto:rmartinez@example.com> List-Id: <hadoop-user.lucene.apache.org> Reply-To: rmartinez@example.com Delivered-To: mailing list rmartinez@example.com Received: (qmail 1859 invoked by uid 99); 1 Mar 2007 09:33:05 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 01:33:05 -0800 X-ASF-Spam-Status: No, hits=2.5 required=10.0 tests=HTML_MESSAGE,NO_REAL_NAME X-Spam-Check-By: apache.org Received-SPF: neutral (herse.apache.org: local policy) Received: from [83.161.120.155] (HELO CTSINTCOSXIA.cts.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 01:32:53 -0800 Received: from ctsinchnsxuh.cts.com ([83.161.120.155]) by mail3.cognizant.com with InterScan Message Security Suite; Thu, 01 Mar 2007 15:02:30 +0530 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C75BE4.87E276D7" Subject: Running hadoop in 2 systems Date: Thu, 1 Mar 2007 15:02:29 +0530 Message-ID: <rmartinez@example.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Running hadoop in 2 systems Thread-Index: Acdb5IfiCVKSoItHQayR6tfY/cXFSg== From: <rmartinez@example.com> To: <rmartinez@example.com> X-Virus-Checked: Checked by ClamAV on apache.org ------_=_NextPart_001_01C75BE4.87E276D7 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello, I'm evaluating Hadoop for a large application. When running the wordcount example, I experience an issue where my =20 master node cannot open a socket to port 50010 of my remote slave node. When I run the example with only my master in the slaves file, it =20 works fine. When I add a second machine, i get the error. Here is my config: Running Hadoop-0.11.0 Server for master (83.161.120.155) Remote slave (83.161.120.155) My conf/slaves file content =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D localhost rmartinez@example.com =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D My masters file content =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D localhost =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D I've set the environmental variable for HADOOP_HOME and JAVA_HOME Using the standard hadoop-default.xml. Here's my hadoop-site.xml (which is the same on both machines): <?xml version=3D"1.0"?> <?xml-stylesheet type=3D"text/xsl" href=3D"configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>fs.default.name</name> <value>83.161.120.155:50010</value> </property> <property> <name>mapred.job.tracker</name> <value>83.161.120.155:50011</value> </property> <property> <name>dfs.replication</name> <value>2</value> </property> <property> <name>dfs.datanode.port</name> <value>50010</value> </property> <property> <name>dfs.info.port</name> <value>50070</value> </property> <property> <name>dfs.name.dir</name> <value>/tmp/hadoop-146736/dfs/name</value> </property> <property> <name>dfs.data.dir</name> <value>/tmp/hadoop-146736/dfs/data</value> </property> <property> <name>dfs.client.buffer.dir</name> <value>/tmp/hadoop-146736/dfs/tmp</value> </property> <property> <name>mapred.local.dir</name> <value>/tmp/hadoop-jaya/mapred/local</value> </property> <property> <name>mapred.system.dir</name> <value>/tmp/hadoop-jaya/mapred/system</value> </property> <property> <name>mapred.temp.dir</name> <value>/tmp/hadoop-jaya/mapred/temp</value> </property> <property> <name>mapred.job.tracker.info.port</name> <value>50030</value> </property> <property> <name>mapred.task.tracker.report.port</name> <value>50050</value> </property> </configuration> Here is the terminal output on the server: 146736@localhost hadoop-0.11.0]$ bin/hadoop dfs input 07/03/01 14:43:22 INFO ipc.Client: Retrying connect to server:= /83.161.120.155:50010. Already tried 1 time(s). 07/03/01 14:43:23 INFO ipc.Client: Retrying connect to server:= /83.161.120.155:50010. Already tried 2 time(s). 07/03/01 14:43:24 INFO ipc.Client: Retrying connect to server:= /83.161.120.155:50010. Already tried 3 time(s). 07/03/01 14:43:25 INFO ipc.Client: Retrying connect to server:= /83.161.120.155:50010. Already tried 4 time(s). 07/03/01 14:43:26 INFO ipc.Client: Retrying connect to server:= /83.161.120.155:50010. Already tried 5 time(s). 07/03/01 14:43:27 INFO ipc.Client: Retrying connect to server:= /83.161.120.155:50010. Already tried 6 time(s). 07/03/01 14:43:28 INFO ipc.Client: Retrying connect to server:= /83.161.120.155:50010. Already tried 7 time(s). 07/03/01 14:43:29 INFO ipc.Client: Retrying connect to server:= /83.161.120.155:50010. Already tried 8 time(s). 07/03/01 14:43:30 INFO ipc.Client: Retrying connect to server:= /83.161.120.155:50010. Already tried 9 time(s). 07/03/01 14:43:31 INFO ipc.Client: Retrying connect to server:= /83.161.120.155:50010. Already tried 10 time(s). Bad connection to FS. command aborted. Can anyone help me identify the issue? am i not doing any other work which= is required... Please help me finding and solving the issue... Thanks & Regards, Jayalakshmi This e-mail and any files transmitted with it are for the sole use of the= intended recipient(s) and may contain confidential and privileged= information. If you are not the intended recipient, please contact the sender by reply= e-mail and destroy all copies of the original message.=20 Any unauthorized review, use, disclosure, dissemination, forwarding,= printing or copying of this email or any action taken in reliance on this= e-mail is strictly=20 prohibited and may be unlawful. ------_=_NextPart_001_01C75BE4.87E276D7-- From hadoop-userrmartinez@example.com Thu Mar 01 11:16:27 2007 Return-Path: <hadoop-userrmartinez@example.com> Delivered-To: rmartinez@example.com Received: (qmail 45496 invoked from network); 1 Mar 2007 11:16:26 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Mar 2007 11:16:26 -0000 Received: (qmail 80169 invoked by uid 500); 1 Mar 2007 11:16:31 -0000 Delivered-To: rmartinez@example.com Received: (qmail 80055 invoked by uid 500); 1 Mar 2007 11:16:30 -0000 Mailing-List: contact rmartinez@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:rmartinez@example.com> List-Unsubscribe: <mailto:rmartinez@example.com> List-Post: <mailto:rmartinez@example.com> List-Id: <hadoop-user.lucene.apache.org> Reply-To: rmartinez@example.com Delivered-To: mailing list rmartinez@example.com Received: (qmail 80018 invoked by uid 99); 1 Mar 2007 11:16:30 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 03:16:30 -0800 X-ASF-Spam-Status: No, hits=2.0 required=10.0 tests=DNS_FROM_SECURITYSAGE X-Spam-Check-By: apache.org Received-SPF: pass (herse.apache.org: local policy) Received: from [83.161.120.155] (HELO ironport2.alibaba-inc.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 03:16:19 -0800 X-IronPort-AV: i="4.14,234,1170604800"; d="scan'208"; a="106564591:sNHT26070560" From: =?gb2312?B?1cXDr8mt?= <rmartinez@example.com> To: <rmartinez@example.com> References: <rmartinez@example.com> Subject: =?gb2312?B?tPC4tDogUnVubmluZyBoYWRvb3AgaW4gMiBzeXN0ZW1z?= Date: Thu, 1 Mar 2007 19:15:54 +0800 Message-ID: <rmartinez@example.com> MIME-Version: 1.0 Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 11 In-Reply-To: <rmartinez@example.com> Thread-Index: Acdb5IfiCVKSoItHQayR6tfY/cXFSgAA3LYw X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3028 X-Virus-Checked: Checked by ClamAV on apache.org Do you have made the ssh-key pair and send the public key to the slave? -----=D3=CA=BC=FE=D4=AD=BC=FE----- =B7=A2=BC=FE=C8=CB: rmartinez@example.com [mailto:rmartinez@example.com]=20 =B7=A2=CB=CD=CA=B1=BC=E4: 2007=C4=EA3=D4=C21=C8=D5 17:32 =CA=D5=BC=FE=C8=CB: rmartinez@example.com =D6=F7=CC=E2: Running hadoop in 2 systems Hello, I'm evaluating Hadoop for a large application. When running the wordcount example, I experience an issue where my =20 master node cannot open a socket to port 50010 of my remote slave node. When I run the example with only my master in the slaves file, it =20 works fine. When I add a second machine, i get the error. Here is my config: Running Hadoop-0.11.0 Server for master (83.161.120.155) Remote slave (83.161.120.155) My conf/slaves file content =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D localhost rmartinez@example.com =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D My masters file content =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D localhost =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D I've set the environmental variable for HADOOP_HOME and JAVA_HOME Using the standard hadoop-default.xml. Here's my hadoop-site.xml (which is the same on both machines): <?xml version=3D"1.0"?> <?xml-stylesheet type=3D"text/xsl" href=3D"configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>fs.default.name</name> <value>83.161.120.155:50010</value> </property> <property> <name>mapred.job.tracker</name> <value>83.161.120.155:50011</value> </property> <property> <name>dfs.replication</name> <value>2</value> </property> <property> <name>dfs.datanode.port</name> <value>50010</value> </property> <property> <name>dfs.info.port</name> <value>50070</value> </property> <property> <name>dfs.name.dir</name> <value>/tmp/hadoop-146736/dfs/name</value> </property> <property> <name>dfs.data.dir</name> <value>/tmp/hadoop-146736/dfs/data</value> </property> <property> <name>dfs.client.buffer.dir</name> <value>/tmp/hadoop-146736/dfs/tmp</value> </property> <property> <name>mapred.local.dir</name> <value>/tmp/hadoop-jaya/mapred/local</value> </property> <property> <name>mapred.system.dir</name> <value>/tmp/hadoop-jaya/mapred/system</value> </property> <property> <name>mapred.temp.dir</name> <value>/tmp/hadoop-jaya/mapred/temp</value> </property> <property> <name>mapred.job.tracker.info.port</name> <value>50030</value> </property> <property> <name>mapred.task.tracker.report.port</name> <value>50050</value> </property> </configuration> Here is the terminal output on the server: 146736@localhost hadoop-0.11.0]$ bin/hadoop dfs input 07/03/01 14:43:22 INFO ipc.Client: Retrying connect to server: /83.161.120.155:50010. Already tried 1 time(s). 07/03/01 14:43:23 INFO ipc.Client: Retrying connect to server: /83.161.120.155:50010. Already tried 2 time(s). 07/03/01 14:43:24 INFO ipc.Client: Retrying connect to server: /83.161.120.155:50010. Already tried 3 time(s). 07/03/01 14:43:25 INFO ipc.Client: Retrying connect to server: /83.161.120.155:50010. Already tried 4 time(s). 07/03/01 14:43:26 INFO ipc.Client: Retrying connect to server: /83.161.120.155:50010. Already tried 5 time(s). 07/03/01 14:43:27 INFO ipc.Client: Retrying connect to server: /83.161.120.155:50010. Already tried 6 time(s). 07/03/01 14:43:28 INFO ipc.Client: Retrying connect to server: /83.161.120.155:50010. Already tried 7 time(s). 07/03/01 14:43:29 INFO ipc.Client: Retrying connect to server: /83.161.120.155:50010. Already tried 8 time(s). 07/03/01 14:43:30 INFO ipc.Client: Retrying connect to server: /83.161.120.155:50010. Already tried 9 time(s). 07/03/01 14:43:31 INFO ipc.Client: Retrying connect to server: /83.161.120.155:50010. Already tried 10 time(s). Bad connection to FS. command aborted. Can anyone help me identify the issue? am i not doing any other work = which is required... Please help me finding and solving the issue... Thanks & Regards, Jayalakshmi This e-mail and any files transmitted with it are for the sole use of = the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by = reply e-mail and destroy all copies of the original message.=20 Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email or any action taken in reliance on = this e-mail is strictly=20 prohibited and may be unlawful. From hadoop-userrmartinez@example.com Thu Mar 01 15:32:52 2007 Return-Path: <hadoop-userrmartinez@example.com> Delivered-To: rmartinez@example.com Received: (qmail 51638 invoked from network); 1 Mar 2007 15:32:51 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Mar 2007 15:32:51 -0000 Received: (qmail 44087 invoked by uid 500); 1 Mar 2007 15:32:58 -0000 Delivered-To: rmartinez@example.com Received: (qmail 44063 invoked by uid 500); 1 Mar 2007 15:32:58 -0000 Mailing-List: contact rmartinez@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:rmartinez@example.com> List-Unsubscribe: <mailto:rmartinez@example.com> List-Post: <mailto:rmartinez@example.com> List-Id: <hadoop-user.lucene.apache.org> Reply-To: rmartinez@example.com Delivered-To: mailing list rmartinez@example.com Received: (qmail 44054 invoked by uid 99); 1 Mar 2007 15:32:58 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 07:32:58 -0800 X-ASF-Spam-Status: No, hits=2.0 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (herse.apache.org: domain of rmartinez@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO nz-out-0506.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 07:32:47 -0800 Received: by nz-out-0506.google.com with SMTP id i1so473255nzh for <rmartinez@example.com>; Thu, 01 Mar 2007 07:32:26 -0800 (PST) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:in-reply-to:mime-version:content-type:references:x-google-sender-auth; b=d6FEMFCu+3MZjn1KsC9UStvDFw+/jUd55igcnYIX8pR3/QY7Wzl9sT3owbKfpmYcXPpsl1lQlQLnapHh+10wBeYlRSjPKvY4hbLor2RONhCvMmXYP0273r3xRqwWEKhBJA+2q8gyXPEGOyL5qQuz2Czc+9WOHpf8BzLn8SCM93w= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:in-reply-to:mime-version:content-type:references:x-google-sender-auth; b=Xs1ismWXaCOj9GXg+ZO6NUKTPziLB+ZTsZsNxg/Y8jya/atzeKelTX1/EDMBKZzYwtH+ejUtk0/csmIWwGrrKb4I6qPE4gereRtA9Yz+cTR/2e/hq3/vpGrfKjNCzNm/7so8QD/yc6ItpC/uH1kTKrMANMtI6srNLUokGSsRIhc= Received: by 83.161.120.155 with SMTP id c1mr69790wae.1172763136075; Thu, 01 Mar 2007 07:32:16 -0800 (PST) Received: by 83.161.120.155 with HTTP; Thu, 1 Mar 2007 07:32:15 -0800 (PST) Message-ID: <rmartinez@example.com> Date: Thu, 1 Mar 2007 10:32:15 -0500 From: "Eugene Weinstein" <rmartinez@example.com> Sender: rmartinez@example.com To: rmartinez@example.com Subject: Re: Running hadoop in 2 systems In-Reply-To: <rmartinez@example.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_62016_16297749.1172763135989" References: <rmartinez@example.com> X-Google-Sender-Auth: f188dea82d60da79 X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_62016_16297749.1172763135989 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline It could be a firewall issue -- for me the firewall was blocking port 50010. Eugene On 3/1/07, rmartinez@example.com < rmartinez@example.com> wrote: > > > > Hello, > > I'm evaluating Hadoop for a large application. > > When running the wordcount example, I experience an issue where my > master node cannot open a socket to port 50010 of my remote slave node. > > When I run the example with only my master in the slaves file, it > works fine. When I add a second machine, i get the error. > > Here is my config: > > Running Hadoop-0.11.0 > > Server for master (83.161.120.155) > Remote slave (83.161.120.155) > > My conf/slaves file content > > =================== > localhost > rmartinez@example.com > ==================== > > My masters file content > > ============== > localhost > ============== > > I've set the environmental variable for HADOOP_HOME and JAVA_HOME > > > Using the standard hadoop-default.xml. > > Here's my hadoop-site.xml (which is the same on both machines): > > <?xml version="1.0"?> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > > <!-- Put site-specific property overrides in this file. --> > > <configuration> > > <property> > <name>fs.default.name</name> > <value>83.161.120.155:50010</value> > </property> > > <property> > <name>mapred.job.tracker</name> > <value>83.161.120.155:50011</value> > </property> > > <property> > <name>dfs.replication</name> > <value>2</value> > </property> > > <property> > <name>dfs.datanode.port</name> > <value>50010</value> > </property> > > <property> > <name>dfs.info.port</name> > <value>50070</value> > </property> > > <property> > <name>dfs.name.dir</name> > <value>/tmp/hadoop-146736/dfs/name</value> > </property> > > <property> > <name>dfs.data.dir</name> > <value>/tmp/hadoop-146736/dfs/data</value> > </property> > > <property> > <name>dfs.client.buffer.dir</name> > <value>/tmp/hadoop-146736/dfs/tmp</value> > </property> > > <property> > <name>mapred.local.dir</name> > <value>/tmp/hadoop-jaya/mapred/local</value> > </property> > > <property> > <name>mapred.system.dir</name> > <value>/tmp/hadoop-jaya/mapred/system</value> > </property> > > <property> > <name>mapred.temp.dir</name> > <value>/tmp/hadoop-jaya/mapred/temp</value> > </property> > > <property> > <name>mapred.job.tracker.info.port</name> > <value>50030</value> > </property> > > <property> > <name>mapred.task.tracker.report.port</name> > <value>50050</value> > </property> > > </configuration> > > Here is the terminal output on the server: > > 146736@localhost hadoop-0.11.0]$ bin/hadoop dfs input > 07/03/01 14:43:22 INFO ipc.Client: Retrying connect to server: > /83.161.120.155:50010. Already tried 1 time(s). > 07/03/01 14:43:23 INFO ipc.Client: Retrying connect to server: > /83.161.120.155:50010. Already tried 2 time(s). > 07/03/01 14:43:24 INFO ipc.Client: Retrying connect to server: > /83.161.120.155:50010. Already tried 3 time(s). > 07/03/01 14:43:25 INFO ipc.Client: Retrying connect to server: > /83.161.120.155:50010. Already tried 4 time(s). > 07/03/01 14:43:26 INFO ipc.Client: Retrying connect to server: > /83.161.120.155:50010. Already tried 5 time(s). > 07/03/01 14:43:27 INFO ipc.Client: Retrying connect to server: > /83.161.120.155:50010. Already tried 6 time(s). > 07/03/01 14:43:28 INFO ipc.Client: Retrying connect to server: > /83.161.120.155:50010. Already tried 7 time(s). > 07/03/01 14:43:29 INFO ipc.Client: Retrying connect to server: > /83.161.120.155:50010. Already tried 8 time(s). > 07/03/01 14:43:30 INFO ipc.Client: Retrying connect to server: > /83.161.120.155:50010. Already tried 9 time(s). > 07/03/01 14:43:31 INFO ipc.Client: Retrying connect to server: > /83.161.120.155:50010. Already tried 10 time(s). > Bad connection to FS. command aborted. > > Can anyone help me identify the issue? am i not doing any other work which > is required... Please help me finding and solving the issue... > > Thanks & Regards, > Jayalakshmi > > > This e-mail and any files transmitted with it are for the sole use of the > intended recipient(s) and may contain confidential and privileged > information. > If you are not the intended recipient, please contact the sender by reply > e-mail and destroy all copies of the original message. > Any unauthorized review, use, disclosure, dissemination, forwarding, > printing or copying of this email or any action taken in reliance on this > e-mail is strictly > prohibited and may be unlawful. ------=_Part_62016_16297749.1172763135989-- From hadoop-userrmartinez@example.com Fri Mar 02 04:10:57 2007 Return-Path: <hadoop-userrmartinez@example.com> Delivered-To: rmartinez@example.com Received: (qmail 26766 invoked from network); 2 Mar 2007 04:10:56 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Mar 2007 04:10:56 -0000 Received: (qmail 19555 invoked by uid 500); 2 Mar 2007 04:11:04 -0000 Delivered-To: rmartinez@example.com Received: (qmail 19430 invoked by uid 500); 2 Mar 2007 04:11:04 -0000 Mailing-List: contact rmartinez@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:rmartinez@example.com> List-Unsubscribe: <mailto:rmartinez@example.com> List-Post: <mailto:rmartinez@example.com> List-Id: <hadoop-user.lucene.apache.org> Reply-To: rmartinez@example.com Delivered-To: mailing list rmartinez@example.com Received: (qmail 19421 invoked by uid 99); 2 Mar 2007 04:11:03 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 20:11:03 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (herse.apache.org: domain of rmartinez@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO talk.nabble.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 20:10:51 -0800 Received: from [83.161.120.155] (helo=jubjub.nabble.com) by talk.nabble.com with esmtp (Exim 4.50) id 1HMz6E-0001ss-OD for rmartinez@example.com; Thu, 01 Mar 2007 20:10:30 -0800 Message-ID: <rmartinez@example.com> Date: Thu, 1 Mar 2007 20:10:30 -0800 (PST) From: jaylac <rmartinez@example.com> To: rmartinez@example.com Subject: Re: Running hadoop in 2 systems In-Reply-To: <rmartinez@example.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: rmartinez@example.com References: <rmartinez@example.com> <rmartinez@example.com> X-Virus-Checked: Checked by ClamAV on apache.org There is no firewall problem.... I've disabled firewall... Can u find the problem? Eugene Weinstein wrote: > > It could be a firewall issue -- for me the firewall was blocking port > 50010. > > Eugene > > On 3/1/07, rmartinez@example.com < > rmartinez@example.com> wrote: >> >> >> >> Hello, >> >> I'm evaluating Hadoop for a large application. >> >> When running the wordcount example, I experience an issue where my >> master node cannot open a socket to port 50010 of my remote slave node. >> >> When I run the example with only my master in the slaves file, it >> works fine. When I add a second machine, i get the error. >> >> Here is my config: >> >> Running Hadoop-0.11.0 >> >> Server for master (83.161.120.155) >> Remote slave (83.161.120.155) >> >> My conf/slaves file content >> >> =================== >> localhost >> rmartinez@example.com >> ==================== >> >> My masters file content >> >> ============== >> localhost >> ============== >> >> I've set the environmental variable for HADOOP_HOME and JAVA_HOME >> >> >> Using the standard hadoop-default.xml. >> >> Here's my hadoop-site.xml (which is the same on both machines): >> >> <?xml version="1.0"?> >> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> >> >> <!-- Put site-specific property overrides in this file. --> >> >> <configuration> >> >> <property> >> <name>fs.default.name</name> >> <value>83.161.120.155:50010</value> >> </property> >> >> <property> >> <name>mapred.job.tracker</name> >> <value>83.161.120.155:50011</value> >> </property> >> >> <property> >> <name>dfs.replication</name> >> <value>2</value> >> </property> >> >> <property> >> <name>dfs.datanode.port</name> >> <value>50010</value> >> </property> >> >> <property> >> <name>dfs.info.port</name> >> <value>50070</value> >> </property> >> >> <property> >> <name>dfs.name.dir</name> >> <value>/tmp/hadoop-146736/dfs/name</value> >> </property> >> >> <property> >> <name>dfs.data.dir</name> >> <value>/tmp/hadoop-146736/dfs/data</value> >> </property> >> >> <property> >> <name>dfs.client.buffer.dir</name> >> <value>/tmp/hadoop-146736/dfs/tmp</value> >> </property> >> >> <property> >> <name>mapred.local.dir</name> >> <value>/tmp/hadoop-jaya/mapred/local</value> >> </property> >> >> <property> >> <name>mapred.system.dir</name> >> <value>/tmp/hadoop-jaya/mapred/system</value> >> </property> >> >> <property> >> <name>mapred.temp.dir</name> >> <value>/tmp/hadoop-jaya/mapred/temp</value> >> </property> >> >> <property> >> <name>mapred.job.tracker.info.port</name> >> <value>50030</value> >> </property> >> >> <property> >> <name>mapred.task.tracker.report.port</name> >> <value>50050</value> >> </property> >> >> </configuration> >> >> Here is the terminal output on the server: >> >> 146736@localhost hadoop-0.11.0]$ bin/hadoop dfs input >> 07/03/01 14:43:22 INFO ipc.Client: Retrying connect to server: >> /83.161.120.155:50010. Already tried 1 time(s). >> 07/03/01 14:43:23 INFO ipc.Client: Retrying connect to server: >> /83.161.120.155:50010. Already tried 2 time(s). >> 07/03/01 14:43:24 INFO ipc.Client: Retrying connect to server: >> /83.161.120.155:50010. Already tried 3 time(s). >> 07/03/01 14:43:25 INFO ipc.Client: Retrying connect to server: >> /83.161.120.155:50010. Already tried 4 time(s). >> 07/03/01 14:43:26 INFO ipc.Client: Retrying connect to server: >> /83.161.120.155:50010. Already tried 5 time(s). >> 07/03/01 14:43:27 INFO ipc.Client: Retrying connect to server: >> /83.161.120.155:50010. Already tried 6 time(s). >> 07/03/01 14:43:28 INFO ipc.Client: Retrying connect to server: >> /83.161.120.155:50010. Already tried 7 time(s). >> 07/03/01 14:43:29 INFO ipc.Client: Retrying connect to server: >> /83.161.120.155:50010. Already tried 8 time(s). >> 07/03/01 14:43:30 INFO ipc.Client: Retrying connect to server: >> /83.161.120.155:50010. Already tried 9 time(s). >> 07/03/01 14:43:31 INFO ipc.Client: Retrying connect to server: >> /83.161.120.155:50010. Already tried 10 time(s). >> Bad connection to FS. command aborted. >> >> Can anyone help me identify the issue? am i not doing any other work >> which >> is required... Please help me finding and solving the issue... >> >> Thanks & Regards, >> Jayalakshmi >> >> >> This e-mail and any files transmitted with it are for the sole use of the >> intended recipient(s) and may contain confidential and privileged >> information. >> If you are not the intended recipient, please contact the sender by reply >> e-mail and destroy all copies of the original message. >> Any unauthorized review, use, disclosure, dissemination, forwarding, >> printing or copying of this email or any action taken in reliance on this >> e-mail is strictly >> prohibited and may be unlawful. > > -- View this message in context: http://www.nabble.com/Running-hadoop-in-2-systems-tf3325881.html#a9263757 Sent from the Hadoop Users mailing list rmartinez@example.com. From hadoop-userrmartinez@example.com Fri Mar 02 04:25:11 2007 Return-Path: <hadoop-userrmartinez@example.com> Delivered-To: rmartinez@example.com Received: (qmail 29944 invoked from network); 2 Mar 2007 04:25:10 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Mar 2007 04:25:10 -0000 Received: (qmail 40116 invoked by uid 500); 2 Mar 2007 04:25:18 -0000 Delivered-To: rmartinez@example.com Received: (qmail 40092 invoked by uid 500); 2 Mar 2007 04:25:18 -0000 Mailing-List: contact rmartinez@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:rmartinez@example.com> List-Unsubscribe: <mailto:rmartinez@example.com> List-Post: <mailto:rmartinez@example.com> List-Id: <hadoop-user.lucene.apache.org> Reply-To: rmartinez@example.com Delivered-To: mailing list rmartinez@example.com Received: (qmail 40079 invoked by uid 99); 2 Mar 2007 04:25:18 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 20:25:18 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (herse.apache.org: domain of rmartinez@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO talk.nabble.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 20:25:04 -0800 Received: from [83.161.120.155] (helo=jubjub.nabble.com) by talk.nabble.com with esmtp (Exim 4.50) id 1HMzK0-000539-4Q for rmartinez@example.com; Thu, 01 Mar 2007 20:24:44 -0800 Message-ID: <rmartinez@example.com> Date: Thu, 1 Mar 2007 20:24:44 -0800 (PST) From: jaylac <rmartinez@example.com> To: rmartinez@example.com Subject: MapReduce MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Nabble-From: rmartinez@example.com X-Virus-Checked: Checked by ClamAV on apache.org Hi I was just going thro abt MapReduce for my final year project work..... I got confused in the middle.... What i thought is "MapReduce deals greatly with key/value pairs only... For fitting a problem into mapreduce we should find the key/value pairs" I want to know whether im right or wrong.... I got confused after looking at the explanation in wikipedia... The following is the content in wikipedia abt mapreduce... =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D "A map function iterates over a list of independent elements and performs a specified operation on each element. The list of answers is stored independently from the original list. Because each element is operated on independently and the original list is not being modified, it is very easy to perform a map operation in parallel. On appropriate hardware this allows extremely large data sets to be processed in short amounts of elapsed time. For example consider a list of test scores where each score has been found to be 1 too high. A map function of s =E2=88=92 1 could be applied to corre= ct every score s. A reduce operation takes a list and combines elements according to some algorithm. Since a reduce always ends up with a single answer, it is not as parallelizable as a map function, but the large number of relatively independent calculations means that reduce functions are still useful in highly parallel environments. Continuing the previous example, what if one wanted to know the average of the test scores? One could define a reduce function which halved the size o= f the list by adding an entry in the list to its neighbor, recursively continuing until there is only one (large) entry, and dividing the total su= m by the original number of elements to get the average." =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Here in map function we are simply adding up the test scores.... we are not using any key/value pair..... Im totally confused.... I might be wrong at any point... please someone help me out..... Am i wrong in the basic understanding of MapReduce itself..... Ill be thankful if anyone explains me clearly... please help me out to successfully complete my final year project.... Jaya --=20 View this message in context: http://www.nabble.com/MapReduce-tf3331603.htm= l#a9263847 Sent from the Hadoop Users mailing list rmartinez@example.com. From hadoop-userrmartinez@example.com Fri Mar 02 07:47:20 2007 Return-Path: <hadoop-userrmartinez@example.com> Delivered-To: rmartinez@example.com Received: (qmail 75665 invoked from network); 2 Mar 2007 07:47:19 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Mar 2007 07:47:19 -0000 Received: (qmail 43406 invoked by uid 500); 2 Mar 2007 07:47:27 -0000 Delivered-To: rmartinez@example.com Received: (qmail 43380 invoked by uid 500); 2 Mar 2007 07:47:27 -0000 Mailing-List: contact rmartinez@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:rmartinez@example.com> List-Unsubscribe: <mailto:rmartinez@example.com> List-Post: <mailto:rmartinez@example.com> List-Id: <hadoop-user.lucene.apache.org> Reply-To: rmartinez@example.com Delivered-To: mailing list rmartinez@example.com Received: (qmail 43371 invoked by uid 99); 2 Mar 2007 07:47:27 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 23:47:27 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (herse.apache.org: domain of rmartinez@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO talk.nabble.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 23:47:14 -0800 Received: from [83.161.120.155] (helo=jubjub.nabble.com) by talk.nabble.com with esmtp (Exim 4.50) id 1HN2Td-00032z-Ld for rmartinez@example.com; Thu, 01 Mar 2007 23:46:53 -0800 Message-ID: <rmartinez@example.com> Date: Thu, 1 Mar 2007 23:46:53 -0800 (PST) From: jaylac <rmartinez@example.com> To: rmartinez@example.com Subject: Detailed steps to run Hadoop in distributed system... MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: rmartinez@example.com X-Virus-Checked: Checked by ClamAV on apache.org Hi Hadoop-Users..... Have anyone successfully tried running hadoop in two systems? I've tried running the wordcount example in one system.. It works fine... But when i try to add nodes to the cluster and run wordcount example, i get errors.... So please let me know the detailed steps to be followed... Though the steps are given in the hadoop website, i need some help from u people... They might have thought some steps to be obvious and would have not stold that in the website... Im new user... So i simply followed the instructions given... I might have overlooked some steps which is necessary to run it.... Another important doubt.... In master node, i have a user name called "jaya"... Is it necessary to create a user name called "jaya" in the slave system also... or we can simply use the user name that exist in the slave machine? Im using two RED HAT LINUX machines... one master(83.161.120.155) and the other slave(83.161.120.155) In master node, the user name is jaya In slave node, the user name is 146736 The steps which i follow is..... Edit /home/jaya/.bashrc file Here ill set the HADOOP_CONF_DIR environment variable MASTER NODE 1. Edit conf/slaves file.... Contents ==================== localhost rmartinez@example.com ==================== 2. Edit conf/hadoop-en.sh file Here ill set the JAVA_HOME environment variable Thats it.... No other changes in this file.... PLEASE LET ME KNOW IF I SHOULD ADD ANYTHING HERE 3. Edit conf/hadoop-site.xml file Contents =========================================== <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>fs.default.name</name> <value>83.161.120.155:50010</value> </property> <property> <name>mapred.job.tracker</name> <value>83.161.120.155:50011</value> </property> <property> <name>dfs.replication</name> <value>2</value> </property> /configuration> ==================================== LET ME KNOW IF I NEED TO ADD ANYTHING HERE.... SLAVE NODE 1. Edit conf/masters file.... Contents ==================== localhost rmartinez@example.com ==================== 2. Edit conf/hadoop-en.sh file Here ill set the JAVA_HOME environment variable Thats it.... No other changes in this file.... PLEASE LET ME KNOW IF I SHOULD ADD ANYTHING HERE 3. Edit conf/hadoop-site.xml file Contents =========================================== <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>fs.default.name</name> <value>83.161.120.155:50010</value> </property> <property> <name>mapred.job.tracker</name> <value>83.161.120.155:50011</value> </property> <property> <name>dfs.replication</name> <value>2</value> </property> /configuration> ==================================== LET ME KNOW IF I NEED TO ADD ANYTHING HERE.... I've already done steps for passwordless login Thats is all........... Then ill perform the following operations.... In the HADOOP_HOME directory, jaya@localhost hadoop-0.11.0]$ bin/hadoop namenode -format Re-format filesystem in /tmp/hadoop-146736/dfs/name ? (Y or N) Y Formatted /tmp/hadoop-146736/dfs/name [jaya@localhost hadoop-0.11.0]$ Then [jaya@localhost hadoop-0.11.0]$ bin/start-all.sh starting namenode, logging to /opt/hadoop-0.11.0/bin/../logs/hadoop-jaya-namenode-localhost.localdomain.out localhost: starting datanode, logging to /opt/hadoop-0.11.0/bin/../logs/hadoop-jaya-datanode-localhost.localdomain.out rmartinez@example.com: ssh: connect to host 83.161.120.155 port 22: No route to host localhost: starting secondarynamenode, logging to /opt/hadoop-0.11.0/bin/../logs/hadoop-jaya-secondarynamenode-localhost.localdomain.out starting jobtracker, logging to /opt/hadoop-0.11.0/bin/../logs/hadoop-jaya-jobtracker-localhost.localdomain.out localhost: starting tasktracker, logging to /opt/hadoop-0.11.0/bin/../logs/hadoop-jaya-tasktracker-localhost.localdomain.out rmartinez@example.com: ssh: connect to host 83.161.120.155 port 22: No route to host [jaya@localhost hadoop-0.11.0]$ [jaya@localhost hadoop-0.11.0]$ mkdir input [jaya@localhost hadoop-0.11.0]$ cp conf/*.xml input [jaya@localhost hadoop-0.11.0]$ [jaya@localhost hadoop-0.11.0]$ bin/hadoop dfs -put input input jaya@localhost hadoop-0.11.0]$ bin/hadoop dfs -lsr / /tmp <dir> /tmp/hadoop-jaya <dir> /tmp/hadoop-jaya/mapred <dir> /tmp/hadoop-jaya/mapred/system <dir> /user <dir> /user/jaya <dir> /user/jaya/input <dir> /user/jaya/input/hadoop-default.xml <r 2> 21708 /user/jaya/input/hadoop-site.xml <r 2> 1333 /user/jaya/input/mapred-default.xml <r 2> 180 [jaya@localhost hadoop-0.11.0]$ [jaya@localhost hadoop-0.11.0]$ bin/hadoop dfs -ls input Found 3 items /user/jaya/input/hadoop-default.xml <r 2> 21708 /user/jaya/input/hadoop-site.xml <r 2> 1333 /user/jaya/input/mapred-default.xml <r 2> 180 [jaya@localhost hadoop-0.11.0]$ bin/hadoop dfs -ls output Found 0 items [jaya@localhost hadoop-0.11.0]$ bin/hadoop jar hadoop-0.11.0-examples.jar wordcount input output java.net.SocketTimeoutException: timed out waiting for rpc rmartinez@example.com(Client.java:469) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164) at $Proxy1.getProtocolVersion(Unknown Source) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:248) at org.apache.hadoop.mapred.JobClient.init(JobClient.java:200) at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:192) at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:381) at org.apache.hadoop.examples.WordCount.main(WordCount.java:143) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71) at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:143) at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:40) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:155) [jaya@localhost hadoop-0.11.0]$ I dont know where the problem is....... I've not created any directory called output.... if at all we need to create one, where should we create? Should i configure some more settings.... Please explain in detail.... Please do help me..... Thanks in advance Jaya -- View this message in context: http://www.nabble.com/Detailed-steps-to-run-Hadoop-in-distributed-system...-tf3332250.html#a9265480 Sent from the Hadoop Users mailing list rmartinez@example.com. From hadoop-userrmartinez@example.com Fri Mar 02 08:42:39 2007 Return-Path: <hadoop-userrmartinez@example.com> Delivered-To: rmartinez@example.com Received: (qmail 95614 invoked from network); 2 Mar 2007 08:42:38 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Mar 2007 08:42:38 -0000 Received: (qmail 97486 invoked by uid 500); 2 Mar 2007 08:42:46 -0000 Delivered-To: rmartinez@example.com Received: (qmail 97463 invoked by uid 500); 2 Mar 2007 08:42:46 -0000 Mailing-List: contact rmartinez@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:rmartinez@example.com> List-Unsubscribe: <mailto:rmartinez@example.com> List-Post: <mailto:rmartinez@example.com> List-Id: <hadoop-user.lucene.apache.org> Reply-To: rmartinez@example.com Delivered-To: mailing list rmartinez@example.com Received: (qmail 97454 invoked by uid 99); 2 Mar 2007 08:42:46 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Mar 2007 00:42:46 -0800 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: neutral (herse.apache.org: local policy) Received: from [83.161.120.155] (HELO rsmtp2.corp.hki.yahoo.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Mar 2007 00:42:35 -0800 Received: from ddaslaptop (ddas-laptop.bangalore.corp.yahoo.com [83.161.120.155]) (authenticated bits=0) by rsmtp2.corp.hki.yahoo.com (8.13.8/8.13.6/y.rout) with ESMTP id l228fwZS015399 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NO) for <rmartinez@example.com>; Fri, 2 Mar 2007 00:42:02 -0800 (PST) DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns; h=from:to:references:subject:date:message-id:mime-version: content-type:content-transfer-encoding:x-mailer:in-reply-to:x-mimeole:thread-index; b=TYzXbNhpawIKYAH/S0DponpEb2SLh0Whd5JGr30B72Xnfb34mi+7cuDpZHydMscv From: "Devaraj Das" <rmartinez@example.com> To: <rmartinez@example.com> References: <rmartinez@example.com> Subject: RE: Detailed steps to run Hadoop in distributed system... Date: Fri, 2 Mar 2007 14:11:57 +0530 Message-ID: <rmartinez@example.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 In-Reply-To: <rmartinez@example.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3028 Thread-Index: AcdcnwZisfvAO2RzRGqJG0LYeTayTQABhu9g X-Virus-Checked: Checked by ClamAV on apache.org > Have anyone successfully tried running hadoop in two systems? Of course! We have Hadoop running on clusters of 900 nodes. > In master node, i have a user name called "jaya"... Is it necessary to > create a user name called "jaya" in the slave system also... or we can > simply use the user name that exist in the slave machine? You should ideally run hadoop as the same user on all machines in the cluster. The shell scripts for starting/stopping hadoop daemons uses ssh to connect to the machines listed in the slaves file. Although you can probably work around that, I would recommend that you have the same user everywhere. >From the log messages, it looks like the host 83.161.120.155 could not communicate with the other host in order to start the hadoop daemons. Please address that issue first. > -----Original Message----- > From: jaylac [mailto:rmartinez@example.com] > Sent: Friday, March 02, 2007 1:17 PM > To: rmartinez@example.com > Subject: Detailed steps to run Hadoop in distributed system... > > > Hi Hadoop-Users..... > > Have anyone successfully tried running hadoop in two systems? > > I've tried running the wordcount example in one system.. It works fine... > But when i try to add nodes to the cluster and run wordcount example, i > get > errors.... > > So please let me know the detailed steps to be followed... > > Though the steps are given in the hadoop website, i need some help from u > people... > > They might have thought some steps to be obvious and would have not stold > that in the website... > > Im new user... So i simply followed the instructions given... I might have > overlooked some steps which is necessary to run it.... > > Another important doubt.... > > In master node, i have a user name called "jaya"... Is it necessary to > create a user name called "jaya" in the slave system also... or we can > simply use the user name that exist in the slave machine? > > > > Im using two RED HAT LINUX machines... one master(83.161.120.155) and the > other > slave(83.161.120.155) > In master node, the user name is jaya > In slave node, the user name is 146736 > > The steps which i follow is..... > > Edit /home/jaya/.bashrc file > Here ill set the HADOOP_CONF_DIR environment variable > > MASTER NODE > > 1. Edit conf/slaves file.... > Contents > ==================== > localhost > rmartinez@example.com > ==================== > > 2. Edit conf/hadoop-en.sh file > Here ill set the JAVA_HOME environment variable > Thats it.... No other changes in this file.... > PLEASE LET ME KNOW IF I SHOULD ADD ANYTHING HERE > > 3. Edit conf/hadoop-site.xml file > Contents > =========================================== > <?xml version="1.0"?> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > > <!-- Put site-specific property overrides in this file. --> > > <configuration> > > <property> > <name>fs.default.name</name> > <value>83.161.120.155:50010</value> > </property> > > <property> > <name>mapred.job.tracker</name> > <value>83.161.120.155:50011</value> > </property> > > <property> > <name>dfs.replication</name> > <value>2</value> > </property> > > /configuration> > ==================================== > > LET ME KNOW IF I NEED TO ADD ANYTHING HERE.... > > SLAVE NODE > > 1. Edit conf/masters file.... > Contents > ==================== > localhost > rmartinez@example.com > ==================== > > 2. Edit conf/hadoop-en.sh file > Here ill set the JAVA_HOME environment variable > Thats it.... No other changes in this file.... > PLEASE LET ME KNOW IF I SHOULD ADD ANYTHING HERE > > 3. Edit conf/hadoop-site.xml file > Contents > =========================================== > <?xml version="1.0"?> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > > <!-- Put site-specific property overrides in this file. --> > > <configuration> > > <property> > <name>fs.default.name</name> > <value>83.161.120.155:50010</value> > </property> > > <property> > <name>mapred.job.tracker</name> > <value>83.161.120.155:50011</value> > </property> > > <property> > <name>dfs.replication</name> > <value>2</value> > </property> > > /configuration> > ==================================== > > LET ME KNOW IF I NEED TO ADD ANYTHING HERE.... > > I've already done steps for passwordless login > > Thats is all........... Then ill perform the following operations.... > > In the HADOOP_HOME directory, > > jaya@localhost hadoop-0.11.0]$ bin/hadoop namenode -format > Re-format filesystem in /tmp/hadoop-146736/dfs/name ? (Y or N) Y > Formatted /tmp/hadoop-146736/dfs/name > [jaya@localhost hadoop-0.11.0]$ > > Then > > [jaya@localhost hadoop-0.11.0]$ bin/start-all.sh > starting namenode, logging to > /opt/hadoop-0.11.0/bin/../logs/hadoop-jaya-namenode- > localhost.localdomain.out > localhost: starting datanode, logging to > /opt/hadoop-0.11.0/bin/../logs/hadoop-jaya-datanode- > localhost.localdomain.out > rmartinez@example.com: ssh: connect to host 83.161.120.155 port 22: No route > to > host > localhost: starting secondarynamenode, logging to > /opt/hadoop-0.11.0/bin/../logs/hadoop-jaya-secondarynamenode- > localhost.localdomain.out > starting jobtracker, logging to > /opt/hadoop-0.11.0/bin/../logs/hadoop-jaya-jobtracker- > localhost.localdomain.out > localhost: starting tasktracker, logging to > /opt/hadoop-0.11.0/bin/../logs/hadoop-jaya-tasktracker- > localhost.localdomain.out > rmartinez@example.com: ssh: connect to host 83.161.120.155 port 22: No route > to > host > [jaya@localhost hadoop-0.11.0]$ > > [jaya@localhost hadoop-0.11.0]$ mkdir input > [jaya@localhost hadoop-0.11.0]$ cp conf/*.xml input > [jaya@localhost hadoop-0.11.0]$ > > [jaya@localhost hadoop-0.11.0]$ bin/hadoop dfs -put input input > jaya@localhost hadoop-0.11.0]$ bin/hadoop dfs -lsr / > /tmp <dir> > /tmp/hadoop-jaya <dir> > /tmp/hadoop-jaya/mapred <dir> > /tmp/hadoop-jaya/mapred/system <dir> > /user <dir> > /user/jaya <dir> > /user/jaya/input <dir> > /user/jaya/input/hadoop-default.xml <r 2> 21708 > /user/jaya/input/hadoop-site.xml <r 2> 1333 > /user/jaya/input/mapred-default.xml <r 2> 180 > [jaya@localhost hadoop-0.11.0]$ > > > > [jaya@localhost hadoop-0.11.0]$ bin/hadoop dfs -ls input > Found 3 items > /user/jaya/input/hadoop-default.xml <r 2> 21708 > /user/jaya/input/hadoop-site.xml <r 2> 1333 > /user/jaya/input/mapred-default.xml <r 2> 180 > [jaya@localhost hadoop-0.11.0]$ bin/hadoop dfs -ls output > Found 0 items > [jaya@localhost hadoop-0.11.0]$ bin/hadoop jar hadoop-0.11.0-examples.jar > wordcount input output > java.net.SocketTimeoutException: timed out waiting for rpc response > at org.apache.hadoop.ipc.Client.call(Client.java:469) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164) > at $Proxy1.getProtocolVersion(Unknown Source) > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:248) > at org.apache.hadoop.mapred.JobClient.init(JobClient.java:200) > at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:192) > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:381) > at org.apache.hadoop.examples.WordCount.main(WordCount.java:143) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: > 39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm > pl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriv > er.java:71) > at > org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:143) > at > org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:40) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: > 39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm > pl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:155) > [jaya@localhost hadoop-0.11.0]$ > > > I dont know where the problem is....... > > I've not created any directory called output.... if at all we need to > create > one, where should we create? > Should i configure some more settings.... Please explain in detail.... > > Please do help me..... > > Thanks in advance > Jaya > -- > View this message in context: http://www.nabble.com/Detailed-steps-to-run- > Hadoop-in-distributed-system...-tf3332250.html#a9265480 > Sent from the Hadoop Users mailing list rmartinez@example.com. From hadoop-userrmartinez@example.com Fri Mar 02 10:33:55 2007 Return-Path: <hadoop-userrmartinez@example.com> Delivered-To: rmartinez@example.com Received: (qmail 41194 invoked from network); 2 Mar 2007 10:33:54 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Mar 2007 10:33:54 -0000 Received: (qmail 39343 invoked by uid 500); 2 Mar 2007 10:34:02 -0000 Delivered-To: rmartinez@example.com Received: (qmail 39325 invoked by uid 500); 2 Mar 2007 10:34:02 -0000 Mailing-List: contact rmartinez@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:rmartinez@example.com> List-Unsubscribe: <mailto:rmartinez@example.com> List-Post: <mailto:rmartinez@example.com> List-Id: <hadoop-user.lucene.apache.org> Reply-To: rmartinez@example.com Delivered-To: mailing list rmartinez@example.com Received: (qmail 39316 invoked by uid 99); 2 Mar 2007 10:34:02 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Mar 2007 02:34:02 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (herse.apache.org: domain of rmartinez@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO talk.nabble.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Mar 2007 02:33:50 -0800 Received: from [83.161.120.155] (helo=jubjub.nabble.com) by talk.nabble.com with esmtp (Exim 4.50) id 1HN54r-0002Kl-Jq for rmartinez@example.com; Fri, 02 Mar 2007 02:33:29 -0800 Message-ID: <rmartinez@example.com> Date: Fri, 2 Mar 2007 02:33:29 -0800 (PST) From: jaylac <rmartinez@example.com> To: rmartinez@example.com Subject: RE: Detailed steps to run Hadoop in distributed system... In-Reply-To: <rmartinez@example.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: rmartinez@example.com References: <rmartinez@example.com> <rmartinez@example.com> X-Virus-Checked: Checked by ClamAV on apache.org What should i do? I dont know what to do..... Devaraj Das wrote: > >> Have anyone successfully tried running hadoop in two systems? > Of course! We have Hadoop running on clusters of 900 nodes. > >> In master node, i have a user name called "jaya"... Is it necessary to >> create a user name called "jaya" in the slave system also... or we can >> simply use the user name that exist in the slave machine? > You should ideally run hadoop as the same user on all machines in the > cluster. The shell scripts for starting/stopping hadoop daemons uses ssh > to > connect to the machines listed in the slaves file. Although you can > probably > work around that, I would recommend that you have the same user > everywhere. > > From the log messages, it looks like the host 83.161.120.155 could not > communicate with the other host in order to start the hadoop daemons. > Please > address that issue first. > >> -----Original Message----- >> From: jaylac [mailto:rmartinez@example.com] >> Sent: Friday, March 02, 2007 1:17 PM >> To: rmartinez@example.com >> Subject: Detailed steps to run Hadoop in distributed system... >> >> >> Hi Hadoop-Users..... >> >> Have anyone successfully tried running hadoop in two systems? >> >> I've tried running the wordcount example in one system.. It works fine... >> But when i try to add nodes to the cluster and run wordcount example, i >> get >> errors.... >> >> So please let me know the detailed steps to be followed... >> >> Though the steps are given in the hadoop website, i need some help from u >> people... >> >> They might have thought some steps to be obvious and would have not stold >> that in the website... >> >> Im new user... So i simply followed the instructions given... I might >> have >> overlooked some steps which is necessary to run it.... >> >> Another important doubt.... >> >> In master node, i have a user name called "jaya"... Is it necessary to >> create a user name called "jaya" in the slave system also... or we can >> simply use the user name that exist in the slave machine? >> >> >> >> Im using two RED HAT LINUX machines... one master(83.161.120.155) and the >> other >> slave(83.161.120.155) >> In master node, the user name is jaya >> In slave node, the user name is 146736 >> >> The steps which i follow is..... >> >> Edit /home/jaya/.bashrc file >> Here ill set the HADOOP_CONF_DIR environment variable >> >> MASTER NODE >> >> 1. Edit conf/slaves file.... >> Contents >> ==================== >> localhost >> rmartinez@example.com >> ==================== >> >> 2. Edit conf/hadoop-en.sh file >> Here ill set the JAVA_HOME environment variable >> Thats it.... No other changes in this file.... >> PLEASE LET ME KNOW IF I SHOULD ADD ANYTHING HERE >> >> 3. Edit conf/hadoop-site.xml file >> Contents >> =========================================== >> <?xml version="1.0"?> >> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> >> >> <!-- Put site-specific property overrides in this file. --> >> >> <configuration> >> >> <property> >> <name>fs.default.name</name> >> <value>83.161.120.155:50010</value> >> </property> >> >> <property> >> <name>mapred.job.tracker</name> >> <value>83.161.120.155:50011</value> >> </property> >> >> <property> >> <name>dfs.replication</name> >> <value>2</value> >> </property> >> >> /configuration> >> ==================================== >> >> LET ME KNOW IF I NEED TO ADD ANYTHING HERE.... >> >> SLAVE NODE >> >> 1. Edit conf/masters file.... >> Contents >> ==================== >> localhost >> rmartinez@example.com >> ==================== >> >> 2. Edit conf/hadoop-en.sh file >> Here ill set the JAVA_HOME environment variable >> Thats it.... No other changes in this file.... >> PLEASE LET ME KNOW IF I SHOULD ADD ANYTHING HERE >> >> 3. Edit conf/hadoop-site.xml file >> Contents >> =========================================== >> <?xml version="1.0"?> >> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> >> >> <!-- Put site-specific property overrides in this file. --> >> >> <configuration> >> >> <property> >> <name>fs.default.name</name> >> <value>83.161.120.155:50010</value> >> </property> >> >> <property> >> <name>mapred.job.tracker</name> >> <value>83.161.120.155:50011</value> >> </property> >> >> <property> >> <name>dfs.replication</name> >> <value>2</value> >> </property> >> >> /configuration> >> ==================================== >> >> LET ME KNOW IF I NEED TO ADD ANYTHING HERE.... >> >> I've already done steps for passwordless login >> >> Thats is all........... Then ill perform the following operations.... >> >> In the HADOOP_HOME directory, >> >> jaya@localhost hadoop-0.11.0]$ bin/hadoop namenode -format >> Re-format filesystem in /tmp/hadoop-146736/dfs/name ? (Y or N) Y >> Formatted /tmp/hadoop-146736/dfs/name >> [jaya@localhost hadoop-0.11.0]$ >> >> Then >> >> [jaya@localhost hadoop-0.11.0]$ bin/start-all.sh >> starting namenode, logging to >> /opt/hadoop-0.11.0/bin/../logs/hadoop-jaya-namenode- >> localhost.localdomain.out >> localhost: starting datanode, logging to >> /opt/hadoop-0.11.0/bin/../logs/hadoop-jaya-datanode- >> localhost.localdomain.out >> rmartinez@example.com: ssh: connect to host 83.161.120.155 port 22: No route >> to >> host >> localhost: starting secondarynamenode, logging to >> /opt/hadoop-0.11.0/bin/../logs/hadoop-jaya-secondarynamenode- >> localhost.localdomain.out >> starting jobtracker, logging to >> /opt/hadoop-0.11.0/bin/../logs/hadoop-jaya-jobtracker- >> localhost.localdomain.out >> localhost: starting tasktracker, logging to >> /opt/hadoop-0.11.0/bin/../logs/hadoop-jaya-tasktracker- >> localhost.localdomain.out >> rmartinez@example.com: ssh: connect to host 83.161.120.155 port 22: No route >> to >> host >> [jaya@localhost hadoop-0.11.0]$ >> >> [jaya@localhost hadoop-0.11.0]$ mkdir input >> [jaya@localhost hadoop-0.11.0]$ cp conf/*.xml input >> [jaya@localhost hadoop-0.11.0]$ >> >> [jaya@localhost hadoop-0.11.0]$ bin/hadoop dfs -put input input >> jaya@localhost hadoop-0.11.0]$ bin/hadoop dfs -lsr / >> /tmp <dir> >> /tmp/hadoop-jaya <dir> >> /tmp/hadoop-jaya/mapred <dir> >> /tmp/hadoop-jaya/mapred/system <dir> >> /user <dir> >> /user/jaya <dir> >> /user/jaya/input <dir> >> /user/jaya/input/hadoop-default.xml <r 2> 21708 >> /user/jaya/input/hadoop-site.xml <r 2> 1333 >> /user/jaya/input/mapred-default.xml <r 2> 180 >> [jaya@localhost hadoop-0.11.0]$ >> >> >> >> [jaya@localhost hadoop-0.11.0]$ bin/hadoop dfs -ls input >> Found 3 items >> /user/jaya/input/hadoop-default.xml <r 2> 21708 >> /user/jaya/input/hadoop-site.xml <r 2> 1333 >> /user/jaya/input/mapred-default.xml <r 2> 180 >> [jaya@localhost hadoop-0.11.0]$ bin/hadoop dfs -ls output >> Found 0 items >> [jaya@localhost hadoop-0.11.0]$ bin/hadoop jar hadoop-0.11.0-examples.jar >> wordcount input output >> java.net.SocketTimeoutException: timed out waiting for rpc response >> at org.apache.hadoop.ipc.Client.call(Client.java:469) >> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164) >> at $Proxy1.getProtocolVersion(Unknown Source) >> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:248) >> at org.apache.hadoop.mapred.JobClient.init(JobClient.java:200) >> at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:192) >> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:381) >> at org.apache.hadoop.examples.WordCount.main(WordCount.java:143) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: >> 39) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm >> pl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at >> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriv >> er.java:71) >> at >> org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:143) >> at >> org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:40) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: >> 39) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm >> pl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at org.apache.hadoop.util.RunJar.main(RunJar.java:155) >> [jaya@localhost hadoop-0.11.0]$ >> >> >> I dont know where the problem is....... >> >> I've not created any directory called output.... if at all we need to >> create >> one, where should we create? >> Should i configure some more settings.... Please explain in detail.... >> >> Please do help me..... >> >> Thanks in advance >> Jaya >> -- >> View this message in context: >> http://www.nabble.com/Detailed-steps-to-run- >> Hadoop-in-distributed-system...-tf3332250.html#a9265480 >> Sent from the Hadoop Users mailing list rmartinez@example.com. > > > > -- View this message in context: http://www.nabble.com/Detailed-steps-to-run-Hadoop-in-distributed-system...-tf3332250.html#a9267312 Sent from the Hadoop Users mailing list rmartinez@example.com. From hadoop-userrmartinez@example.com Fri Mar 02 16:48:38 2007 Return-Path: <hadoop-userrmartinez@example.com> Delivered-To: rmartinez@example.com Received: (qmail 79940 invoked from network); 2 Mar 2007 16:48:37 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Mar 2007 16:48:37 -0000 Received: (qmail 33489 invoked by uid 500); 2 Mar 2007 16:48:44 -0000 Delivered-To: rmartinez@example.com Received: (qmail 33467 invoked by uid 500); 2 Mar 2007 16:48:44 -0000 Mailing-List: contact rmartinez@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:rmartinez@example.com> List-Unsubscribe: <mailto:rmartinez@example.com> List-Post: <mailto:rmartinez@example.com> List-Id: <hadoop-user.lucene.apache.org> Reply-To: rmartinez@example.com Delivered-To: mailing list rmartinez@example.com Received: (qmail 33458 invoked by uid 99); 2 Mar 2007 16:48:44 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Mar 2007 08:48:44 -0800 X-ASF-Spam-Status: No, hits=2.0 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (herse.apache.org: domain of rmartinez@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mu-out-0910.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Mar 2007 08:48:33 -0800 Received: by mu-out-0910.google.com with SMTP id i10so899573mue for <rmartinez@example.com>; Fri, 02 Mar 2007 08:48:11 -0800 (PST) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; b=W/f8zRCfUbbUFriVUSPM75rRoOL4Z3OXLFl20hHr0zbAVUPMHbg0WA+GCAT0fND55SnEkiY5Sy3syPbMUL2r2gEZPzdAqqwCu1T03CvLGNlGmoneAQ31ewRqHNwSsdEGqa0XvM/B6Zm8jQchZtVYrnuAIZmDEFUBfvWJXc0TCsw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; b=XXzZoAuBZ2Dez3LxuLhKJQOWQUnQDAiEIlPHusqx91nFKulLYuqOMDvN8gTGEiQKsTTdxEDWZlGyBOL33+0cATBjElXINnmeq21KA4KGWo+cSKTkl94Dwhjfat06/udcu6hSCK028tTdVkk4uwTKo5sNhVDm07IsrsQo6xQ3qk8= Received: by 83.161.120.155 with SMTP id z3mr1092507bue.1172854090795; Fri, 02 Mar 2007 08:48:10 -0800 (PST) Received: by 83.161.120.155 with HTTP; Fri, 2 Mar 2007 08:48:10 -0800 (PST) Message-ID: <rmartinez@example.com> Date: Fri, 2 Mar 2007 08:48:10 -0800 From: "Albert Chern" <rmartinez@example.com> To: rmartinez@example.com Subject: Re: Detailed steps to run Hadoop in distributed system... In-Reply-To: <rmartinez@example.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_71434_15217365.1172854090409" References: <rmartinez@example.com> X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_71434_15217365.1172854090409 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Your hadoop-site.xml is missing a "<" in the last line, but it looks like you're having a network problem. Can you ping the machines from each other? On 3/1/07, jaylac <rmartinez@example.com> wrote: > > > Hi Hadoop-Users..... > > Have anyone successfully tried running hadoop in two systems? > > I've tried running the wordcount example in one system.. It works fine... > But when i try to add nodes to the cluster and run wordcount example, i > get > errors.... > > So please let me know the detailed steps to be followed... > > Though the steps are given in the hadoop website, i need some help from u > people... > > They might have thought some steps to be obvious and would have not stold > that in the website... > > Im new user... So i simply followed the instructions given... I might have > overlooked some steps which is necessary to run it.... > > Another important doubt.... > > In master node, i have a user name called "jaya"... Is it necessary to > create a user name called "jaya" in the slave system also... or we can > simply use the user name that exist in the slave machine? > > > > Im using two RED HAT LINUX machines... one master(83.161.120.155) and the > other > slave(83.161.120.155) > In master node, the user name is jaya > In slave node, the user name is 146736 > > The steps which i follow is..... > > Edit /home/jaya/.bashrc file > Here ill set the HADOOP_CONF_DIR environment variable > > MASTER NODE > > 1. Edit conf/slaves file.... > Contents > ==================== > localhost > rmartinez@example.com > ==================== > > 2. Edit conf/hadoop-en.sh file > Here ill set the JAVA_HOME environment variable > Thats it.... No other changes in this file.... > PLEASE LET ME KNOW
{'source': 'Apache Software Foundation Public Mail Archives'}