From hama-commits-returnvaughneric@example.org Mon May 2 04:55:48 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id 483903FD5 for <vaughneric@example.org>; Mon, 2 May 2011 04:55:48 +0000 (UTC) Received: (qmail 25276 invoked by uid 500); 2 May 2011 04:55:48 -0000 Delivered-To: vaughneric@example.org Received: (qmail 25244 invoked by uid 500); 2 May 2011 04:55:45 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 25236 invoked by uid 99); 2 May 2011 04:55:43 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 May 2011 04:55:43 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 May 2011 04:55:42 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 9C57A23889C5; Mon, 2 May 2011 04:55:22 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1098478 - /incubator/hama/trunk/CHANGES.txt Date: Mon, 02 May 2011 04:55:22 -0000 To: vaughneric@example.org From: vaughneric@example.org X-Mailer: svnmailer-1.0.8 Message-Id: <vaughneric@example.org> Author: edwardyoon Date: Mon May 2 04:55:22 2011 New Revision: 1098478 URL: http://svn.apache.org/viewvc?rev=1098478&view=rev Log: HAMA-380 is a new feature. Modified: incubator/hama/trunk/CHANGES.txt Modified: incubator/hama/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/incubator/hama/trunk/CHANGES.txt?rev=1098478&r1=1098477&r2=1098478&view=diff ============================================================================== --- incubator/hama/trunk/CHANGES.txt (original) +++ incubator/hama/trunk/CHANGES.txt Mon May 2 04:55:22 2011 @@ -5,14 +5,14 @@ Release 0.3 - Unreleased NEW FEATURES HAMA-374: Add LocalBSPRunner (Thomas Jungblut via edwardyoon) - + HAMA-380: Add BSPMessageBundle to reduce RPC overhead (Miklos Erdelyi via edwardyoon) + BUG FIXES IMPROVEMENTS HAMA-376: Refactor BSPMaster and GroomServer (ChiaHung Lin via edwardyoon) HAMA-382: Refactor HAMA POM (Tommaso Teofili) - HAMA-380: Send messages in batches to reduce RPC overhead (Miklos Erdelyi via edwardyoon) HAMA-362: Re-design a new data structure of BSPMessage (Thomas Jungblut via edwardyoon) HAMA-369: Reduce overhead of local communications (Miklos Erdelyi via edwardyoon) HAMA-371: Add a parameter to build.xml to allow developers deciding to reset content From hama-commits-returnvaughneric@example.org Fri May 6 09:26:08 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id 062103E37 for <vaughneric@example.org>; Fri, 6 May 2011 09:26:08 +0000 (UTC) Received: (qmail 77875 invoked by uid 500); 6 May 2011 09:26:08 -0000 Delivered-To: vaughneric@example.org Received: (qmail 77838 invoked by uid 500); 6 May 2011 09:26:07 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 77828 invoked by uid 99); 6 May 2011 09:26:06 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 May 2011 09:26:06 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 May 2011 09:26:05 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 8671723888FD; Fri, 6 May 2011 09:25:45 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1100148 - in /incubator/hama/trunk: ./ src/java/org/apache/hama/bsp/ src/java/org/apache/hama/util/ src/test/org/apache/hama/ src/test/org/apache/hama/bsp/ Date: Fri, 06 May 2011 09:25:45 -0000 To: vaughneric@example.org From: vaughneric@example.org X-Mailer: svnmailer-1.0.8 Message-Id: <vaughneric@example.org> Author: edwardyoon Date: Fri May 6 09:25:44 2011 New Revision: 1100148 URL: http://svn.apache.org/viewvc?rev=1100148&view=rev Log: Setting default values to job.setNumBspTask() Removed: incubator/hama/trunk/src/test/org/apache/hama/bsp/TestSerializePrinting.java Modified: incubator/hama/trunk/CHANGES.txt incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPJobClient.java incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPMaster.java incubator/hama/trunk/src/java/org/apache/hama/bsp/LocalBSPRunner.java incubator/hama/trunk/src/java/org/apache/hama/util/VersionInfo.java incubator/hama/trunk/src/test/org/apache/hama/HamaClusterTestCase.java incubator/hama/trunk/src/test/org/apache/hama/bsp/TestBSPMasterGroomServer.java Modified: incubator/hama/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/incubator/hama/trunk/CHANGES.txt?rev=1100148&r1=1100147&r2=1100148&view=diff ============================================================================== --- incubator/hama/trunk/CHANGES.txt (original) +++ incubator/hama/trunk/CHANGES.txt Fri May 6 09:25:44 2011 @@ -11,6 +11,7 @@ Release 0.3 - Unreleased IMPROVEMENTS + HAMA-379: Setting default values to job.setNumBspTask() (Thomas Jungblut via edwardyoon) HAMA-376: Refactor BSPMaster and GroomServer (ChiaHung Lin via edwardyoon) HAMA-382: Refactor HAMA POM (Tommaso Teofili) HAMA-362: Re-design a new data structure of BSPMessage (Thomas Jungblut via edwardyoon) Modified: incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPJobClient.java URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPJobClient.java?rev=1100148&r1=1100147&r2=1100148&view=diff ============================================================================== --- incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPJobClient.java (original) +++ incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPJobClient.java Fri May 6 09:25:44 2011 @@ -294,6 +294,18 @@ public class BSPJobClient extends Config */ UnixUserGroupInformation ugi = getUGI(job.getConf()); + ClusterStatus clusterStatus = getClusterStatus(true); + + // check the number of BSP tasks + int tasks = job.getNumBspTask(); + int maxTasks = clusterStatus.getMaxTasks(); + + if (tasks <= 0 || tasks > maxTasks) { + LOG.warn("The number of tasks you've entered was invalid. Using default value of " + + maxTasks + "!"); + job.setNumBspTask(maxTasks); + } + // Create a number of filenames in the BSPMaster's fs namespace FileSystem fs = getFs(); fs.delete(submitJobDir, true); Modified: incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPMaster.java URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPMaster.java?rev=1100148&r1=1100147&r2=1100148&view=diff ============================================================================== --- incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPMaster.java (original) +++ incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPMaster.java Fri May 6 09:25:44 2011 @@ -500,6 +500,10 @@ public class BSPMaster implements JobSub groomPeersMap.put(s.getGroomName(), s.getPeerName()); } } + + // TODO currently we only have one task slot per groom server + this.totalTaskCapacity = numGroomServers; + if (detailed) { return new ClusterStatus(groomPeersMap, totalTasks, totalTaskCapacity, state); Modified: incubator/hama/trunk/src/java/org/apache/hama/bsp/LocalBSPRunner.java URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/java/org/apache/hama/bsp/LocalBSPRunner.java?rev=1100148&r1=1100147&r2=1100148&view=diff ============================================================================== --- incubator/hama/trunk/src/java/org/apache/hama/bsp/LocalBSPRunner.java (original) +++ incubator/hama/trunk/src/java/org/apache/hama/bsp/LocalBSPRunner.java Fri May 6 09:25:44 2011 @@ -122,7 +122,7 @@ public class LocalBSPRunner implements J for (Entry<String, BSPPeerProtocol> entry : localGrooms.entrySet()) { map.put(entry.getKey(), entry.getValue().getPeerName()); } - return new ClusterStatus(map, 0, 1, State.RUNNING); + return new ClusterStatus(map, threadPoolSize, threadPoolSize, State.RUNNING); } @Override Modified: incubator/hama/trunk/src/java/org/apache/hama/util/VersionInfo.java URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/java/org/apache/hama/util/VersionInfo.java?rev=1100148&r1=1100147&r2=1100148&view=diff ============================================================================== --- incubator/hama/trunk/src/java/org/apache/hama/util/VersionInfo.java (original) +++ incubator/hama/trunk/src/java/org/apache/hama/util/VersionInfo.java Fri May 6 09:25:44 2011 @@ -23,6 +23,6 @@ package org.apache.hama.util; public class VersionInfo { public static void main(String[] args) { - System.out.println("Apache Hama - 0.2"); + System.out.println("Apache Hama - 0.3"); } } Modified: incubator/hama/trunk/src/test/org/apache/hama/HamaClusterTestCase.java URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/test/org/apache/hama/HamaClusterTestCase.java?rev=1100148&r1=1100147&r2=1100148&view=diff ============================================================================== --- incubator/hama/trunk/src/test/org/apache/hama/HamaClusterTestCase.java (original) +++ incubator/hama/trunk/src/test/org/apache/hama/HamaClusterTestCase.java Fri May 6 09:25:44 2011 @@ -31,6 +31,7 @@ public abstract class HamaClusterTestCas protected MiniBSPCluster bspCluster; protected MiniZooKeeperCluster zooKeeperCluster; protected boolean startDfs; + protected int numOfGroom = 2; /** default constructor */ public HamaClusterTestCase() { @@ -53,7 +54,7 @@ public abstract class HamaClusterTestCas this.zooKeeperCluster = new MiniZooKeeperCluster(); int clientPort = this.zooKeeperCluster.startup(testDir); conf.set("hama.zookeeper.property.clientPort", Integer.toString(clientPort)); - bspCluster = new MiniBSPCluster(this.conf, 2); + bspCluster = new MiniBSPCluster(this.conf, numOfGroom); bspCluster.startBSPCluster(); } Modified: incubator/hama/trunk/src/test/org/apache/hama/bsp/TestBSPMasterGroomServer.java URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/test/org/apache/hama/bsp/TestBSPMasterGroomServer.java?rev=1100148&r1=1100147&r2=1100148&view=diff ============================================================================== --- incubator/hama/trunk/src/test/org/apache/hama/bsp/TestBSPMasterGroomServer.java (original) +++ incubator/hama/trunk/src/test/org/apache/hama/bsp/TestBSPMasterGroomServer.java Fri May 6 09:25:44 2011 @@ -57,6 +57,7 @@ public class TestBSPMasterGroomServer ex // Set the task size as a number of GroomServer BSPJobClient jobClient = new BSPJobClient(configuration); ClusterStatus cluster = jobClient.getClusterStatus(false); + assertEquals(this.numOfGroom, cluster.getMaxTasks()); bsp.setNumBspTask(cluster.getGroomServers()); FileSystem fileSys = FileSystem.get(conf); From hama-commits-returnvaughneric@example.org Sat May 7 11:50:43 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id 5B96D2ADF for <vaughneric@example.org>; Sat, 7 May 2011 11:50:43 +0000 (UTC) Received: (qmail 2766 invoked by uid 500); 7 May 2011 11:50:43 -0000 Delivered-To: vaughneric@example.org Received: (qmail 2748 invoked by uid 500); 7 May 2011 11:50:42 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 2740 invoked by uid 99); 7 May 2011 11:50:42 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 07 May 2011 11:50:42 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eos.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 07 May 2011 11:50:40 +0000 Received: from eos.apache.org (localhost [83.161.120.155]) by eos.apache.org (Postfix) with ESMTP id D9F32E7F for <vaughneric@example.org>; Sat, 7 May 2011 11:50:19 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Apache Wiki <vaughneric@example.org> To: Apache Wiki <vaughneric@example.org> Date: Sat, 07 May 2011 11:50:19 -0000 Message-ID: <vaughneric@example.org> Subject: =?utf-8?q?=5BHama_Wiki=5D_Trivial_Update_of_=22HowToContribute=22_by_edwa?= =?utf-8?q?rdyoon?= X-Virus-Checked: Checked by ClamAV on apache.org Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hama Wiki" for chan= ge notification. The "HowToContribute" page has been changed by edwardyoon. http://wiki.apache.org/hama/HowToContribute?action=3Ddiff&rev1=3D9&rev2=3D10 -------------------------------------------------- = {{{ > cd hama-trunk - > ant clean test docs -Dforrest.home=3D{$FORREST_HOME} + > mvn install package }}} = After a while, if you see From hama-commits-returnvaughneric@example.org Sat May 7 17:42:33 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id EE76821A2 for <vaughneric@example.org>; Sat, 7 May 2011 17:42:32 +0000 (UTC) Received: (qmail 19337 invoked by uid 500); 7 May 2011 17:42:32 -0000 Delivered-To: vaughneric@example.org Received: (qmail 19303 invoked by uid 500); 7 May 2011 17:42:32 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 19295 invoked by uid 99); 7 May 2011 17:42:32 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 07 May 2011 17:42:32 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eos.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 07 May 2011 17:42:31 +0000 Received: from eos.apache.org (localhost [83.161.120.155]) by eos.apache.org (Postfix) with ESMTP id 7EA73EC4 for <vaughneric@example.org>; Sat, 7 May 2011 17:42:11 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Apache Wiki <vaughneric@example.org> To: Apache Wiki <vaughneric@example.org> Date: Sat, 07 May 2011 17:42:11 -0000 Message-ID: <vaughneric@example.org> Subject: =?utf-8?q?=5BHama_Wiki=5D_Update_of_=22SSSP=22_by_thomasjungblut?= Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hama Wiki" for chan= ge notification. The "SSSP" page has been changed by thomasjungblut. http://wiki.apache.org/hama/SSSP -------------------------------------------------- New page: =3D=3D Single Source Shortest Paths =3D=3D * Uses the SSSP algorithm described in the Google Pregel paper * Introduces partitioning and collective communication * Lets the user submit his/her own SequenceFile to calculate the SSSP's =3D=3D Implementation =3D=3D For detailed questions in terms of implementation have a look at my blog. It describes the algorithm and focuses on the main ideas showing implementa= tion things. http://codingwiththomas.blogspot.com/2011/05/shortest-path-finding-with-apa= che-hama.html =3D=3D Usage =3D=3D {{{ hama/bin/hama jar ../hama-0.x.0-examples.jar sssp <name of the start vertex= > <optional: output path> <optional: path of your own sequencefile> }}} Change "x" to the version you are using! Note: If you provide your own sequencefile, make sure you've set the output= path. '''The output path should never be the root path!''' =3D=3D Submit your own Graph =3D=3D You can transform your graph as a adjacency list to fit into the input whic= h Hama is going to parse and calculate the shortest paths between the verti= ces. The file that Hama can successfully parse is a SequenceFile that contains b= oth Key and Value as a Text. = {{{ K / V = Vertex[Text] / AdjacentVertex : Weight [Text] }}} A vertex typically contains a name that uniquely identifies a vertex. So yo= u are associating a key vertex that just contains a name to another vertex = (that contains its name) to the weight. Both seperated by ":". Let's look at this sample: {{{ K / V Berlin / Paris : 25 Berlin / London : 40 London / Paris : 10 }}} This will adjacent Berlin to Paris and London, and London to Paris with the= given weights. If you are familiar with MapReduce, this looks like a mapper output that ca= n be easily reduced. = '''Notes:''' Make sure... * your names are unique! Otherwise it will result in strange output. * it is only ONE file! * Key and Value are Text.class fields! * Value always contains only ONE vertex * Value is separated by ":" to split the name and the weight Have fun! If you are facing problems, feel free to ask questions on the off= icial mailing list. From hama-commits-returnvaughneric@example.org Sat May 7 17:43:02 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id 93FB221B2 for <vaughneric@example.org>; Sat, 7 May 2011 17:43:02 +0000 (UTC) Received: (qmail 19487 invoked by uid 500); 7 May 2011 17:43:02 -0000 Delivered-To: vaughneric@example.org Received: (qmail 19470 invoked by uid 500); 7 May 2011 17:43:02 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 19462 invoked by uid 99); 7 May 2011 17:43:02 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 07 May 2011 17:43:02 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eos.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 07 May 2011 17:43:01 +0000 Received: from eos.apache.org (localhost [83.161.120.155]) by eos.apache.org (Postfix) with ESMTP id A944CEC5; Sat, 7 May 2011 17:42:41 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Apache Wiki <vaughneric@example.org> To: Apache Wiki <vaughneric@example.org> Date: Sat, 07 May 2011 17:42:41 -0000 Message-ID: <vaughneric@example.org> Subject: =?utf-8?q?=5BHama_Wiki=5D_Update_of_=22FrontPage=22_by_thomasjungblut?= Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hama Wiki" for chan= ge notification. The "FrontPage" page has been changed by thomasjungblut. http://wiki.apache.org/hama/FrontPage?action=3Ddiff&rev1=3D93&rev2=3D94 -------------------------------------------------- * Examples * [[SerializePrinting|Serialize Printing Of "Hello BSP"]] * [[PiEstimator|BSP based Pi Estimator]] + * [[SSSP|Single Source Shortest Paths]] = =3D=3D Developer Documentation =3D=3D =20 From hama-commits-returnvaughneric@example.org Sun May 8 17:43:39 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id 53C8336B0 for <vaughneric@example.org>; Sun, 8 May 2011 17:43:39 +0000 (UTC) Received: (qmail 76298 invoked by uid 500); 8 May 2011 17:43:39 -0000 Delivered-To: vaughneric@example.org Received: (qmail 76266 invoked by uid 500); 8 May 2011 17:43:38 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 76258 invoked by uid 99); 8 May 2011 17:43:38 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 08 May 2011 17:43:38 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED,MIME_QP_LONG_LINE X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eos.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 08 May 2011 17:43:37 +0000 Received: from eos.apache.org (localhost [83.161.120.155]) by eos.apache.org (Postfix) with ESMTP id 94EF5F90 for <vaughneric@example.org>; Sun, 8 May 2011 17:43:17 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Apache Wiki <vaughneric@example.org> To: Apache Wiki <vaughneric@example.org> Date: Sun, 08 May 2011 17:43:17 -0000 Message-ID: <vaughneric@example.org> Subject: =?utf-8?q?=5BHama_Wiki=5D_Update_of_=22SSSP=22_by_thomasjungblut?= Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hama Wiki" for chan= ge notification. The "SSSP" page has been changed by thomasjungblut. http://wiki.apache.org/hama/SSSP?action=3Ddiff&rev1=3D1&rev2=3D2 -------------------------------------------------- * Value always contains only ONE vertex * Value is separated by ":" to split the name and the weight = + =3D=3D Sample adjacency list SequenceFile =3D=3D + = + You can download a adjacencylist SequenceFile containing 2,031,215 vertic= es and 24,793,713 edges, named with cities in the world and random weights. = + It is arround 1.14GB large and can be downloaded here: + = + http://code.google.com/p/hama-shortest-paths/source/browse/trunk/hama-gso= c/files/cities-adjacencylist/adjacencylist.seq2 + = + You can run it with + = + {{{ + hama/bin/hama jar ../hama-0.x.0-examples.jar sssp Klewno sample/output PA= TH_TO_THE_SEQUENCEFILE + }}} + = + Obviously you have to replace "PATH_TO_THE_SEQUENCEFILE" with the path wh= ere the sample file is stored. + = + You can adjust the starting vertex city (here: "Klewno") to the city you = want, I'm pretty sure the graph contains it. + = + = Have fun! If you are facing problems, feel free to ask questions on the o= fficial mailing list. =20 From hama-commits-returnvaughneric@example.org Sun May 8 17:46:56 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id 9175B348B for <vaughneric@example.org>; Sun, 8 May 2011 17:46:56 +0000 (UTC) Received: (qmail 76496 invoked by uid 500); 8 May 2011 17:46:56 -0000 Delivered-To: vaughneric@example.org Received: (qmail 76474 invoked by uid 500); 8 May 2011 17:46:56 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 76466 invoked by uid 99); 8 May 2011 17:46:56 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 08 May 2011 17:46:56 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED,MIME_QP_LONG_LINE X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eos.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 08 May 2011 17:46:55 +0000 Received: from eos.apache.org (localhost [83.161.120.155]) by eos.apache.org (Postfix) with ESMTP id 9C864F91 for <vaughneric@example.org>; Sun, 8 May 2011 17:46:35 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Apache Wiki <vaughneric@example.org> To: Apache Wiki <vaughneric@example.org> Date: Sun, 08 May 2011 17:46:35 -0000 Message-ID: <vaughneric@example.org> Subject: =?utf-8?q?=5BHama_Wiki=5D_Update_of_=22SSSP=22_by_thomasjungblut?= Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hama Wiki" for chan= ge notification. The "SSSP" page has been changed by thomasjungblut. http://wiki.apache.org/hama/SSSP?action=3Ddiff&rev1=3D2&rev2=3D3 -------------------------------------------------- K / V Berlin / Paris : 25 Berlin / London : 40 - London / Paris : 10 + London / Paris : 10 = + Paris / Berlin : 25 }}} = This will adjacent Berlin to Paris and London, and London to Paris with t= he given weights. @@ -56, +57 @@ = Make sure... = - * your names are unique! Otherwise it will result in strange output. + * your names are unique! Otherwise it will result in strange output or e= xceptions. + * every adjacent vertex can be somewhere found within the SequenceFile a= s a key. * it is only ONE file! * Key and Value are Text.class fields! * Value always contains only ONE vertex @@ -64, +66 @@ = =3D=3D Sample adjacency list SequenceFile =3D=3D = - You can download a adjacencylist SequenceFile containing 2,031,215 vertic= es and 24,793,713 edges, named with cities in the world and random weights. = + You can download an adjacencylist SequenceFile containing 2,031,215 verti= ces and 24,793,713 edges, named with cities in the world and random weights= . = It is arround 1.14GB large and can be downloaded here: = http://code.google.com/p/hama-shortest-paths/source/browse/trunk/hama-gso= c/files/cities-adjacencylist/adjacencylist.seq2 From hama-commits-returnvaughneric@example.org Sun May 8 17:53:54 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id 57A5F3B1A for <vaughneric@example.org>; Sun, 8 May 2011 17:53:54 +0000 (UTC) Received: (qmail 81295 invoked by uid 500); 8 May 2011 17:53:54 -0000 Delivered-To: vaughneric@example.org Received: (qmail 81281 invoked by uid 500); 8 May 2011 17:53:54 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 81264 invoked by uid 99); 8 May 2011 17:53:54 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 08 May 2011 17:53:54 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eos.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 08 May 2011 17:53:53 +0000 Received: from eos.apache.org (localhost [83.161.120.155]) by eos.apache.org (Postfix) with ESMTP id 268F1F92 for <vaughneric@example.org>; Sun, 8 May 2011 17:53:33 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Apache Wiki <vaughneric@example.org> To: Apache Wiki <vaughneric@example.org> Date: Sun, 08 May 2011 17:53:32 -0000 Message-ID: <vaughneric@example.org> Subject: =?utf-8?q?=5BHama_Wiki=5D_Update_of_=22SSSP=22_by_thomasjungblut?= Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hama Wiki" for chan= ge notification. The "SSSP" page has been changed by thomasjungblut. http://wiki.apache.org/hama/SSSP?action=3Ddiff&rev1=3D3&rev2=3D4 -------------------------------------------------- = {{{ K / V - Berlin / Paris : 25 + Berlin / Paris:25 - Berlin / London : 40 + Berlin / London:40 - London / Paris : 10 = + London / Paris:10 = - Paris / Berlin : 25 + Paris / Berlin:25 }}} = This will adjacent Berlin to Paris and London, and London to Paris with t= he given weights. @@ -58, +58 @@ Make sure... = * your names are unique! Otherwise it will result in strange output or e= xceptions. + * you watch out for leading and trailing whitespaces in the names, this = will cause vertices to not find each other! For instance "_Paris" is not "P= aris". * every adjacent vertex can be somewhere found within the SequenceFile a= s a key. * it is only ONE file! * Key and Value are Text.class fields! From hama-commits-returnvaughneric@example.org Wed May 11 02:28:56 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id D81DD46FE for <vaughneric@example.org>; Wed, 11 May 2011 02:28:56 +0000 (UTC) Received: (qmail 51809 invoked by uid 500); 11 May 2011 02:28:56 -0000 Delivered-To: vaughneric@example.org Received: (qmail 51777 invoked by uid 500); 11 May 2011 02:28:56 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 51768 invoked by uid 99); 11 May 2011 02:28:56 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 May 2011 02:28:56 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eos.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 May 2011 02:28:49 +0000 Received: from eos.apache.org (localhost [83.161.120.155]) by eos.apache.org (Postfix) with ESMTP id D1ED51DC for <vaughneric@example.org>; Wed, 11 May 2011 02:28:28 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Apache Wiki <vaughneric@example.org> To: Apache Wiki <vaughneric@example.org> Date: Wed, 11 May 2011 02:28:28 -0000 Message-ID: <vaughneric@example.org> Subject: =?utf-8?q?=5BHama_Wiki=5D_Update_of_=22GroomServerFaultTolerance=22_by_Ch?= =?utf-8?q?iaHungLin?= X-Virus-Checked: Checked by ClamAV on apache.org Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hama Wiki" for chan= ge notification. The "GroomServerFaultTolerance" page has been changed by ChiaHungLin. http://wiki.apache.org/hama/GroomServerFaultTolerance?action=3Ddiff&rev1=3D= 6&rev2=3D7 -------------------------------------------------- = =3D=3D=3D Literature Review =3D=3D=3D = - In general, a system designed to deal with failures largely bases on the = concepts including unit of mitigation, redundancy, fault observer[4]. = + In general, a system designed to deal with failures usually need to apply= techniques including unit of mitigation, redundancy, fault detection, fau= lt recovery[4], and so on. = = = - The architecture defines the basic unit which performs functions of a sys= tem according to requirements. = + Unit of mitigation: GroomServer(s)/ BSPMaster = - Providing redundant units. = + Redundant units: GroomServer(s) = - Fault observers are designed to detect fault or error in an earlier stage= so that other strategies, such as error recovery can be employed to correc= t the problem. = + Fault detection: System monitor, heartbeat. = - = + Fault recovery: Fail over = =3D=3D=3D Architecture =3D=3D=3D '''Task Failure''' From hama-commits-returnvaughneric@example.org Fri May 13 04:54:34 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id B6B834BCB for <vaughneric@example.org>; Fri, 13 May 2011 04:54:34 +0000 (UTC) Received: (qmail 17824 invoked by uid 500); 13 May 2011 04:54:34 -0000 Delivered-To: vaughneric@example.org Received: (qmail 17792 invoked by uid 500); 13 May 2011 04:54:33 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 17778 invoked by uid 99); 13 May 2011 04:54:32 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 13 May 2011 04:54:32 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 13 May 2011 04:54:29 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id D4CF22388901; Fri, 13 May 2011 04:54:07 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1102562 - in /incubator/hama/trunk: CHANGES.txt src/java/org/apache/hama/bsp/BSPPeer.java src/java/org/apache/hama/bsp/ByteMessage.java src/test/org/apache/hama/bsp/TestBSPPeer.java src/test/org/apache/hama/bsp/TestMessages.java Date: Fri, 13 May 2011 04:54:07 -0000 To: vaughneric@example.org From: vaughneric@example.org X-Mailer: svnmailer-1.0.8 Message-Id: <vaughneric@example.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: edwardyoon Date: Fri May 13 04:54:07 2011 New Revision: 1102562 URL: http://svn.apache.org/viewvc?rev=1102562&view=rev Log: Removing 'System.arraycopy(...)' call to reduce memory/garbage collection usage Added: incubator/hama/trunk/src/test/org/apache/hama/bsp/TestMessages.java Modified: incubator/hama/trunk/CHANGES.txt incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPPeer.java incubator/hama/trunk/src/java/org/apache/hama/bsp/ByteMessage.java incubator/hama/trunk/src/test/org/apache/hama/bsp/TestBSPPeer.java Modified: incubator/hama/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/incubator/hama/trunk/CHANGES.txt?rev=1102562&r1=1102561&r2=1102562&view=diff ============================================================================== --- incubator/hama/trunk/CHANGES.txt (original) +++ incubator/hama/trunk/CHANGES.txt Fri May 13 04:54:07 2011 @@ -11,6 +11,7 @@ Release 0.3 - Unreleased IMPROVEMENTS + HAMA-386: Removing 'System.arraycopy(...)' call from ByteMessage (edwardyoon) HAMA-379: Setting default values to job.setNumBspTask() (Thomas Jungblut via edwardyoon) HAMA-376: Refactor BSPMaster and GroomServer (ChiaHung Lin via edwardyoon) HAMA-382: Refactor HAMA POM (Tommaso Teofili) Modified: incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPPeer.java URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPPeer.java?rev=1102562&r1=1102561&r2=1102562&view=diff ============================================================================== --- incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPPeer.java (original) +++ incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPPeer.java Fri May 13 04:54:07 2011 @@ -35,6 +35,7 @@ import org.apache.hadoop.conf.Configurat import org.apache.hadoop.ipc.RPC; import org.apache.hadoop.ipc.RPC.Server; import org.apache.hama.Constants; +import org.apache.hama.util.Bytes; import org.apache.zookeeper.CreateMode; import org.apache.zookeeper.KeeperException; import org.apache.zookeeper.WatchedEvent; Modified: incubator/hama/trunk/src/java/org/apache/hama/bsp/ByteMessage.java URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/java/org/apache/hama/bsp/ByteMessage.java?rev=1102562&r1=1102561&r2=1102562&view=diff ============================================================================== --- incubator/hama/trunk/src/java/org/apache/hama/bsp/ByteMessage.java (original) +++ incubator/hama/trunk/src/java/org/apache/hama/bsp/ByteMessage.java Fri May 13 04:54:07 2011 @@ -32,22 +32,18 @@ public class ByteMessage extends BSPMess public ByteMessage(byte[] tag, byte[] data) { super(); - this.tag = new byte[tag.length]; - this.data = new byte[data.length]; - System.arraycopy(tag, 0, this.tag, 0, tag.length); - System.arraycopy(data, 0, this.data, 0, data.length); + this.tag = tag; + this.data = data; } @Override public byte[] getTag() { - byte[] result = this.tag; - return result; + return this.tag; } @Override public byte[] getData() { - byte[] result = this.data; - return result; + return this.data; } @Override Modified: incubator/hama/trunk/src/test/org/apache/hama/bsp/TestBSPPeer.java URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/test/org/apache/hama/bsp/TestBSPPeer.java?rev=1102562&r1=1102561&r2=1102562&view=diff ============================================================================== --- incubator/hama/trunk/src/test/org/apache/hama/bsp/TestBSPPeer.java (original) +++ incubator/hama/trunk/src/test/org/apache/hama/bsp/TestBSPPeer.java Fri May 13 04:54:07 2011 @@ -47,7 +47,7 @@ import org.apache.zookeeper.data.Stat; public class TestBSPPeer extends HamaCluster implements Watcher { private Log LOG = LogFactory.getLog(TestBSPPeer.class); - private static final int NUM_PEER = 10; + private static final int NUM_PEER = 3; private static final int ROUND = 3; private static final int PAYLOAD = 1024; // 1kb in default List<BSPPeerThread> list = new ArrayList<BSPPeerThread>(NUM_PEER); @@ -105,16 +105,16 @@ public class TestBSPPeer extends HamaClu @Override public void runTest() throws AssertionFailedError { int randomTime; - byte[] dummyData = new byte[PAYLOAD]; - BSPMessage msg = null; for (int i = 0; i < ROUND; i++) { randomTime = r.nextInt(MAXIMUM_DURATION) + 5; - for (int j = 0; j < 10; j++) { + for (int j = 0; j < 3; j++) { + byte[] dummyData = new byte[PAYLOAD]; r.nextBytes(dummyData); - msg = new ByteMessage(Bytes.tail(dummyData, 128), dummyData); String peerName = "localhost:" + (30000 + j); + ByteMessage msg = new ByteMessage(Bytes.tail(dummyData, 128), + dummyData); try { peer.send(peerName, msg); } catch (IOException e) { @@ -149,8 +149,8 @@ public class TestBSPPeer extends HamaClu LOG.info("[" + peer.getPeerName() + "] verifying " + numMessages + " messages at " + round + " round"); - if (lastTwoDigitsOfPort < 10) { - assertEquals(10, numMessages); + if (lastTwoDigitsOfPort < 3) { + assertEquals(3, numMessages); } else { assertEquals(0, numMessages); } Added: incubator/hama/trunk/src/test/org/apache/hama/bsp/TestMessages.java URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/test/org/apache/hama/bsp/TestMessages.java?rev=1102562&view=auto ============================================================================== --- incubator/hama/trunk/src/test/org/apache/hama/bsp/TestMessages.java (added) +++ incubator/hama/trunk/src/test/org/apache/hama/bsp/TestMessages.java Fri May 13 04:54:07 2011 @@ -0,0 +1,21 @@ +package org.apache.hama.bsp; + +import junit.framework.TestCase; + +import org.apache.hama.util.Bytes; + +public class TestMessages extends TestCase { + + public void testByteMessage() { + int dataSize = (int) (Runtime.getRuntime().maxMemory() * 0.60); + ByteMessage msg = new ByteMessage(Bytes.toBytes("tag"), new byte[dataSize]); + assertEquals(msg.getData().length, dataSize); + msg = null; + + byte[] dummyData = new byte[1024]; + ByteMessage msg2 = new ByteMessage(Bytes.tail(dummyData, 128), dummyData); + assertEquals( + Bytes.compareTo(msg2.getTag(), 0, 128, msg2.getData(), + msg2.getData().length - 128, 128), 0); + } +} From hama-commits-returnvaughneric@example.org Mon May 16 00:13:50 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id BD22F41D4 for <vaughneric@example.org>; Mon, 16 May 2011 00:13:50 +0000 (UTC) Received: (qmail 2334 invoked by uid 500); 16 May 2011 00:13:50 -0000 Delivered-To: vaughneric@example.org Received: (qmail 2301 invoked by uid 500); 16 May 2011 00:13:50 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 2293 invoked by uid 99); 16 May 2011 00:13:50 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 16 May 2011 00:13:50 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 16 May 2011 00:13:47 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 1764523888E4; Mon, 16 May 2011 00:13:26 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1103580 - /incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPPeer.java Date: Mon, 16 May 2011 00:13:26 -0000 To: vaughneric@example.org From: vaughneric@example.org X-Mailer: svnmailer-1.0.8 Message-Id: <vaughneric@example.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: edwardyoon Date: Mon May 16 00:13:25 2011 New Revision: 1103580 URL: http://svn.apache.org/viewvc?rev=1103580&view=rev Log: Remove unused import Modified: incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPPeer.java Modified: incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPPeer.java URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPPeer.java?rev=1103580&r1=1103579&r2=1103580&view=diff ============================================================================== --- incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPPeer.java (original) +++ incubator/hama/trunk/src/java/org/apache/hama/bsp/BSPPeer.java Mon May 16 00:13:25 2011 @@ -23,9 +23,9 @@ import java.util.Collection; import java.util.Iterator; import java.util.List; import java.util.Map; +import java.util.Map.Entry; import java.util.SortedSet; import java.util.TreeSet; -import java.util.Map.Entry; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentLinkedQueue; @@ -35,13 +35,12 @@ import org.apache.hadoop.conf.Configurat import org.apache.hadoop.ipc.RPC; import org.apache.hadoop.ipc.RPC.Server; import org.apache.hama.Constants; -import org.apache.hama.util.Bytes; import org.apache.zookeeper.CreateMode; import org.apache.zookeeper.KeeperException; import org.apache.zookeeper.WatchedEvent; import org.apache.zookeeper.Watcher; -import org.apache.zookeeper.ZooKeeper; import org.apache.zookeeper.ZooDefs.Ids; +import org.apache.zookeeper.ZooKeeper; import org.apache.zookeeper.data.Stat; /** From hama-commits-returnvaughneric@example.org Mon May 16 05:08:10 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id 1126D6AB7 for <vaughneric@example.org>; Mon, 16 May 2011 05:08:10 +0000 (UTC) Received: (qmail 30362 invoked by uid 500); 16 May 2011 05:08:08 -0000 Delivered-To: vaughneric@example.org Received: (qmail 30315 invoked by uid 500); 16 May 2011 05:08:03 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 30304 invoked by uid 99); 16 May 2011 05:08:01 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 16 May 2011 05:08:01 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eos.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 16 May 2011 05:07:58 +0000 Received: from eos.apache.org (localhost [83.161.120.155]) by eos.apache.org (Postfix) with ESMTP id BBDCF965 for <vaughneric@example.org>; Mon, 16 May 2011 05:07:37 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Apache Wiki <vaughneric@example.org> To: Apache Wiki <vaughneric@example.org> Date: Mon, 16 May 2011 05:07:37 -0000 Message-ID: <vaughneric@example.org> Subject: =?utf-8?q?=5BHama_Wiki=5D_Trivial_Update_of_=22HowToContribute=22_by_edwa?= =?utf-8?q?rdyoon?= X-Virus-Checked: Checked by ClamAV on apache.org Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hama Wiki" for chan= ge notification. The "HowToContribute" page has been changed by edwardyoon. http://wiki.apache.org/hama/HowToContribute?action=3Ddiff&rev1=3D10&rev2=3D= 11 -------------------------------------------------- {{{ > cd hama-trunk > mvn install package + > mvn -Pexamples package + > mvn -Pforrest package -Dforrest.home=3D{$FORREST_HOME} }}} = After a while, if you see From hama-commits-returnvaughneric@example.org Mon May 16 08:58:06 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id 0D5C366E7 for <vaughneric@example.org>; Mon, 16 May 2011 08:58:06 +0000 (UTC) Received: (qmail 26693 invoked by uid 500); 16 May 2011 08:58:06 -0000 Delivered-To: vaughneric@example.org Received: (qmail 26661 invoked by uid 500); 16 May 2011 08:58:05 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 26652 invoked by uid 99); 16 May 2011 08:58:05 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 16 May 2011 08:58:05 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eos.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 16 May 2011 08:58:04 +0000 Received: from eos.apache.org (localhost [83.161.120.155]) by eos.apache.org (Postfix) with ESMTP id 733FB9DE for <vaughneric@example.org>; Mon, 16 May 2011 08:57:44 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Apache Wiki <vaughneric@example.org> To: Apache Wiki <vaughneric@example.org> Date: Mon, 16 May 2011 08:57:44 -0000 Message-ID: <vaughneric@example.org> Subject: =?utf-8?q?=5BHama_Wiki=5D_Trivial_Update_of_=22FrontPage=22_by_edwardyoon?= Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hama Wiki" for chan= ge notification. The "FrontPage" page has been changed by edwardyoon. The comment on this change is: add status . http://wiki.apache.org/hama/FrontPage?action=3Ddiff&rev1=3D94&rev2=3D95 -------------------------------------------------- * Examples * [[SerializePrinting|Serialize Printing Of "Hello BSP"]] * [[PiEstimator|BSP based Pi Estimator]] - * [[SSSP|Single Source Shortest Paths]] + * [[SSSP|Single Source Shortest Paths]] -- Work in progress = =3D=3D Developer Documentation =3D=3D =20 From hama-commits-returnvaughneric@example.org Tue May 17 00:13:51 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id AB3346DAA for <vaughneric@example.org>; Tue, 17 May 2011 00:13:51 +0000 (UTC) Received: (qmail 49462 invoked by uid 500); 17 May 2011 00:13:51 -0000 Delivered-To: vaughneric@example.org Received: (qmail 49429 invoked by uid 500); 17 May 2011 00:13:51 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 49421 invoked by uid 99); 17 May 2011 00:13:51 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 May 2011 00:13:51 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 May 2011 00:13:48 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id DA7F423888FE; Tue, 17 May 2011 00:13:26 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1103959 - in /incubator/hama/trunk: CHANGES.txt conf/hama-default.xml src/java/org/apache/hama/bsp/LocalBSPRunner.java Date: Tue, 17 May 2011 00:13:26 -0000 To: vaughneric@example.org From: vaughneric@example.org X-Mailer: svnmailer-1.0.8 Message-Id: <vaughneric@example.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: edwardyoon Date: Tue May 17 00:13:26 2011 New Revision: 1103959 URL: http://svn.apache.org/viewvc?rev=1103959&view=rev Log: Make the thread pool size user configurable Modified: incubator/hama/trunk/CHANGES.txt incubator/hama/trunk/conf/hama-default.xml incubator/hama/trunk/src/java/org/apache/hama/bsp/LocalBSPRunner.java Modified: incubator/hama/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/incubator/hama/trunk/CHANGES.txt?rev=1103959&r1=1103958&r2=1103959&view=diff ============================================================================== --- incubator/hama/trunk/CHANGES.txt (original) +++ incubator/hama/trunk/CHANGES.txt Tue May 17 00:13:26 2011 @@ -11,6 +11,7 @@ Release 0.3 - Unreleased IMPROVEMENTS + HAMA-389: Make the thread pool size user configurable (Thomas Jungblut via edwardyoon) HAMA-386: Removing 'System.arraycopy(...)' call from ByteMessage (edwardyoon) HAMA-379: Setting default values to job.setNumBspTask() (Thomas Jungblut via edwardyoon) HAMA-376: Refactor BSPMaster and GroomServer (ChiaHung Lin via edwardyoon) Modified: incubator/hama/trunk/conf/hama-default.xml URL: http://svn.apache.org/viewvc/incubator/hama/trunk/conf/hama-default.xml?rev=1103959&r1=1103958&r2=1103959&view=diff ============================================================================== --- incubator/hama/trunk/conf/hama-default.xml (original) +++ incubator/hama/trunk/conf/hama-default.xml Tue May 17 00:13:26 2011 @@ -83,6 +83,11 @@ maximum virtual memory of the child processes. </description> </property> + <property> + <name>bsp.local.tasks.maximum</name> + <value>20</value> + <description>Number of tasks that run in parallel when in local mode.</description> + </property> <!-- Beginning of properties that are directly mapped from ZooKeeper's zoo.cfg. Modified: incubator/hama/trunk/src/java/org/apache/hama/bsp/LocalBSPRunner.java URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/java/org/apache/hama/bsp/LocalBSPRunner.java?rev=1103959&r1=1103958&r2=1103959&view=diff ============================================================================== --- incubator/hama/trunk/src/java/org/apache/hama/bsp/LocalBSPRunner.java (original) +++ incubator/hama/trunk/src/java/org/apache/hama/bsp/LocalBSPRunner.java Tue May 17 00:13:26 2011 @@ -50,7 +50,7 @@ public class LocalBSPRunner implements J private static final String IDENTIFIER = "localrunner"; private static String WORKING_DIR = "/user/hama/bsp/"; protected static volatile ThreadPoolExecutor threadPool; - protected static final int threadPoolSize; + protected static int threadPoolSize; protected static final LinkedList<Future<BSP>> futureList = new LinkedList<Future<BSP>>(); protected static CyclicBarrier barrier; @@ -70,13 +70,6 @@ public class LocalBSPRunner implements J protected Configuration conf; protected FileSystem fs; - { - for (int i = 0; i < threadPoolSize; i++) { - String name = IDENTIFIER + " " + i; - localGrooms.put(name, new LocalGroom(name)); - } - } - public LocalBSPRunner(Configuration conf) throws IOException { super(); this.conf = conf; @@ -84,6 +77,18 @@ public class LocalBSPRunner implements J String path = conf.get("bsp.local.dir"); if (path != null && !path.isEmpty()) WORKING_DIR = path; + + int overridenSize = conf.getInt("bsp.local.tasks.maximum", 20); + threadPoolSize = overridenSize; + threadPool = (ThreadPoolExecutor) Executors + .newFixedThreadPool(threadPoolSize); + barrier = new CyclicBarrier(threadPoolSize); + + for (int i = 0; i < threadPoolSize; i++) { + String name = IDENTIFIER + " " + i; + localGrooms.put(name, new LocalGroom(name)); + } + } @Override From hama-commits-returnvaughneric@example.org Tue May 17 02:05:47 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id B19614B16 for <vaughneric@example.org>; Tue, 17 May 2011 02:05:47 +0000 (UTC) Received: (qmail 50017 invoked by uid 500); 17 May 2011 02:05:47 -0000 Delivered-To: vaughneric@example.org Received: (qmail 49998 invoked by uid 500); 17 May 2011 02:05:47 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 49990 invoked by uid 99); 17 May 2011 02:05:47 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 May 2011 02:05:47 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 May 2011 02:05:45 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 9763E23888DD; Tue, 17 May 2011 02:05:24 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1103982 - /incubator/hama/trunk/conf/hama-default.xml Date: Tue, 17 May 2011 02:05:24 -0000 To: vaughneric@example.org From: vaughneric@example.org X-Mailer: svnmailer-1.0.8 Message-Id: <vaughneric@example.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: edwardyoon Date: Tue May 17 02:05:24 2011 New Revision: 1103982 URL: http://svn.apache.org/viewvc?rev=1103982&view=rev Log: re-formatting Modified: incubator/hama/trunk/conf/hama-default.xml Modified: incubator/hama/trunk/conf/hama-default.xml URL: http://svn.apache.org/viewvc/incubator/hama/trunk/conf/hama-default.xml?rev=1103982&r1=1103981&r2=1103982&view=diff ============================================================================== --- incubator/hama/trunk/conf/hama-default.xml (original) +++ incubator/hama/trunk/conf/hama-default.xml Tue May 17 02:05:24 2011 @@ -83,7 +83,7 @@ maximum virtual memory of the child processes. </description> </property> - <property> + <property> <name>bsp.local.tasks.maximum</name> <value>20</value> <description>Number of tasks that run in parallel when in local mode.</description> From hama-commits-returnvaughneric@example.org Tue May 17 07:24:09 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id 9DEF34445 for <vaughneric@example.org>; Tue, 17 May 2011 07:24:09 +0000 (UTC) Received: (qmail 86736 invoked by uid 500); 17 May 2011 07:24:09 -0000 Delivered-To: vaughneric@example.org Received: (qmail 86626 invoked by uid 500); 17 May 2011 07:24:05 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 86617 invoked by uid 99); 17 May 2011 07:24:03 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 May 2011 07:24:03 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 May 2011 07:24:02 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 2AA9E23888EA; Tue, 17 May 2011 07:23:42 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1104034 - /incubator/hama/trunk/src/java/org/apache/hama/bsp/LocalBSPRunner.java Date: Tue, 17 May 2011 07:23:42 -0000 To: vaughneric@example.org From: vaughneric@example.org X-Mailer: svnmailer-1.0.8 Message-Id: <vaughneric@example.org> Author: edwardyoon Date: Tue May 17 07:23:41 2011 New Revision: 1104034 URL: http://svn.apache.org/viewvc?rev=1104034&view=rev Log: a trivial code change. Modified: incubator/hama/trunk/src/java/org/apache/hama/bsp/LocalBSPRunner.java Modified: incubator/hama/trunk/src/java/org/apache/hama/bsp/LocalBSPRunner.java URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/java/org/apache/hama/bsp/LocalBSPRunner.java?rev=1104034&r1=1104033&r2=1104034&view=diff ============================================================================== --- incubator/hama/trunk/src/java/org/apache/hama/bsp/LocalBSPRunner.java (original) +++ incubator/hama/trunk/src/java/org/apache/hama/bsp/LocalBSPRunner.java Tue May 17 07:23:41 2011 @@ -78,8 +78,7 @@ public class LocalBSPRunner implements J if (path != null && !path.isEmpty()) WORKING_DIR = path; - int overridenSize = conf.getInt("bsp.local.tasks.maximum", 20); - threadPoolSize = overridenSize; + threadPoolSize = conf.getInt("bsp.local.tasks.maximum", 20); threadPool = (ThreadPoolExecutor) Executors .newFixedThreadPool(threadPoolSize); barrier = new CyclicBarrier(threadPoolSize); From hama-commits-returnvaughneric@example.org Wed May 18 06:14:00 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id 0D7826D82 for <vaughneric@example.org>; Wed, 18 May 2011 06:14:00 +0000 (UTC) Received: (qmail 9510 invoked by uid 500); 18 May 2011 06:13:59 -0000 Delivered-To: vaughneric@example.org Received: (qmail 9478 invoked by uid 500); 18 May 2011 06:13:58 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 9470 invoked by uid 99); 18 May 2011 06:13:55 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 May 2011 06:13:55 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 May 2011 06:13:52 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 0AE5823888EA; Wed, 18 May 2011 06:13:32 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1104694 - in /incubator/hama/trunk: CHANGES.txt conf/hama-env.sh.template Date: Wed, 18 May 2011 06:13:31 -0000 To: vaughneric@example.org From: vaughneric@example.org X-Mailer: svnmailer-1.0.8 Message-Id: <vaughneric@example.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: edwardyoon Date: Wed May 18 06:13:31 2011 New Revision: 1104694 URL: http://svn.apache.org/viewvc?rev=1104694&view=rev Log: Add environment variable 'HAMA_HEAPSIZE' to hama-env.sh Modified: incubator/hama/trunk/CHANGES.txt incubator/hama/trunk/conf/hama-env.sh.template Modified: incubator/hama/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/incubator/hama/trunk/CHANGES.txt?rev=1104694&r1=1104693&r2=1104694&view=diff ============================================================================== --- incubator/hama/trunk/CHANGES.txt (original) +++ incubator/hama/trunk/CHANGES.txt Wed May 18 06:13:31 2011 @@ -9,6 +9,8 @@ Release 0.3 - Unreleased BUG FIXES + HAMA-393: Add environment variable 'HAMA_HEAPSIZE' to hama-env.sh (edwardyoon) + IMPROVEMENTS HAMA-389: Make the thread pool size user configurable (Thomas Jungblut via edwardyoon) Modified: incubator/hama/trunk/conf/hama-env.sh.template URL: http://svn.apache.org/viewvc/incubator/hama/trunk/conf/hama-env.sh.template?rev=1104694&r1=1104693&r2=1104694&view=diff ============================================================================== --- incubator/hama/trunk/conf/hama-env.sh.template (original) +++ incubator/hama/trunk/conf/hama-env.sh.template Wed May 18 06:13:31 2011 @@ -27,6 +27,9 @@ # Where log files are stored. $HAMA_HOME/logs by default. # export HAMA_LOG_DIR=${HAMA_HOME}/logs +# The maximum amount of heap to use, in MB. Default is 1000. +# export HAMA_HEAPSIZE=2000 + # Extra ssh options. Empty by default. # export HAMA_SSH_OPTS="-o ConnectTimeout=1 -o SendEnv=HAMA_CONF_DIR" From hama-commits-returnvaughneric@example.org Wed May 18 10:46:10 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id 4ECDC6548 for <vaughneric@example.org>; Wed, 18 May 2011 10:46:10 +0000 (UTC) Received: (qmail 14950 invoked by uid 500); 18 May 2011 10:46:10 -0000 Delivered-To: vaughneric@example.org Received: (qmail 14918 invoked by uid 500); 18 May 2011 10:46:09 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 14910 invoked by uid 99); 18 May 2011 10:46:09 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 May 2011 10:46:09 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 May 2011 10:46:08 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id AE09923888FD; Wed, 18 May 2011 10:45:48 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1124181 - in /incubator/hama/trunk: CHANGES.txt src/java/org/apache/hama/zookeeper/QuorumPeer.java Date: Wed, 18 May 2011 10:45:48 -0000 To: vaughneric@example.org From: vaughneric@example.org X-Mailer: svnmailer-1.0.8 Message-Id: <vaughneric@example.org> Author: edwardyoon Date: Wed May 18 10:45:48 2011 New Revision: 1124181 URL: http://svn.apache.org/viewvc?rev=1124181&view=rev Log: Make zk session-timeout configurable Modified: incubator/hama/trunk/CHANGES.txt incubator/hama/trunk/src/java/org/apache/hama/zookeeper/QuorumPeer.java Modified: incubator/hama/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/incubator/hama/trunk/CHANGES.txt?rev=1124181&r1=1124180&r2=1124181&view=diff ============================================================================== --- incubator/hama/trunk/CHANGES.txt (original) +++ incubator/hama/trunk/CHANGES.txt Wed May 18 10:45:48 2011 @@ -13,6 +13,7 @@ Release 0.3 - Unreleased IMPROVEMENTS + HAMA-392: Make zk session-timeout configurable (edwardyoon) HAMA-389: Make the thread pool size user configurable (Thomas Jungblut via edwardyoon) HAMA-386: Removing 'System.arraycopy(...)' call from ByteMessage (edwardyoon) HAMA-379: Setting default values to job.setNumBspTask() (Thomas Jungblut via edwardyoon) Modified: incubator/hama/trunk/src/java/org/apache/hama/zookeeper/QuorumPeer.java URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/java/org/apache/hama/zookeeper/QuorumPeer.java?rev=1124181&r1=1124180&r2=1124181&view=diff ============================================================================== --- incubator/hama/trunk/src/java/org/apache/hama/zookeeper/QuorumPeer.java (original) +++ incubator/hama/trunk/src/java/org/apache/hama/zookeeper/QuorumPeer.java Wed May 18 10:45:48 2011 @@ -178,6 +178,10 @@ public class QuorumPeer implements Const // Otherwise, use the configuration options from Hama's XML files. Properties zkProperties = new Properties(); + // Set the max session timeout from the provided client-side timeout + zkProperties.setProperty("maxSessionTimeout", + conf.get("hama.zookeeper.session.timeout", "1200000")); + // Directly map all of the hama.zookeeper.property.KEY properties. for (Entry<String, String> entry : conf) { String key = entry.getKey(); From hama-commits-returnvaughneric@example.org Wed May 18 11:07:08 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id 9D9B76CA4 for <vaughneric@example.org>; Wed, 18 May 2011 11:07:08 +0000 (UTC) Received: (qmail 43761 invoked by uid 500); 18 May 2011 11:07:08 -0000 Delivered-To: vaughneric@example.org Received: (qmail 43744 invoked by uid 500); 18 May 2011 11:07:08 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 43736 invoked by uid 99); 18 May 2011 11:07:08 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 May 2011 11:07:08 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 May 2011 11:07:03 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id B2ABC2388897; Wed, 18 May 2011 11:06:43 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1124187 - in /incubator/hama/trunk: CHANGES.txt LICENSE.txt NOTICE.txt build.xml lib/ pom.xml src/test/org/apache/hama/bsp/TestBSPPeer.java Date: Wed, 18 May 2011 11:06:43 -0000 To: vaughneric@example.org From: vaughneric@example.org X-Mailer: svnmailer-1.0.8 Message-Id: <vaughneric@example.org> Author: edwardyoon Date: Wed May 18 11:06:43 2011 New Revision: 1124187 URL: http://svn.apache.org/viewvc?rev=1124187&view=rev Log: Add a profile 'forrest' to generate the website Removed: incubator/hama/trunk/build.xml incubator/hama/trunk/lib/ incubator/hama/trunk/src/test/org/apache/hama/bsp/TestBSPPeer.java Modified: incubator/hama/trunk/CHANGES.txt incubator/hama/trunk/LICENSE.txt incubator/hama/trunk/NOTICE.txt incubator/hama/trunk/pom.xml Modified: incubator/hama/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/incubator/hama/trunk/CHANGES.txt?rev=1124187&r1=1124186&r2=1124187&view=diff ============================================================================== --- incubator/hama/trunk/CHANGES.txt (original) +++ incubator/hama/trunk/CHANGES.txt Wed May 18 11:06:43 2011 @@ -13,6 +13,7 @@ Release 0.3 - Unreleased IMPROVEMENTS + HAMA-388: Add a profile 'forrest' to generate the website (edwardyoon) HAMA-392: Make zk session-timeout configurable (edwardyoon) HAMA-389: Make the thread pool size user configurable (Thomas Jungblut via edwardyoon) HAMA-386: Removing 'System.arraycopy(...)' call from ByteMessage (edwardyoon) Modified: incubator/hama/trunk/LICENSE.txt URL: http://svn.apache.org/viewvc/incubator/hama/trunk/LICENSE.txt?rev=1124187&r1=1124186&r2=1124187&view=diff ============================================================================== --- incubator/hama/trunk/LICENSE.txt (original) +++ incubator/hama/trunk/LICENSE.txt Wed May 18 11:06:43 2011 @@ -201,60 +201,6 @@ See the License for the specific language governing permissions and limitations under the License. -=================================================== -The following license applies to the UmlGraph package -=================================================== -Copyright (c) 2002-2007, Diomidis Spinellis -All rights reserved. - -Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions are -met: - - * Redistributions of source code must retain the above copyright - notice, this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright - notice, this list of conditions and the following disclaimer in the - documentation and/or other materials provided with the distribution. - * Neither the name of Diomidis Spinellis nor the names of its - contributors may be used to endorse or promote products derived from - this software without specific prior written permission. - -THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS -"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT -LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR -A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT -OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, -SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED -TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR -PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF -LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING -NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS -SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -========================================================== -The following license applies to the GroboUtils package -========================================================== -The MIT License - (http://groboutils.sourceforge.net/license.html) - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in -all copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -THE SOFTWARE. - ========================================================== The following license applies to the JUnit package ========================================================== Modified: incubator/hama/trunk/NOTICE.txt URL: http://svn.apache.org/viewvc/incubator/hama/trunk/NOTICE.txt?rev=1124187&r1=1124186&r2=1124187&view=diff ============================================================================== --- incubator/hama/trunk/NOTICE.txt (original) +++ incubator/hama/trunk/NOTICE.txt Wed May 18 11:06:43 2011 @@ -4,11 +4,5 @@ Copyright 2008-2011 The Apache Software This product includes software developed at The Apache Software Foundation (http://www.apache.org/). -This product includes UmlGraph http://www.umlgraph.org/ -Copyright (c) 2002-2007, Diomidis Spinellis - -This product includes GroboTestingJUnit http://groboutils.sourceforge.net/ -Copyright (c) 2002-2003, Matt Albrecht - This product includes JUnit under the Common Public License Version 1.0 See http://www.junit.org/ \ No newline at end of file Modified: incubator/hama/trunk/pom.xml URL: http://svn.apache.org/viewvc/incubator/hama/trunk/pom.xml?rev=1124187&r1=1124186&r2=1124187&view=diff ============================================================================== --- incubator/hama/trunk/pom.xml (original) +++ incubator/hama/trunk/pom.xml Wed May 18 11:06:43 2011 @@ -107,13 +107,6 @@ <version>1.0.4</version> </dependency> <dependency> - <groupId>grobo-testing-junit</groupId> - <artifactId>grobo-testing-junit-core</artifactId> - <version>1.2.1</version> - <scope>system</scope> - <systemPath>${project.basedir}/lib/GroboTestingJUnit-1.2.1-core.jar</systemPath> - </dependency> - <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-core</artifactId> <version>0.20.2</version> @@ -159,11 +152,6 @@ <version>2.5</version> </dependency> <dependency> - <groupId>gr.spinellis</groupId> - <artifactId>UmlGraph</artifactId> - <version>4.6</version> - </dependency> - <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>zookeeper</artifactId> <version>3.3.1</version> @@ -259,6 +247,34 @@ </dependency> </dependencies> </profile> + <profile> + <id>forrest</id> + <build> + <plugins> + <plugin> + <artifactId>maven-antrun-plugin</artifactId> + <executions> + <execution> + <phase>package</phase> + <configuration> + <tasks> + <exec dir="${basedir}/src/docs" executable="${forrest.home}/bin/forrest" failonerror="true"> + </exec> + <copy todir="${project.build.directory}/site"> + <fileset dir="${basedir}/src/docs/build/site/" /> + </copy> + <delete dir="${basedir}/src/docs/build/" /> + </tasks> + </configuration> + <goals> + <goal>run</goal> + </goals> + </execution> + </executions> + </plugin> + </plugins> + </build> + </profile> </profiles> </project> From hama-commits-returnvaughneric@example.org Thu May 19 02:16:23 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id 1B7C948E3 for <vaughneric@example.org>; Thu, 19 May 2011 02:16:23 +0000 (UTC) Received: (qmail 88924 invoked by uid 500); 19 May 2011 02:16:23 -0000 Delivered-To: vaughneric@example.org Received: (qmail 88904 invoked by uid 500); 19 May 2011 02:16:23 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 88896 invoked by uid 99); 19 May 2011 02:16:22 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 19 May 2011 02:16:22 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eos.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 19 May 2011 02:16:20 +0000 Received: from eos.apache.org (localhost [83.161.120.155]) by eos.apache.org (Postfix) with ESMTP id A46CBE58 for <vaughneric@example.org>; Thu, 19 May 2011 02:15:59 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Apache Wiki <vaughneric@example.org> To: Apache Wiki <vaughneric@example.org> Date: Thu, 19 May 2011 02:15:59 -0000 Message-ID: <vaughneric@example.org> Subject: =?utf-8?q?=5BHama_Wiki=5D_Trivial_Update_of_=22HowToContribute=22_by_edwa?= =?utf-8?q?rdyoon?= X-Virus-Checked: Checked by ClamAV on apache.org Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hama Wiki" for chan= ge notification. The "HowToContribute" page has been changed by edwardyoon. http://wiki.apache.org/hama/HowToContribute?action=3Ddiff&rev1=3D11&rev2=3D= 12 -------------------------------------------------- {{{ > cd hama-trunk > mvn install package + = + # build examples jar file > mvn -Pexamples package + # build website > mvn -Pforrest package -Dforrest.home=3D{$FORREST_HOME} }}} =20 From hama-commits-returnvaughneric@example.org Thu May 19 08:35:31 2011 Return-Path: <hama-commits-returnvaughneric@example.org> X-Original-To: vaughneric@example.org Delivered-To: vaughneric@example.org Received: from mail.apache.org (hermes.apache.org [83.161.120.155]) by minotaur.apache.org (Postfix) with SMTP id 612A14EB1 for <vaughneric@example.org>; Thu, 19 May 2011 08:35:31 +0000 (UTC) Received: (qmail 77911 invoked by uid 500); 19 May 2011 08:35:31 -0000 Delivered-To: vaughneric@example.org Received: (qmail 77879 invoked by uid 500); 19 May 2011 08:35:30 -0000 Mailing-List: contact vaughneric@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:vaughneric@example.org> List-Unsubscribe: <mailto:vaughneric@example.org> List-Post: <mailto:vaughneric@example.org> List-Id: <hama-commits.incubator.apache.org> Reply-To: vaughneric@example.org Delivered-To: mailing list vaughneric@example.org Received: (qmail 77871 invoked by uid 99); 19 May 2011 08:35:30 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 19 May 2011 08:35:30 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 19 May 2011 08:35:28 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id DE1FC23888FE; Thu, 19 May 2011 08:35:06 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1124594 - in /incubator/hama/trunk: CHANGES.txt bin/hama pom.xml Date: Thu, 19 May 2011 08:35:06 -0000 To: vaughneric@example.org From: vaughneric@example.org X-Mailer: svnmailer-1.0.8 Message-Id: <vaughneric@example.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: edwardyoon Date: Thu May 19 08:35:06 2011 New Revision: 1124594 URL: http://svn.apache.org/viewvc?rev=1124594&view=rev Log: Refactor 'hama' script and pom.xml Modified: incubator/hama/trunk/CHANGES.txt incubator/hama/trunk/bin/hama incubator/hama/trunk/pom.xml Modified: incubator/hama/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/incubator/hama/trunk/CHANGES.txt?rev=1124594&r1=1124593&r2=1124594&view=diff ============================================================================== --- incubator/hama/trunk/CHANGES.txt (original) +++ incubator/hama/trunk/CHANGES.txt Thu May 19 08:35:06 2011 @@ -13,6 +13,7 @@ Release 0.3 - Unreleased IMPROVEMENTS + HAMA-394: Refactor 'hama' script and pom.xml (edwardyoon) HAMA-388: Add a profile 'forrest' to generate the
{'source': 'Apache Software Foundation Public Mail Archives'}