From zookeeper-user-returnlopeztyler@example.org Mon Mar 01 08:06:11 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 29899 invoked from network); 1 Mar 2010 08:06:11 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 1 Mar 2010 08:06:11 -0000 Received: (qmail 44161 invoked by uid 500); 28 Feb 2010 03:39:30 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 44064 invoked by uid 500); 28 Feb 2010 03:39:29 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 44055 invoked by uid 99); 28 Feb 2010 03:39:29 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 28 Feb 2010 03:39:29 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of lopeztyler@example.org designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-iw0-f172.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 28 Feb 2010 03:39:21 +0000 Received: by iwn2 with SMTP id 2so1542326iwn.29 for <lopeztyler@example.org>; Sat, 27 Feb 2010 19:39:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=iGhLfOsLMYwCyCfXCJFXfBFDsQg04MZXKfwXSmUyLcg=; b=Q2A+Y2RQ4rvvf2KxUaLke5Vq1pEl6Tvyg5ayAwA5QhpyI2u7AzP4Nx4MxgTDl4hgs4 ppU+RYBcfgpbuDrXUSSZ3lTgMQyYozjK2vKcH9oyP0brOYa31Qryw4LE0mHXI8XyamKA fEbUE3GV2C35HJu52F4KKAR+Ao+Zb8GVj2oyg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=hMJIDlA2tMG8gYLzMGQ9o24vuafZ451lDmuKTNUb/Jp5hTZ9xLvnhum/WoMtQf01fv P7Y6VZuJVxJlW3tdRhK0nwbzeiFShicqq8DahK2xcPvjuXv9SI5nSwfXukOsANqRKYs7 vofI4Hy/60OhoQTx44OmF/231H4h8CbM9qx68= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id l20mr785371ibv.77.1267328340133; Sat, 27 Feb 2010 19:39:00 -0800 (PST) Date: Sat, 27 Feb 2010 19:39:00 -0800 Message-ID: <lopeztyler@example.org> Subject: Usage of myId From: Avinash Lakshman <lopeztyler@example.org> To: lopeztyler@example.org Content-Type: multipart/alternative; boundary=0016e6498a5c870c5b0480a0dfa3 X-Virus-Checked: Checked by ClamAV on apache.org --0016e6498a5c870c5b0480a0dfa3 Content-Type: text/plain; charset=ISO-8859-1 Why is this important? What breaks down if I have 2 servers with the same myId? Cheers A --0016e6498a5c870c5b0480a0dfa3-- From zookeeper-user-returnlopeztyler@example.org Mon Mar 01 08:07:13 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 32699 invoked from network); 1 Mar 2010 08:07:13 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 1 Mar 2010 08:07:13 -0000 Received: (qmail 89083 invoked by uid 500); 28 Feb 2010 07:07:12 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 89036 invoked by uid 500); 28 Feb 2010 07:07:11 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 89028 invoked by uid 99); 28 Feb 2010 07:07:11 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 28 Feb 2010 07:07:11 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of lopeztyler@example.org designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-iw0-f172.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 28 Feb 2010 07:07:03 +0000 Received: by iwn2 with SMTP id 2so1625049iwn.29 for <lopeztyler@example.org>; Sat, 27 Feb 2010 23:06:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=NTeOa1mrswg4FyQZElLeuuxHZpi0UDjBLa7v/LXTVeE=; b=vRHnkeQXQrG9uZG6sq+mPtlVCx/fxMQSUkw2nRzmdt9rN65b3fpRiS4PEfh5nwDnbY G6dmrOWtMPXfdm9VKJ8jc/aX0mEdRIPDvwgcGRRxG5AdIXsEx9Hmr2u7dtBT7jpdSOdu fPhSdWLcVKiAxkqTcUXC/jpCEJ959jrFMlOgk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=GVwF75Qaucl8I5bxVLuLsZ3bgx7JSVm+OkculVG1LgTcND1nia6ohIW74t/GtrQmwf 8oA4pxqs80l72AjIOQ2MBw1v24i27QA/dhOKozjM3BpHgDoqljBMAB+ufdFtrANQXnPt Lvqi9vVpI7zqz3rO9VqVH51y4C8bmw/olXnzQ= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id g15mr962318ibv.49.1267340801972; Sat, 27 Feb 2010 23:06:41 -0800 (PST) In-Reply-To: <lopeztyler@example.org> References: <lopeztyler@example.org> Date: Sun, 28 Feb 2010 15:06:41 +0800 Message-ID: <lopeztyler@example.org> Subject: Re: Usage of myId From: Qian Ye <lopeztyler@example.org> To: lopeztyler@example.org Content-Type: multipart/alternative; boundary=0016e647601e4f9b8e0480a3c6b6 X-Virus-Checked: Checked by ClamAV on apache.org --0016e647601e4f9b8e0480a3c6b6 Content-Type: text/plain; charset=UTF-8 myid is used to identify your service instance, with its help, it is possible to start more than one Zookeeper service on one computer. If the configuration of myid is wrong, the service can not be started properly. On Sun, Feb 28, 2010 at 11:39 AM, Avinash Lakshman < lopeztyler@example.org> wrote: > Why is this important? What breaks down if I have 2 servers with the same > myId? > > Cheers > A > -- With Regards! Ye, Qian --0016e647601e4f9b8e0480a3c6b6-- From zookeeper-user-returnlopeztyler@example.org Tue Mar 02 02:57:22 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 16485 invoked from network); 2 Mar 2010 02:57:21 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Mar 2010 02:57:21 -0000 Received: (qmail 73129 invoked by uid 500); 2 Mar 2010 02:57:19 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 73100 invoked by uid 500); 2 Mar 2010 02:57:18 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 73088 invoked by uid 99); 2 Mar 2010 02:57:17 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Mar 2010 02:57:17 +0000 X-ASF-Spam-Status: No, hits=3.4 required=10.0 tests=HTML_MESSAGE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO mail-wy0-f176.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Mar 2010 02:57:11 +0000 Received: by wye20 with SMTP id 20so353239wye.35 for <lopeztyler@example.org>; Mon, 01 Mar 2010 18:56:49 -0800 (PST) MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id e48mr3737475wem.0.1267498609272; Mon, 01 Mar 2010 18:56:49 -0800 (PST) In-Reply-To: <lopeztyler@example.org> References: <lopeztyler@example.org> <lopeztyler@example.org> Date: Mon, 1 Mar 2010 18:56:49 -0800 Message-ID: <lopeztyler@example.org> Subject: Re: Usage of myId From: Henry Robinson <lopeztyler@example.org> To: lopeztyler@example.org Content-Type: multipart/alternative; boundary=001485f274c85bea510480c884fa --001485f274c85bea510480c884fa Content-Type: text/plain; charset=ISO-8859-1 If you have two servers with the same myid, two servers will identify themselves as the 'same' machine X in a ZooKeeper ensemble. This id is used to map onto a hostname / port pair where messages for a given server are sent. Assuming a consistent quorum specification across all machines, messages for server X will only go to one machine and the other will think itself partitioned from the network. Servers need ids to distinguish themselves from other servers in order to break symmetry and successfully elect a leader. Henry On 27 February 2010 23:06, Qian Ye <lopeztyler@example.org> wrote: > myid is used to identify your service instance, with its help, it is > possible to start more than one Zookeeper service on one computer. If the > configuration of myid is wrong, the service can not be started properly. > > On Sun, Feb 28, 2010 at 11:39 AM, Avinash Lakshman < > lopeztyler@example.org> wrote: > > > Why is this important? What breaks down if I have 2 servers with the same > > myId? > > > > Cheers > > A > > > > > > -- > With Regards! > > Ye, Qian > -- Henry Robinson Software Engineer Cloudera +1-225-644-7005 --001485f274c85bea510480c884fa-- From zookeeper-user-returnlopeztyler@example.org Tue Mar 02 21:05:00 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 60134 invoked from network); 2 Mar 2010 21:05:00 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Mar 2010 21:05:00 -0000 Received: (qmail 87006 invoked by uid 500); 2 Mar 2010 21:04:55 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 86956 invoked by uid 500); 2 Mar 2010 21:04:54 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 86948 invoked by uid 99); 2 Mar 2010 21:04:54 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Mar 2010 21:04:54 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO homiemail-a3.g.dreamhost.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Mar 2010 21:04:45 +0000 Received: from darsys9 (cpe-66-108-142-190.nyc.res.rr.com [83.161.120.155]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by homiemail-a3.g.dreamhost.com (Postfix) with ESMTP id 0FEE4C5CE0 for <lopeztyler@example.org>; Tue, 2 Mar 2010 13:04:24 -0800 (PST) Received: from [83.161.120.155] by darsys9 with esmtp (Exim 4.71) (envelope-from <lopeztyler@example.org>) id 1NmZGP-0006Dg-Gc for lopeztyler@example.org; Tue, 02 Mar 2010 16:04:21 -0500 Message-ID: <lopeztyler@example.org> Date: Tue, 02 Mar 2010 16:04:51 -0500 From: David Rosenstrauch <lopeztyler@example.org> User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:83.161.120.155) Gecko/20100301 Shredder/3.0.3 MIME-Version: 1.0 To: lopeztyler@example.org Subject: zookeeper utils Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org Was reading through the zookeeper docs on the web - specifically the "recipes and solutions" page (as well as comments elsewhere inviting additional such contributions from the community) and was wondering: Is there a library of higher-level zookeeper utilities that people have contributed, beyond the barrier and queue examples provided in the docs? Thanks, DR From zookeeper-user-returnlopeztyler@example.org Tue Mar 02 22:39:12 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 87338 invoked from network); 2 Mar 2010 22:39:12 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Mar 2010 22:39:12 -0000 Received: (qmail 25711 invoked by uid 500); 2 Mar 2010 22:39:07 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 25692 invoked by uid 500); 2 Mar 2010 22:39:07 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 25684 invoked by uid 99); 2 Mar 2010 22:39:07 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Mar 2010 22:39:07 +0000 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO mrout2-b.corp.re1.yahoo.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Mar 2010 22:38:57 +0000 Received: from SNV-EXPF01.ds.corp.yahoo.com (snv-expf01.ds.corp.yahoo.com [83.161.120.155]) by mrout2-b.corp.re1.yahoo.com (8.13.8/8.13.8/y.out) with ESMTP id o22MbfYO099347 for <lopeztyler@example.org>; Tue, 2 Mar 2010 14:37:56 -0800 (PST) DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns; h=received:user-agent:date:subject:from:to:message-id: thread-topic:thread-index:in-reply-to:mime-version:content-type: content-transfer-encoding:x-originalarrivaltime; b=ajdpPvzNznsYgwm+XLnZn5WYQoPA+f+JQD7Q6GGTIqvBvseyPBV88ZENY4/gUijM Received: from SNV-EXVS09.ds.corp.yahoo.com ([83.161.120.155]) by SNV-EXPF01.ds.corp.yahoo.com with Microsoft SMTPSVC+1-225-644-7005); Tue, 2 Mar 2010 14:37:44 -0800 Received: from 83.161.120.155 ([83.161.120.155]) by SNV-EXVS09.ds.corp.yahoo.com ([83.161.120.155]) via Exchange Front-End Server snv-webmail.corp.yahoo.com ([83.161.120.155]) with Microsoft Exchange Server HTTP-DAV ; Tue, 2 Mar 2010 22:37:13 +0000 User-Agent: Microsoft-Entourage/83.161.120.155605 Date: Tue, 02 Mar 2010 14:37:13 -0800 Subject: Re: zookeeper utils From: Mahadev Konar <lopeztyler@example.org> To: <lopeztyler@example.org> Message-ID: <lopeztyler@example.org> Thread-Topic: zookeeper utils Thread-Index: Acq6WOcQRKlSXkXjiky09CacEEx2fw== In-Reply-To: <lopeztyler@example.org> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit X-OriginalArrivalTime: 02 Mar 2010 22:37:44.0816 (UTC) FILETIME=[FA076F00:01CABA58] Hi David, There is an implementation for locks and queues in src/recipes. The documentation residres in src/recipes/{lock/queue}/README.txt. Thanks mahadev On 3/2/10 1:04 PM, "David Rosenstrauch" <lopeztyler@example.org> wrote: > Was reading through the zookeeper docs on the web - specifically the > "recipes and solutions" page (as well as comments elsewhere inviting > additional such contributions from the community) and was wondering: > > Is there a library of higher-level zookeeper utilities that people have > contributed, beyond the barrier and queue examples provided in the docs? > > Thanks, > > DR From zookeeper-user-returnlopeztyler@example.org Tue Mar 02 22:53:07 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 90983 invoked from network); 2 Mar 2010 22:53:07 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 2 Mar 2010 22:53:07 -0000 Received: (qmail 51777 invoked by uid 500); 2 Mar 2010 22:53:02 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 51730 invoked by uid 500); 2 Mar 2010 22:53:02 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 51722 invoked by uid 99); 2 Mar 2010 22:53:02 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Mar 2010 22:53:02 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of lopeztyler@example.org designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-pw0-f48.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Mar 2010 22:52:55 +0000 Received: by pwi6 with SMTP id 6so430723pwi.35 for <lopeztyler@example.org>; Tue, 02 Mar 2010 14:52:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :from:date:message-id:subject:to:content-type; bh=hZUkX+tKui0MQV3XfamG++8P4Zg5afysURutJcKbm4E=; b=NX5vLS+Z3NHPUPzv3iR7lOtsNec8CglQdpZnNjRQsFQYoAZev8eoxK0Bd5y5CKLL0W b+qWRCzt85yzbC5BztRB2uX7oQ/Fh7V/pzAAAccrKf9/E+VOg7iiMaa74/LeW/ENeZX3 OqUP+XeFeBuyYUzaAWENbYgPoKtuSk5af/XBg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=MpHzpxuhBVFvxkAo47tVjw1+O8E0jc8MmmTS1n7qgzf8iThwGHUuMORmhTWWAAYTFN G8Fh5ILfPa3BMApblDc65SJnh2ue8/qpyQZQ5GucpYHUuHwB0ebm2dTCwDnPx1QCMwNs IByRZ9ewAWcmPGIZ9nEsbylQtRExm1rWzzdaI= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id p29mr3660802rvq.133.1267570355075; Tue, 02 Mar 2010 14:52:35 -0800 (PST) In-Reply-To: <lopeztyler@example.org> References: <lopeztyler@example.org> From: Ted Dunning <lopeztyler@example.org> Date: Tue, 2 Mar 2010 14:52:15 -0800 Message-ID: <lopeztyler@example.org> Subject: Re: zookeeper utils To: lopeztyler@example.org Content-Type: multipart/alternative; boundary=000e0cd1a9a0bdfb660480d938e5 --000e0cd1a9a0bdfb660480d938e5 Content-Type: text/plain; charset=UTF-8 What other examples are you looking for? On Tue, Mar 2, 2010 at 1:04 PM, David Rosenstrauch <lopeztyler@example.org>wrote: > Is there a library of higher-level zookeeper utilities that people have > contributed, beyond the barrier and queue examples provided in the docs? -- Ted Dunning, CTO DeepDyve --000e0cd1a9a0bdfb660480d938e5-- From zookeeper-user-returnlopeztyler@example.org Wed Mar 03 04:15:54 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 800 invoked from network); 3 Mar 2010 04:15:54 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 3 Mar 2010 04:15:54 -0000 Received: (qmail 48197 invoked by uid 500); 3 Mar 2010 04:15:48 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 48075 invoked by uid 500); 3 Mar 2010 04:15:48 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 48067 invoked by uid 99); 3 Mar 2010 04:15:48 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Mar 2010 04:15:48 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO homiemail-a37.g.dreamhost.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Mar 2010 04:15:40 +0000 Received: from darsys9 (cpe-66-108-142-190.nyc.res.rr.com [83.161.120.155]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by homiemail-a37.g.dreamhost.com (Postfix) with ESMTP id 0A9A04D3C0 for <lopeztyler@example.org>; Tue, 2 Mar 2010 20:15:19 -0800 (PST) Received: from [83.161.120.155] by darsys9 with esmtp (Exim 4.71) (envelope-from <lopeztyler@example.org>) id 1NmfzR-0007wp-Td for lopeztyler@example.org; Tue, 02 Mar 2010 23:15:17 -0500 Message-ID: <lopeztyler@example.org> Date: Tue, 02 Mar 2010 23:15:41 -0500 From: David Rosenstrauch <lopeztyler@example.org> User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:83.161.120.155) Gecko/20100301 Shredder/3.0.3 MIME-Version: 1.0 To: lopeztyler@example.org Subject: Re: zookeeper utils References: <lopeztyler@example.org> In-Reply-To: <lopeztyler@example.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Thanks, I'll take a look. DR On 03/02/2010 05:37 PM, Mahadev Konar wrote: > Hi David, > There is an implementation for locks and queues in src/recipes. The > documentation residres in src/recipes/{lock/queue}/README.txt. > > Thanks > mahadev > > > On 3/2/10 1:04 PM, "David Rosenstrauch"<lopeztyler@example.org> wrote: > >> Was reading through the zookeeper docs on the web - specifically the >> "recipes and solutions" page (as well as comments elsewhere inviting >> additional such contributions from the community) and was wondering: >> >> Is there a library of higher-level zookeeper utilities that people have >> contributed, beyond the barrier and queue examples provided in the docs? >> >> Thanks, >> >> DR > From zookeeper-user-returnlopeztyler@example.org Wed Mar 03 04:18:33 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 6204 invoked from network); 3 Mar 2010 04:18:33 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 3 Mar 2010 04:18:33 -0000 Received: (qmail 49967 invoked by uid 500); 3 Mar 2010 04:18:27 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 49951 invoked by uid 500); 3 Mar 2010 04:18:26 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 49943 invoked by uid 99); 3 Mar 2010 04:18:26 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Mar 2010 04:18:26 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO homiemail-a1.g.dreamhost.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Mar 2010 04:18:18 +0000 Received: from darsys9 (cpe-66-108-142-190.nyc.res.rr.com [83.161.120.155]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by homiemail-a1.g.dreamhost.com (Postfix) with ESMTP id 594CF119DB6 for <lopeztyler@example.org>; Tue, 2 Mar 2010 20:17:58 -0800 (PST) Received: from [83.161.120.155] by darsys9 with esmtp (Exim 4.71) (envelope-from <lopeztyler@example.org>) id 1Nmg21-0007xN-64 for lopeztyler@example.org; Tue, 02 Mar 2010 23:17:57 -0500 Message-ID: <lopeztyler@example.org> Date: Tue, 02 Mar 2010 23:18:20 -0500 From: David Rosenstrauch <lopeztyler@example.org> User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:83.161.120.155) Gecko/20100301 Shredder/3.0.3 MIME-Version: 1.0 To: lopeztyler@example.org Subject: Re: zookeeper utils References: <lopeztyler@example.org> <lopeztyler@example.org> In-Reply-To: <lopeztyler@example.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 03/02/2010 05:52 PM, Ted Dunning wrote: > What other examples are you looking for? > > On Tue, Mar 2, 2010 at 1:04 PM, David Rosenstrauch<lopeztyler@example.org>wrote: > >> Is there a library of higher-level zookeeper utilities that people have >> contributed, beyond the barrier and queue examples provided in the docs? Well, first off, I'm trying to get familiar with Zookeeper's capabilities, and I figured that'd be a good place to start. Aside from that, though, we're going to need something like AtomicInteger for an app I'm about to start working on, so I was looking to see if there was already some code out there that got me all or part of the way there. DR From zookeeper-user-returnlopeztyler@example.org Wed Mar 03 07:08:21 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 12867 invoked from network); 3 Mar 2010 06:44:18 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 3 Mar 2010 06:44:18 -0000 Received: (qmail 7691 invoked by uid 500); 3 Mar 2010 06:44:11 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 7579 invoked by uid 500); 3 Mar 2010 06:44:11 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 7571 invoked by uid 99); 3 Mar 2010 06:44:11 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Mar 2010 06:44:11 +0000 X-ASF-Spam-Status: No, hits=3.4 required=10.0 tests=HTML_MESSAGE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO mail-ww0-f48.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Mar 2010 06:44:04 +0000 Received: by wwb29 with SMTP id 29so535774wwb.35 for <lopeztyler@example.org>; Tue, 02 Mar 2010 22:43:42 -0800 (PST) MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id g10mr4323831wef.217.1267598622304; Tue, 02 Mar 2010 22:43:42 -0800 (PST) In-Reply-To: <lopeztyler@example.org> References: <lopeztyler@example.org> <lopeztyler@example.org> <lopeztyler@example.org> Date: Tue, 2 Mar 2010 22:43:42 -0800 Message-ID: <lopeztyler@example.org> Subject: Re: zookeeper utils From: Henry Robinson <lopeztyler@example.org> To: lopeztyler@example.org Content-Type: multipart/alternative; boundary=0016e6d77e3c99aa7c0480dfcd2b --0016e6d77e3c99aa7c0480dfcd2b Content-Type: text/plain; charset=ISO-8859-1 Just to illustrate one of the primitives you're looking for: an AtomicInteger equivalent would be fairly easy to construct, with nearly identical semantics to the Java version. Let's say a given znode has four bytes of data that represent an integer value. Get operations or set operations are easy, as ZK will make sure that all operations are atomic, so they happen in a linearizable order. A get-and-set operation can be performed by reading the znode via getData, and performing a conditional update using the version number of the znode that was returned as part of the getData operation. ZooKeeper's setData operation takes an optional version number (set it to -1 to ignore it) which tells the operation to succeed only if the znode's version hasn't changed since. Other operations can use this procedure as a base. This is exactly how Java's getAndSet is implemented - neither implementation is wait-free, but they are still lock-free: some process will always make progress. Hope this helps - let me know if you'd like more detail on exactly how to build this. Henry -- Henry Robinson Software Engineer Cloudera +1-225-644-7005 On 2 March 2010 20:18, David Rosenstrauch <lopeztyler@example.org> wrote: > On 03/02/2010 05:52 PM, Ted Dunning wrote: > >> What other examples are you looking for? >> >> On Tue, Mar 2, 2010 at 1:04 PM, David Rosenstrauch<lopeztyler@example.org >> >wrote: >> >> Is there a library of higher-level zookeeper utilities that people have >>> contributed, beyond the barrier and queue examples provided in the docs? >>> >> > Well, first off, I'm trying to get familiar with Zookeeper's capabilities, > and I figured that'd be a good place to start. > > Aside from that, though, we're going to need something like AtomicInteger > for an app I'm about to start working on, so I was looking to see if there > was already some code out there that got me all or part of the way there. > > DR > --0016e6d77e3c99aa7c0480dfcd2b-- From zookeeper-user-returnlopeztyler@example.org Wed Mar 03 17:06:42 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 90424 invoked from network); 3 Mar 2010 17:06:42 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 3 Mar 2010 17:06:42 -0000 Received: (qmail 97098 invoked by uid 500); 3 Mar 2010 17:06:34 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 97039 invoked by uid 500); 3 Mar 2010 17:06:34 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 97027 invoked by uid 99); 3 Mar 2010 17:06:34 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Mar 2010 17:06:34 +0000 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO mrout1.yahoo.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Mar 2010 17:06:26 +0000 Received: from [83.161.120.155] (snvvpn2-10-73-152-c84.hq.corp.yahoo.com [83.161.120.155]) by mrout1.yahoo.com (8.13.6/8.13.6/y.out) with ESMTP id o23H5shT097626; Wed, 3 Mar 2010 09:05:55 -0800 (PST) Message-ID: <lopeztyler@example.org> Date: Wed, 03 Mar 2010 09:05:54 -0800 From: Patrick Hunt <lopeztyler@example.org> User-Agent: Thunderbird 83.161.120.155 (X11/20090817) MIME-Version: 1.0 To: lopeztyler@example.org Subject: Re: zookeeper utils References: <lopeztyler@example.org> <lopeztyler@example.org> <lopeztyler@example.org> <lopeztyler@example.org> In-Reply-To: <lopeztyler@example.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org This is def. something we should add to the recipes (docs & code lib), Henry/David can you create a jira for this? Patrick Henry Robinson wrote: > Just to illustrate one of the primitives you're looking for: an > AtomicInteger equivalent would be fairly easy to construct, with nearly > identical semantics to the Java version. > > Let's say a given znode has four bytes of data that represent an integer > value. Get operations or set operations are easy, as ZK will make sure that > all operations are atomic, so they happen in a linearizable order. A > get-and-set operation can be performed by reading the znode via getData, and > performing a conditional update using the version number of the znode that > was returned as part of the getData operation. ZooKeeper's setData operation > takes an optional version number (set it to -1 to ignore it) which tells the > operation to succeed only if the znode's version hasn't changed since. Other > operations can use this procedure as a base. > > This is exactly how Java's getAndSet is implemented - neither implementation > is wait-free, but they are still lock-free: some process will always make > progress. > > Hope this helps - let me know if you'd like more detail on exactly how to > build this. > > Henry > From zookeeper-user-returnlopeztyler@example.org Wed Mar 03 22:41:47 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 5467 invoked from network); 3 Mar 2010 22:41:47 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 3 Mar 2010 22:41:47 -0000 Received: (qmail 57373 invoked by uid 500); 3 Mar 2010 22:41:39 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 57278 invoked by uid 500); 3 Mar 2010 22:41:38 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 57270 invoked by uid 99); 3 Mar 2010 22:41:38 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Mar 2010 22:41:38 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO homiemail-a10.g.dreamhost.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Mar 2010 22:41:29 +0000 Received: from darsys9 (cpe-66-108-142-190.nyc.res.rr.com [83.161.120.155]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by homiemail-a10.g.dreamhost.com (Postfix) with ESMTPSA id F315E28006B for <lopeztyler@example.org>; Wed, 3 Mar 2010 14:41:07 -0800 (PST) Received: from [83.161.120.155] by darsys9 with esmtp (Exim 4.71) (envelope-from <lopeztyler@example.org>) id 1NmxFa-0003y3-MQ for lopeztyler@example.org; Wed, 03 Mar 2010 17:41:06 -0500 Message-ID: <lopeztyler@example.org> Date: Wed, 03 Mar 2010 17:41:14 -0500 From: David Rosenstrauch <lopeztyler@example.org> User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:83.161.120.155) Gecko/20100301 Shredder/3.0.3 MIME-Version: 1.0 To: lopeztyler@example.org Subject: Re: zookeeper utils References: <lopeztyler@example.org> <lopeztyler@example.org> <lopeztyler@example.org> <lopeztyler@example.org> In-Reply-To: <lopeztyler@example.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org On 03/03/2010 01:43 AM, Henry Robinson wrote: > Just to illustrate one of the primitives you're looking for: an > AtomicInteger equivalent would be fairly easy to construct, with nearly > identical semantics to the Java version. > > Let's say a given znode has four bytes of data that represent an integer > value. Get operations or set operations are easy, as ZK will make sure that > all operations are atomic, so they happen in a linearizable order. A > get-and-set operation can be performed by reading the znode via getData, and > performing a conditional update using the version number of the znode that > was returned as part of the getData operation. ZooKeeper's setData operation > takes an optional version number (set it to -1 to ignore it) which tells the > operation to succeed only if the znode's version hasn't changed since. Other > operations can use this procedure as a base. > > This is exactly how Java's getAndSet is implemented - neither implementation > is wait-free, but they are still lock-free: some process will always make > progress. > > Hope this helps - let me know if you'd like more detail on exactly how to > build this. > > Henry Ah! Didn't see that "version" parm on the setData call. Yes, that makes atomic operations incredibly easy! Thanks much for the explanation, Henry! DR From zookeeper-user-returnlopeztyler@example.org Wed Mar 03 22:49:11 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 10368 invoked from network); 3 Mar 2010 22:49:11 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 3 Mar 2010 22:49:11 -0000 Received: (qmail 76972 invoked by uid 500); 3 Mar 2010 22:49:03 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 76949 invoked by uid 500); 3 Mar 2010 22:49:03 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 76929 invoked by uid 99); 3 Mar 2010 22:49:03 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Mar 2010 22:49:03 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO homiemail-a38.g.dreamhost.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Mar 2010 22:48:55 +0000 Received: from darsys9 (cpe-66-108-142-190.nyc.res.rr.com [83.161.120.155]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by homiemail-a38.g.dreamhost.com (Postfix) with ESMTP id 41E54D4DA6 for <lopeztyler@example.org>; Wed, 3 Mar 2010 14:48:34 -0800 (PST) Received: from [83.161.120.155] by darsys9 with esmtp (Exim 4.71) (envelope-from <lopeztyler@example.org>) id 1NmxMm-0003zk-UD for lopeztyler@example.org; Wed, 03 Mar 2010 17:48:32 -0500 Message-ID: <lopeztyler@example.org> Date: Wed, 03 Mar 2010 17:48:40 -0500 From: David Rosenstrauch <lopeztyler@example.org> User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:83.161.120.155) Gecko/20100301 Shredder/3.0.3 MIME-Version: 1.0 To: lopeztyler@example.org Subject: Re: zookeeper utils References: <lopeztyler@example.org> <lopeztyler@example.org> <lopeztyler@example.org> <lopeztyler@example.org> <lopeztyler@example.org> In-Reply-To: <lopeztyler@example.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 03/03/2010 12:05 PM, Patrick Hunt wrote: > This is def. something we should add to the recipes (docs & code lib), > Henry/David can you create a jira for this? > > Patrick Done. https://issues.apache.org/jira/browse/ZOOKEEPER-686 Thanks, DR From zookeeper-user-returnlopeztyler@example.org Sat Mar 06 00:27:08 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 19244 invoked from network); 6 Mar 2010 00:27:08 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 6 Mar 2010 00:27:08 -0000 Received: (qmail 51281 invoked by uid 500); 6 Mar 2010 00:26:52 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 51260 invoked by uid 500); 6 Mar 2010 00:26:51 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 51252 invoked by uid 99); 6 Mar 2010 00:26:51 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 06 Mar 2010 00:26:51 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of lopeztyler@example.org designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-px0-f201.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 06 Mar 2010 00:26:47 +0000 Received: by pxi39 with SMTP id 39so1441281pxi.29 for <lopeztyler@example.org>; Fri, 05 Mar 2010 16:26:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=Yl11WaEObhSiwakeUdr1Kgdys3lC3D9SsF5NQtVFbX0=; b=VgmXKVzaimFACQ/RmlbZunuiaHadlJmwr+EJZSNI2T3hTLoDKdqGPrr/L6sNxvC4Sl mPKpOK7Oh2G4hUTM9q88uKKaPaepRUo6ym5oyutU/wNE/m3UV6f+IH0Qnz99nEKv2m8y pmIWN7VXCjV5mzDNfLe9fKnXtsYUnXzumCeCI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=JcMaqi+tbPjhV1/NEgrhAPHlvUr0M9aU7kUAl1wHuYJ1FRGcilSZXkbkhUGjIhnnQK fuRIUy11XWaYXttiG8XCWB0Q/gy50laG6gsw5PC70BGYg0E9NE5BOWOGRS1+psZvwHBh 64PFCPMZvrfS8sby+HOZcT4fSuRIn8TRYnzGk= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id d13mr1127490wfj.0.1267835186946; Fri, 05 Mar 2010 16:26:26 -0800 (PST) In-Reply-To: <lopeztyler@example.org> References: <lopeztyler@example.org> <lopeztyler@example.org> <lopeztyler@example.org> Date: Fri, 5 Mar 2010 16:26:26 -0800 Message-ID: <lopeztyler@example.org> Subject: Re: Usage of myId From: Nick Dimiduk <lopeztyler@example.org> To: lopeztyler@example.org Content-Type: multipart/alternative; boundary=001636e0ac8af3ade9048116e164 --001636e0ac8af3ade9048116e164 Content-Type: text/plain; charset=UTF-8 I think automating this assignment has been discussed in the past. IIRC, some combination of MAC address and pid or timestamp would be sufficient to uniquely identify a ZK instance. There may be some detail I'm forgetting about maintaining the ID between process restarts, making these ephemeral values unacceptable. Cheers, -Nick On Mon, Mar 1, 2010 at 6:56 PM, Henry Robinson <lopeztyler@example.org> wrote: > If you have two servers with the same myid, two servers will identify > themselves as the 'same' machine X in a ZooKeeper ensemble. This id is used > to map onto a hostname / port pair where messages for a given server are > sent. Assuming a consistent quorum specification across all machines, > messages for server X will only go to one machine and the other will think > itself partitioned from the network. > > Servers need ids to distinguish themselves from other servers in order to > break symmetry and successfully elect a leader. > > Henry > > On 27 February 2010 23:06, Qian Ye <lopeztyler@example.org> wrote: > > > myid is used to identify your service instance, with its help, it is > > possible to start more than one Zookeeper service on one computer. If the > > configuration of myid is wrong, the service can not be started properly. > > > > On Sun, Feb 28, 2010 at 11:39 AM, Avinash Lakshman < > > lopeztyler@example.org> wrote: > > > > > Why is this important? What breaks down if I have 2 servers with the > same > > > myId? > > > > > > Cheers > > > A > > > > > > > > > > > -- > > With Regards! > > > > Ye, Qian > > > > > > -- > Henry Robinson > Software Engineer > Cloudera > +1-225-644-7005 > --001636e0ac8af3ade9048116e164-- From zookeeper-user-returnlopeztyler@example.org Sat Mar 06 00:49:58 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 29513 invoked from network); 6 Mar 2010 00:49:58 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 6 Mar 2010 00:49:58 -0000 Received: (qmail 62949 invoked by uid 500); 6 Mar 2010 00:49:42 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 62928 invoked by uid 500); 6 Mar 2010 00:49:42 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 62915 invoked by uid 99); 6 Mar 2010 00:49:42 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 06 Mar 2010 00:49:42 +0000 X-ASF-Spam-Status: No, hits=0.7 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO mrout2.yahoo.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 06 Mar 2010 00:49:41 +0000 Received: from [83.161.120.155] (wifi-e-135-250.corp.yahoo.com [83.161.120.155]) by mrout2.yahoo.com (8.13.6/8.13.6/y.out) with ESMTP id o260n6rB050240; Fri, 5 Mar 2010 16:49:07 -0800 (PST) Message-ID: <lopeztyler@example.org> Date: Fri, 05 Mar 2010 16:49:07 -0800 From: Patrick Hunt <lopeztyler@example.org> User-Agent: Thunderbird 83.161.120.155 (X11/20090817) MIME-Version: 1.0 To: lopeztyler@example.org Subject: Re: Usage of myId References: <lopeztyler@example.org> <lopeztyler@example.org> <lopeztyler@example.org> <lopeztyler@example.org> In-Reply-To: <lopeztyler@example.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit re previous discussion see this thread: http://lopeztyler@example.org/msg00916.html Patrick Nick Dimiduk wrote: > I think automating this assignment has been discussed in the past. IIRC, > some combination of MAC address and pid or timestamp would be sufficient to > uniquely identify a ZK instance. There may be some detail I'm forgetting > about maintaining the ID between process restarts, making these ephemeral > values unacceptable. > > Cheers, > -Nick > > On Mon, Mar 1, 2010 at 6:56 PM, Henry Robinson <lopeztyler@example.org> wrote: > >> If you have two servers with the same myid, two servers will identify >> themselves as the 'same' machine X in a ZooKeeper ensemble. This id is used >> to map onto a hostname / port pair where messages for a given server are >> sent. Assuming a consistent quorum specification across all machines, >> messages for server X will only go to one machine and the other will think >> itself partitioned from the network. >> >> Servers need ids to distinguish themselves from other servers in order to >> break symmetry and successfully elect a leader. >> >> Henry >> >> On 27 February 2010 23:06, Qian Ye <lopeztyler@example.org> wrote: >> >>> myid is used to identify your service instance, with its help, it is >>> possible to start more than one Zookeeper service on one computer. If the >>> configuration of myid is wrong, the service can not be started properly. >>> >>> On Sun, Feb 28, 2010 at 11:39 AM, Avinash Lakshman < >>> lopeztyler@example.org> wrote: >>> >>>> Why is this important? What breaks down if I have 2 servers with the >> same >>>> myId? >>>> >>>> Cheers >>>> A >>>> >>> >>> >>> -- >>> With Regards! >>> >>> Ye, Qian >>> >> >> >> -- >> Henry Robinson >> Software Engineer >> Cloudera >> +1-225-644-7005 >> > From zookeeper-user-returnlopeztyler@example.org Sat Mar 06 09:50:48 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 81623 invoked from network); 6 Mar 2010 09:50:48 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 6 Mar 2010 09:50:48 -0000 Received: (qmail 98670 invoked by uid 500); 6 Mar 2010 09:50:32 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 98521 invoked by uid 500); 6 Mar 2010 09:50:31 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 98511 invoked by uid 99); 6 Mar 2010 09:50:31 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 06 Mar 2010 09:50:31 +0000 X-ASF-Spam-Status: No, hits=4.4 required=10.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of lopeztyler@example.org designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-fx0-f213.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 06 Mar 2010 09:50:26 +0000 Received: by fxm5 with SMTP id 5so4911783fxm.29 for <lopeztyler@example.org>; Sat, 06 Mar 2010 01:50:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=CF7skxVPNcnMDVJeNlA/NbgypVnWH7MiT4KIHR4djYE=; b=L0rREQSUiz8dAhuUdLmf4BAsylfAidjuaAbdq5WkvlfVkjlKWjhWDw3gsWmvhtzYXX UHzp0CSLfzSmA8ArtMZFIn3967kuFv1pR6xl6sSgf4HKbdzkgwJ1fSSAOMeGkGd3cRFq KjudVSDZs/BWnYq77hS9YwU2SkZyr2Sz2exUg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=R5A5Ibps7RthpOPpKw6+UDuFYWPxkNM0dV8TAVZRGbFIm4PgTbLbBadxYJflJK5VgI TdUZkkosRGUxTPR8WHjfxLKA/+BxLxN2K0L4CHIHnzu84HPVg159dNrQ1ywasATvDyyw if5VkRj9F93GY3ggUU09u+JCV9w0KG8HTJWW0= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id y3mr1235745mug.4.1267869004734; Sat, 06 Mar 2010 01:50:04 -0800 (PST) Date: Sat, 6 Mar 2010 09:50:04 +0000 Message-ID: <lopeztyler@example.org> Subject: Managing multi-site clusters with Zookeeper From: Martin Waite <lopeztyler@example.org> To: lopeztyler@example.org Content-Type: multipart/alternative; boundary=0016364c781ba6237004811ec10a --0016364c781ba6237004811ec10a Content-Type: text/plain; charset=ISO-8859-1 Hi, We're attempting to build a multi-site cluster: 1. web-tier and application tier is active in all sites 2. only one database is active at a time- normally in the designated "primary" site We want to use 3 sites to maintain a quorum. So, if the Primary site loses sight of both of the other sites, it will close down itself down. If the other sites both lose sight of the Primary site, they will co-operate in electing one of the pair as the new primary, and bring up the database services. I am thinking that in each site, a number of sentinel processes could hold open ephemeral znodes flagging that the site is up - with names like "site1/sentinel-1". These sentinels could be plugged into local health monitoring, and when the site falls into dis-repair, remove themselves. If links between sites fail, then the ephemeral nodes would disappear too. Each site would have a process that periodically checks the presence of the sentinel znodes of the other sites. If all disappear, then the site knows it is in a minority partition, and shuts down services as required. Is this a viable approach, or am I taking Zookeeper out of its application domain and just asking for trouble ? regards, Martin --0016364c781ba6237004811ec10a-- From zookeeper-user-returnlopeztyler@example.org Sat Mar 06 18:30:47 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 85471 invoked from network); 6 Mar 2010 18:30:47 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 6 Mar 2010 18:30:47 -0000 Received: (qmail 5499 invoked by uid 500); 6 Mar 2010 18:30:30 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 5441 invoked by uid 500); 6 Mar 2010 18:30:29 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 5433 invoked by uid 99); 6 Mar 2010 18:30:28 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 06 Mar 2010 18:30:28 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of lopeztyler@example.org designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-px0-f175.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 06 Mar 2010 18:30:23 +0000 Received: by pxi5 with SMTP id 5so1568427pxi.21 for <lopeztyler@example.org>; Sat, 06 Mar 2010 10:30:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :from:date:message-id:subject:to:content-type; bh=gIRO85izJeZTscTDLpmrbNZuRjhtpb5yi8u/4iiFmvE=; b=ml/v4F7nQkF9pqQpZ7+QPwsiC2cC2/rWTyk33BvJt1toj54g9qkkOFH3JdzcDiM5w8 7EWG473oNrZ3wugVNqlQ77JbRxUKVjFq+eXBA6C7+4UxDD7lRNDb7lq85nWasolhRjFa NfaoTbBJGl+uSzURcK2JFpd6LIU7+33oxqe6Q= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=bFb2RvvrBUzmf9Y7r8TYpmTP/Kvi1P/Rkdk207O7NZcCi+ck1CuGtKg6uFjYuR+SMc 6NeLKM3uihq7Ghma5Pb/0/103VaBsRrNMRo0cTH9/vMgJD9TcdDtbLcwURvKc5HReptb LOytqOGhjHd/zNcrzBgtykRbkVIfxg973NoYc= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id z12mr1688043rvi.229.1267900203049; Sat, 06 Mar 2010 10:30:03 -0800 (PST) In-Reply-To: <lopeztyler@example.org> References: <lopeztyler@example.org> From: Ted Dunning <lopeztyler@example.org> Date: Sat, 6 Mar 2010 10:29:43 -0800 Message-ID: <lopeztyler@example.org> Subject: Re: Managing multi-site clusters with Zookeeper To: lopeztyler@example.org Content-Type: multipart/alternative; boundary=000e0cd20d6236a5680481260523 --000e0cd20d6236a5680481260523 Content-Type: text/plain; charset=UTF-8 What you describe is relatively reasonable, even though Zookeeper is not normally distributed across multiple data centers with all members getting full votes. If you account for the limited throughput that this will impose on your applications that use ZK, then I think that this can work well. Probably, you would have local ZK clusters for higher transaction rate applications. You should also consider very carefully whether having multiple data centers increases or decreases your overall reliability. Unless you design very carefully, this will normally substantially degrade reliability. Making sure that it increases reliability is a really big task that involves a lot of surprising (it was to me) considerations and considerable hardware and time investments. Good luck! On Sat, Mar 6, 2010 at 1:50 AM, Martin Waite <lopeztyler@example.org>wrote: > Is this a viable approach, or am I taking Zookeeper out of its application > domain and just asking for trouble ? > -- Ted Dunning, CTO DeepDyve --000e0cd20d6236a5680481260523-- From zookeeper-user-returnlopeztyler@example.org Sat Mar 06 20:02:04 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 13892 invoked from network); 6 Mar 2010 20:02:04 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 6 Mar 2010 20:02:04 -0000 Received: (qmail 61841 invoked by uid 500); 6 Mar 2010 20:01:46 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 61824 invoked by uid 500); 6 Mar 2010 20:01:46 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 61816 invoked by uid 99); 6 Mar 2010 20:01:46 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 06 Mar 2010 20:01:46 +0000 X-ASF-Spam-Status: No, hits=4.4 required=10.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of lopeztyler@example.org designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO fg-out-1718.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 06 Mar 2010 20:01:42 +0000 Received: by fg-out-1718.google.com with SMTP id d23so1075867fga.11 for <lopeztyler@example.org>; Sat, 06 Mar 2010 12:01:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=sKEVWjrPMaliTzhH1Ai0p0Ro+f6dZ5PP2FHAZ+qSzRA=; b=PCfd99bw+hz9iIeb81ITgcB3q52At/vBoEAOxJ73rGKpuNcdFWxmdI+4I8vyOUzP4Z qAvROgL3J2NPM3RNAsQMKfOHHKh46NZKrCdUfcT5A/TIX024P2tYLWFWRMwIds01bnhZ h78RF07JO22yToTY/aNhCfkiFFHRglnHWeFuU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=DLzlJMpAeKQKL/2Yy1DJeXIIwijAZKw3BrFj6j1EgSQboRTsvlRcWJEJ34ODUKM4x3 exVcpThgvy5B53roVacHUWl5Kcx3ZlrGHdQmkVTbErRajiN9jSSnQvvrWSW1GyKdZe4q thiFhGoNQVjC5uKw4VfqXyEPcqkKU0Y/upXwc= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id m17mr1814951mul.6.1267905680153; Sat, 06 Mar 2010 12:01:20 -0800 (PST) In-Reply-To: <lopeztyler@example.org> References: <lopeztyler@example.org> <lopeztyler@example.org> Date: Sat, 6 Mar 2010 20:01:19 +0000 Message-ID: <lopeztyler@example.org> Subject: Re: Managing multi-site clusters with Zookeeper From: Martin Waite <lopeztyler@example.org> To: lopeztyler@example.org Content-Type: multipart/alternative; boundary=0016e65b635cacb8b50481274bdc --0016e65b635cacb8b50481274bdc Content-Type: text/plain; charset=ISO-8859-1 I take your point about reliability, but I have no option other than finding a multi-site solution. Unfortunately, in my experience sites are much less reliable than individual machines, and so in a way coping with site failure is more important than individual machine failure. I imagine that the risk profile changes according to the number of machines you have, however. Thanks for the input Martin On 6 March 2010 18:29, Ted Dunning <lopeztyler@example.org> wrote: > What you describe is relatively reasonable, even though Zookeeper is not > normally distributed across multiple data centers with all members getting > full votes. If you account for the limited throughput that this will > impose > on your applications that use ZK, then I think that this can work well. > Probably, you would have local ZK clusters for higher transaction rate > applications. > > You should also consider very carefully whether having multiple data > centers > increases or decreases your overall reliability. Unless you design very > carefully, this will normally substantially degrade reliability. Making > sure that it increases reliability is a really big task that involves a lot > of surprising (it was to me) considerations and considerable hardware and > time investments. > > Good luck! > > On Sat, Mar 6, 2010 at 1:50 AM, Martin Waite <lopeztyler@example.org > >wrote: > > > Is this a viable approach, or am I taking Zookeeper out of its > application > > domain and just asking for trouble ? > > > > > > -- > Ted Dunning, CTO > DeepDyve > --0016e65b635cacb8b50481274bdc-- From zookeeper-user-returnlopeztyler@example.org Sat Mar 06 20:06:09 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 15068 invoked from network); 6 Mar 2010 20:06:08 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 6 Mar 2010 20:06:08 -0000 Received: (qmail 64423 invoked by uid 500); 6 Mar 2010 20:05:50 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 64395 invoked by uid 500); 6 Mar 2010 20:05:50 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 64387 invoked by uid 99); 6 Mar 2010 20:05:50 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 06 Mar 2010 20:05:50 +0000 X-ASF-Spam-Status: No, hits=4.4 required=10.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of lopeztyler@example.org designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-fx0-f213.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 06 Mar 2010 20:05:44 +0000 Received: by fxm5 with SMTP id 5so5204428fxm.29 for <lopeztyler@example.org>; Sat, 06 Mar 2010 12:05:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=RXVGtLfc59HVF2CUcwZoBClyvTYUZk5yCFln2smG3tw=; b=o8bm/cFi4Lm5QKTw3s1nnmceHckCO1uWlma6Ag6ys9ECBLjWDtgnzAi44Nynedi8Pb evsVtWPI/5JzRmTzt7axapPec6RIWivTzlNZj9eH7oObj4ngK+7Fh44LCGu0BWBKco9X WjqOr3ZXmGfJ90k2W0aA/qW0kfMQa40s89d1M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=gq85Hz9t/kGGHet6yJmHgUzVVrXjgieBnYo3ha2bf2sw0Z6E6p0rWdPp2Z39JrhBL/ aUzXQriVdGV6VbY34wsrXwpT8klFcY/wx7xoHrnlZIaX+bM4dx3z2IdimCTDPOASejy/ np87fhBTzAfBbiLvcp+vpmhMp6oVdA6iUsQzs= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id d37mr1636178mul.105.1267905924330; Sat, 06 Mar 2010 12:05:24 -0800 (PST) Date: Sat, 6 Mar 2010 20:05:24 +0000 Message-ID: <lopeztyler@example.org> Subject: network requirements From: Martin Waite <lopeztyler@example.org> To: lopeztyler@example.org Content-Type: multipart/alternative; boundary=0016e65bc0bc3a78070481275ad6 X-Virus-Checked: Checked by ClamAV on apache.org --0016e65bc0bc3a78070481275ad6 Content-Type: text/plain; charset=ISO-8859-1 Hi, Is there any documentation describing the network requirements for running Zookeeper ? Our network admin has baffled me with all the reasons that multicast might not work in our environment, and then I realised that I do not know what network support Zookeeper requires. regards, Martin --0016e65bc0bc3a78070481275ad6-- From zookeeper-user-returnlopeztyler@example.org Sat Mar 06 23:04:40 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 63937 invoked from network); 6 Mar 2010 23:04:40 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 6 Mar 2010 23:04:40 -0000 Received: (qmail 64981 invoked by uid 500); 6 Mar 2010 23:04:21 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 64960 invoked by uid 500); 6 Mar 2010 23:04:21 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 64952 invoked by uid 99); 6 Mar 2010 23:04:21 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 06 Mar 2010 23:04:21 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of lopeztyler@example.org designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-pv0-f176.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 06 Mar 2010 23:04:17 +0000 Received: by pvc7 with SMTP id 7so1382200pvc.35 for <lopeztyler@example.org>; Sat, 06 Mar 2010 15:03:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :from:date:message-id:subject:to:content-type; bh=lGoxX95WgQt7ITjF7PBs9Wx44IA4PNsdcHOyet4eipw=; b=WSlayd2hTczC+tD0r0qRBHFVJmSipGbLDQsCqVa++sjlpoNkAaA8MuzBj/SFwg8tcj 1S7hw7Dfg7ZSDJa2Yg0AOsU47l23UtNaFtNFohZuLZL8ZLXnoToELCtjwfPjfQrKF1Aj lBrpckzW1q30YnHh45WWRjUHi+ANKhdm/Dz0E= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=jVuKkIe04ggsklx512PhujdFHzLAzVkZhAtgymZ5TVdU+njRoYvXkj+4KJjS0eydWr CIps+vyHJ5twMVK1zohHjY2v591oMke1ZPFsNi4nCzCeFgbVOAtx969ada1rxDtyywE/ hFcHsh/x3gJB03HBIZPWVJ6xM8k/Loa59Vh9Q= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id z18mr1854043rvi.66.1267916637052; Sat, 06 Mar 2010 15:03:57 -0800 (PST) In-Reply-To: <lopeztyler@example.org> References: <lopeztyler@example.org> From: Ted Dunning <lopeztyler@example.org> Date: Sat, 6 Mar 2010 15:03:37 -0800 Message-ID: <lopeztyler@example.org> Subject: Re: network requirements To: lopeztyler@example.org Content-Type: multipart/alternative; boundary=000e0cd17a12c1a1dd048129d8a0 --000e0cd17a12c1a1dd048129d8a0 Content-Type: text/plain; charset=UTF-8 Your network admin is correct. Multicast often doesn't work. ZK does not use multicast at the network level. Where events or notifications must go to many places (that SOUNDS like multicast, I know) it uses very standard TCP connections. For almost any known modern network, ZK should be just fine. Unless you are running massive numbers of operations through ZK and you are only using a dial-up connection, you should be fine. On Sat, Mar 6, 2010 at 12:05 PM, Martin Waite <lopeztyler@example.org>wrote: > > Our network admin has baffled me with all the reasons that multicast might > not work in our environment, and then I realised that I do not know what > network support Zookeeper requires. > --000e0cd17a12c1a1dd048129d8a0-- From zookeeper-user-returnlopeztyler@example.org Sun Mar 07 04:33:03 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 5762 invoked from network); 7 Mar 2010 04:33:02 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 7 Mar 2010 04:33:02 -0000 Received: (qmail 56357 invoked by uid 500); 7 Mar 2010 04:32:43 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 56255 invoked by uid 500); 7 Mar 2010 04:32:43 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 56247 invoked by uid 99); 7 Mar 2010 04:32:43 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 07 Mar 2010 04:32:43 +0000 X-ASF-Spam-Status: No, hits=4.4 required=10.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of lopeztyler@example.org designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-fx0-f213.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 07 Mar 2010 04:32:36 +0000 Received: by fxm5 with SMTP id 5so5362922fxm.29 for <lopeztyler@example.org>; Sat, 06 Mar 2010 20:32:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=Fbzx1rfT5X9BCvouCGo8v8WGRkTXketvlGdyc+avUXw=; b=MDc2ul5q3oHPHM9dDOWyhgpWaTlK2Fh9tYb3PRVin6x3GdQfs9ot4f/2ZGMKH/gRq3 Faxz6gBlFLPOKCPubCwaANMYS4ofUA8WDQpoxo4HFDeJIoEVv+ybPxEWrFpgDRZ8y4WL 3mejfGtK3P3RrDBTLLWLK3PkZouWsmNVKURY0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=GiW69+lbFCvP3Q3g76mX7/h25bimY3nQCAWsHpWn+/+tj8RuFp6V3zDT9r0OY4Y7Gq feMevYr5Krw/QvqbMEScqJbt4JP6ES6buL7hIhl6w00Xe7oqZLLdYut6cpOBXtiILtco RI/CeuaxnnVn4ox3fMw0/P9sQeUGcOPC1ud4E= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id j17mr1941240mul.101.1267936335682; Sat, 06 Mar 2010 20:32:15 -0800 (PST) In-Reply-To: <lopeztyler@example.org> References: <lopeztyler@example.org> <lopeztyler@example.org> Date: Sun, 7 Mar 2010 04:32:15 +0000 Message-ID: <lopeztyler@example.org> Subject: Re: network requirements From: Martin Waite <lopeztyler@example.org> To: lopeztyler@example.org Content-Type: multipart/alternative; boundary=0016e65b64b4e2deea04812e6ee7 X-Virus-Checked: Checked by ClamAV on apache.org --0016e65b64b4e2deea04812e6ee7 Content-Type: text/plain; charset=ISO-8859-1 That's good news. Thanks again Ted. regards, Martin On 6 March 2010 23:03, Ted Dunning <lopeztyler@example.org> wrote: > Your network admin is correct. Multicast often doesn't work. > > ZK does not use multicast at the network level. Where events or > notifications must go to many places (that SOUNDS like multicast, I know) > it > uses very standard TCP connections. > > For almost any known modern network, ZK should be just fine. Unless you > are > running massive numbers of operations through ZK and you are only using a > dial-up connection, you should be fine. > > On Sat, Mar 6, 2010 at 12:05 PM, Martin Waite <lopeztyler@example.org > >wrote: > > > > > Our network admin has baffled me with all the reasons that multicast > might > > not work in our environment, and then I realised that I do not know what > > network support Zookeeper requires. > > > --0016e65b64b4e2deea04812e6ee7-- From zookeeper-user-returnlopeztyler@example.org Sun Mar 07 21:45:00 2010 Return-Path: <zookeeper-user-returnlopeztyler@example.org> Delivered-To: lopeztyler@example.org Received: (qmail 85485 invoked from network); 7 Mar 2010 21:45:00 -0000 Received: from unknown (HELO mail.apache.org) (83.161.120.155) by 83.161.120.155 with SMTP; 7 Mar 2010 21:45:00 -0000 Received: (qmail 25415 invoked by uid 500); 7 Mar 2010 21:44:39 -0000 Delivered-To: lopeztyler@example.org Received: (qmail 25359 invoked by uid 500); 7 Mar 2010 21:44:38 -0000 Mailing-List: contact lopeztyler@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:lopeztyler@example.org> List-Unsubscribe: <mailto:lopeztyler@example.org> List-Post: <mailto:lopeztyler@example.org> List-Id: <zookeeper-user.hadoop.apache.org> Reply-To: lopeztyler@example.org Delivered-To: mailing list lopeztyler@example.org Received: (qmail 25346 invoked by uid 99); 7 Mar 2010 21:44:38 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 07 Mar 2010 21:44:38 +0000 X-ASF-Spam-Status: No, hits=0.7 required=10.0 tests=RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO mrout3.yahoo.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 07 Mar 2010 21:44:31 +0000 Received: from SNV-EXBH01.ds.corp.yahoo.com (snv-exbh01.ds.corp.yahoo.com [83.161.120.155]) by mrout3.yahoo.com (8.13.6/8.13.6/y.out) with ESMTP id o27Li0ME069244 for <lopeztyler@example.org>; Sun, 7 Mar 2010 13:44:00 -0800 (PST) DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns; h=received:user-agent:date:subject:from:to:message-id: thread-topic:thread-index:in-reply-to:mime-version:content-type: content-transfer-encoding:x-originalarrivaltime; b=FNwSv3X2tpHKUaqj1bxTbom7oT3TqOZo7LyosVdbeTxiFboGykKzn/foEmFQDQqU Received: from SNV-EXVS09.ds.corp.yahoo.com ([83.161.120.155]) by SNV-EXBH01.ds.corp.yahoo.com with Microsoft SMTPSVC+1-225-644-7005); Sun, 7 Mar 2010 13:44:00 -0800 Received: from 83.161.120.155 ([83.161.120.155]) by SNV-EXVS09.ds.corp.yahoo.com ([83.161.120.155]) via Exchange Front-End Server snv-webmail.corp.yahoo.com ([83.161.120.155]) with Microsoft Exchange Server HTTP-DAV ; Sun, 7 Mar 2010 21:43:59 +0000 User-Agent: Microsoft-Entourage/83.161.120.155605 Date: Sun, 07 Mar 2010 13:43:56 -0800 Subject: Re: Managing multi-site clusters with Zookeeper From: Mahadev Konar <lopeztyler@example.org> To: <lopeztyler@example.org> Message-ID: <lopeztyler@example.org> Thread-Topic: Managing multi-site clusters with Zookeeper Thread-Index: Acq+P0mRkVQuXzqxKku90AJ9pU/trg== In-Reply-To: <lopeztyler@example.org> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit X-OriginalArrivalTime: 07 Mar 2010 21:44:00.0522 (UTC) FILETIME=[4C43F6A0:01CABE3F] X-Virus-Checked: Checked by ClamAV on apache.org Hi Martin, As Ted rightly mentions that ZooKeeper usually is run within a colo because of the low latency requirements of applications that it supports. Its definitely reasnoble to use it in a multi data center environments but you should realize the implications of it. The high latency/low throughput means that you should make minimal use of such a ZooKeeper ensemble. Also, there are things like the tick Time, the syncLimit and others (setup parameters for ZooKeeper in config) which you will need to tune a little to get ZooKeeper running without many hiccups in this environment. Thanks mahadev On 3/6/10 10:29 AM, "Ted Dunning" <lopeztyler@example.org> wrote: > What you describe is relatively reasonable, even though Zookeeper is not > normally distributed across multiple data centers with all members getting > full votes.
{'source': 'Apache Software Foundation Public Mail Archives'}