From testscottflores@example.net Thu Aug 01 16:23:10 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 24365 invoked by uid 500); 1 Aug 2002 16:23:09 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 24354 invoked from network); 1 Aug 2002 16:23:09 -0000 Date: Thu, 1 Aug 2002 19:21:16 +0200 From: Jacek Prucia <scottflores@example.net> To: scottflores@example.net Subject: [PATCH] Misformed URLs cause segfault Message-Id: <scottflores@example.net> In-Reply-To: <scottflores@example.net> References: <scottflores@example.net> Organization: 7bulls.com S.A. X-Mailer: Sylpheed version 0.7.7 (GTK+ 1.2.10; i686-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N On Wed, 31 Jul 2002 23:45:33 -0400 Rodent of Unusual Size <scottflores@example.net> wrote: [...] > * Misformed URLs cause a segfault. When we fix this we should > make sure to print out a helpful message when this occurs. > (This is not something that can be detected with a validating > XML parser, unfortunately.) Here's the patch that fixes this problem. Well... at least the segfault, because i have no idea how helpful we might be in case of misformed URL. Right now bogus URL is sent to stderr, but it could be eventually acompanied by line number in config file (if posible at all). This might be nice with oversized flood config files :) Look's like this block of code needs a few more tests. In particular i think flood only cares about http:// schema, so everything else is wrong. There could be more checks, but this is all i can come up with right now. I'll post a patch when i'll have all tests figured out. One more thing. Right now i'm exiting flood with exit(). This schould be no problem, since no matter what resources we have allocated, they will be released. OTOH maybe there is something i'm missing, and it has to be done other way... Anyway here's the patch for RBC (Review, Bless, Commit) regards, -- Jacek Prucia 7bulls.com S.A. http://www.7bulls.com/ --- flood_round_robin.c.orig 2002-08-01 12:50:15.000000000 +0200 +++ flood_round_robin.c 2002-08-01 18:59:39.000000000 +0200 @@ -788,6 +788,10 @@ r->parsed_uri = apr_pcalloc(rp->pool, sizeof(apr_uri_t)); apr_uri_parse(rp->pool, r->uri, r->parsed_uri); + if (r->parsed_uri->scheme == NULL || r->parsed_uri->hostname == NULL) { + apr_file_printf (local_stderr, "Misformed URL '%s'\n", r->uri); + exit (APR_EGENERAL); + } if (!r->parsed_uri->port) { r->parsed_uri->port = From testscottflores@example.net Thu Aug 01 21:12:06 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 71548 invoked by uid 500); 1 Aug 2002 21:12:03 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Delivered-To: moderator for scottflores@example.net Received: (qmail 90473 invoked from network); 30 Jul 2002 11:05:45 -0000 Message-ID: <scottflores@example.net> Date: Tue, 30 Jul 2002 12:05:44 +0100 From: Reini Urban <scottflores@example.net> Organization: inode.at User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; de-AT; rv:1.0.0) Gecko/20020530 X-Accept-Language: de-at, de, en-us, en MIME-Version: 1.0 To: scottflores@example.net Subject: cygwin notes: needs symlinks Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N just configured the latest CVS flood and arp for the latest cygwin (not from CVS) using autotool-devel: (there's also a new autotool-stable, which needs similar symlinks) cd /usr/share/aclocal for s in /usr/autotool/devel/share/aclocal/* ; do ln -s $s; done cd /usr/src/apache/httpd-test/flood ./buildconf cd apr ./buildconf cd .. ./configure --enable-ssl --with-capath=/usr/ssl/certs \ --disable-shared make all works okay. I'm not subscribed to test-dev. This patch below is a minor aesthetic fix for the cygwin /usr/bin/echo which doesn't print "\t". $ diff -u examples/analyze-relative~ examples/analyze-relative --- examples/analyze-relative~ 2001-08-28 00:06:42.000000000 +0000 +++ examples/analyze-relative 2002-07-30 10:57:37.000000000 +0000 @@ -9,7 +9,7 @@ fi echo "Slowest pages on average (worst 5):" echo " Average times (sec)" -echo "connect\twrite\tread\tclose\thits\tURL" +gawk 'BEGIN{printf("connect\twrite\tread\tclose\thits\tURL\n");}' tail +1 $1 | gawk ' /OK.*https?/ { split($8, urlarray, /\?/); url = urlarray[1]; ht[url] = url; -- Reini Urban - Programmer - http://inode.at From testscottflores@example.net Thu Aug 01 22:16:41 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 22899 invoked by uid 500); 1 Aug 2002 22:16:40 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 22888 invoked from network); 1 Aug 2002 22:16:39 -0000 Date: Thu, 1 Aug 2002 15:16:46 -0700 From: Aaron Bannert <scottflores@example.net> To: scottflores@example.net Cc: Reini Urban <scottflores@example.net> Subject: Re: cygwin notes: needs symlinks Message-ID: <scottflores@example.net> References: <scottflores@example.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <scottflores@example.net> User-Agent: Mutt/1.4i X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N On Tue, Jul 30, 2002 at 12:05:44PM +0100, Reini Urban wrote: > just configured the latest CVS flood and arp for the latest cygwin > (not from CVS) > > using autotool-devel: (there's also a new autotool-stable, which needs > similar symlinks) > > cd /usr/share/aclocal > for s in /usr/autotool/devel/share/aclocal/* ; do ln -s $s; done > > cd /usr/src/apache/httpd-test/flood > ./buildconf > cd apr > ./buildconf > cd .. > ./configure --enable-ssl --with-capath=/usr/ssl/certs \ > --disable-shared > make all > > works okay. Awesome! Thanks for the feedback. Any idea why your version of autoconf looks for those files in /usr/share/aclocal instead of in the full path? Maybe a mismatched binary? Or is something in flood causing it? > This patch below is a minor aesthetic fix for the cygwin /usr/bin/echo > which doesn't print "\t". Thanks, I'll apply this right away. (My shell doesn't interpolate \t either, so I don't know how that snuck in there... maybe we should just do literal ascii tabs?) -aaron From testscottflores@example.net Thu Aug 08 03:45:17 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 87914 invoked by uid 500); 8 Aug 2002 03:45:16 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 87852 invoked from network); 8 Aug 2002 03:45:15 -0000 Date: Wed, 7 Aug 2002 23:45:28 -0400 Message-Id: <scottflores@example.net> From: Rodent of Unusual Size <scottflores@example.net> To: Apache test suite <scottflores@example.net> Subject: [STATUS] (flood) Wed Aug 7 23:45:28 EDT 2002 X-Note: This is an automated message. X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N flood STATUS: -*-text-*- Last modified at [$Date: 2002/08/01 22:28:51 $] Release: 1.0: Released July 23, 2002 milestone-03: Tagged January 16, 2002 ASF-transfer: Released July 17, 2001 milestone-02: Tagged August 13, 2001 milestone-01: Tagged July 11, 2001 (tag lost during transfer) RELEASE SHOWSTOPPERS: * "Everything needs to work perfectly" Other bugs that need fixing: * I get a SIGBUS on Darwin with our examples/round-robin-ssl.xml config, on the second URL. I'm using OpenSSL 0.9.6c 21 dec 2001. * DNS lookup failures in any <urllist> cause segfault. Justin says: Wow. Why? Aaron says: Is this still happening? It's been awhile. * iPlanet sends "Content-length" - there is a hack in there now to recognize it. However, all HTTP headers need to be normalized before checking their values. This isn't easy to do. Grr. * OpenSSL 0.9.6 Segfaults under high load. Upgrade to OpenSSL 0.9.6b. Aaron says: I just found a big bug that might have been causing this all along (we weren't closing ssl sockets). How can I reproduce the problem you were seeing to verify if this was the fix? * SEGVs when /tmp/.rnd doesn't exist are bad. Make it configurable and at least bomb with a good error message. (See Doug's patch.) Status: This is fixed, no? * If APR has disabled threads, flood should as well. We might want to have an enable/disable parameter that does this also, providing an error if threads are desired but not available. * flood needs to clear pools more often. With a long running test it can chew up memory very quickly. We should just bite the bullet and create/destroy/clear pools for each level of our model: farm, farmer, profile, url/request-cycle, etc. * APR needs to have a unified interface for ephemeral port exhaustion, but aparently Solaris and Linux return different errors at the moment. Fix this in APR then take advantage of it in flood. * The examples/analyze-relative scripts fail when there are less than 5 unique URLs. Other features that need writing: * It would be nice if a url in a urllist could handle basic authentication. * More analysis and graphing scripts are needed * Write robust tool (using tethereal perhaps) to take network dumps and convert them to flood's XML format. Status: Justin volunteers. Aaron had a script somewhere that is a start. * Get chunked encoding support working. Status: Justin volunteers. He got sidetracked by the httpd implementation of input filtering and never finished this. This is a stopgap until apr-serf is completed. * Maybe we should make randfile and capath runtime directives that come out of the XML, instead of autoconf parameters. * We are using apr_os_thread_current() and getpid() in some places when what we really want is a GUID. The GUID will be used to correlate raw output data with each farmer. We may wish to print a unique ID for each of farm, farmer, profile, and url to help in postprocessing. * We are using strtol() in some places and strtoll() in others. Pick one (Aaron says strtol(), but he's not sure). * Validation of responses (known C-L, specific strings in response) Status: Justin volunteers * HTTP error codes (ie. teach it about 302s) Justin says: Yeah, this won't be with round_robin as implemented. Need a linked list-based profile where we can insert new URLs into the sequence. * Farmer (Single-thread, multiple profiles) Status: Aaron says: If you have threads, then any Farmer can be run as part of any Farm. If you don't have threads, you can currently only run one Farmer named "Joe" right now (this will be changed so that if you don't have threads, flood will attempt to run all Farmers in serial under one process). * Collective (Single-host, multiple farms) This is a number of Farms that have been fork()ed into child processes. * Megaconglomerate (Multiple hosts each running a collective) This is a number of Collectives running on a number of hosts, invoked via RSH/SSH or maybe even some proprietary mechanism. * Other types of urllists a) Random / Random-weighted b) Sequenced (useful with cookie propogation) c) Round-robin d) Chaining of the above strategies Status: Round-robin is complete. * Other types of reports Status: Aaron says: "simple" reports are functional. Justin added a new type that simply prints the approx. timestamp when the test was run, and the result as OK/FAIL; it is called "easy reports" (see flood_easy_reports.h). Furthermore, simple_reports and easy_reports both print out the current requesting URI line. Documentation that needs writing: * Documentation? What documentation? RTFS? Status: Justin volunteers. He'll probably use Anakia for user docs and doxygen for source code comments. * Feature set We'll have to eventually write down all the features we support, which will most likely come out to be the various XML parameters we support. Available Patches: Open issues: * Ponder using apr-serf in flood Status: This requires apr-serf to be written. Chicken and egg. Ideally, apr-serf would handle buckets, filters, and other cool stuff. However, this isn't the highest of priorities. * Validating XML Parser? Justin says: I don't think we want this. We want a standalone validator. Aaron says: I strongly feel we should have this turned on at least for the invoking "Megaconglomerate". The added overhead for validating when we're already parsing will be minimal. It will not affect the results of the tests. It only happens when a new flood process is created (directly or via rsh/ssh), not when invoking a collective, farm, farmer, or profile. What it gives us from the user's standpoint is far greater than the impact of the added startup overhead. (Maybe a flood argument that disables it for when some UI wrapper already did the checking or if it's being invoked as part of a megaconglomerate (rsh/ssh).) * Supporting use of installed APR / APR-util Justin says: Requires changes to APR/APR-util to make it install the right stuff. We currently rely on the source. * Mandrake Linux 8.0 and OpenSSL 0.9.6a just aren't scottflores@example.net. The problem seems to be that OpenSSL is refusing to initialize the PRNG. Go figure. I give up on this for now, but it is a big nasty bug in something somewhere. This code works on Solaris/Intel. That's all I'll say on the matter for now. * Report Aggregation We're not sure how to handle reporting quite yet. We'd like to keep it open ended and flexible, but that will be difficult to do while maintaining the ability to run a huge variety of tests. * WebLogic 5.10 Service Pack 9 No one told BEA how to make an HTTP server. Send it Connection: Close and a cookie, and it'll respond with Connection: Keep-Alive (no cookie and it honors Connection: Close). That's wrong. From testscottflores@example.net Thu Aug 08 03:45:18 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 88033 invoked by uid 500); 8 Aug 2002 03:45:18 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 87965 invoked from network); 8 Aug 2002 03:45:17 -0000 Date: Wed, 7 Aug 2002 23:45:29 -0400 Message-Id: <scottflores@example.net> From: Rodent of Unusual Size <scottflores@example.net> To: Apache test suite <scottflores@example.net> Subject: [STATUS] (perl-framework) Wed Aug 7 23:45:29 EDT 2002 X-Note: This is an automated message. X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N httpd-test/perl-framework STATUS: -*-text-*- Last modified at [$Date: 2002/03/09 05:22:48 $] Stuff to do: * finish the t/TEST exit code issue (ORed with 0x2C if framework failed) * change existing tests that frob the DocumentRoot (e.g., t/modules/access.t) to *not* do that; instead, have Makefile.PL prepare appropriate subdirectory configs for them. Why? So t/TEST can be used to test a remote server. * problems with -d perl mode, doesn't work as documented Message-ID: <scottflores@example.net> Date: Sat, 20 Oct 2001 12:58:33 +0800 Subject: Re: perldb Tests to be written: * t/apache - simulations of network failures (incomplete POST bodies, chunked and unchunked; missing POST bodies; slooow client connexions, such as taking 1 minute to send 1KiB; ...) * t/modules/autoindex - something seems possibly broken with inheritance on 2.0 * t/ssl - SSLPassPhraseDialog exec: - SSLRandomSeed exec: From testscottflores@example.net Thu Aug 15 03:46:01 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 69624 invoked by uid 500); 15 Aug 2002 03:46:01 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 69499 invoked from network); 15 Aug 2002 03:45:59 -0000 Date: Wed, 14 Aug 2002 23:46:10 -0400 Message-Id: <scottflores@example.net> From: Rodent of Unusual Size <scottflores@example.net> To: Apache test suite <scottflores@example.net> Subject: [STATUS] (flood) Wed Aug 14 23:46:10 EDT 2002 X-Note: This is an automated message. X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N flood STATUS: -*-text-*- Last modified at [$Date: 2002/08/01 22:28:51 $] Release: 1.0: Released July 23, 2002 milestone-03: Tagged January 16, 2002 ASF-transfer: Released July 17, 2001 milestone-02: Tagged August 13, 2001 milestone-01: Tagged July 11, 2001 (tag lost during transfer) RELEASE SHOWSTOPPERS: * "Everything needs to work perfectly" Other bugs that need fixing: * I get a SIGBUS on Darwin with our examples/round-robin-ssl.xml config, on the second URL. I'm using OpenSSL 0.9.6c 21 dec 2001. * DNS lookup failures in any <urllist> cause segfault. Justin says: Wow. Why? Aaron says: Is this still happening? It's been awhile. * iPlanet sends "Content-length" - there is a hack in there now to recognize it. However, all HTTP headers need to be normalized before checking their values. This isn't easy to do. Grr. * OpenSSL 0.9.6 Segfaults under high load. Upgrade to OpenSSL 0.9.6b. Aaron says: I just found a big bug that might have been causing this all along (we weren't closing ssl sockets). How can I reproduce the problem you were seeing to verify if this was the fix? * SEGVs when /tmp/.rnd doesn't exist are bad. Make it configurable and at least bomb with a good error message. (See Doug's patch.) Status: This is fixed, no? * If APR has disabled threads, flood should as well. We might want to have an enable/disable parameter that does this also, providing an error if threads are desired but not available. * flood needs to clear pools more often. With a long running test it can chew up memory very quickly. We should just bite the bullet and create/destroy/clear pools for each level of our model: farm, farmer, profile, url/request-cycle, etc. * APR needs to have a unified interface for ephemeral port exhaustion, but aparently Solaris and Linux return different errors at the moment. Fix this in APR then take advantage of it in flood. * The examples/analyze-relative scripts fail when there are less than 5 unique URLs. Other features that need writing: * It would be nice if a url in a urllist could handle basic authentication. * More analysis and graphing scripts are needed * Write robust tool (using tethereal perhaps) to take network dumps and convert them to flood's XML format. Status: Justin volunteers. Aaron had a script somewhere that is a start. * Get chunked encoding support working. Status: Justin volunteers. He got sidetracked by the httpd implementation of input filtering and never finished this. This is a stopgap until apr-serf is completed. * Maybe we should make randfile and capath runtime directives that come out of the XML, instead of autoconf parameters. * We are using apr_os_thread_current() and getpid() in some places when what we really want is a GUID. The GUID will be used to correlate raw output data with each farmer. We may wish to print a unique ID for each of farm, farmer, profile, and url to help in postprocessing. * We are using strtol() in some places and strtoll() in others. Pick one (Aaron says strtol(), but he's not sure). * Validation of responses (known C-L, specific strings in response) Status: Justin volunteers * HTTP error codes (ie. teach it about 302s) Justin says: Yeah, this won't be with round_robin as implemented. Need a linked list-based profile where we can insert new URLs into the sequence. * Farmer (Single-thread, multiple profiles) Status: Aaron says: If you have threads, then any Farmer can be run as part of any Farm. If you don't have threads, you can currently only run one Farmer named "Joe" right now (this will be changed so that if you don't have threads, flood will attempt to run all Farmers in serial under one process). * Collective (Single-host, multiple farms) This is a number of Farms that have been fork()ed into child processes. * Megaconglomerate (Multiple hosts each running a collective) This is a number of Collectives running on a number of hosts, invoked via RSH/SSH or maybe even some proprietary mechanism. * Other types of urllists a) Random / Random-weighted b) Sequenced (useful with cookie propogation) c) Round-robin d) Chaining of the above strategies Status: Round-robin is complete. * Other types of reports Status: Aaron says: "simple" reports are functional. Justin added a new type that simply prints the approx. timestamp when the test was run, and the result as OK/FAIL; it is called "easy reports" (see flood_easy_reports.h). Furthermore, simple_reports and easy_reports both print out the current requesting URI line. Documentation that needs writing: * Documentation? What documentation? RTFS? Status: Justin volunteers. He'll probably use Anakia for user docs and doxygen for source code comments. * Feature set We'll have to eventually write down all the features we support, which will most likely come out to be the various XML parameters we support. Available Patches: Open issues: * Ponder using apr-serf in flood Status: This requires apr-serf to be written. Chicken and egg. Ideally, apr-serf would handle buckets, filters, and other cool stuff. However, this isn't the highest of priorities. * Validating XML Parser? Justin says: I don't think we want this. We want a standalone validator. Aaron says: I strongly feel we should have this turned on at least for the invoking "Megaconglomerate". The added overhead for validating when we're already parsing will be minimal. It will not affect the results of the tests. It only happens when a new flood process is created (directly or via rsh/ssh), not when invoking a collective, farm, farmer, or profile. What it gives us from the user's standpoint is far greater than the impact of the added startup overhead. (Maybe a flood argument that disables it for when some UI wrapper already did the checking or if it's being invoked as part of a megaconglomerate (rsh/ssh).) * Supporting use of installed APR / APR-util Justin says: Requires changes to APR/APR-util to make it install the right stuff. We currently rely on the source. * Mandrake Linux 8.0 and OpenSSL 0.9.6a just aren't scottflores@example.net. The problem seems to be that OpenSSL is refusing to initialize the PRNG. Go figure. I give up on this for now, but it is a big nasty bug in something somewhere. This code works on Solaris/Intel. That's all I'll say on the matter for now. * Report Aggregation We're not sure how to handle reporting quite yet. We'd like to keep it open ended and flexible, but that will be difficult to do while maintaining the ability to run a huge variety of tests. * WebLogic 5.10 Service Pack 9 No one told BEA how to make an HTTP server. Send it Connection: Close and a cookie, and it'll respond with Connection: Keep-Alive (no cookie and it honors Connection: Close). That's wrong. From testscottflores@example.net Thu Aug 15 03:46:02 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 69750 invoked by uid 500); 15 Aug 2002 03:46:02 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 69626 invoked from network); 15 Aug 2002 03:46:01 -0000 Date: Wed, 14 Aug 2002 23:46:14 -0400 Message-Id: <scottflores@example.net> From: Rodent of Unusual Size <scottflores@example.net> To: Apache test suite <scottflores@example.net> Subject: [STATUS] (perl-framework) Wed Aug 14 23:46:13 EDT 2002 X-Note: This is an automated message. X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N httpd-test/perl-framework STATUS: -*-text-*- Last modified at [$Date: 2002/03/09 05:22:48 $] Stuff to do: * finish the t/TEST exit code issue (ORed with 0x2C if framework failed) * change existing tests that frob the DocumentRoot (e.g., t/modules/access.t) to *not* do that; instead, have Makefile.PL prepare appropriate subdirectory configs for them. Why? So t/TEST can be used to test a remote server. * problems with -d perl mode, doesn't work as documented Message-ID: <scottflores@example.net> Date: Sat, 20 Oct 2001 12:58:33 +0800 Subject: Re: perldb Tests to be written: * t/apache - simulations of network failures (incomplete POST bodies, chunked and unchunked; missing POST bodies; slooow client connexions, such as taking 1 minute to send 1KiB; ...) * t/modules/autoindex - something seems possibly broken with inheritance on 2.0 * t/ssl - SSLPassPhraseDialog exec: - SSLRandomSeed exec: From testscottflores@example.net Fri Aug 16 14:04:37 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 25312 invoked by uid 500); 16 Aug 2002 14:04:37 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 25301 invoked from network); 16 Aug 2002 14:04:36 -0000 Sender: scottflores@example.net Message-ID: <scottflores@example.net> Date: Fri, 16 Aug 2002 10:01:04 -0400 From: Greg Ames <scottflores@example.net> X-Mailer: Mozilla 4.77 [en] (X11; U; Linux 2.2.19-10mdk i686) MIME-Version: 1.0 To: Chris Johnson <scottflores@example.net>, scottflores@example.net Subject: Re: mod_specweb99 debugging... References: <scottflores@example.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N Chris Johnson wrote: > > Hi: > I'm using your specweb99 modules to examine apache performance, and was > wondering if you have newer versions handy - I noticed that the 1.3 > version did not compile on rh7.2 > (the SEMCNTR define needed to be disabled); it seemed to work > afterwards. Would you mind sending a patch to scottflores@example.net? We like diff -u. You might want to subscribe to scottflores@example.net (developer discussions) and scottflores@example.net (automatic cvs commit notifications). There are a couple of other subprojects supported there, but they are both very low volume mailing lists. > The 2.0 version has problems with dynamic content (lots of invalid and expired > ads). I fixed that one. You see those messages on the very first pass after the client tests the various types of requests, and then they go away on subsequent passes, right? If so, you can just do a "cvs up" on httpd-test to get that fix. I assume you know how to use anonymous cvs? > Being new to apache debugging, any info on debugging these modules would > be greatly appreciated. > > thanks > > P.S. I'm hoping to analyze kernel bottlenecks (using lockmeter, cpu > statistics, etc.) after examining optimizing httpd in user space. Sounds great! I hope you have a *lot* of client horsepower and bandwidth, or else a very wimpy server with a lot of disk that can run current software. Otherwise it's hard to drive the server to saturation with this workload. Naturally, we'd like to see any interesting user space results you come up with on scottflores@example.net (especially for 2.0) or scottflores@example.net, as appropriate. Some Linux kernel hackers I work with at IBM are also doing specweb99 benchmarks with 2.0. They are seeing returnHTMLPageWithBuffer (a mod_specweb99 function) as the hottest user space function. But they also have issues with oprofile overflowing some hash table, so I don't know whether to believe it or not. No smoking guns jump out at me when I inspect that code, so independent confirmation would be nice. Greg From testscottflores@example.net Fri Aug 16 19:05:14 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 19288 invoked by uid 500); 16 Aug 2002 19:05:11 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 19264 invoked from network); 16 Aug 2002 19:05:11 -0000 Sender: scottflores@example.net Message-ID: <scottflores@example.net> Date: Fri, 16 Aug 2002 15:01:49 -0400 From: Greg Ames <scottflores@example.net> X-Mailer: Mozilla 4.77 [en] (X11; U; Linux 2.2.19-10mdk i686) MIME-Version: 1.0 To: Ian Holsman <scottflores@example.net> CC: scottflores@example.net Subject: Re: mod_specweb99 debugging... References: <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N Ian Holsman wrote: > i'm just about to talk to ***** who are going to do some static specweb > tests with apache2 > any tips you would like to share with them ??? yeah. * When you source the shrc file, don't expect to see any output. It just sets some envvars. * Look at the SPECWeb99 results page to see how some of the winning entries have tuned the servers. Also notice how many client boxes and how many/what kind of NICs they are using. * The harder you drive the server, the more disk space you will need. SPECWeb99 touches more files as you ramp up the number of concurrent connections. There is a formula here: http://www.spec.org/osg/web99/docs/users_guide.html#Pre-Inst Plus you need disk space for logs proportional to the load. * I wouldn't be surprised if there are bugs or bottlenecks in mod_specweb99 itself, since it was only recently open sourced. Please feed back any info/patches to this list. But you don't need mod_specweb99 if you are only doing static content. Greg From testscottflores@example.net Fri Aug 16 19:26:15 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 42171 invoked by uid 500); 16 Aug 2002 19:26:14 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 42156 invoked from network); 16 Aug 2002 19:26:13 -0000 Date: Fri, 16 Aug 2002 21:26:15 +0200 (CEST) From: Dirk-Willem van Gulik <scottflores@example.net> X-X-Sender: scottflores@example.net Reply-To: scottflores@example.net To: scottflores@example.net cc: Ian Holsman <scottflores@example.net>, Sander Temme <scottflores@example.net> Subject: Re: mod_specweb99 debugging... In-Reply-To: <scottflores@example.net> Message-ID: <scottflores@example.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N > * The harder you drive the server, the more disk space you will need. SPECWeb99 > touches more files as you ramp up the number of concurrent connections. There > is a formula here: http://www.spec.org/osg/web99/docs/users_guide.html#Pre-Inst > Plus you need disk space for logs proportional to the load. > * I wouldn't be surprised if there are bugs or bottlenecks in mod_specweb99 > itself, since it was only recently open sourced. Please feed back any > info/patches to this list. But you don't need mod_specweb99 if you are only > doing static content. SpecWeb right now follows the run book very liberally - and there are certainly shortcuts you could take whilst staying within the run rules. If you want to I can dig out a list. Also - hack the conf to do short runs first ! You are going to find plenty of network, kernel and OS bottlenecks. It is for a good reason that those setups you find on the spec.org side are such massive beasts. If you clients are not on windows - you may want to talk to Sander Temme - Covalent had to hack them on FreeBSD to get the byte counts done reliably DW From testscottflores@example.net Fri Aug 16 20:12:39 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 84618 invoked by uid 500); 16 Aug 2002 20:12:39 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 84552 invoked from network); 16 Aug 2002 20:12:38 -0000 Sender: scottflores@example.net Message-ID: <scottflores@example.net> Date: Fri, 16 Aug 2002 16:09:13 -0400 From: Greg Ames <scottflores@example.net> X-Mailer: Mozilla 4.77 [en] (X11; U; Linux 2.2.19-10mdk i686) MIME-Version: 1.0 To: scottflores@example.net Subject: Re: mod_specweb99 debugging... References: <scottflores@example.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N Dirk-Willem van Gulik wrote: > SpecWeb right now follows the run book very liberally - and there are > certainly shortcuts you could take whilst staying within the run rules. If > you want to I can dig out a list. It would be great if you could post such a list. Looking at SPECWeb99 dynamic content code for other web servers gives me a few ideas too. Since we have a number of people doing SPECWeb99 benchmarking of Apache 2 at the moment, and httpd itself is relatively stable (knock on wood), it's a good time to focus on making mod_specweb99 better. I'd love to see winning results posted to the SPECWeb99 site mentioning Apache 2, or understand why we can't do it. > Also - hack the conf to do short runs first ! Oh yeah, excellent point. That's the SPEC rc file. Look for *_TIME, ITERATIONS, and SIMULTANEOUS_CONNECTIONS. Greg From testscottflores@example.net Fri Aug 16 20:20:15 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 94495 invoked by uid 500); 16 Aug 2002 20:20:14 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 94479 invoked from network); 16 Aug 2002 20:20:14 -0000 User-Agent: Microsoft-Entourage/83.161.120.1556 Date: Fri, 16 Aug 2002 13:20:20 -0700 Subject: Re: mod_specweb99 debugging... From: Sander Temme <scottflores@example.net> To: "test dev httpd.apache.org" <scottflores@example.net> Message-ID: <scottflores@example.net> In-Reply-To: <scottflores@example.net> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N >> Also - hack the conf to do short runs first ! > > Oh yeah, excellent point. That's the SPEC rc file. Look for *_TIME, > ITERATIONS, and SIMULTANEOUS_CONNECTIONS. My rc files usually have trial (short) values for *_TIME and a single iteration. You can force all those in line for a compliant run with the -C option on the command line. The only time parameter that is not affected by -C is WARMUP_TIME. Note that when you do short runs and get bandwidth misreporting, increasing the run length (like, from two to four or five minutes) will allow you to go further. Also, I have seen divide-by-zero problems on extremely short runs. S. -- Covalent Technologies scottflores@example.net Engineering group Voice: (415) 536 5214 645 Howard St. Fax: (336)537-3130 San Francisco CA 94105 PGP Fingerprint: 1E74 4E58 DFAC 2CF5 6A03 5531 AFB1 96AF B584 0AB1 ======================================================= This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message ======================================================= From testscottflores@example.net Mon Aug 19 15:20:23 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 87941 invoked by uid 500); 19 Aug 2002 15:20:22 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 87930 invoked from network); 19 Aug 2002 15:20:22 -0000 Message-ID: <scottflores@example.net> Date: Mon, 19 Aug 2002 08:20:28 -0700 From: Ian Holsman <scottflores@example.net> User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.0) Gecko/20020713 X-Accept-Language: en MIME-Version: 1.0 To: scottflores@example.net Subject: Re: mod_specweb99 debugging... References: <scottflores@example.net> <scottflores@example.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N Dirk-Willem van Gulik wrote: >>* The harder you drive the server, the more disk space you will need. SPECWeb99 >>touches more files as you ramp up the number of concurrent connections. There >>is a formula here: http://www.spec.org/osg/web99/docs/users_guide.html#Pre-Inst >>Plus you need disk space for logs proportional to the load. > > Thanks... what I was after was more hints on configuring HTTP. ie.. make sure FollowSymLinks is On & AllowOverride off (to avoid unnessecary fileops) and things like this >>* I wouldn't be surprised if there are bugs or bottlenecks in mod_specweb99 >>itself, since it was only recently open sourced. Please feed back any >>info/patches to this list. But you don't need mod_specweb99 if you are only >>doing static content. > > > SpecWeb right now follows the run book very liberally - and there are > certainly shortcuts you could take whilst staying within the run rules. If > you want to I can dig out a list. > > Also - hack the conf to do short runs first ! > > You are going to find plenty of network, kernel and OS bottlenecks. It is > for a good reason that those setups you find on the spec.org side are such > massive beasts. If you clients are not on windows - you may want to talk > to Sander Temme - Covalent had to hack them on FreeBSD to get the byte > counts done reliably > > DW > From testscottflores@example.net Mon Aug 19 17:23:35 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 9094 invoked by uid 500); 19 Aug 2002 17:23:35 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 9021 invoked from network); 19 Aug 2002 17:23:34 -0000 Sender: scottflores@example.net Message-ID: <scottflores@example.net> Date: Mon, 19 Aug 2002 13:20:09 -0400 From: Greg Ames <scottflores@example.net> X-Mailer: Mozilla 4.77 [en] (X11; U; Linux 2.2.19-10mdk i686) MIME-Version: 1.0 To: scottflores@example.net Subject: Re: mod_specweb99 debugging... References: <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N Ian Holsman wrote: > Thanks... > what I was after was more hints on configuring HTTP. > ie.. make sure FollowSymLinks is On & AllowOverride off > (to avoid unnessecary fileops) and things like this Oh, OK, then http://httpd.apache.org/docs-2.0/misc/perf-tuning.html would be relevant. I set up my specweb99 DocumentRoot subtree to be as flat as possible, thinking that would cut down on cycles spent walking the directory paths. I can't say for certain that it makes a significant difference on Linux; the kernel has an efficient filepath cache. Another area for potential improvement here is the use of file caching. I tried using mod_cache's fd caching, but quickly ran into problems with the Linux per-process fd limits because SPECWeb99 accesses so many different files. It's possible that caching the file contents in memory might allow more files to be cached. But any attempt to cache files in userland will tend to negate the kernel's file caching, which is pretty efficient on Linux. The choice of MPMs makes a difference too. I get better results with prefork on Linux, with worker not too far behind. With a good userland file caching strategy, worker might do better if it has a higher cache hit ratio and lower overall RAM usage. Greg From testscottflores@example.net Mon Aug 19 17:42:20 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 70765 invoked by uid 500); 19 Aug 2002 17:42:20 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 70718 invoked from network); 19 Aug 2002 17:42:18 -0000 Message-ID: <scottflores@example.net> Date: Mon, 19 Aug 2002 10:42:26 -0700 From: Ian Holsman <scottflores@example.net> User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.0) Gecko/20020713 X-Accept-Language: en MIME-Version: 1.0 To: scottflores@example.net Subject: Re: mod_specweb99 debugging... References: <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N Greg Ames wrote: > Ian Holsman wrote: > > >>Thanks... >>what I was after was more hints on configuring HTTP. >>ie.. make sure FollowSymLinks is On & AllowOverride off >>(to avoid unnessecary fileops) and things like this > > > Oh, OK, then http://httpd.apache.org/docs-2.0/misc/perf-tuning.html would be > relevant. yeah.. I need to read it again, and see if I can add any perls > > I set up my specweb99 DocumentRoot subtree to be as flat as possible, thinking > that would cut down on cycles spent walking the directory paths. I can't say > for certain that it makes a significant difference on Linux; the kernel has an > efficient filepath cache. > > Another area for potential improvement here is the use of file caching. I tried > using mod_cache's fd caching, but quickly ran into problems with the Linux > per-process fd limits because SPECWeb99 accesses so many different files. It's > possible that caching the file contents in memory might allow more files to be > cached. But any attempt to cache files in userland will tend to negate the > kernel's file caching, which is pretty efficient on Linux. hmm.. did you use a recent version of mod_mem_cache.. you should be able to set the maximum number of fd's to be under the limits (not sure if that would help.. depends on the caching) > > The choice of MPMs makes a difference too. I get better results with prefork on > Linux, with worker not too far behind. With a good userland file caching > strategy, worker might do better if it has a higher cache hit ratio and lower > overall RAM usage. > yes.. prefork with threading completly disabled helps > Greg > From testscottflores@example.net Mon Aug 19 19:31:19 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 12337 invoked by uid 500); 19 Aug 2002 19:31:19 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 12323 invoked from network); 19 Aug 2002 19:31:18 -0000 Sender: scottflores@example.net Message-ID: <scottflores@example.net> Date: Mon, 19 Aug 2002 15:27:53 -0400 From: Greg Ames <scottflores@example.net> X-Mailer: Mozilla 4.77 [en] (X11; U; Linux 2.2.19-10mdk i686) MIME-Version: 1.0 To: scottflores@example.net Subject: Re: mod_specweb99 debugging... References: <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N Ian Holsman wrote: > > Another area for potential improvement here is the use of file caching. I tried > > using mod_cache's fd caching, but quickly ran into problems with the Linux > > per-process fd limits because SPECWeb99 accesses so many different files. > hmm.. did you use a recent version of mod_mem_cache.. you should be able > to set the maximum number of fd's to be under the limits (not sure if > that would help.. depends on the caching) I had a current mod_mem_cache at the time I tried it (2-3 months ago?). The problem is that when I tell SPECWeb99 that I'm shooting for around 250 concurrent connections on my old ThinkPad, it wants to access 6,768 different files. My ulimit -n on Linux is 1024 and sysctl doesn't like me trying to raise it. So without a lot of pain, I can only cache less than 1/6 of the files using fd caching. As I recall, I restricted the fd caching to the most popular range of files sizes to keep it under the 1k limit, and then proceeded to run into system wide fd limits with prefork. Worker was a lot better of course with respect to system wide fd usage. The current record holder achieved 5000 concurrent connections, which means that the SPECWeb99 clients accessed about 119,700 different files, so the problem gets harder as the server gets more powerful. Greg From testscottflores@example.net Wed Aug 21 15:05:09 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 70233 invoked by uid 500); 21 Aug 2002 15:05:08 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 70221 invoked from network); 21 Aug 2002 15:05:08 -0000 Message-ID: <scottflores@example.net> From: "Michaud,Pierre [CMC]" <scottflores@example.net> To: "'scottflores@example.net'" <scottflores@example.net> Subject: flood round-robin-dynamic.xml problem Date: Wed, 21 Aug 2002 12:04:57 -0300 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="ISO-8859-1" X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N Hi, I've just installed flood on my linux machine (red hat 7.3). I've followed all the steps indicated at http://httpd.apache.org/test/flood/building.html . I've tried successfully the round-robin-example.xml example. But, when I try the round-robin-dynamic.xml example, it is not working. Here is what I get: [asdf@ghjk flood]$ ./flood examples/round-robin-dynamic.xml > time.out flood: flood_round_robin.c:867: round_robin_postprocess: Assertion `status == 0' failed. Abort I've put in comments the lines that targets www.apachelabs.org (this site is not working for now) and activated the lines that generate the foo search on google. Any idea of what is happening? Thanks, Newbie Pierre From testscottflores@example.net Wed Aug 21 17:27:01 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 54129 invoked by uid 500); 21 Aug 2002 17:26:59 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 53986 invoked from network); 21 Aug 2002 17:26:56 -0000 X-Authentication-Warning: cancer.clove.org: jerenk set sender to scottflores@example.net using -f Date: Wed, 21 Aug 2002 10:26:58 -0700 From: Justin Erenkrantz <scottflores@example.net> To: scottflores@example.net Subject: Re: flood round-robin-dynamic.xml problem Message-ID: <scottflores@example.net> Mail-Followup-To: Justin Erenkrantz <scottflores@example.net>, scottflores@example.net References: <scottflores@example.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <scottflores@example.net> User-Agent: Mutt/1.4i X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N On Wed, Aug 21, 2002 at 12:04:57PM -0300, Michaud,Pierre [CMC] wrote: > [asdf@ghjk flood]$ ./flood examples/round-robin-dynamic.xml > time.out > flood: flood_round_robin.c:867: round_robin_postprocess: Assertion `status > == 0' failed. > Abort > > I've put in comments the lines that targets www.apachelabs.org (this site is > not working for now) and activated the lines that generate the foo search on > google. Google changed its result format (which means that the postprocessing will fail as the response isn't what it expects) and apachelabs.org is now dead. In short, flood is doing the right thing - round-robin-dynamic.xml would need to be rewritten to match Google's new result format. Namely the responsetemplate parameter would have to be written to be a valid regexp on the response from Google. -- justin From testscottflores@example.net Wed Aug 21 18:04:03 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 93339 invoked by uid 500); 21 Aug 2002 18:04:03 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 93328 invoked from network); 21 Aug 2002 18:04:02 -0000 Date: Wed, 21 Aug 2002 11:04:06 -0700 From: Aaron Bannert <scottflores@example.net> To: scottflores@example.net Subject: Re: flood round-robin-dynamic.xml problem Message-ID: <scottflores@example.net> References: <scottflores@example.net> <scottflores@example.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <scottflores@example.net> User-Agent: Mutt/1.4i X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N On Wed, Aug 21, 2002 at 10:26:58AM -0700, Justin Erenkrantz wrote: > Google changed its result format (which means that the postprocessing > will fail as the response isn't what it expects) and apachelabs.org > is now dead. > > In short, flood is doing the right thing - round-robin-dynamic.xml > would need to be rewritten to match Google's new result format. > > Namely the responsetemplate parameter would have to be written to > be a valid regexp on the response from Google. -- justin Are we throwing an exception when the regexp doesn't match? -aaron From testscottflores@example.net Wed Aug 21 18:05:58 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 95107 invoked by uid 500); 21 Aug 2002 18:05:58 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 95096 invoked from network); 21 Aug 2002 18:05:57 -0000 X-Authentication-Warning: cancer.clove.org: jerenk set sender to scottflores@example.net using -f Date: Wed, 21 Aug 2002 11:06:02 -0700 From: Justin Erenkrantz <scottflores@example.net> To: scottflores@example.net Subject: Re: flood round-robin-dynamic.xml problem Message-ID: <scottflores@example.net> Mail-Followup-To: Justin Erenkrantz <scottflores@example.net>, scottflores@example.net References: <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <scottflores@example.net> User-Agent: Mutt/1.4i X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N On Wed, Aug 21, 2002 at 11:04:06AM -0700, Aaron Bannert wrote: > Are we throwing an exception when the regexp doesn't match? That's what is happening - we throw an assertion when the regexp fails. -- justin From testscottflores@example.net Wed Aug 21 20:59:12 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 71649 invoked by uid 500); 21 Aug 2002 20:59:11 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 71615 invoked from network); 21 Aug 2002 20:59:09 -0000 Date: Wed, 21 Aug 2002 13:59:15 -0700 From: Aaron Bannert <scottflores@example.net> To: scottflores@example.net Subject: Re: flood round-robin-dynamic.xml problem Message-ID: <scottflores@example.net> References: <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <scottflores@example.net> User-Agent: Mutt/1.4i X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N On Wed, Aug 21, 2002 at 11:06:02AM -0700, Justin Erenkrantz wrote: > On Wed, Aug 21, 2002 at 11:04:06AM -0700, Aaron Bannert wrote: > > Are we throwing an exception when the regexp doesn't match? > > That's what is happening - we throw an assertion when the regexp > fails. -- justin So we need to fix that, right? An unmatched regexp shouldn't be fatal. -aaron From testscottflores@example.net Wed Aug 21 21:02:27 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 75463 invoked by uid 500); 21 Aug 2002 21:02:27 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 75452 invoked from network); 21 Aug 2002 21:02:27 -0000 X-Authentication-Warning: cancer.clove.org: jerenk set sender to scottflores@example.net using -f Date: Wed, 21 Aug 2002 14:02:32 -0700 From: Justin Erenkrantz <scottflores@example.net> To: scottflores@example.net Subject: Re: flood round-robin-dynamic.xml problem Message-ID: <scottflores@example.net> Mail-Followup-To: Justin Erenkrantz <scottflores@example.net>, scottflores@example.net References: <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <scottflores@example.net> User-Agent: Mutt/1.4i X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N On Wed, Aug 21, 2002 at 01:59:15PM -0700, Aaron Bannert wrote: > So we need to fix that, right? An unmatched regexp shouldn't be fatal. The thought was that if the regexp failed, some expectation of how the site was designed has changed. So, rather than continuing on, abort. The only way to correct the problem would be to alter the profile to correct the regexp. It's one of the few places where I like assertions as this is user-correctable. =) -- justin From testscottflores@example.net Wed Aug 21 21:05:52 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 78181 invoked by uid 500); 21 Aug 2002 21:05:52 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 78163 invoked from network); 21 Aug 2002 21:05:51 -0000 Date: Wed, 21 Aug 2002 14:05:57 -0700 From: Aaron Bannert <scottflores@example.net> To: scottflores@example.net Subject: Re: flood round-robin-dynamic.xml problem Message-ID: <scottflores@example.net> References: <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <scottflores@example.net> User-Agent: Mutt/1.4i X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N On Wed, Aug 21, 2002 at 02:02:32PM -0700, Justin Erenkrantz wrote: > On Wed, Aug 21, 2002 at 01:59:15PM -0700, Aaron Bannert wrote: > > So we need to fix that, right? An unmatched regexp shouldn't be fatal. > > The thought was that if the regexp failed, some expectation of how > the site was designed has changed. So, rather than continuing on, > abort. The only way to correct the problem would be to alter the > profile to correct the regexp. Sure, but this isn't informative: flood: flood_round_robin.c:867: round_robin_postprocess: Assertion `status == 0' failed. Besides, if the user disables debugging, the assertions will be disabled. If this is a fatal error (and I'm still on the fence here) then we should detect that case and print a nice message. -aaron From testscottflores@example.net Wed Aug 21 21:08:50 2002 Return-Path: <testscottflores@example.net> Delivered-To: scottflores@example.net Received: (qmail 80200 invoked by uid 500); 21 Aug 2002 21:08:50 -0000 Mailing-List: contact scottflores@example.net; run by ezmlm Precedence: bulk Reply-To: scottflores@example.net list-help: <mailto:scottflores@example.net> list-unsubscribe: <mailto:scottflores@example.net> list-post: <mailto:scottflores@example.net> Delivered-To: mailing list scottflores@example.net Received: (qmail 80189 invoked from network); 21 Aug 2002 21:08:50 -0000 Date: Wed, 21 Aug 2002 14:08:56 -0700 From: Aaron Bannert <scottflores@example.net> To: scottflores@example.net Subject: Re: flood round-robin-dynamic.xml problem Message-ID: <scottflores@example.net> References: <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> <scottflores@example.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <scottflores@example.net> User-Agent: Mutt/1.4i X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N > Besides, if the user disables debugging, the assertions will be > disabled. If this is a fatal error (and I'm still on the fence here) > then we should detect that case and print a nice message. Since I don't plan on fixing this, and since we both seem to be in a mostly maintenance mode, I've added this to the STATUS in case anyone else wishes to contribute the fix. -aaron
{'source': 'Apache Software Foundation Public Mail Archives'}