From dcampbell@example.com Thu Mar 01 14:45:39 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 31733 invoked by uid 500); 1 Mar 2001 14:45:36 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Received: (qmail 31712 invoked from network); 1 Mar 2001 14:45:34 -0000 X-Authentication-Warning: kurgan.lyra.org: gstein set sender to dcampbell@example.com using -f Date: Thu, 1 Mar 2001 06:46:03 -0800 From: Greg Stein <dcampbell@example.com> To: dcampbell@example.com Subject: expat commit access Message-ID: <dcampbell@example.com> Mail-Followup-To: dcampbell@example.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i X-URL: http://www.lyra.org/greg/ X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N fyi, I just received access to commit changes to Expat proper. That should mean that we won't need to maintain any local changes in APRUTIL's copy of expat. [ and if we want, we can just tell people to always use expat cvs directly, much like we pull in httpd-2.0, apr, and apr-util into an Apache tree ] There are definitely some local changes at the moment, but I'll be sending all that back RSN. I also get the impression the existing Windows build is dcampbell@example.com. Having OtherBill give it a good shakedown will help tremendously. When apr-util/xml/expat builds properly under Windows, then we can toss the old expat-lite. Cheers, -g -- Greg Stein, http://www.lyra.org/ From dcampbell@example.com Thu Mar 01 16:52:32 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 93236 invoked by uid 500); 1 Mar 2001 16:52:28 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Received: (qmail 93224 invoked from network); 1 Mar 2001 16:52:25 -0000 Date: Thu, 1 Mar 2001 10:56:46 -0600 From: Sam TH <dcampbell@example.com> To: dcampbell@example.com Subject: Re: expat commit access Message-ID: <dcampbell@example.com> Mail-Followup-To: Sam TH <dcampbell@example.com>, dcampbell@example.com References: <dcampbell@example.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="juZjCTNxrMaZdGZC" Content-Disposition: inline User-Agent: Mutt/1.3.15i In-Reply-To: <dcampbell@example.com>; from dcampbell@example.com on Thu, Mar 01, 2001 at 06:46:03AM -0800 X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N --juZjCTNxrMaZdGZC Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Mar 01, 2001 at 06:46:03AM -0800, Greg Stein wrote: > I also get the impression the existing Windows build is dcampbell@example.com. > Having OtherBill give it a good shakedown will help tremendously. >=20 > When apr-util/xml/expat builds properly under Windows, then we can toss t= he > old expat-lite. Using unix-style tools (via cygwin) but with VC++, that expat source builds under windows (modulo the patches I have already sent) about once an hour, on the AbiWord tinderbox host. So it does compile properly. The only question is the integration with the VC++ build system.=20 =20 sam th =20 dcampbell@example.com http://www.abisource.com/~sam/ GnuPG Key: =20 http://www.abisource.com/~sam/key --juZjCTNxrMaZdGZC Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6nn9Ot+kM0Mq9M/wRAnBEAKDJmFxwIKVt5t1jTupKGjmaTUgFNQCgx64n 33Uk+bNhzQEP1qTiZN4nMDY= =2cO6 -----END PGP SIGNATURE----- --juZjCTNxrMaZdGZC-- From dcampbell@example.com Thu Mar 01 18:58:59 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 48784 invoked by uid 500); 1 Mar 2001 18:58:54 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Received: (qmail 48764 invoked from network); 1 Mar 2001 18:58:51 -0000 Date: Fri, 2 Mar 2001 05:44:39 +1100 From: Luke Kenneth Casson Leighton <dcampbell@example.com> X-Sender: dcampbell@example.com To: Jeremy Allison <dcampbell@example.com> Cc: Multiple recipients of list <dcampbell@example.com>, dcampbell@example.com, dcampbell@example.com, dcampbell@example.com Subject: Re: FW: RPC restructuring finished (mostly). In-Reply-To: <dcampbell@example.com> Message-ID: <dcampbell@example.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N [cc'd to apr dev because of reference to apr's pool destroy capabilities, which provide a means to call clean-up functions on sections of memory / etc]. On Thu, 1 Mar 2001, Jeremy Allison wrote: > > > long lived daemon. > > > > ... then why not fix those few? :) > > That's what I'm doing :-) :-). *grin* > > so. important question. > > > > you can guarantee that in both the client-side and the server-side code, > > the user-or-consumer (being client and server respectively) is responsible > > for freeing memory, and that the marshalling/unmarshalling's use of memory > > is therefore effectively transparent? > > > > remember the NetApiBufferFree() function that has to be called for the > > usage of the NET* API? > > The idea will be that the caller (client or server) will > hand a talloc context to the RPC code, and then copy out > of the talloced memory any structures that need to be > long lived then destroy the context (freeing everything > allocated in the call). the caller? you mean, in the case of the client-side usage, the caller is the user of the rpc function, e.g. samr_enum_users(), and the "user" of samr_enum_users() must be aware of the use of talloc? the caller? you mean, in the case of the server-side usage, the caller is the _samr_enum_users() function, and the "user" of _samr_enum_users() must be aware of the use of talloc? if you are in any way exposing the use of talloc to the "callers" - the users, i seriously recommend that you reconsider. anyone wishing to plug-in the OSF/DCE IDL compiler into your modified version is _not_ going to consider being forced to use talloc in their client / server application. i've been slowly working / massaging the codebase towards this end [plugging in the OSF IDL compiler or equivalent]. this requires that you provide, on a per-RPC-service basis, a per-handle "freeing" function, a memory-alloc function and a memory-free function. the OSF/DCE code "tracks" all handles, and when they are freed, it automatically calls the appropriate per-handle "freeing" function - a bit like apr_pool_destroy does at the moment. > That way error returns get much easier, as you can just > return without needing to remember and free everything > you allocated on the way down the call chain. yes it does. however, there are still memory management issues to be dealt with that i don't think you have addressed. yes, you've dealt with the marshalling/unmarshalling, but what about server-side and client-side memory usage? what happens, in your proposal, if you do this: #define samr_connect _samr_connect #define samr_enum_users _samr_enum_users ... etc and then TOTALLY remove all the marshalling / unmarshalling code sections altogether? will it work? will there be any memory leaks? _that's_ the whole point of dce/rpc: the marshalling / unmarshalling must be totally transparent. luke > -------------------------------------------------------- > Buying an operating system without source is like buying > a self-assembly Space Shuttle with no instructions. > -------------------------------------------------------- > ----- Luke Kenneth Casson Leighton <dcampbell@example.com> ----- "i want a world of dreams, run by near-sighted visionaries" "good. that's them sorted out. now, on _this_ world..." From dcampbell@example.com Thu Mar 01 19:35:04 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 93885 invoked by uid 500); 1 Mar 2001 19:34:54 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Delivered-To: moderator for dcampbell@example.com Received: (qmail 65566 invoked from network); 1 Mar 2001 19:17:13 -0000 Date: Thu, 1 Mar 2001 20:17:10 +0100 From: Elrond <dcampbell@example.com> To: Sander Striker <dcampbell@example.com>, dcampbell@example.com, dcampbell@example.com Subject: Re: [RFC] Network Abstraction Layer Message-ID: <dcampbell@example.com> References: <dcampbell@example.com> <dcampbell@example.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <dcampbell@example.com>; from dcampbell@example.com on Wed, Feb 28, 2001 at 07:40:21AM -0800 X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N Hi, On Wed, Feb 28, 2001 at 07:40:21AM -0800, Greg Stein wrote: [...] > >... > > Below is a simple draft of the NAL. The goal is to be able to > > add transports easily, even when not present in the kernel. > > Also this allows an easy way of protocol/transport stacking. > > A 'transport' can also do buffering, authentication, filtering. > > It might be interesting to examine the filters that we have in Apache right > now. They provide for the protocol-stacking, buffering, and (hey!) > filtering. The filters are currently in Apache itself? That might explain, why I didn't find anything relating to filter in apr-util. If filters actualy are, what we're looking for, it would be nice, if the base of the filters (not the filters themselves) be moved into apr-util. This might be a good idea anyway. > Apache handles authentication outside of the filter stacks. (two: one stack > for input, one for output) > > That said: what you outlined is reasonably close to a concept we had a while > back called IOLs. (Input/Output Layers) They were tossed about six months > back in favor of the filter stacks. > > Filters are actually very little code. The heavy lifting is all in the > buckets and the brigades (see APRUTIL's "buckets" directory). For the most > part, implementing a filter stack is mostly design policy rather than code. > What I'm trying to say :-), is that almost everything is already there. If > there *is* something missing, then we can shove it down from Apache. I've read the complete apr_buckets.h. >From a _very_ high point of view, bucket brigades are a list of buckets, and buckets are simply referring to static read-only data, which was (or will be) created. So this is a nice way of moving around data basicaly. > The essence is that you create a brigade of some data. Then you pass that > brigade through N functions which each manipulate the brigade in interesting > ways. Let's take a simple case of generating a file request over an > encrypted pipe: > > *) Samba creates a brigade containing two buckets: a FILE bucket for the > request pipe, and an EOS bucket to signify "end of stream" > *) the brigade is sent into the "output stack" associated with the current > request > *) the output stack happens to have two filters in it: an encryption filter, > and a network filter (the latter places the brigade contents onto the > network). > *) the encryption filter sequences through the brigade, rewriting the > contents for the requested encryption. it periodically flushes chunks of > work to the next filter in the stack. > *) the network filter places the content of any brigades passed to right > onto the network. > > So... that is the design we happen to be using in Apache, and is emboded in > the various APR and APRUTIL bits. It might be interesting to determine > whether a similar design would work for Samba's needs. Some of our protocol layers might be actualy doable as filters. (SMB itself quiet likely, at least for the fileserving part, SMB does more than fileserving, but it probably resembles http in some ways) The part, where Sander's NAL comes in, is the one filter that writes stuff to the network: In Apache this is most likely a simple unix tcp socket, into which you write data. NAL tries to abstract sockets. The issue here is, that the protocols, that are below SMB, are more like sockets. From a clean point of view, they should be implemented in the kernel, but none of us is willing to write kerneldrivers for each OS out there. If you look through Sander's proposal, nearly all typical Unix socket operations are represened as function pointers, and actualy nearly each of them needs to be implemented, and most of them will actualy do something complex. I see one correlation point to filters: Maybe there should be an optional "append_output_filters_to_filterstack" and "prepend_input_filters_to_filterstack". That way, if the socket can be represented as filters, and the socket-writer has the energy to write those filters, that can be optimized. Functions are needed, because the actual used filters can be quiet dynamic and could depend on the remote system, that we connect to, or if someone actualy implemented those sockets in the kernel. The fallback is more or less clear: append_output_filters_to_filterstack defaults to a simple filter, that takes the whole brigade and write()s it into the socket. prepend_input_filters_to_filterstack defaults to the opposite. Maybe we also might have a apr_bucket_NAL, which is a bucket, that reads its data from a NAL socket. I'm not too sure, which of the last ones is actualy the right thing to do. The next interesting point is, that NALs might have their own idea, of how select()/poll() should be handled for them. I hope, that explains somewhat the intention behind NAL. Here are some examples of NALs, that are either interesting to us, or just fictitious: - NetBIOS - unix domain sockets (with authentication) - NT named Pipes (those are like Unix domain sockets and include authentication already, while the unix side needs to add that) - The former two can be wrapped in a single "IPC"-Facility. Now some fictitious ones: - IPX using raw sockets - NetBEUI using raw sockets > > Ignore name clashes with current APR code please. > > Also, I haven't looked into the apr_pool package to understand > > how it works quite yet. I want it to be _possible_ to tie transports > > to memory management (so we can dump all memory that we used for: > > a session, a request, a reply, etc). > [... explanation about pools ...] I haven't looked at pools right now, so if you don't feel like answering, simple "RTFM" is okay. BTW: Do the things in the pool have reference counts? I mean, is the pool itself offering refcounting? And I've also seen, that the included objects are included with a destroy-function, so one can have pools in pools, right? > > I'll post a memory management draft aswell in a few days. It will > > be similar to this when it comes abstraction. > > Look forward to it :-) > > Cheers, > -g Elrond > > -- > Greg Stein, http://www.lyra.org/ From dcampbell@example.com Thu Mar 01 19:36:33 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 94548 invoked by uid 500); 1 Mar 2001 19:36:30 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Delivered-To: moderator for dcampbell@example.com Received: (qmail 75517 invoked from network); 1 Mar 2001 19:25:32 -0000 Errors-To: <dcampbell@example.com> Message-ID: <dcampbell@example.com> Reply-To: "William A. Rowe, Jr." <dcampbell@example.com> From: "William A. Rowe, Jr." <dcampbell@example.com> To: <dcampbell@example.com>, <dcampbell@example.com> References: <dcampbell@example.com> <dcampbell@example.com> Subject: Re: some reasons why Apache 2.0 threaded is slower than prefork Date: Thu, 1 Mar 2001 13:24:56 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N From: "Bill Stoddard" <dcampbell@example.com> Sent: Thursday, March 01, 2001 12:47 PM > FWIW, last week I wrote a very simple memory allocator (apr_malloc(), > apr_calloc(), apr_free()) and replaced all the malloc/calloc/free calls in the > apr-util/buckets with the apr_* calls. It was good for a 10% performance > boost serving static pages on Windows. My allocator used intraprocess apr > mutexs which are implemented as Win32 CriticalSections. There are probably > better sync objects available (compare and swap) which would be good fora few > more %. > > If anyone is interested, I'd be happy to post it to the list in all it's > unfinished/unrefined glory. +1! This offers us the opportunity to add all sorts of validation to the server. However... we once discussed the possibility of extending the 'pool' concept to wrap any sort of memory, in a manner that cleanups could always be introduced. Creating a context (I think this is why we renamed to _ctx_ for a while) would provide a scope. apr_free on a pool would be a noop, on a malloc would free. Can we adopt the following? apr_mem_alloc apr_mem_alloc_clear apr_mem_free (mem_ could be m_ or simply m with no trailing underscore, your choice Bill). The upshot? apr_ctx_alloc, free, etc (probably in apr-util) could delegate to the appropriate apr_pool, apr_mem, or whichever implemented allocation schema we have implemented. But FirstBill, your functions appear very platform specific so they belong in apr. From dcampbell@example.com Thu Mar 01 19:40:13 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 3978 invoked by uid 500); 1 Mar 2001 19:40:08 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Received: (qmail 3967 invoked from network); 1 Mar 2001 19:40:06 -0000 Date: Fri, 2 Mar 2001 06:25:56 +1100 From: Luke Kenneth Casson Leighton <dcampbell@example.com> X-Sender: dcampbell@example.com To: Elrond <dcampbell@example.com> Cc: Sander Striker <dcampbell@example.com>, dcampbell@example.com Subject: Re: [RFC] Network Abstraction Layer In-Reply-To: <dcampbell@example.com> Message-ID: <dcampbell@example.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N > The issue here is, that the protocols, that are below SMB, > are more like sockets. From a clean point of view, they > should be implemented in the kernel, but none of us is > willing to write kerneldrivers for each OS out there. however, on NT, some functions already exist, such as CreateNamedPipe. and there is no way in hell i'm writing a kerneldriver for NT when there's a perfectly good function already out there. however, i _am_ prepared to "mirror" that functionality in systems that _don't_ have a CreateNamedPipe function. so, the implementation of apr_nal_create_socket("namedpipe:foo", ...) on NT will call CreateNamedPipe. which i should imagine will be utterly trivial to do. and the implementation on non-NT systems will go through hoops and layers like there's no tomorrow [i.e. emulating what NT does in the kernel. *yuck* :)]. if the buckets principle can hide this from a developer, so be it. > - NT named Pipes (those are like Unix domain sockets and > include authentication already, while the unix side needs > to add that) ----- Luke Kenneth Casson Leighton <dcampbell@example.com> ----- "i want a world of dreams, run by near-sighted visionaries" "good. that's them sorted out. now, on _this_ world..." From dcampbell@example.com Thu Mar 01 23:00:59 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 86856 invoked by uid 500); 1 Mar 2001 23:00:58 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Received: (qmail 86845 invoked from network); 1 Mar 2001 23:00:58 -0000 Message-ID: <015d01c0a2a2$a6fb1b90$011aa8c0@godzilla> From: "David Reid" <dcampbell@example.com> To: "APR Development List" <dcampbell@example.com> Subject: Header strangeness... Date: Thu, 1 Mar 2001 22:54:59 -0000 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N OK, so my laptop is now running FreeBSD 4-STABLE and I'm seeing some really strange stuff with headers. I committed a patch that gets mktemp.c building but now I have a similar problem with groupinfo.c where the definition of HAVE_GRP_H isn't being seen (as far as I can tell the apr_private.h file isn't even being read) so it fails to build. Anyone any ideas why? I'd carry on digging but it's time to sleep now... :) david From dcampbell@example.com Thu Mar 01 23:05:49 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 98933 invoked by uid 500); 1 Mar 2001 23:05:48 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Received: (qmail 98906 invoked from network); 1 Mar 2001 23:05:48 -0000 Message-ID: <018301c0a2a3$552ceb30$011aa8c0@godzilla> From: "David Reid" <dcampbell@example.com> To: "APR Development List" <dcampbell@example.com> References: <015d01c0a2a2$a6fb1b90$011aa8c0@godzilla> Subject: Re: Header strangeness... Date: Thu, 1 Mar 2001 22:59:51 -0000 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N OK, my bad - an old copy of apr_private.h was in the include directory! Doh! Removed it and it's building OK now :) Time I got some sleep. david ----- Original Message ----- From: "David Reid" <dcampbell@example.com> To: "APR Development List" <dcampbell@example.com> Sent: Thursday, March 01, 2001 10:54 PM Subject: Header strangeness... > OK, so my laptop is now running FreeBSD 4-STABLE and I'm seeing some really > strange stuff with headers. I committed a patch that gets mktemp.c building > but now I have a similar problem with groupinfo.c where the definition of > HAVE_GRP_H isn't being seen (as far as I can tell the apr_private.h file > isn't even being read) so it fails to build. > > Anyone any ideas why? I'd carry on digging but it's time to sleep now... :) > > david > From dcampbell@example.com Fri Mar 02 02:33:36 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 55308 invoked by uid 500); 2 Mar 2001 02:33:32 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Received: (qmail 55294 invoked from network); 2 Mar 2001 02:33:29 -0000 Date: Thu, 1 Mar 2001 18:32:02 -0800 From: "Roy T. Fielding" <dcampbell@example.com> To: Jim Jagielski <dcampbell@example.com> Cc: dcampbell@example.com Subject: Re: FreeBSD and gnu m4 Message-ID: <dcampbell@example.com> Mail-Followup-To: "Roy T. Fielding" <dcampbell@example.com>, Jim Jagielski <dcampbell@example.com>, dcampbell@example.com References: <dcampbell@example.com> <dcampbell@example.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.13-current-20010115i In-Reply-To: <dcampbell@example.com>; from dcampbell@example.com on Tue, Feb 27, 2001 at 02:38:10PM -0500 X-AntiVirus: scanned for viruses by AMaViS 0.2.1-pre3 (http://amavis.org/) X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N Correct me if I am wrong, but all these changes and problems are simply to reduce the three parameters of a standard autoconf macro to one parameter in an APR-specific macro? If so, then -1 --> that kind of change is why our buildconf setup is so much less portable than typical configure scripts, and what I have been trying to get rid of over the past few weeks. Do it the autoconf way or don't do dcampbell@example.com. ....Roy From dcampbell@example.com Fri Mar 02 02:39:40 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 57394 invoked by uid 500); 2 Mar 2001 02:39:36 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Received: (qmail 57165 invoked from network); 2 Mar 2001 02:39:32 -0000 X-Authentication-Warning: cobra.cs.Virginia.EDU: jcw5q owned process doing -bs Date: Thu, 1 Mar 2001 21:39:22 -0500 (EST) From: Cliff Woolley <dcampbell@example.com> X-X-Sender: <dcampbell@example.com> To: "Roy T. Fielding" <dcampbell@example.com> cc: Jim Jagielski <dcampbell@example.com>, <dcampbell@example.com> Subject: Re: FreeBSD and gnu m4 In-Reply-To: <dcampbell@example.com> Message-ID: <dcampbell@example.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N On Thu, 1 Mar 2001, Roy T. Fielding wrote: > Correct me if I am wrong, but all these changes and problems are > simply to reduce the three parameters of a standard autoconf macro > to one parameter in an APR-specific macro? That's one way to look at it, yeah, but that doesn't capture all of the benefits. Those three parameters have given us *numerous* subtle bugs due to typos and the like that go undetected on the committer's platform but cause complete build breakage on other platforms. Reducing to one parameter saves the work that causes these bugs. --Cliff From dcampbell@example.com Fri Mar 02 03:15:37 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 83237 invoked by uid 500); 2 Mar 2001 03:15:31 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Received: (qmail 83220 invoked from network); 2 Mar 2001 03:15:28 -0000 Date: Thu, 1 Mar 2001 19:14:26 -0800 From: "Roy T. Fielding" <dcampbell@example.com> To: Cliff Woolley <dcampbell@example.com> Cc: dcampbell@example.com Subject: Re: FreeBSD and gnu m4 Message-ID: <dcampbell@example.com> Mail-Followup-To: "Roy T. Fielding" <dcampbell@example.com>, Cliff Woolley <dcampbell@example.com>, dcampbell@example.com References: <dcampbell@example.com> <dcampbell@example.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.13-current-20010115i In-Reply-To: <dcampbell@example.com>; from dcampbell@example.com on Thu, Mar 01, 2001 at 09:39:22PM -0500 X-AntiVirus: scanned for viruses by AMaViS 0.2.1-pre3 (http://amavis.org/) X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N On Thu, Mar 01, 2001 at 09:39:22PM -0500, Cliff Woolley wrote: > On Thu, 1 Mar 2001, Roy T. Fielding wrote: > > > Correct me if I am wrong, but all these changes and problems are > > simply to reduce the three parameters of a standard autoconf macro > > to one parameter in an APR-specific macro? > > That's one way to look at it, yeah, but that doesn't capture all of the > benefits. Those three parameters have given us *numerous* subtle bugs due > to typos and the like that go undetected on the committer's platform but > cause complete build breakage on other platforms. Reducing to one > parameter saves the work that causes these bugs. Then we should fix the cause of that goofyness, which is due to all APR defines being set to "0" or "1" instead of undefined/defined. Fixing that would remove the cause of all those other subtle typos and bugs, in addition to these above, and leave us with a standard autoconf function with one parameter. ....Roy From dcampbell@example.com Fri Mar 02 03:58:12 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 63329 invoked by uid 500); 2 Mar 2001 03:58:09 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Received: (qmail 63305 invoked from network); 2 Mar 2001 03:58:07 -0000 Date: Thu, 1 Mar 2001 19:59:00 -0800 (PST) From: <dcampbell@example.com> X-Sender: <rbb@koj> To: "Roy T. Fielding" <dcampbell@example.com> Cc: Cliff Woolley <dcampbell@example.com>, <dcampbell@example.com> Subject: Re: FreeBSD and gnu m4 In-Reply-To: <dcampbell@example.com> Message-ID: <Pine.LNX.4.30.0103011955220.29136-100000@koj> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N On Thu, 1 Mar 2001, Roy T. Fielding wrote: > On Thu, Mar 01, 2001 at 09:39:22PM -0500, Cliff Woolley wrote: > > On Thu, 1 Mar 2001, Roy T. Fielding wrote: > > > > > Correct me if I am wrong, but all these changes and problems are > > > simply to reduce the three parameters of a standard autoconf macro > > > to one parameter in an APR-specific macro? > > > > That's one way to look at it, yeah, but that doesn't capture all of the > > benefits. Those three parameters have given us *numerous* subtle bugs due > > to typos and the like that go undetected on the committer's platform but > > cause complete build breakage on other platforms. Reducing to one > > parameter saves the work that causes these bugs. > > Then we should fix the cause of that goofyness, which is due to all > APR defines being set to "0" or "1" instead of undefined/defined. > Fixing that would remove the cause of all those other subtle typos and > bugs, in addition to these above, and leave us with a standard autoconf > function with one parameter. Changing from "1" and "0" to def, undef wouldn't change dcampbell@example.com. The exact same logic is required, and the same general variables are required. The differences between the two are incredibly minor, because we want to use namespace protected macros. Ryan _______________________________________________________________________________ Ryan Bloom dcampbell@example.com 406 29th St. San Francisco, CA 94131 ------------------------------------------------------------------------------- From dcampbell@example.com Fri Mar 02 05:36:28 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 18879 invoked by uid 500); 2 Mar 2001 05:36:28 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Received: (qmail 18868 invoked from network); 2 Mar 2001 05:36:27 -0000 Date: Thu, 1 Mar 2001 21:35:36 -0800 From: "Roy T. Fielding" <dcampbell@example.com> To: dcampbell@example.com Cc: Cliff Woolley <dcampbell@example.com>, dcampbell@example.com Subject: Re: FreeBSD and gnu m4 Message-ID: <dcampbell@example.com> Mail-Followup-To: "Roy T. Fielding" <dcampbell@example.com>, dcampbell@example.com, Cliff Woolley <dcampbell@example.com>, dcampbell@example.com References: <dcampbell@example.com> <Pine.LNX.4.30.0103011955220.29136-100000@koj> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.13-current-20010115i In-Reply-To: <Pine.LNX.4.30.0103011955220.29136-100000@koj>; from dcampbell@example.com on Thu, Mar 01, 2001 at 07:59:00PM -0800 X-AntiVirus: scanned for viruses by AMaViS 0.2.1-pre3 (http://amavis.org/) X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N > Changing from "1" and "0" to def, undef wouldn't change dcampbell@example.com. > The exact same logic is required, and the same general variables are > required. The differences between the two are incredibly minor, because > we want to use namespace protected macros. If we changed to def/undef and we used the standard autoconf macros with the default behavior, then we don't need namespace protected symbols. Defining them as "0" or "1" doesn't allow us to use #ifndef HAVE_UNISTD_H #undef HAVE_UNISTD_H #endif as the template for the generated file, which gets translated to #ifndef HAVE_UNISTD_H #define HAVE_UNISTD_H 1 #endif by configure if AC_CHECK_HEADERS(unistd.h) succeeds, which won't result in errors if the same symbol is defined by includes in other packages, which was the original justification for the APR_ prefix on those symbols. Of course, this means we'd have to use a template instead of autoheader, but that is how I prefer to set up autoconf in any case, since it also allows us to add documentation to the generated file. Doing it this way has the nice side-effect of allowing the user to override a configure definition by simply doing a env CFLAGS="-DHAVE_UNISTD_H=1" make I wasn't kidding when I said that our problems with autoconf are due to the way we are using it. ....Roy From dcampbell@example.com Fri Mar 02 05:40:02 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 21897 invoked by uid 500); 2 Mar 2001 05:40:02 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Received: (qmail 21886 invoked from network); 2 Mar 2001 05:40:01 -0000 Date: Thu, 1 Mar 2001 21:40:44 -0800 (PST) From: <dcampbell@example.com> X-Sender: <rbb@koj> To: "Roy T. Fielding" <dcampbell@example.com> Cc: <dcampbell@example.com>, Cliff Woolley <dcampbell@example.com>, <dcampbell@example.com> Subject: Re: FreeBSD and gnu m4 In-Reply-To: <dcampbell@example.com> Message-ID: <Pine.LNX.4.30.0103012138200.5582-100000@koj> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N We still need to APR namespace protection. We tried to not namespace protect things to begin with, and Apache and APR were conflicting horribly. Add to that, that we define our own macros that no other package should have. We have been down the road of exposing non-namespace protected macros before, it didn't work then, and I don't believe it will work now. Please take a look at the archive from when we first put autoconf into APR, and how it conflicted with Apache. Ryan On Thu, 1 Mar 2001, Roy T. Fielding wrote: > > Changing from "1" and "0" to def, undef wouldn't change dcampbell@example.com. > > The exact same logic is required, and the same general variables are > > required. The differences between the two are incredibly minor, because > > we want to use namespace protected macros. > > If we changed to def/undef and we used the standard autoconf macros > with the default behavior, then we don't need namespace protected symbols. > Defining them as "0" or "1" doesn't allow us to use > > #ifndef HAVE_UNISTD_H > #undef HAVE_UNISTD_H > #endif > > as the template for the generated file, which gets translated to > > #ifndef HAVE_UNISTD_H > #define HAVE_UNISTD_H 1 > #endif > > by configure if AC_CHECK_HEADERS(unistd.h) succeeds, which won't result > in errors if the same symbol is defined by includes in other packages, > which was the original justification for the APR_ prefix on those symbols. > > Of course, this means we'd have to use a template instead of autoheader, > but that is how I prefer to set up autoconf in any case, since it also > allows us to add documentation to the generated file. Doing it this way > has the nice side-effect of allowing the user to override a configure > definition by simply doing a > > env CFLAGS="-DHAVE_UNISTD_H=1" make > > I wasn't kidding when I said that our problems with autoconf > are due to the way we are using it. > > ....Roy > > _______________________________________________________________________________ Ryan Bloom dcampbell@example.com 406 29th St. San Francisco, CA 94131 ------------------------------------------------------------------------------- From dcampbell@example.com Fri Mar 02 06:12:29 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 33631 invoked by uid 500); 2 Mar 2001 06:12:28 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Received: (qmail 33606 invoked from network); 2 Mar 2001 06:12:27 -0000 Date: Thu, 1 Mar 2001 22:10:49 -0800 From: "Roy T. Fielding" <dcampbell@example.com> To: dcampbell@example.com Cc: dcampbell@example.com Subject: Re: FreeBSD and gnu m4 Message-ID: <dcampbell@example.com> Mail-Followup-To: "Roy T. Fielding" <dcampbell@example.com>, dcampbell@example.com, dcampbell@example.com References: <dcampbell@example.com> <Pine.LNX.4.30.0103012138200.5582-100000@koj> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.13-current-20010115i In-Reply-To: <Pine.LNX.4.30.0103012138200.5582-100000@koj>; from dcampbell@example.com on Thu, Mar 01, 2001 at 09:40:44PM -0800 X-AntiVirus: scanned for viruses by AMaViS 0.2.1-pre3 (http://amavis.org/) X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N On Thu, Mar 01, 2001 at 09:40:44PM -0800, dcampbell@example.com wrote: > > We still need to APR namespace protection. We tried to not namespace > protect things to begin with, and Apache and APR were conflicting > horribly. Because the method I described was not used. > Add to that, that we define our own macros that no other > package should have. That is a separate issue -- anything that is defined by an APR-specific macro should be name-protected. I am only talking about the standard names that every autoconf package defines. > We have been down the road of exposing non-namespace protected macros > before, it didn't work then, and I don't believe it will work now. Please > take a look at the archive from when we first put autoconf into APR, > and how it conflicted with Apache. As I mentioned when I started the build blues thread, I read the archives first. Most of the decisions back then were made because APR had to be integrated with a non-configure-based httpd. I have the benefit of hindsight and a holistic view of the build system, so it shouldn't be too surprising that I can think of a solution that may not have been possible back then. The reason I bring it up right now is because every step we have taken away from a standard autoconf setup has resulted in significant problems on one platform or another, leading to another step away and more problems, ad nauseum. Some steps, like not using the standard generating mechanism for the Makefile files, have benefits that outweigh the pain. Others do not. ....Roy From dcampbell@example.com Fri Mar 02 06:31:44 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 45840 invoked by uid 500); 2 Mar 2001 06:31:43 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Received: (qmail 45829 invoked from network); 2 Mar 2001 06:31:43 -0000 Date: Thu, 1 Mar 2001 22:32:21 -0800 (PST) From: <dcampbell@example.com> X-Sender: <rbb@koj> To: "Roy T. Fielding" <dcampbell@example.com> Cc: <dcampbell@example.com>, <dcampbell@example.com> Subject: Re: FreeBSD and gnu m4 In-Reply-To: <dcampbell@example.com> Message-ID: <Pine.LNX.4.30.0103012218430.5670-100000@koj> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N On Thu, 1 Mar 2001, Roy T. Fielding wrote: > On Thu, Mar 01, 2001 at 09:40:44PM -0800, dcampbell@example.com wrote: > > We still need to APR namespace protection. We tried to not namespace > > protect things to begin with, and Apache and APR were conflicting > > horribly. > > Because the method I described was not used. Doesn't your model require that all APR applications define their configuration macros the same way? If an APR application is forced to do all of their macros in a given way, then I am against this model. If this works regardless of how the app defines it macro's, then cool. > > Add to that, that we define our own macros that no other > > package should have. > > That is a separate issue -- anything that is defined by an APR-specific > macro should be name-protected. I am only talking about the standard > names that every autoconf package defines. What happens with a package that doesn't use autoconf? > > We have been down the road of exposing non-namespace protected macros > > before, it didn't work then, and I don't believe it will work now. Please > > take a look at the archive from when we first put autoconf into APR, > > and how it conflicted with Apache. > > As I mentioned when I started the build blues thread, I read the archives > first. Most of the decisions back then were made because APR had to be > integrated with a non-configure-based httpd. I have the benefit of That's not true. Most of the problems weren't even discovered until we integrated APR into a configure-based Apache. > hindsight and a holistic view of the build system, so it shouldn't be > too surprising that I can think of a solution that may not have been > possible back then. I hope you have found new ideas, but the current APR configuration system is much more autoconf-like than the Apache one, and most of the problems haven't been with the APR configure system. > The reason I bring it up right now is because every step we have taken away > from a standard autoconf setup has resulted in significant problems on > one platform or another, leading to another step away and more problems, > ad nauseum. Some steps, like not using the standard generating mechanism > for the Makefile files, have benefits that outweigh the pain. Others do not. I look at the APR configuration system, and I see a system that is very close to what autoconf expects. Apache, on the other hand is completely non-autoconf. However, this model is not completely broken, and it is what PHP uses, which is why it was originally chosen. I am 100% for fixing this mess, but I am concerned by the number of times that this issue has been raised, and never resolved. Ryan _______________________________________________________________________________ Ryan Bloom dcampbell@example.com 406 29th St. San Francisco, CA 94131 ------------------------------------------------------------------------------- From dcampbell@example.com Fri Mar 02 08:41:05 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 10990 invoked by uid 500); 2 Mar 2001 08:41:01 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Received: (qmail 10538 invoked from network); 2 Mar 2001 08:40:53 -0000 X-Authentication-Warning: kurgan.lyra.org: gstein set sender to dcampbell@example.com using -f Date: Fri, 2 Mar 2001 00:41:28 -0800 From: Greg Stein <dcampbell@example.com> To: dcampbell@example.com Subject: Re: FreeBSD and gnu m4 Message-ID: <dcampbell@example.com> Mail-Followup-To: dcampbell@example.com References: <dcampbell@example.com> <Pine.LNX.4.30.0103012218430.5670-100000@koj> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <Pine.LNX.4.30.0103012218430.5670-100000@koj>; from dcampbell@example.com on Thu, Mar 01, 2001 at 10:32:21PM -0800 X-URL: http://www.lyra.org/greg/ X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N On Thu, Mar 01, 2001 at 10:32:21PM -0800, dcampbell@example.com wrote: > On Thu, 1 Mar 2001, Roy T. Fielding wrote: > > On Thu, Mar 01, 2001 at 09:40:44PM -0800, dcampbell@example.com wrote: > > > We still need to APR namespace protection. We tried to not namespace > > > protect things to begin with, and Apache and APR were conflicting > > > horribly. > > > > Because the method I described was not used. Goody for you, Oh Omniscient One. We are just poor souls who don't have your wisdom, so we fucked up the code. > Doesn't your model require that all APR applications define their > configuration macros the same way? If an APR application is forced to do > all of their macros in a given way, then I am against this model. If this > works regardless of how the app defines it macro's, then cool. If an application includes an APR header, and then includes its own header, then it will receive errors/warnings. Invariably, I always structure my headers (say) in Subversion like so: #include <some_apr_header.h> #include <another_apr_header.h> #include "public_svn_header.h" #include "another.h" #include "module_private_header.h" In the above scenario, my SVN headers would create duplicates. "So fix SVN." Okay, I happen to be able to do that. Now, let's add an Expat include in there. Ooops. Hey, I have privs to fix that, too. Now SVN includes Berkeley DB headers. Fuck. autoconf headers are just not meant to be mixed. If APR is intending to export the values, then they must be namespace protected. That implies that a plain old AC_CHECK_HEADER(foo.h) is not sufficient. We need to follow that up with setting a shell variable (fooh=0/1), then using that to def/undef or 0/1 an APR-namespace symbol. > > > Add to that, that we define our own macros that no other > > > package should have. > > > > That is a separate issue -- anything that is defined by an APR-specific > > macro should be name-protected. I am only talking about the standard > > names that every autoconf package defines. We don't have a problem with the standard names. Those go into include/arch/unix/apr_private.h(.in). The issue is the generation of APR_HAVE_FOO_H in the public apr.h header. For those, we need logic beyond a simple AC_CHECK_HEADER(). > What happens with a package that doesn't use autoconf? Presumbly, they wouldn't be using HAVE_STDIO_H, so a conflict won't occur. >... > > As I mentioned when I started the build blues thread, I read the archives > > first. Most of the decisions back then were made because APR had to be > > integrated with a non-configure-based httpd. I have the benefit of > > That's not true. Most of the problems weren't even discovered until we > integrated APR into a configure-based Apache. Exactly. Mix one autoconf'd guy with another, and you're in for trouble. > > hindsight and a holistic view of the build system, so it shouldn't be > > too surprising that I can think of a solution that may not have been > > possible back then. And you're saying that we don't have that either? That only you possess that knowledge? That isn't fair, Roy. The current spate of problems was due to some M4 coding that I was doing to simplify and optimize our configuration. Unfortunately, despite M4's age, it appears to be rather non-portable and prone to flakiness. We backed off a bit, but we're still in a very good and stable spot. No more deep magic, and it should all work quite fine. We don't do anything beyond normal autoconf setups. > I hope you have found new ideas, but the current APR configuration system > is much more autoconf-like than the Apache one, and most of the problems > haven't been with the APR configure system. Exactly. This recent spate was M4. My fault. Our autoconf system has been working quite well. > > The reason I bring it up right now is because every step we have taken away > > from a standard autoconf setup has resulted in significant problems on > > one platform or another, leading to another step away and more problems, > > ad nauseum. Some steps, like not using the standard generating mechanism > > for the Makefile files, have benefits that outweigh the pain. Others do not. > > I look at the APR configuration system, and I see a system that is very > close to what autoconf expects. Exactly. It is much more complex than many because it touches "all" of an operating system's features. Many apps just don't have that kind of breadth, so their configure scripts are quite short (look at Expat). > Apache, on the other hand is completely > non-autoconf. However, this model is not completely broken, and it is > what PHP uses, which is why it was originally chosen. I am 100% for > fixing this mess, but I am concerned by the number of times that this > issue has been raised, and never resolved. When the Apache stuff was first put in, based on the PHP stuff, I wasn't paying much attention to it. It seemed to work, so that was fine. But over the past year or so (or however long it has been), I've had to deal with it more and more. Gads. It definitely does things "different" in an attempt to do things faster. The whole APACHE_SUBST and APACHE_FAST_OUTPUT and APACHE_OUTPUT was done to avoid variable substitution. Yet I find that to be a poor tradeoff (I could go either way with this; sounds like Roy likes it; with a general cleaning of the Apache autoconf, it might be more obvious and useful to use). There was also a lot of crud remaining in Apache simply because it was autoconf'd in a big whack before we settled on what should be done by APR and what should be done by Apache. And the build stuff is just messy (quick: what is the difference between library.mk ltlib.mk program.mk and special.mk?) And the whole build2.mk thang. I'm a big proponent of revamping the build system for Apache (more like APR(UTIL)), and a basic cleaning of its config system. Cheers, -g -- Greg Stein, http://www.lyra.org/ From dcampbell@example.com Fri Mar 02 10:09:23 2001 Return-Path: <dcampbell@example.com> Delivered-To: dcampbell@example.com Received: (qmail 65163 invoked by uid 500); 2 Mar 2001 10:07:35 -0000 Mailing-List: contact dcampbell@example.com; run by ezmlm Precedence: bulk List-Post: <mailto:dcampbell@example.com> List-Help: <mailto:dcampbell@example.com> List-Unsubscribe: <mailto:dcampbell@example.com> List-Subscribe: <mailto:dcampbell@example.com> Delivered-To: mailing list dcampbell@example.com Received: (qmail 64429 invoked from network); 2 Mar 2001 10:06:46 -0000 X-Authentication-Warning: kurgan.lyra.org: gstein set sender to dcampbell@example.com using -f Date: Fri, 2 Mar 2001 02:06:17 -0800 From: Greg Stein <dcampbell@example.com> To: Elrond <dcampbell@example.com> Cc: Sander Striker <dcampbell@example.com>, dcampbell@example.com, dcampbell@example.com Subject: Re: [RFC] Network Abstraction Layer Message-ID: <dcampbell@example.com> Mail-Followup-To: Elrond <dcampbell@example.com>, Sander Striker <dcampbell@example.com>, dcampbell@example.com, dcampbell@example.com References: <dcampbell@example.com> <dcampbell@example.com> <dcampbell@example.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <dcampbell@example.com>; from dcampbell@example.com on Thu, Mar 01, 2001 at 08:17:10PM +0100 X-URL: http://www.lyra.org/greg/ X-Spam-Rating: h31.sny.collab.net 1.6.2 0/1000/N Getting long here. Watch out! :-) On Thu, Mar 01, 2001 at 08:17:10PM +0100, Elrond wrote: > On Wed, Feb 28, 2001 at 07:40:21AM -0800, Greg Stein wrote: >... > > It might be interesting to examine the filters that we have in Apache right > > now. They provide for the protocol-stacking, buffering, and (hey!) > > filtering. > > The filters are currently in Apache itself? Yup. > That might explain, why I didn't find anything relating to > filter in apr-util. > > If filters actualy are, what we're looking for, it would be > nice, if the base of the filters (not the filters > themselves) be moved into apr-util. > This might be a good idea anyway. As I stated elsewhere, there is actually not a lot of stuff related to filters. Most of the code deals with registration of the filter, rather than their use. I'm not sure how much has broad utility. In a nutshell, a filter function has this prototype: typedef apr_status_t (*ap_out_filter_func)(ap_filter_t *f, apr_bucket_brigade *bb); ap_filter_t has a couple pieces of metadata, the filter function pointer, and a pointer to the "next" filter. A filter function looks something like: filter_func(f, bb) { /* manipulate bb in interesting ways */ return ap_pass_brigade(f->next, bb); } Where ap_pass_brigade is essentially: { return f->func(f, bb); } The meat of the filter code deals with naming the filters and then creating the linked list of filters (we insert by name, and filters have an inherent ordering). >... > I've read the complete apr_buckets.h. > From a _very_ high point of view, bucket brigades are a > list of buckets, and buckets are simply referring to static > read-only data, which was (or will be) created. This is correct. A brigade's contents are read-only, but may be generated *during* a bucket's read() function (e.g. read the content from a pipe, socket, or file). "Manipulating" a brigade involves splitting buckets, inserting or removing portions, or simply replacing portions. For example, let's say you are a filter handling server-side-includes (SSI), and you get a brigade with a single bucket with the string: BRIGADE \ BUCKET "duration = <!--#var foo --> seconds" The filter will split this into three buckets: BRIGADE \ BUCKET ---------- BUCKET -------------- BUCKET "duration = " "<!--#var foo -->" " seconds" It then replaces the middle bucket with a new bucket representing the request values. Note that the first and last buckets do not copy the value; they simply point to different parts of the original bucket's value. > So this is a nice way of moving around data basicaly. It is a way to do zero-copy of content (presuming that is feasible within the constraints of the filters' manipulations; a gzip filter simply can't retain the original data :-) Since one of the bucket types represents a FILE, you can also inject a file into the brigade and send that down the output stack. Let's say that nothing in the chain happens to need the file contents. The file then arrives at the bottom "filter" of the chain, and sendfile() is used to shove the thing out onto the network. This zero-copy is quite feasible when you consider a brigade transform such as this: BRIGADE = { FILE, EOS } # FILE bucket and EOS (End Of Stream) bucket becomes BRIGADE = { "packet header bytes", FILE, EOS } We inserted a header without touching the file. The output filter generates a sendfile which contains an iovec for those header bytes. Blam! Out the network it goes... :-) >... > Some of our protocol layers might be actualy doable as > filters. (SMB itself quiet likely, at least for the > fileserving part, SMB does more than fileserving, but it > probably resembles http in some ways) Possibly doable. I merely bring it up as something for you to look at. The brigades are a great way to move around
{'source': 'Apache Software Foundation Public Mail Archives'}