From commonsstephanie15@example.org Mon May 05 22:38:49 2003
Return-Path: <commonsstephanie15@example.org>
Delivered-To: stephanie15@example.org
Received: (qmail 38742 invoked by uid 500); 5 May 2003 22:38:47 -0000
Mailing-List: contact stephanie15@example.org; run by ezmlm
Precedence: bulk
list-help: <mailto:stephanie15@example.org>
list-unsubscribe: <mailto:stephanie15@example.org>
list-post: <mailto:stephanie15@example.org>
Delivered-To: mailing list stephanie15@example.org
Received: (qmail 38619 invoked from network); 5 May 2003 22:38:46 -0000
Received: from e1.ny.us.ibm.com (83.161.120.155)
  by daedalus.apache.org with SMTP; 5 May 2003 22:38:46 -0000
Received: from northrelay02.pok.ibm.com (northrelay02.pok.ibm.com [83.161.120.155])
	by e1.ny.us.ibm.com (8.12.9/8.12.2) with ESMTP id h45McpWn187296;
	Mon, 5 May 2003 18:38:51 -0400
Received: from d25ml04.torolab.ibm.com (d01av02.pok.ibm.com [83.161.120.155])
	by northrelay02.pok.ibm.com (8.12.9/NCO/VER6.5) with ESMTP id h45McmaT062200;
	Mon, 5 May 2003 18:38:49 -0400
Subject: Re: [ANN] XInclude processor for xml-commons
To: Stefano Mazzocchi <stephanie15@example.org>
Cc: stephanie15@example.org, stephanie15@example.org,
   stephanie15@example.org
X-Mailer: Lotus Notes Release 5.0.7  March 21, 2001
Message-ID: <stephanie15@example.org>
From: "Neil Graham" <stephanie15@example.org>
Date: Mon, 5 May 2003 18:38:47 -0400
X-MIMETrack: Serialize by Router on D25ML04/25/M/IBM(Release 5.0.9a |January 7, 2002) at
 05/05/2003 06:38:50 PM
MIME-Version: 1.0
Content-type: text/plain; charset=us-ascii
X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N

Hi Stefano,

Sorry about the delay; I was really hoping that Andy Clark (the "Father of
XNI") would pick up this thread.  He'd do a much better job of contrasting
XNI with SAX, and especially giving you the historical angle (I came on the
scene a bit after XNI had gotten out of the crib, so by then the decision
to break from SAX was already made).

> Can you tell us why SAX is not powerful enough for what you need?

Well it's turned out to be pretty handy to be able to pass "augmentations"
along with the events representing the infoset.  SAX doesn't have any kind
of configuration-management infrastructure:  If you want to set up a
pipeline of XMLFilters then you have to manage propagation of features and
properties to the components that comprise the pipeline on your own; XNI
sorts this out.  SAX is also somewhat impoverished for people who care a
great deal about the lexical layout of DTD's; there isn't much you can't
get in this respect from XNI.  Although SAX 1.1 extentions (still in
alpha!) could rectify this, as it stands it's not possible to determine the
version or encoding of a document with "stable" SAX interfaces.

> It does. But Cocoon is *entirely* built around SAX. Moving to XNI is an
> incredibly difficult task. Nobody will do that just for the sake of it.

I can understand that.

> And since Cocoon almost never validates and cocoon already has xinclude
> transformers built around SAX. I really don't see a need for such a
> massive transition.

I think my question revolved around Joerg's assertion that the spec thinks
of XInclude processing being done before validation.  If you don't do
validation of any kind anyway, then clearly this won't disturb you; but if
you did--or wanted to follow as closely to the spec as possible--then you'd
be confronted by the fact that most SAX parsers have validation built in,
not built as a separate module that you can drop an XInclude processor in
front of.

> Now, please, help us understand: what are the differences between XNI
> and SAX and what would we gain basing the entire cocoon architecture
> around XNI events instead of SAX events?

At this stage, I'm afraid the biggest pro might be that SAX just doesn't
seem to be all that healthy these days; I don't know if you lurk on its
development lists, but they've been pretty much dead for quite some time...

But I'll leave the hard job of selling XNI to Andy; it works well for us,
but perhaps SAX is good enough for what you need it to do.  Cocoon being
the huge project it is, I certainly wouldn't blame you for needing some
very solid reasons to migrate to a different pipeline framework!

Cheers,
Neil
Neil Graham
XML Parser Development
IBM Toronto Lab
Phone:  (812)350-0098, T/L 969-3519
E-mail:  stephanie15@example.org




|---------+---------------------------->
|         |           Stefano Mazzocchi|
|         |           <stephanie15@example.org|
|         |           rg>              |
|         |                            |
|         |           04/30/2003 11:04 |
|         |           AM               |
|         |                            |
|---------+---------------------------->
  >---------------------------------------------------------------------------------------------------------------------------------------------|
  |                                                                                                                                             |
  |       To:       stephanie15@example.org                                                                                                   |
  |       cc:       stephanie15@example.org, stephanie15@example.org                                                                     |
  |       Subject:  Re: [ANN] XInclude processor for xml-commons                                                                                |
  |                                                                                                                                             |
  |                                                                                                                                             |
  >---------------------------------------------------------------------------------------------------------------------------------------------|



on 4/29/03 6:41 PM Neil Graham wrote:

> I see what you're saying, I think:  you like the idea of a componentized
> parser that operates in a pipeline, but you wish that SAX had been used
as
> the glue for such a beast, rather than its own API.  Right?
>
> And if SAX had seemed rich and complete enough, and had it supported the
> kind of configuration-management facilities that are needed for such an
> arrangement, I imagine that's the road we would have gone down.  But it
> isn't/doesn't, so what was there to do?  :)

Can you tell us why SAX is not powerful enough for what you need?
Caution: I'm not sarcastic, just curious.

> So I guess I'd turn the question around:  If using SAX means you're stuck
> with monolithic-looking parsers, but XNI would give you a parser with all
> kinds of flexibility, maybe using XNI might merit consideration?  :)

It does. But Cocoon is *entirely* built around SAX. Moving to XNI is an
incredibly difficult task. Nobody will do that just for the sake of it.
And since Cocoon almost never validates and cocoon already has xinclude
transformers built around SAX. I really don't see a need for such a
massive transition.

of course, this might well be blindness from our side or simply
ignorance on the XNI paradigms.

> Sure
> it would bind you to a specific parser (until other parsers begin to use
> XNI :)) ), but if you write your own SAX-glued parser then you're stuck
to
> that anyway...

SAX is much more used than XNI and this will containue to be so for a
long while, independently of its technical merits. This is a fact we
simply cannot forget to take into consideration.

>>Dumb proposal: why not a XNI2SAX Filter that uses an XNI processor for
>>SAX?
>
>
> We already have one:  org.apache.xerces.parsers.SAXParser; heck, it even
> understands both SAX1 and SAX2!  :)  The trouble is that, once you're
> emitting SAX, you're not in the Xerces pipeline world anymore; so this
guy
> is only useful at the end of such a pipeline (whatever XNI components
that
> pipeline has in it).  So there's no way of putting a Xerces validator
after
> that component, for instance (to do so would kind of defeat the point of
> XNI).  Does that explain anything?

Yes and it's obviuos that such a XNI2SAX filter existed because Xerces
has to emit SAX events.

Now, please, help us understand: what are the differences between XNI
and SAX and what would we gain basing the entire cocoon architecture
around XNI events instead of SAX events?

Also, would would be the cons?

TIA for this.

--
Stefano.






From commonsstephanie15@example.org Mon May 05 22:46:24 2003
Return-Path: <commonsstephanie15@example.org>
Delivered-To: stephanie15@example.org
Received: (qmail 49042 invoked by uid 500); 5 May 2003 22:46:23 -0000
Mailing-List: contact stephanie15@example.org; run by ezmlm
Precedence: bulk
list-help: <mailto:stephanie15@example.org>
list-unsubscribe: <mailto:stephanie15@example.org>
list-post: <mailto:stephanie15@example.org>
Delivered-To: mailing list stephanie15@example.org
Received: (qmail 48991 invoked from network); 5 May 2003 22:46:23 -0000
Received: from e1.ny.us.ibm.com (83.161.120.155)
  by daedalus.apache.org with SMTP; 5 May 2003 22:46:23 -0000
Received: from northrelay04.pok.ibm.com (northrelay04.pok.ibm.com [83.161.120.155])
	by e1.ny.us.ibm.com (8.12.9/8.12.2) with ESMTP id h45MkSWn110554;
	Mon, 5 May 2003 18:46:28 -0400
Received: from d25ml04.torolab.ibm.com (d01av02.pok.ibm.com [83.161.120.155])
	by northrelay04.pok.ibm.com (8.12.9/NCO/VER6.5) with ESMTP id h45MkQ81230214;
	Mon, 5 May 2003 18:46:26 -0400
Subject: Re: [ANN] XInclude processor for xml-commons
To: stephanie15@example.org
Cc: stephanie15@example.org, stephanie15@example.org,
   stephanie15@example.org
X-Mailer: Lotus Notes Release 5.0.7  March 21, 2001
Message-ID: <stephanie15@example.org>
From: "Neil Graham" <stephanie15@example.org>
Date: Mon, 5 May 2003 18:46:25 -0400
X-MIMETrack: Serialize by Router on D25ML04/25/M/IBM(Release 5.0.9a |January 7, 2002) at
 05/05/2003 06:46:27 PM
MIME-Version: 1.0
Content-type: text/plain; charset=us-ascii
X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N

Hi Nicola Ken,

Some of this I covered in the note I just wrote to Stefano; but a couple of
brief comments:

> ...which brings us to the real catch22 problem, as Xerces needs XNI but

> all systems out use SAX. Why not only extend SAX? (ignorant mode)

Well, SAX is its own community and will evolve (or not...) as it will.  We
needed a new kind of entity resolver, for instance; we could have extended
SAX's, but then SAX decided to define EntityResolver2.  This would have
been a pretty ugly mess as soon as SAX 1.1 extensions become finalized!

Besides, with things like the introduction of Augmentation parameters
everywhere in XNI, I doubt we share many method signatures with SAX
anymore.

> Is it impossible to do validation of a document from a SAX stream? Or is
> XNI compulsory?

Well, it's compulsory for a component that lives in Xerces.  I'm not
entirely clear how you'd build a DTD grammar out of the information you get
from SAX, but I've not tried to do this so it well might be possible.  But
of course it would be easy (well, as easy as it ever is...) to build a
representation for an XML Schema grammar from a SAX stream.

> Is it possible to do XInclude from a SAX stream? Or is XNI compulsory?

Well, if you want to go by the letter of the spec, which Joerg tells us
assumes validation to happen after XInclude processing, then it's only
possible if you find a validator that takes SAX events as input.  You could
certainly build such a thing, but I'm not sure many exist ATM.

Anyway, before responding any further on these threads, I need to spend
some quality time with XInclude, and hopefully also have a look at Joerg's
code.

Cheers!
Neil
Neil Graham
XML Parser Development
IBM Toronto Lab
Phone:  (812)350-0098, T/L 969-3519
E-mail:  stephanie15@example.org




|---------+---------------------------->
|         |           Nicola Ken       |
|         |           Barozzi          |
|         |           <nicolaken@apache|
|         |           .org>            |
|         |                            |
|         |           04/30/2003 06:21 |
|         |           AM               |
|         |           Please respond to|
|         |           nicolaken        |
|         |                            |
|---------+---------------------------->
  >---------------------------------------------------------------------------------------------------------------------------------------------|
  |                                                                                                                                             |
  |       To:                                                                                                                                   |
  |       cc:       stephanie15@example.org, stephanie15@example.org, stephanie15@example.org                                          |
  |       Subject:  Re: [ANN] XInclude processor for xml-commons                                                                                |
  |                                                                                                                                             |
  |                                                                                                                                             |
  >---------------------------------------------------------------------------------------------------------------------------------------------|




Neil Graham wrote, On 30/04/2003 1.41:
> Hi Nicola Ken,
>
>>>In your original post, you'd mentioned that you'd implemented this as a
>>>"SAX filter"; did you mean that it's an XMLFilter implementation?  If
>>> so,
>>>and you're using a standard XMLReader implementation of some sort, then
>>>how do you do XInclude processing before validation?
>
>>Ooooh, opening a can of worms ;-)
>
> Both commons-dev and xerces-j-dev have been really quiet lately; about
time
> for some activity!  :)

:-D

>>Yes having used XInclude I can solemly state that XInclude has to occur
>>before validation. The question is: should a parser validate? ;-)
>
>>The fact is that Processors like Cocoon will always have other methods
>>to use before a validation, so what we (Cocoon) need is instead a
>>validating transformer, or something like that.
>
>>Not that I'm defining solutions, just that Xerces is monolithic seen
>>from a SAX-adhering processor world.
>
>
> I see what you're saying, I think:  you like the idea of a componentized
> parser that operates in a pipeline, but you wish that SAX had been used
as
> the glue for such a beast, rather than its own API.  Right?
>
> And if SAX had seemed rich and complete enough, and had it supported the
> kind of configuration-management facilities that are needed for such an
> arrangement, I imagine that's the road we would have gone down.  But it
> isn't/doesn't, so what was there to do?  :)

Yeah, I know... I read about XNI, so I'm aware of the reasons it was
created...

> So I guess I'd turn the question around:  If using SAX means you're stuck
> with monolithic-looking parsers, but XNI would give you a parser with all
> kinds of flexibility, maybe using XNI might merit consideration?  :)
Sure
> it would bind you to a specific parser (until other parsers begin to use
> XNI :)) ), but if you write your own SAX-glued parser then you're stuck
to
> that anyway...

...which brings us to the real catch22 problem, as Xerces needs XNI but
all systems out use SAX. Why not only extend SAX? (ignorant mode)

>>Dumb proposal: why not a XNI2SAX Filter that uses an XNI processor for
>>SAX?
>
> We already have one:  org.apache.xerces.parsers.SAXParser; heck, it even
> understands both SAX1 and SAX2!  :)  The trouble is that, once you're
> emitting SAX, you're not in the Xerces pipeline world anymore; so this
guy
> is only useful at the end of such a pipeline (whatever XNI components
that
> pipeline has in it).  So there's no way of putting a Xerces validator
after
> that component, for instance (to do so would kind of defeat the point of
> XNI).  Does that explain anything?

Is it impossible to do validation of a document from a SAX stream? Or is
XNI compulsory?

Is it possible to do XInclude from a SAX stream? Or is XNI compulsory?

Is there a real possibility of hacking the SAX pipelines to make this
happen for *some* components like validation or XInclude?

> Here's hoping I haven't let too many more worms out of that can...

No, you have been very on-the-spot :-)

--
Nicola Ken Barozzi                   stephanie15@example.org
             - verba volant, scripta manent -
    (discussions get forgotten, just code remains)
---------------------------------------------------------------------





From commonsstephanie15@example.org Tue May 06 03:27:54 2003
Return-Path: <commonsstephanie15@example.org>
Delivered-To: stephanie15@example.org
Received: (qmail 90042 invoked by uid 500); 6 May 2003 03:27:52 -0000
Mailing-List: contact stephanie15@example.org; run by ezmlm
Precedence: bulk
list-help: <mailto:stephanie15@example.org>
list-unsubscribe: <mailto:stephanie15@example.org>
list-post: <mailto:stephanie15@example.org>
Delivered-To: mailing list stephanie15@example.org
Received: (qmail 90002 invoked from network); 6 May 2003 03:27:50 -0000
Received: from unknown (HELO pulse.betaversion.org) (83.161.120.155)
  by daedalus.apache.org with SMTP; 6 May 2003 03:27:50 -0000
Received: (qmail 16450 invoked from network); 6 May 2003 03:27:58 -0000
Received: from unknown (HELO apache.org) (stephanie15@example.org)
  by pulse.betaversion.org with SMTP; 6 May 2003 03:27:58 -0000
Message-ID: <stephanie15@example.org>
Date: Mon, 05 May 2003 22:28:07 -0500
From: Stefano Mazzocchi <stephanie15@example.org>
User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.3) Gecko/20030312
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: Neil Graham <stephanie15@example.org>
CC:  stephanie15@example.org,  stephanie15@example.org, 
 stephanie15@example.org
Subject: Re: [ANN] XInclude processor for xml-commons
References: <stephanie15@example.org>
In-Reply-To: <stephanie15@example.org>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N

on 5/5/03 5:38 PM Neil Graham wrote:

> Hi Stefano,

Hi Neil,

thanks very much for answering this.

> Sorry about the delay; I was really hoping that Andy Clark (the "Father of
> XNI") would pick up this thread.  He'd do a much better job of contrasting
> XNI with SAX, and especially giving you the historical angle (I came on the
> scene a bit after XNI had gotten out of the crib, so by then the decision
> to break from SAX was already made).

 :-)

>>Can you tell us why SAX is not powerful enough for what you need?
> 
> 
> Well it's turned out to be pretty handy to be able to pass "augmentations"
> along with the events representing the infoset. 

Can you make an explicit example of this? that would help us understand
better.

> SAX doesn't have any kind
> of configuration-management infrastructure:  If you want to set up a
> pipeline of XMLFilters then you have to manage propagation of features and
> properties to the components that comprise the pipeline on your own; 

Hmmm, I see, but why do you pass such information thru the pipeline and
not as a pipeline context?

> XNI
> sorts this out.  SAX is also somewhat impoverished for people who care a
> great deal about the lexical layout of DTD's; there isn't much you can't
> get in this respect from XNI.  Although SAX 1.1 extentions (still in
> alpha!) could rectify this, as it stands it's not possible to determine the
> version or encoding of a document with "stable" SAX interfaces.

I see. But the very fact that I, for one, never noticed those
limitations might seem to advocate against such a move.

>>It does. But Cocoon is *entirely* built around SAX. Moving to XNI is an
>>incredibly difficult task. Nobody will do that just for the sake of it.
> 
> 
> I can understand that.
> 
> 
>>And since Cocoon almost never validates and cocoon already has xinclude
>>transformers built around SAX. I really don't see a need for such a
>>massive transition.
> 
> 
> I think my question revolved around Joerg's assertion that the spec thinks
> of XInclude processing being done before validation.  

I'm a follower of the JClark-ish school of thought that validation
should be orthogonal on respect of the infoset. I heard that this school
of thought is penetrating the validation circles at W3C, which is, IMO,
a good thing even if it moves the problem on the a pipeline definition
language and the XPipe note, well, it cries for abuse.

So, for now, what I personally advocate is to avoid the use of
infoset-messing validation as much as possible.

> If you don't do
> validation of any kind anyway, then clearly this won't disturb you; but if
> you did--or wanted to follow as closely to the spec as possible--then you'd
> be confronted by the fact that most SAX parsers have validation built in,
> not built as a separate module that you can drop an XInclude processor in
> front of.

Yes, that is the reason why I advocate not to validate at all using
infoset-messing schematas. In fact, for schema validation needs, I
advocate the use of a RelaxNG SAX filter and along with the cocoon
sitemap syntax, you can precisely describe how your pipeline behaves
(and you don't have those nasty pre/post-schema-infoset issues)

>>Now, please, help us understand: what are the differences between XNI
>>and SAX and what would we gain basing the entire cocoon architecture
>>around XNI events instead of SAX events?
> 
> 
> At this stage, I'm afraid the biggest pro might be that SAX just doesn't
> seem to be all that healthy these days; 

While I don't question your assertion, I think it might be misleading:
there are APIs which became solid out of lack of necessity to improve.
>From a parser-writer POV SAX might seem dead while from a non-validating
SAX user POV (as a cocoon developer is), SAX is just complete.

But I do see your point.

> I don't know if you lurk on its
> development lists, but they've been pretty much dead for quite some time...

No, I don't, nor ever felt the need to, to be honest.

> But I'll leave the hard job of selling XNI to Andy; it works well for us,
> but perhaps SAX is good enough for what you need it to do.  

Yes. I can only speak for myself (and I encouradge others to speak up if
I'm mistaken) but I never felt the need for something better. (I do have
some issues with the fact that SAX is lossy in respect of the original
whitespace between attributes and attribute order, but that's not an
issue for cocoon)

> Cocoon being
> the huge project it is, I certainly wouldn't blame you for needing some
> very solid reasons to migrate to a different pipeline framework!

Yes. we would need *incredibly* solid arguments to do such a transition
without risking a huge fork that would kill us. This is why I asked: in
all honesty, it's much easier for us to turn off parser validation
entirely and provide a Jing SAX filter than moving everything to XNI to
make internal Xerces filters into cocoon pipeline components.

And if the need ever emerged, we could use the XNI/SAX adapters that
already exist, like we do for Andy's HTML parser.

Anyway, thanks for anwering. There hasn't been much communication
between the cocoon and xerces communities but both are well funded on
xml-event-driven pipelines, even if seen from different points of view,
so any idea/suggestion/criticism exchange can only be a good thing for both.

Thanks.

-- 
Stefano.



From commonsstephanie15@example.org Tue May 06 14:43:13 2003
Return-Path: <commonsstephanie15@example.org>
Delivered-To: stephanie15@example.org
Received: (qmail 98160 invoked by uid 500); 6 May 2003 14:43:12 -0000
Mailing-List: contact stephanie15@example.org; run by ezmlm
Precedence: bulk
list-help: <mailto:stephanie15@example.org>
list-unsubscribe: <mailto:stephanie15@example.org>
list-post: <mailto:stephanie15@example.org>
Delivered-To: mailing list stephanie15@example.org
Delivered-To: moderator for stephanie15@example.org
Received: (qmail 25404 invoked from network); 6 May 2003 12:59:45 -0000
From: "Arnaud  Le Hors" <stephanie15@example.org>
To: <stephanie15@example.org>
Cc: <stephanie15@example.org>,
	<stephanie15@example.org>
Subject: RE: [ANN] XInclude processor for xml-commons
Date: Tue, 6 May 2003 14:59:11 +0200
Message-ID: <stephanie15@example.org>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="US-ASCII"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
Importance: Normal
In-Reply-To: <stephanie15@example.org>
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N

> Well, if you want to go by the letter of the spec, which Joerg tells us
> assumes validation to happen after XInclude processing

This is not true. XInclude is defined to work with an infoset. How you get
this first infoset and what you did with it (transformed, validated, etc.)
before it is passed to the XInclude processor is totally open.

It is true that the spec doesn't request XInclude processors to deal with
extensions, such as those found in a PSVI. XInclude processors are only
required to produce an infoset. So, you may waste your time validating
before, if what you're interested in is extensions of the infoset that get
dropped by the XInclude processor. But it's up to the user to decide.

The bottom line is that there is no universal answer to the question of when
XInclude should be processed. Different users may have different needs and
answer it differently.

Now, the main use of XInclude that we anticipate is basically to replace
external entities references. In this case one is likely to want to process
it fairly early on. Most likely before validation, especially XML Schema
validation. So, from a Xerces point of view, we would want to support it by
providing an XNI component that people can put in the parser pipeline
typically, but not only, before the validator. Cocoon which looks at
documents at a different level may want it at the SAX level.

To end, I'll add a note on the debate over the "infoset-messing validation".
Here again there is no universal solution. If all you are dealing with is
XML documents - this is text -, validation merely means checking that your
document validates against a set of constraints. You couldn't care less
about any infoset augmentations in an environment where everything is text.
On the other hand, if what you are dealing with is XML data - this is
objects serialized in XML -, validation is the way you reconcile serialized
objects with their real type. Infoset augmentations are then fundamental and
can hardly be considered as messy... So, beware of over simplistic
characterizations... :-)
--
Arnaud  Le Hors - IBM, XML Standards Strategy Group / W3C AC Rep.


From commonsstephanie15@example.org Tue May 06 20:48:07 2003
Return-Path: <commonsstephanie15@example.org>
Delivered-To: stephanie15@example.org
Received: (qmail 43896 invoked by uid 500); 6 May 2003 20:48:06 -0000
Mailing-List: contact stephanie15@example.org; run by ezmlm
Precedence: bulk
list-help: <mailto:stephanie15@example.org>
list-unsubscribe: <mailto:stephanie15@example.org>
list-post: <mailto:stephanie15@example.org>
Delivered-To: mailing list stephanie15@example.org
Received: (qmail 43875 invoked from network); 6 May 2003 20:48:05 -0000
Received: from mail.datazug.ch (83.161.120.155)
  by daedalus.apache.org with SMTP; 6 May 2003 20:48:05 -0000
Received: from yahoo.de [83.161.120.155] by mail.datazug.ch with ESMTP
  (SMTPD32-7.15) id AF94BDDE0208; Tue, 06 May 2003 22:48:20 +0200
Message-ID: <stephanie15@example.org>
Date: Tue, 06 May 2003 22:48:07 +0200
From: "J.Pietschmann" <stephanie15@example.org>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3) Gecko/20030312
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: Neil Graham <stephanie15@example.org>
CC: Stefano Mazzocchi <stephanie15@example.org>,  stephanie15@example.org
Subject: Re: [ANN] XInclude processor for xml-commons
References: <stephanie15@example.org>
In-Reply-To: <stephanie15@example.org>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N

Neil Graham wrote:
> I think my question revolved around Joerg's assertion that the spec thinks
> of XInclude processing being done before validation.

Sorry, I wrote
 >>> The REC itself is deliberately neutral.
XInclude is defined in terms of "merging infosets", and it is briefly
and non-normatively) discussed what this could mean if validation
occurred before or afterwards.

I personally see there are use cases for either case:
1. XInclude as a substitute for external entities for physically
   structuring documents. This is useful if the document should
   be validated against a schema, where entities are not organically
   available. Naturally, the post-XInclude infoset has to be
   validated.
2. Validation before XInclude. This could in turn mean the including
   document is validated, or the included documents, all documents
   or arbitrary combinations. This use case happens for formats which
   are meant to aggregate content, for example imagine a portal where
   you aggregate vastly different content, like an article, stock
   quotes and ad streams. While each of the aggregated content may
   have an attached schema and therefore may be easy to vgalidate,
   the aggregated content may prove more diffucult in this respect
   (imagine if they use different schema languages).

Therefore parsers with an integrated XInclude stage (before validation)
are useful (remember libxml), but standalone XInclude processors make
sense as well.

J.Pietschmann


From commonsstephanie15@example.org Tue May 06 20:57:17 2003
Return-Path: <commonsstephanie15@example.org>
Delivered-To: stephanie15@example.org
Received: (qmail 68998 invoked by uid 500); 6 May 2003 20:57:17 -0000
Mailing-List: contact stephanie15@example.org; run by ezmlm
Precedence: bulk
list-help: <mailto:stephanie15@example.org>
list-unsubscribe: <mailto:stephanie15@example.org>
list-post: <mailto:stephanie15@example.org>
Delivered-To: mailing list stephanie15@example.org
Received: (qmail 68950 invoked from network); 6 May 2003 20:57:15 -0000
Received: from mail.datazug.ch (83.161.120.155)
  by daedalus.apache.org with SMTP; 6 May 2003 20:57:15 -0000
Received: from yahoo.de [83.161.120.155] by mail.datazug.ch with ESMTP
  (SMTPD32-7.15) id A1AEF73101D6; Tue, 06 May 2003 22:57:18 +0200
Message-ID: <stephanie15@example.org>
Date: Tue, 06 May 2003 22:57:05 +0200
From: "J.Pietschmann" <stephanie15@example.org>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3) Gecko/20030312
X-Accept-Language: en-us, en
MIME-Version: 1.0
To:  stephanie15@example.org
CC: Neil Graham <stephanie15@example.org>,  stephanie15@example.org
Subject: Re: [ANN] XInclude processor for xml-commons
References: <stephanie15@example.org> <stephanie15@example.org>
In-Reply-To: <stephanie15@example.org>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N

Stefano Mazzocchi wrote:
> Yes. I can only speak for myself (and I encouradge others to speak up if
> I'm mistaken) but I never felt the need for something better. (I do have
> some issues with the fact that SAX is lossy in respect of the original
> whitespace between attributes and attribute order, but that's not an
> issue for cocoon)

If
1. XPath/XSLT 2.0 proves to be the Next Big Thing and
2. people start to match on *datatypes* rather then element names and/or
3. Cocoon gets used as web services provider,
then you'll get pressure to adapt something which can actually
supply said data types from the schema validator or XQuery generator
to the XSLT processor or SOAP serializer.

J.Pietschmann


From commonsstephanie15@example.org Tue May 06 22:03:05 2003
Return-Path: <commonsstephanie15@example.org>
Delivered-To: stephanie15@example.org
Received: (qmail 21500 invoked by uid 500); 6 May 2003 22:03:03 -0000
Mailing-List: contact stephanie15@example.org; run by ezmlm
Precedence: bulk
list-help: <mailto:stephanie15@example.org>
list-unsubscribe: <mailto:stephanie15@example.org>
list-post: <mailto:stephanie15@example.org>
Delivered-To: mailing list stephanie15@example.org
Received: (qmail 21471 invoked from network); 6 May 2003 22:03:02 -0000
Received: from unknown (HELO pulse.betaversion.org) (83.161.120.155)
  by daedalus.apache.org with SMTP; 6 May 2003 22:03:02 -0000
Received: (qmail 3771 invoked from network); 6 May 2003 22:03:06 -0000
Received: from unknown (HELO apache.org) (stephanie15@example.org)
  by pulse.betaversion.org with SMTP; 6 May 2003 22:03:06 -0000
Message-ID: <stephanie15@example.org>
Date: Tue, 06 May 2003 17:03:09 -0500
From: Stefano Mazzocchi <stephanie15@example.org>
User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.3) Gecko/20030312
X-Accept-Language: en-us, en
MIME-Version: 1.0
Newsgroups: gmane.text.xml.cocoon.devel,gmane.comp.jakarta.commons.devel
To:  stephanie15@example.org
CC:  stephanie15@example.org
Subject: Re: [ANN] XInclude processor for xml-commons
References: <stephanie15@example.org> <stephanie15@example.org>
In-Reply-To: <stephanie15@example.org>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N

on 5/6/03 7:59 AM Arnaud Le Hors wrote:

> To end, I'll add a note on the debate over the "infoset-messing validation".
> Here again there is no universal solution. 

True.

> If all you are dealing with is
> XML documents - this is text -, validation merely means checking that your
> document validates against a set of constraints. You couldn't care less
> about any infoset augmentations in an environment where everything is text.
> On the other hand, if what you are dealing with is XML data - this is
> objects serialized in XML -, validation is the way you reconcile serialized
> objects with their real type. Infoset augmentations are then fundamental and
> can hardly be considered as messy... 

Wait a second.

I agree with this vision, but I wouldn't call it "infoset augmentation"
but rather "infoset normalization". "augmenting" means to "add
information". This is what I personally consider bad because it alters
the infoset (read: "information set") of the XML stream.

just like DOM Node.normalize() changes the tree but doesn't change the
information included in the tree, your 'normalization' of types changes
the tree but doesn't change the information it includes.

Why am I so picky on this? consider caching: when the infoset is
normalized, the information contained is not changed, just morphed. So,
if an XML stream wasn't valid before validation, it's not valid after.
but if it was valid before validation, it is valid after.

This means: normalizing the infoset doesn't influence the ergodic period
of that infoset.

But if I have "infoset augmentation" (say, external entity evaluation),
this cannot be considered true anymore.

This is why DTD are and must be considered harmful in a heavily
cache-based environment like Cocoon.

yes, there are way to work around this, but none are as elegant as
separating concerns between infoset-normalizing pipeline stages and
infoset augmenting pipeline stages.

> So, beware of over simplistic characterizations... :-)

I hope this helps outlining my point on this.

-- 
Stefano.



From commonsstephanie15@example.org Thu May 08 21:40:40 2003
Return-Path: <commonsstephanie15@example.org>
Delivered-To: stephanie15@example.org
Received: (qmail 21140 invoked by uid 500); 8 May 2003 21:40:39 -0000
Mailing-List: contact stephanie15@example.org; run by ezmlm
Precedence: bulk
list-help: <mailto:stephanie15@example.org>
list-unsubscribe: <mailto:stephanie15@example.org>
list-post: <mailto:stephanie15@example.org>
Delivered-To: mailing list stephanie15@example.org
Delivered-To: moderator for stephanie15@example.org
Received: (qmail 1363 invoked from network); 8 May 2003 19:55:20 -0000
Message-ID: <stephanie15@example.org>
Date: Thu, 08 May 2003 12:54:55 -0700
From: Andy Clark <stephanie15@example.org>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3b) Gecko/20030210
X-Accept-Language: en-us, en, ja
MIME-Version: 1.0
To: stephanie15@example.org
CC: stephanie15@example.org, stephanie15@example.org
Subject: Re: [ANN] XInclude processor for xml-commons
References: <stephanie15@example.org> <stephanie15@example.org>
In-Reply-To: <stephanie15@example.org>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N

Stefano Mazzocchi wrote:
>>Well it's turned out to be pretty handy to be able to 
 >>pass "augmentations" along with the events representing
 >>the infoset.
> 
> Can you make an explicit example of this? that would 
 > help us understand better.

In general, the augmentations allow us to extend the
existing framework (in certain ways) within breaking
the method prototypes. But they also have other uses.
For example, in my NekoHTML parser, I use them as a
way of communicating whether elements were specified
in the original document stream or were synthesized
by the tag balancer.

However, the augmentations will most often be used
for passing the PSVI information. And back when we
were debating augmentations, this was the major topic
of discussion. I pushed back from adding XML Schema
related information to XNI in favor of a more generic
system that allowed us to add XML Schema information
as well as any other information we wanted.

>>SAX doesn't have any kind of configuration-management 
 >>infrastructure:  If you want to set up a pipeline of
 >>XMLFilters then you have to manage propagation of
 >>features and properties to the components that comprise
 >>the pipeline on your own;
> 
> Hmmm, I see, but why do you pass such information thru 
 > the pipeline and not as a pipeline context?

When the information is directly associated to the data
being passed through the pipline, then it makes sense to
pass that information via augmentations of that data. In
addition, XNI is designed so that components can alter
the data passed through the pipeline. If an individual
component were to remove information from the pipeline,
it may not be able to understand which associated items
of information to remove if that information is only
passed via a pipeline context.

And it should be noted that we also have the concept
of a pipeline context within XNI. If a component in the
pipeline is interested in the overall settings of the
pipeline, then it can implement XMLComponent. Then,
before the parse it is notified and passed a reference
to the component manager which stores the context. So
it requests just those settings that it understands.

>>SAX is also somewhat impoverished for people who care 
 >>a great deal about the lexical layout of DTD's; there
 >>isn't much you can't get in this respect from XNI.
 >>Although SAX 1.1 extentions (still in alpha!) could
 >>rectify this, as it stands it's not possible to determine
 >>the version or encoding of a document with "stable" SAX
 >>interfaces.
> 
> I see. But the very fact that I, for one, never noticed those
> limitations might seem to advocate against such a move.

Neil is correct when he said that SAX is "impoverished".
And that is one of the major reasons that we made XNI in
the first place -- SAX simply didn't communicate the
information that we needed. Another problem was that
the information was read-only (e.g. Attributes). This
situation may change but at the time we were designing
XNI it was insufficient for our needs.

Also, XNI fulfills a different need than SAX. XNI is
for building different kinds of parsers whereas SAX
is for communicating document information to the app.
XNI could also be used for this purpose but that is
not the primary goal. And we've never marketed XNI
that way.

-- 
Andy Clark * stephanie15@example.org



{'source': 'Apache Software Foundation Public Mail Archives'}