| 1 |
769 |
jeremybenn |
<html><head><title>
|
| 2 |
|
|
blah
|
| 3 |
|
|
<!--
|
| 4 |
|
|
/*
|
| 5 |
|
|
* Copyright (C) 1999-2001 The Free Software Foundation, Inc.
|
| 6 |
|
|
*/
|
| 7 |
|
|
-->
|
| 8 |
|
|
</title></head><body>
|
| 9 |
|
|
|
| 10 |
|
|
<p>This package exposes a kind of XML processing pipeline, based on sending
|
| 11 |
|
|
SAX events, which can be used as components of application architectures.
|
| 12 |
|
|
Pipelines are used to convey streams of processing events from a producer
|
| 13 |
|
|
to one or more consumers, and to let each consumer control the data seen by
|
| 14 |
|
|
later consumers.
|
| 15 |
|
|
|
| 16 |
|
|
<p> There is a <a href="PipelineFactory.html">PipelineFactory</a> class which
|
| 17 |
|
|
accepts a syntax describing how to construct some simple pipelines. Strings
|
| 18 |
|
|
describing such pipelines can be used in command line tools (see the
|
| 19 |
|
|
<a href="../util/DoParse.html">DoParse</a> class)
|
| 20 |
|
|
and in other places that it is
|
| 21 |
|
|
useful to let processing be easily reconfigured. Pipelines can of course
|
| 22 |
|
|
be constructed programmatically, providing access to options that the
|
| 23 |
|
|
factory won't.
|
| 24 |
|
|
|
| 25 |
|
|
<p> Web applications are supported by making it easy for servlets (or
|
| 26 |
|
|
non-Java web application components) to be part of a pipeline. They can
|
| 27 |
|
|
originate XML (or XHTML) data through an <em>InputSource</em> or in
|
| 28 |
|
|
response to XML messages sent from clients using <em>CallFilter</em>
|
| 29 |
|
|
pipeline stages. Such facilities are available using the simple syntax
|
| 30 |
|
|
for pipeline construction.
|
| 31 |
|
|
|
| 32 |
|
|
|
| 33 |
|
|
<h2> Programming Models </h2>
|
| 34 |
|
|
|
| 35 |
|
|
<p> Pipelines should be simple to understand.
|
| 36 |
|
|
|
| 37 |
|
|
<ul>
|
| 38 |
|
|
<li> XML content, typically entire documents,
|
| 39 |
|
|
is pushed through consumers by producers.
|
| 40 |
|
|
|
| 41 |
|
|
<li> Pipelines are basically about consuming SAX2 callback events,
|
| 42 |
|
|
where the events encapsulate XML infoset-level data.<ul>
|
| 43 |
|
|
|
| 44 |
|
|
<li> Pipelines are constructed by taking one or more consumer
|
| 45 |
|
|
stages and combining them to produce a composite consumer.
|
| 46 |
|
|
|
| 47 |
|
|
<li> A pipeline is presumed to have pending tasks and state from
|
| 48 |
|
|
the beginning of its ContentHandler.startDocument() callback until
|
| 49 |
|
|
it's returned from its ContentHandler.doneDocument() callback.
|
| 50 |
|
|
|
| 51 |
|
|
<li> Pipelines may have multiple output stages ("fan-out")
|
| 52 |
|
|
or multiple input stages ("fan-in") when appropriate.
|
| 53 |
|
|
|
| 54 |
|
|
<li> Pipelines may be long-lived, but need not be.
|
| 55 |
|
|
|
| 56 |
|
|
</ul>
|
| 57 |
|
|
|
| 58 |
|
|
<li> There is flexibility about event production. <ul>
|
| 59 |
|
|
|
| 60 |
|
|
<li> SAX2 XMLReader objects are producers, which
|
| 61 |
|
|
provide a high level "pull" model: documents (text or DOM) are parsed,
|
| 62 |
|
|
and the parser pushes individual events through the pipeline.
|
| 63 |
|
|
|
| 64 |
|
|
<li> Events can be pushed directly to event consumer components
|
| 65 |
|
|
by application modules, if they invoke SAX2 callbacks directly.
|
| 66 |
|
|
That is, application modules use the XML Infoset as exposed
|
| 67 |
|
|
through SAX2 event callbacks.
|
| 68 |
|
|
|
| 69 |
|
|
</ul>
|
| 70 |
|
|
|
| 71 |
|
|
<li> Multiple producer threads may concurrently access a pipeline,
|
| 72 |
|
|
if they coordinate appropriately.
|
| 73 |
|
|
|
| 74 |
|
|
<li> Pipeline processing is not the only framework applications
|
| 75 |
|
|
will use.
|
| 76 |
|
|
|
| 77 |
|
|
</ul>
|
| 78 |
|
|
|
| 79 |
|
|
|
| 80 |
|
|
<h3> Producers: XMLReader or Custom </h3>
|
| 81 |
|
|
|
| 82 |
|
|
<p> Many producers will be SAX2 XMLReader objects, and
|
| 83 |
|
|
will read (pull) data which is then written (pushed) as events.
|
| 84 |
|
|
Typically these will parse XML text (acquired from
|
| 85 |
|
|
<code>org.xml.sax.helpers.XMLReaderFactory</code>) or a DOM tree
|
| 86 |
|
|
(using a <code><a href="../util/DomParser.html">DomParser</a></code>)
|
| 87 |
|
|
These may be bound to event consumer using a convenience routine,
|
| 88 |
|
|
<em><a href="EventFilter.html">EventFilter</a>.bind()</em>.
|
| 89 |
|
|
Once bound, these producers may be given additional documents to
|
| 90 |
|
|
sent through its pipeline.
|
| 91 |
|
|
|
| 92 |
|
|
<p> In other cases, you will write producers yourself. For example, some
|
| 93 |
|
|
data structures might know how to write themselves out using one or
|
| 94 |
|
|
more XML models, expressed as sequences of SAX2 event callbacks.
|
| 95 |
|
|
An application module might
|
| 96 |
|
|
itself be a producer, issuing startDocument and endDocument events
|
| 97 |
|
|
and then asking those data structures to write themselves out to a
|
| 98 |
|
|
given EventConsumer, or walking data structures (such as JDBC query
|
| 99 |
|
|
results) and applying its own conversion rules. WAP format XML
|
| 100 |
|
|
(WBMXL) can be directly converted to producer output.
|
| 101 |
|
|
|
| 102 |
|
|
<p> SAX2 introduced an "XMLFilter" interface, which is a kind of XMLReader.
|
| 103 |
|
|
It is most useful in conjunction with its XMLFilterImpl helper class;
|
| 104 |
|
|
see the <em><a href="EventFilter.html">EventFilter</a></em> javadoc
|
| 105 |
|
|
for information contrasting that XMLFilterImpl approach with the
|
| 106 |
|
|
relevant parts of this pipeline framework. Briefly, such XMLFilterImpl
|
| 107 |
|
|
children can be either producers or consumers, and are more limited in
|
| 108 |
|
|
configuration flexibility. In this framework, the focus of filters is
|
| 109 |
|
|
on the EventConsumer side; see the section on
|
| 110 |
|
|
<a href="#fitting">pipe fitting</a> below.
|
| 111 |
|
|
|
| 112 |
|
|
|
| 113 |
|
|
<h3> Consume to Standard or Custom Data Representations </h3>
|
| 114 |
|
|
|
| 115 |
|
|
<p> Many consumers will be used to create standard representations of XML
|
| 116 |
|
|
data. The <a href="TextConsumer.html">TextConsumer</a> takes its events
|
| 117 |
|
|
and writes them as text for a single XML document,
|
| 118 |
|
|
using an internal <a href="../util/XMLWriter.html">XMLWriter</a>.
|
| 119 |
|
|
The <a href="DomConsumer.html">DomConsumer</a> takes its events and uses
|
| 120 |
|
|
them to create and populate a DOM Document.
|
| 121 |
|
|
|
| 122 |
|
|
<p> In other cases, you will write consumers yourself. For example,
|
| 123 |
|
|
you might use a particular unmarshaling filter to produce objects
|
| 124 |
|
|
that fit your application's requirements, instead of using DOM.
|
| 125 |
|
|
Such consumers work at the level of XML data models, rather than with
|
| 126 |
|
|
specific representations such as XML text or a DOM tree. You could
|
| 127 |
|
|
convert your output directly to WAP format data (WBXML).
|
| 128 |
|
|
|
| 129 |
|
|
|
| 130 |
|
|
<h3><a name="fitting">Pipe Fitting</a></h3>
|
| 131 |
|
|
|
| 132 |
|
|
<p> Pipelines are composite event consumers, with each stage having
|
| 133 |
|
|
the opportunity to transform the data before delivering it to any
|
| 134 |
|
|
subsequent stages.
|
| 135 |
|
|
|
| 136 |
|
|
<p> The <a href="PipelineFactory.html">PipelineFactory</a> class
|
| 137 |
|
|
provides access to much of this functionality through a simple syntax.
|
| 138 |
|
|
See the table in that class's javadoc describing a number of standard
|
| 139 |
|
|
components. Direct API calls are still needed for many of the most
|
| 140 |
|
|
interesting pipeline configurations, including ones leveraging actual
|
| 141 |
|
|
or logical concurrency.
|
| 142 |
|
|
|
| 143 |
|
|
<p> Four basic types of pipe fitting are directly supported. These may
|
| 144 |
|
|
be used to construct complex pipeline networks. <ul>
|
| 145 |
|
|
|
| 146 |
|
|
<li> <a href="TeeConsumer.html">TeeConsumer</a> objects split event
|
| 147 |
|
|
flow so it goes to two two different consumers, one before the other.
|
| 148 |
|
|
This is a basic form of event fan-out; you can use this class to
|
| 149 |
|
|
copy events to any number of output pipelines.
|
| 150 |
|
|
|
| 151 |
|
|
<li> Clients can call remote components through HTTP or HTTPS using
|
| 152 |
|
|
the <a href="CallFilter.html">CallFilter</a> component, and Servlets
|
| 153 |
|
|
can implement such components by extending the
|
| 154 |
|
|
<a href="XmlServlet.html">XmlServlet</a> component. Java is not
|
| 155 |
|
|
required on either end, and transport protocols other than HTTP may
|
| 156 |
|
|
also be used.
|
| 157 |
|
|
|
| 158 |
|
|
<li> <a href="EventFilter.html">EventFilter</a> objects selectively
|
| 159 |
|
|
provide handling for callbacks, and can pass unhandled ones to a
|
| 160 |
|
|
subsequent stage. They are often subclassed, since much of the
|
| 161 |
|
|
basic filtering machinery is already in place in the base class.
|
| 162 |
|
|
|
| 163 |
|
|
<li> Applications can merge two event flows by just using the same
|
| 164 |
|
|
consumer in each one. If multiple threads are in use, synchronization
|
| 165 |
|
|
needs to be addressed by the appropriate application level policy.
|
| 166 |
|
|
|
| 167 |
|
|
</ul>
|
| 168 |
|
|
|
| 169 |
|
|
<p> Note that filters can be as complex as
|
| 170 |
|
|
<a href="XsltFilter.html">XSLT transforms</a>
|
| 171 |
|
|
available) on input data, or as simple as removing simple syntax data
|
| 172 |
|
|
such as ignorable whitespace, comments, and CDATA delimiters.
|
| 173 |
|
|
Some simple "built-in" filters are part of this package.
|
| 174 |
|
|
|
| 175 |
|
|
|
| 176 |
|
|
<h3> Coding Conventions: Filter and Terminus Stages</h3>
|
| 177 |
|
|
|
| 178 |
|
|
<p> If you follow these coding conventions, your classes may be used
|
| 179 |
|
|
directly (give the full class name) in pipeline descriptions as understood
|
| 180 |
|
|
by the PipelineFactory. There are four constructors the factory may
|
| 181 |
|
|
try to use; in order of decreasing numbers of parameters, these are: <ul>
|
| 182 |
|
|
|
| 183 |
|
|
<li> Filters that need a single String setup parameter should have
|
| 184 |
|
|
a public constructor with two parameters: that string, then the
|
| 185 |
|
|
EventConsumer holding the "next" consumer to get events.
|
| 186 |
|
|
|
| 187 |
|
|
<li> Filters that don't need setup parameters should have a public
|
| 188 |
|
|
constructor that accepts a single EventConsumer holding the "next"
|
| 189 |
|
|
consumer to get events when they are done.
|
| 190 |
|
|
|
| 191 |
|
|
<li> Terminus stages may have a public constructor taking a single
|
| 192 |
|
|
paramter: the string value of that parameter.
|
| 193 |
|
|
|
| 194 |
|
|
<li> Terminus stages may have a public no-parameters constructor.
|
| 195 |
|
|
|
| 196 |
|
|
</ul>
|
| 197 |
|
|
|
| 198 |
|
|
<p> Of course, classes may support more than one such usage convention;
|
| 199 |
|
|
if they do, they can automatically be used in multiple modes. If you
|
| 200 |
|
|
try to use a terminus class as a filter, and that terminus has a constructor
|
| 201 |
|
|
with the appropriate number of arguments, it is automatically wrapped in
|
| 202 |
|
|
a "tee" filter.
|
| 203 |
|
|
|
| 204 |
|
|
|
| 205 |
|
|
<h2> Debugging Tip: "Tee" Joints can Snapshot Data</h2>
|
| 206 |
|
|
|
| 207 |
|
|
<p> It can sometimes be hard to see what's happening, when something
|
| 208 |
|
|
goes wrong. Easily fixed: just snapshot the data. Then you can find
|
| 209 |
|
|
out where things start to go wrong.
|
| 210 |
|
|
|
| 211 |
|
|
<p> If you're using pipeline descriptors so that they're easily
|
| 212 |
|
|
administered, just stick a <em>write ( filename )</em>
|
| 213 |
|
|
filter into the pipeline at an appropriate point.
|
| 214 |
|
|
|
| 215 |
|
|
<p> Inside your programs, you can do the same thing directly: perhaps
|
| 216 |
|
|
by saving a Writer (perhaps a StringWriter) in a variable, using that
|
| 217 |
|
|
to create a TextConsumer, and making that the first part of a tee --
|
| 218 |
|
|
splicing that into your pipeline at a convenient location.
|
| 219 |
|
|
|
| 220 |
|
|
<p> You can also use a DomConsumer to buffer the data, but remember
|
| 221 |
|
|
that DOM doesn't save all the information that XML provides, so that DOM
|
| 222 |
|
|
snapshots are relatively low fidelity. They also are substantially more
|
| 223 |
|
|
expensive in terms of memory than a StringWriter holding similar data.
|
| 224 |
|
|
|
| 225 |
|
|
<h2> Debugging Tip: Non-XML Producers</h2>
|
| 226 |
|
|
|
| 227 |
|
|
<p> Producers in pipelines don't need to start from XML
|
| 228 |
|
|
data structures, such as text in XML syntax (likely coming
|
| 229 |
|
|
from some <em>XMLReader</em> that parses XML) or a
|
| 230 |
|
|
DOM representation (perhaps with a
|
| 231 |
|
|
<a href="../util/DomParser.html">DomParser</a>).
|
| 232 |
|
|
|
| 233 |
|
|
<p> One common type of event producer will instead make
|
| 234 |
|
|
direct calls to SAX event handlers returned from an
|
| 235 |
|
|
<a href="EventConsumer.html">EventConsumer</a>.
|
| 236 |
|
|
For example, making <em>ContentHandler.startElement</em>
|
| 237 |
|
|
calls and matching <em>ContentHandler.endElement</em> calls.
|
| 238 |
|
|
|
| 239 |
|
|
<p> Applications making such calls can catch certain
|
| 240 |
|
|
common "syntax errors" by using a
|
| 241 |
|
|
<a href="WellFormednessFilter.html">WellFormednessFilter</a>.
|
| 242 |
|
|
That filter will detect (and report) erroneous input data
|
| 243 |
|
|
such as mismatched document, element, or CDATA start/end calls.
|
| 244 |
|
|
Use such a filter near the head of the pipeline that your
|
| 245 |
|
|
producer feeds, at least while debugging, to help ensure that
|
| 246 |
|
|
you're providing legal XML Infoset data.
|
| 247 |
|
|
|
| 248 |
|
|
<p> You can also arrange to validate data on the fly.
|
| 249 |
|
|
For DTD validation, you can configure a
|
| 250 |
|
|
<a href="ValidationConsumer.html">ValidationConsumer</a>
|
| 251 |
|
|
to work as a filter, using any DTD you choose.
|
| 252 |
|
|
Other validation schemes can be handled with other
|
| 253 |
|
|
validation filters.
|
| 254 |
|
|
|
| 255 |
|
|
</body></html>
|
