1 <?xml version="1.0" standalone="no"?>
2 <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1//EN"
3 "http://www.oasis-open.org/docbook/xml/4.1/docbookx.dtd"
5 <!ENTITY % local SYSTEM "local.ent">
7 <!ENTITY manref SYSTEM "manref.xml">
8 <!ENTITY progref SYSTEM "progref.xml">
9 <!ENTITY % common SYSTEM "common/common.ent">
11 <!-- Next line allows imagedata/@format="PDF" and is taken from
12 http://lists.oasis-open.org/archives/docbook/200303/msg00163.html
14 <!ENTITY % local.notation.class "| PDF">
15 <!-- Next line is necessary for some XML parsers, for reasons I
16 don't understand. I got this from
17 http://lists.oasis-open.org/archives/docbook/200303/msg00180.html
19 <!NOTATION PDF SYSTEM "PDF">
21 <!-- $Id: book.xml,v 1.49 2007-01-12 15:43:00 adam Exp $ -->
24 <title>Metaproxy - User's Guide and Reference</title>
26 <firstname>Adam</firstname><surname>Dickmeiss</surname>
29 <firstname>Marc</firstname><surname>Cromme</surname>
32 <firstname>Mike</firstname><surname>Taylor</surname>
35 <year>2005-2007</year>
36 <holder>Index Data ApS</holder>
40 This manual is part of Metaproxy version &version;.
43 Metaproxy is a universal router, proxy and encapsulated
44 metasearcher for information retrieval protocols. It accepts,
45 processes, interprets and redirects requests from IR clients using
46 standard protocols such as
47 <ulink url="&url.z39.50;">ANSI/NISO Z39.50</ulink>
48 (and in the future <ulink url="&url.sru;">SRU</ulink>
49 and <ulink url="&url.srw;">SRW</ulink>), as
50 well as functioning as a limited
51 <ulink url="&url.http;">HTTP</ulink> server.
52 Metaproxy is configured by an XML file which
53 specifies how the software should function in terms of routes that
54 the request packets can take through the proxy, each step on a
55 route being an instantiation of a filter. Filters come in many
56 types, one for each operation: accepting Z39.50 packets, logging,
57 query transformation, multiplexing, etc. Further filter-types can
58 be added as loadable modules to extend Metaproxy functionality,
62 Metaproxy is <emphasis>not</emphasis> open-source software, but
63 may be freely downloaded, unpacked, inspected, built and run for
64 evaluation purposes. Deployment requires a separate, commercial,
70 <imagedata fileref="common/id.png" format="PNG"/>
73 <imagedata fileref="common/id.eps" format="EPS"/>
80 <chapter id="introduction">
81 <title>Introduction</title>
85 <ulink url="&url.metaproxy;">Metaproxy</ulink>
86 is a stand alone program that acts as a universal router, proxy and
87 encapsulated metasearcher for information retrieval protocols such
88 as <ulink url="&url.z39.50;">Z39.50</ulink>, and in the future
89 <ulink url="&url.sru;">SRU</ulink> and <ulink url="&url.srw;">SRW</ulink>.
90 To clients, it acts as a server of these protocols: it can be searched,
91 records can be retrieved from it, etc.
92 To servers, it acts as a client: it searches in them,
93 retrieves records from them, etc. it satisfies its clients'
94 requests by transforming them, multiplexing them, forwarding them
95 on to zero or more servers, merging the results, transforming
96 them, and delivering them back to the client. In addition, it
97 acts as a simple <ulink url="&url.http;">HTTP</ulink> server; support
98 for further protocols can be added in a modular fashion, through the
99 creation of new filters.
104 Fish, bananas, cold pyjamas,
105 Mutton, beef and trout!
106 - attributed to Cole Porter.
109 Metaproxy is a more capable alternative to
110 <ulink url="&url.yazproxy;">YAZ Proxy</ulink>,
111 being more powerful, flexible, configurable and extensible. Among
112 its many advantages over the older, more pedestrian work are
113 support for multiplexing (encapsulated metasearching), routing by
114 database name, authentication and authorization and serving local
115 files via HTTP. Equally significant, its modular architecture
116 facilitites the creation of pluggable modules implementing further
120 This manual will describe how to install Metaproxy
121 before giving an overview of its architecture, then discussing the
122 key concept of a filter in some depth and giving an overview of
123 the various filter types, then discussing the configuration file
124 format. After this come several optional chapters which may be
125 freely skipped: a detailed discussion of virtual databases and
126 multi-database searching, some notes on writing extensions
127 (additional filter types) and a high-level description of the
128 source code. Finally comes the reference guide, which contains
129 instructions for invoking the <command>metaproxy</command>
130 program, and detailed information on each type of filter,
135 <chapter id="license">
136 <title>The Metaproxy License</title>
137 <orderedlist numeration="arabic">
140 You are allowed to download this software for evaluation purposes.
141 You can unpack it, build it, run it, see how it works and how it fits
142 your needs, all at zero cost.
147 You may NOT deploy the software. For the purposes of this license,
148 deployment means running it for any purpose other than evaluation,
149 whether or not you or anyone else makes a profit from doing so. If
150 you wish to deploy the software, you must first contact Index Data and
151 arrange to purchase a DEPLOYMENT LICENCE. If you are unsure
152 whether or not your proposed use of the software constitutes
153 deployment, email us at <literal>info@indexdata.com</literal>
159 You may modify your copy of the software (fix bugs, add features)
160 if you need to. We encourage you to send your changes back to us for
161 integration into the master copy, but you are not obliged to do so. You
162 may NOT pass your changes on to any other party.
167 There is NO WARRANTY for this software, to the extent permitted by
168 applicable law. We provide the software ``as is'' without warranty of
169 any kind, either expressed or implied, including, but not limited to, the
170 implied warranties of MERCHANTABILITY and FITNESS FOR A
171 PARTICULAR PURPOSE. The entire risk as to the quality and
172 performance of the software is with you. Should the software prove
173 defective, you assume the cost of all necessary servicing, repair or
174 correction. In no event unless required by applicable law will we be
175 liable to you for damages, arising out of the use of the software,
176 including but not limited to loss of data or data being rendered
182 All rights to the software are reserved by Index Data except where
183 this license explicitly says otherwise.
189 <chapter id="installation">
190 <title>Installation</title>
192 Metaproxy depends on the following tools/libraries:
194 <varlistentry><term><ulink url="&url.yazplusplus;">YAZ++</ulink></term>
197 This is a C++ library based on <ulink url="&url.yaz;">YAZ</ulink>.
201 <varlistentry><term><ulink url="&url.libxslt;">Libxslt</ulink></term>
203 <para>This is an XSLT processor - based on
204 <ulink url="&url.libxml2;">Libxml2</ulink>. Both Libxml2 and
205 Libxslt must be installed with the development components
206 (header files, etc.) as well as the run-time libraries.
210 <varlistentry><term><ulink url="&url.boost;">Boost</ulink></term>
213 The popular C++ library. Initial versions of Metaproxy
214 was built with 1.33.0. Version 1.33.1 works too.
221 In order to compile Metaproxy a modern C++ compiler is
222 required. Boost, in particular, requires the C++ compiler
223 to facilitate the newest features. Refer to Boost
224 <ulink url="&url.boost.compilers.status;">Compiler Status</ulink>
225 for more information.
228 We have successfully built Metaproxy using the compilers
229 <ulink url="&url.gcc;">GCC</ulink> version 4.0 and
230 <ulink url="&url.vstudio;">Microsoft Visual Studio</ulink> 2003/2005.
233 <section id="installation.unix">
234 <title>Installation on Unix (from Source)</title>
236 Here is a quick step-by-step guide on how to compile all the
237 tools that Metaproxy uses. Only few systems have none of the required
238 tools binary packages. If, for example, Libxml2/libxslt are already
239 installed as development packages use those (and omit compilation).
246 gunzip -c libxml2-version.tar.gz|tar xf -
254 gunzip -c libxslt-version.tar.gz|tar xf -
265 gunzip -c yaz-version.tar.gz|tar xf -
273 gunzip -c yazpp-version.tar.gz|tar xf -
284 gunzip -c boost-version.tar.gz|tar xf -
295 gunzip -c metaproxy-version.tar.gz|tar xf -
304 <section id="installation.debian">
305 <title>Installation on Debian GNU/Linux</title>
307 All dependencies for Metaproxy are available as
308 <ulink url="&url.debian;">Debian</ulink>
309 packages for the sarge (stable in 2005) and etch (testing in 2005)
313 The procedures for Debian based systems, such as
314 <ulink url="&url.ubuntu;">Ubuntu</ulink> is probably similar
317 There is currently no official Debian package for YAZ++.
318 And the Debian package for YAZ is probably too old.
319 Update the <filename>/etc/apt/sources.list</filename>
320 to include the Index Data repository.
321 See YAZ' <ulink url="&url.yaz.download.debian;">Download Debian</ulink>
322 for more information.
325 apt-get install libxslt1-dev
326 apt-get install libyazpp-dev
327 apt-get install libboost-dev
328 apt-get install libboost-thread-dev
329 apt-get install libboost-date-time-dev
330 apt-get install libboost-program-options-dev
331 apt-get install libboost-test-dev
334 With these packages installed, the usual configure + make
335 procedure can be used for Metaproxy as outlined in
336 <xref linkend="installation.unix"/>.
340 <section id="installation.windows">
341 <title>Installation on Windows</title>
343 Metaproxy can be compiled with Microsoft
344 <ulink url="&url.vstudio;">Visual Studio</ulink>.
345 Version 2003 (C 7.1) and 2005 (C 8.0) is known to work.
347 <section id="installation.windows.boost">
350 Get Boost from its <ulink url="&url.boost;">home page</ulink>.
351 You also need Boost Jam (an alternative to make).
352 That's also available from the Boost home page.
353 The files to be downloaded are called something like:
354 <filename>boost_1_33-1.exe</filename>
356 <filename>boost-jam-3.1.12-1-ntx86.zip</filename>.
357 Unpack Boost Jam first. Put <filename>bjam.exe</filename>
358 in your system path. Make a command prompt and ensure
359 it can be found automatically. If not check the PATH.
360 The Boost .exe is a self-extracting exe with
361 complete source for Boost. Compile that source with
362 Boost Jam (An alternative to Make).
363 The compilation takes a while.
364 For Visual Studio 2003, use
366 bjam "-sTOOLS=vc-7_1"
368 Here <literal>vc-7_1</literal> refers to a "Toolset" (compiler system).
369 For Visual Studio 2005, use
371 bjam "-sTOOLS=vc-8_0"
373 To install the libraries in a common place, use
375 bjam "-sTOOLS=vc-7_1" install
377 (or vc-8_0 for VS 2005).
380 By default, the Boost build process installs the resulting
381 libraries + header files in
382 <literal>\boost\lib</literal>, <literal>\boost\include</literal>.
385 For more information about installing Boost refer to the
386 <ulink url="&url.boost.getting.started;">getting started</ulink>
391 <section id="installation.windows.libxslt">
392 <title>Libxslt</title>
394 <ulink url="&url.libxslt;">Libxslt</ulink> can be downloaded
396 <ulink url="&url.libxml2.download.win32;">here</ulink>.
399 Libxslt has other dependencies, but these can all be downloaded
400 from the same site. Get the following:
401 iconv, zlib, libxml2, libxslt.
405 <section id="installation.windows.yaz">
408 <ulink url="&url.yaz;">YAZ</ulink> can be downloaded
410 <ulink url="&url.yaz.download.win32;">here</ulink>.
414 <section id="installation.windows.yazplusplus">
417 Get <ulink url="&url.yazplusplus;">YAZ++</ulink> as well.
418 Version 1.0 or later is required. For now get it from
420 <ulink url="&url.snapshot.download;">Snapshot area</ulink>.
423 YAZ++ includes NMAKE makefiles, similar to those found in the
428 <section id="installation.windows.metaproxy">
429 <title>Metaproxy</title>
431 Metaproxy is shipped with NMAKE makefiles as well - similar
432 to those found in the YAZ++/YAZ packages. Adjust this Makefile
433 to point to the proper locations of Boost, Libxslt, Libxml2,
434 zlib, iconv, yaz and yazpp.
438 <varlistentry><term><literal>DEBUG</literal></term>
440 If set to 1, the software is
441 compiled with debugging libraries (code generation is
442 multi-threaded debug DLL).
443 If set to 0, the software is compiled with release libraries
444 (code generation is multi-threaded DLL).
449 <term><literal>BOOST</literal></term>
452 Boost install location
458 <term><literal>BOOST_VERSION</literal></term>
461 Boost version (replace . with _).
467 <term><literal>BOOST_TOOLSET</literal></term>
476 <term><literal>LIBXSLT_DIR</literal>,
477 <literal>LIBXML2_DIR</literal> ..</term>
480 Specify the locations of Libxslt, libiconv, libxml2 and
489 After successful compilation you'll find
490 <literal>metaproxy.exe</literal> in the
491 <literal>bin</literal> directory.
499 <chapter id="architecture">
500 <title>The Metaproxy Architecture</title>
502 The Metaproxy architecture is based on three concepts:
503 the <emphasis>package</emphasis>,
504 the <emphasis>route</emphasis>
505 and the <emphasis>filter</emphasis>.
509 <term>Packages</term>
512 A package is request or response, encoded in some protocol,
513 issued by a client, making its way through Metaproxy, send to or
514 received from a server, or sent back to the client.
517 The core of a package is the protocol unit - for example, a
518 Z39.50 Init Request or Search Response, or an SRU searchRetrieve
519 URL or Explain Response. In addition to this core, a package
520 also carries some extra information added and used by Metaproxy
524 In general, packages are doctored as they pass through
525 Metaproxy. For example, when the proxy performs authentication
526 and authorization on a Z39.50 Init request, it removes the
527 authentication credentials from the package so that they are not
528 passed onto the back-end server; and when search-response
529 packages are obtained from multiple servers, they are merged
530 into a single unified package that makes its way back to the
539 Packages make their way through routes, which can be thought of
540 as programs that operate on the package data-type. Each
541 incoming package initially makes its way through a default
542 route, but may be switched to a different route based on various
543 considerations. Routes are made up of sequences of filters (see
552 Filters provide the individual instructions within a route, and
553 effect the necessary transformations on packages. A particular
554 configuration of Metaproxy is essentially a set of filters,
555 described by configuration details and arranged in order in one
556 or more routes. There are many kinds of filter - about a dozen
557 at the time of writing with more appearing all the time - each
558 performing a specific function and configured by different
562 The word ``filter'' is sometimes used rather loosely, in two
563 different ways: it may be used to mean a particular
564 <emphasis>type</emphasis> of filter, as when we speak of ``the
565 auth_simple filter'' or ``the multi filter''; or it may be used
566 to be a specific <emphasis>instance</emphasis> of a filter
567 within a Metaproxy configuration. For example, a single
568 configuration will often contain multiple instances of the
569 <literal>z3950_client</literal> filter. In
570 operational terms, of these is a separate filter. In practice,
571 context always make it clear which sense of the word ``filter''
575 Extensibility of Metaproxy is primarily through the creation of
576 plugins that provide new filters. The filter API is small and
577 conceptually simple, but there are many details to master. See
579 <link linkend="extensions">extensions</link>.
585 Since packages are created and handled by the system itself, and
586 routes are conceptually simple, most of the remainder of this
587 document concentrates on filters. After a brief overview of the
588 filter types follows, along with some thoughts on possible future
595 <chapter id="filters">
596 <title>Filters</title>
600 <title>Introductory notes</title>
602 It's useful to think of Metaproxy as an interpreter providing a small
603 number of primitives and operations, but operating on a very
604 complex data type, namely the ``package''.
607 A package represents a Z39.50 or SRU/W request (whether for Init,
608 Search, Scan, etc.) together with information about where it came
609 from. Packages are created by front-end filters such as
610 <literal>frontend_net</literal> (see below), which reads them from
611 the network; other front-end filters are possible. They then pass
612 along a route consisting of a sequence of filters, each of which
613 transforms the package and may also have side-effects such as
614 generating logging. Eventually, the route will yield a response,
615 which is sent back to the origin.
618 There are many kinds of filter: some that are defined statically
619 as part of Metaproxy, and others may be provided by third parties
620 and dynamically loaded. They all conform to the same simple API
621 of essentially two methods: <function>configure()</function> is
622 called at startup time, and is passed an XML DOM tree representing that
623 part of the configuration file that pertains to this filter
624 instance: it is expected to walk that tree extracting relevant
625 information; and <function>process()</function> is called every
626 time the filter has to processes a package.
629 While all filters provide the same API, there are different modes
630 of functionality. Some filters are sources: they create
632 (<literal>frontend_net</literal>);
633 others are sinks: they consume packages and return a result
634 (<literal>backend_test</literal>,
635 <literal>bounce</literal>,
636 <literal>http_file</literal>,
637 <literal>z3950_client</literal>);
638 the others are true filters, that read, process and pass on the
639 packages they are fed
640 (<literal>auth_simple</literal>,
641 <literal>log</literal>,
642 <literal>multi</literal>,
643 <literal>query_rewrite</literal>,
644 <literal>record_transform</literal>,
645 <literal>session_shared</literal>,
646 <literal>sru_z3950</literal>,
647 <literal>template</literal>,
648 <literal>virt_db</literal>).
653 <section id="overview.filter.types">
654 <title>Overview of filter types</title>
656 We now briefly consider each of the types of filter supported by
657 the core Metaproxy binary. This overview is intended to give a
658 flavor of the available functionality; more detailed information
659 about each type of filter is included below in
660 <xref linkend="reference"/>.
663 The filters are here named by the string that is used as the
664 <literal>type</literal> attribute of a
665 <literal><filter></literal> element in the configuration
666 file to request them, with the name of the class that implements
667 them in parentheses. (The classname is not needed for normal
668 configuration and use of Metaproxy; it is useful only to
672 The filters are here listed in alphabetical order:
679 New virt_db-alike that does inteligent peer choice, explain merging,
680 adds FD&N to explain. Keeps init responses (like "virt_db Classic"),
681 makes routing choices based on local explain knowledge. Ref IDDI
684 Filter to convert Explain Classic to ZeeRex.
686 CQL2PQF (which needs augmented ZeeRex) - MARC for Talis.
690 Figure out what additional information we need in:
691 ZeeRex (check against D3.1)
692 Init request (e.g. loop detection)
693 Query package (e.g. number of hops)
694 Query response (e.g. record source)
699 <title><literal>auth_simple</literal>
700 (mp::filter::AuthSimple)</title>
702 Simple authentication and authorization. The configuration
703 specifies the name of a file that is the user register, which
704 lists <varname>username</varname>:<varname>password</varname>
705 pairs, one per line, colon separated. When a session begins, it
706 is rejected unless username and passsword are supplied, and match
707 a pair in the register. The configuration file may also specific
708 the name of another file that is the target register: this lists
709 lists <varname>username</varname>:<varname>dbname</varname>,<varname>dbname</varname>...
710 sets, one per line, with multiple database names separated by
711 commas. When a search is processed, it is rejected unless the
712 database to be searched is one of those listed as available to
718 <title><literal>backend_test</literal>
719 (mp::filter::Backend_test)</title>
721 A partial sink that provides dummy responses in the manner of the
722 <literal>yaz-ztest</literal> Z39.50 server. This is useful only
723 for testing. Seriously, you don't need this. Pretend you didn't
724 even read this section.
729 <title><literal>bounce</literal>
730 (mp::filter::Bounce)</title>
732 A sink that swallows <emphasis>all packages</emphasis>,
733 and returns them almost unprocessed.
734 It never sends any package of any type further down the row, but
735 sets Z39.50 packages to Z_Close, and HTTP_Request packages to
736 HTTP_Response err code 400 packages, and adds a suitable bounce
738 The bounce filter is usually added at end of each filter chain
739 config.xml to prevent infinite hanging of for example HTTP
740 requests packages when only the Z39.50 client partial sink
741 filter is found in the
747 <title><literal>frontend_net</literal>
748 (mp::filter::FrontendNet)</title>
750 A source that accepts Z39.50 connections from a port
751 specified in the configuration, reads protocol units, and
752 feeds them into the next filter in the route. When the result is
753 received, it is returned to the original origin.
758 <title><literal>http_file</literal>
759 (mp::filter::HttpFile)</title>
761 A partial sink which swallows only HTTP_Request packages, and
762 returns the contents of files from the local
763 filesystem in response to HTTP requests.
764 It lets Z39.50 packages and all other forthcoming package types
767 does mean that Metaproxy is also a Web-server in its spare time. So
768 far it does not contain either an email-reader or a Lisp
769 interpreter, but that day is surely coming.)
774 <title><literal>load_balance</literal>
775 (mp::filter::LoadBalance)</title>
777 Performs load balancing for incoming Z39.50 init requests.
778 It is used together with the <literal>virt_db</literal> filter,
779 but unlike the <literal>multi</literal> filter it does send an
780 entire session to only one of the virtual backends. The
781 <literal>load_balance</literal> filter is assuming that
782 all backend targets have equal content, and chooses the backend
783 with least load cost for a new session.
786 This filter is experimental and yet not mature for heavy load
794 <title><literal>log</literal>
795 (mp::filter::Log)</title>
797 Writes logging information to standard output, and passes on
798 the package unchanged. A log file name can be specified, as well
799 as multiple different logging formats.
804 <title><literal>multi</literal>
805 (mp::filter::Multi)</title>
807 Performs multi-database searching.
809 <link linkend="multidb">the extended discussion</link>
810 of virtual databases and multi-database searching below.
815 <title><literal>query_rewrite</literal>
816 (mp::filter::QueryRewrite)</title>
818 Rewrites Z39.50 Type-1 and Type-101 (``RPN'') queries by a
819 three-step process: the query is transliterated from Z39.50
820 packet structures into an XML representation; that XML
821 representation is transformed by an XSLT stylesheet; and the
822 resulting XML is transliterated back into the Z39.50 packet
829 <title><literal>record_transform</literal>
830 (mp::filter::RecordTransform)</title>
832 This filter acts only on Z3950 present requests, and let all
833 other types of packages and requests pass untouched. It's use is
834 twofold: blocking Z3950 present requests, which the backend
835 server does not understand and can not honor, and transforming
836 the present syntax and elementset name according to the rules
837 specified, to fetch only existing record formats, and transform
838 them on the fly to requested record syntaxes.
843 <title><literal>session_shared</literal>
844 (mp::filter::SessionShared)</title>
846 When this is finished, it will implement global sharing of
847 result sets (i.e. between threads and therefore between
848 clients), yielding performance improvements especially when
849 incoming requests are from a stateless environment such as a
850 web-server, in which the client process representing a session
851 might be any one of many. However:
855 This filter is not yet completed.
861 <title><literal>sru_z3950</literal>
862 (mp::filter::SRUtoZ3950)</title>
864 This filter transforms valid
865 SRU GET/POST/SOAP searchRetrieve requests to Z3950 init, search,
866 and present requests, and wraps the
867 received hit counts and XML records into suitable SRU response
869 The <literal>sru_z3950</literal> filter processes also SRU
870 GET/POST/SOAP explain requests, returning
871 either the absolute minimum required by the standard, or a full
872 pre-defined ZeeReX explain record.
874 <ulink url="&url.zeerex.explain;">ZeeReX Explain</ulink>
875 standard pages and the
876 <ulink url="&url.sru.explain;">SRU Explain</ulink> pages
877 for more information on the correct explain syntax.
878 SRU scan requests are not supported yet.
883 <title><literal>template</literal>
884 (mp::filter::Template)</title>
886 Does nothing at all, merely passing the packet on. (Maybe it
887 should be called <literal>nop</literal> or
888 <literal>passthrough</literal>?) This exists not to be used, but
889 to be copied - to become the skeleton of new filters as they are
890 written. As with <literal>backend_test</literal>, this is not
891 intended for civilians.
896 <title><literal>virt_db</literal>
897 (mp::filter::VirtualDB)</title>
899 Performs virtual database selection: based on the name of the
900 database in the search request, a server is selected, and its
901 address added to the request in a <literal>VAL_PROXY</literal>
902 otherInfo packet. It will subsequently be used by a
903 <literal>z3950_client</literal> filter.
905 <link linkend="multidb">the extended discussion</link>
906 of virtual databases and multi-database searching below.
911 <title><literal>z3950_client</literal>
912 (mp::filter::Z3950Client)</title>
914 A partial sink which swallows only Z39.50 packages.
915 It performs Z39.50 searching and retrieval by proxying the
916 packages that are passed to it. Init requests are sent to the
917 address specified in the <literal>VAL_PROXY</literal> otherInfo
918 attached to the request: this may have been specified by client,
919 or generated by a <literal>virt_db</literal> filter earlier in
920 the route. Subsequent requests are sent to the same address,
921 which is remembered at Init time in a Session object.
922 HTTP_Request packages and all other forthcoming package types
923 are passed untouched.
929 <title><literal>zeerex_explain</literal>
930 (mp::filter::ZeerexExplain)</title>
932 This filter acts as a sink for
933 Z39.50 explain requests, returning a static ZeeReX
934 Explain XML record from the config section. All other packages
937 <ulink url="&url.zeerex.explain;">ZeeReX Explain</ulink>
939 for more information on the correct explain syntax.
943 This filter is not yet completed.
952 <section id="future.directions">
953 <title>Future directions</title>
955 Some other filters that do not yet exist, but which would be
956 useful, are briefly described. These may be added in future
957 releases (or may be created by third parties, as loadable
963 <term><literal>frontend_cli</literal> (source)</term>
966 Command-line interface for generating requests.
971 <term><literal>sru_client</literal> (sink)</term>
974 SRU/GET and SRU/SOAP searching and retrieval.
979 <term><literal>opensearch_client</literal> (sink)</term>
982 A9 OpenSearch searching and retrieval.
992 <chapter id="configuration">
993 <title>Configuration: the Metaproxy configuration file format</title>
997 <title>Introductory notes</title>
999 If Metaproxy is an interpreter providing operations on packages, then
1000 its configuration file can be thought of as a program for that
1001 interpreter. Configuration is by means of a single XML file, the name
1002 of which is supplied as the sole command-line argument to the
1003 <command>metaproxy</command> program. (See
1004 <xref linkend="reference"/> below for more information on invoking
1009 <section id="overview.xml.structure">
1010 <title>Overview of the config file XML structure</title>
1012 All elements and attributes are in the namespace
1013 <ulink url="http://indexdata.com/metaproxy"/>.
1014 This is most easily achieved by setting the default namespace on
1015 the top-level element, as here:
1018 <metaproxy xmlns="http://indexdata.com/metaproxy" version="1.0">
1021 The top-level element is <metaproxy>. This contains a
1022 <start> element, a <filters> element and a
1023 <routes> element, in that order. <filters> is
1024 optional; the other two are mandatory. All three are
1028 The <start> element is empty, but carries a
1029 <literal>route</literal> attribute, whose value is the name of
1030 route at which to start running - analogous to the name of the
1031 start production in a formal grammar.
1034 If present, <filters> contains zero or more <filter>
1035 elements. Each filter carries a <literal>type</literal> attribute
1036 which specifies what kind of filter is being defined
1037 (<literal>frontend_net</literal>, <literal>log</literal>, etc.)
1038 and contain various elements that provide suitable configuration
1039 for a filter of its type. The filter-specific elements are
1041 <xref linkend="reference"/>.
1042 Filters defined in this part of the file must carry an
1043 <literal>id</literal> attribute so that they can be referenced
1047 <routes> contains one or more <route> elements, each
1048 of which must carry an <literal>id</literal> element. One of the
1049 routes must have the ID value that was specified as the start
1050 route in the <start> element's <literal>route</literal>
1051 attribute. Each route contains zero or more <filter>
1052 elements. These are of two types. They may be empty, but carry a
1053 <literal>refid</literal> attribute whose value is the same as the
1054 <literal>id</literal> of a filter previously defined in the
1055 <filters> section. Alternatively, a route within a filter
1056 may omit the <literal>refid</literal> attribute, but contain
1057 configuration elements similar to those used for filters defined
1058 in the <filters> section. (In other words, each filter in a
1059 route may be included either by reference or by physical
1065 <section id="example.configuration">
1066 <title>An example configuration</title>
1068 The following is a small, but complete, Metaproxy configuration
1069 file (included in the distribution as
1070 <literal>metaproxy/etc/config1.xml</literal>).
1071 This file defines a very simple configuration that simply proxies
1072 to whatever back-end server the client requests, but logs each
1073 request and response. This can be useful for debugging complex
1074 client-server dialogues.
1076 <screen><![CDATA[<?xml version="1.0"?>
1077 <metaproxy xmlns="http://indexdata.com/metaproxy" version="1.0">
1078 <start route="start"/>
1080 <filter id="frontend" type="frontend_net">
1083 <filter id="backend" type="z3950_client">
1088 <filter refid="frontend"/>
1089 <filter type="log"/>
1090 <filter refid="backend"/>
1091 <filter type="bounce"/>
1097 It works by defining a single route, called
1098 <literal>start</literal>, which consists of a sequence of four
1099 filters. The first and last of these are included by reference:
1100 their <literal><filter></literal> elements have
1101 <literal>refid</literal> attributes that refer to filters defined
1102 within the prior <literal><filters></literal> section. The
1103 middle filter is included inline in the route.
1106 The four filters in the route are as follows: first, a
1107 <literal>frontend_net</literal> filter accepts Z39.50 requests
1108 from any host on port 9000; then these requests are passed through
1109 a <literal>log</literal> filter that emits a message for each
1110 request; they are then fed into a <literal>z3950_client</literal>
1111 filter, which forwards all Z39.50 requests to the client-specified
1112 back-end Z39.509 server. Those Z39.50 packages are returned by the
1113 <literal>z3950_client</literal> filter, with the response data
1114 filled by the external Z39.50 server targeted.
1115 All non-Z39.50 packages are passed through to the
1116 <literal>bounce</literal> filter, which definitely bounces
1117 everything, including fish, bananas, cold pyjamas,
1118 mutton, beef and trout packages.
1119 When the response arrives, it is handed
1120 back to the <literal>log</literal> filter, which emits another
1121 message; and then to the <literal>frontend_net</literal> filter,
1122 which returns the response to the client.
1125 <section id="checking.xml.syntax">
1126 <title>Config file syntax checking</title>
1128 The distribution contains RelaxNG Compact and XML syntax checking
1129 files, as well as XML Schema files. These are found in the
1132 xml/schema/metaproxy.rnc
1133 xml/schema/metaproxy.rng
1134 xml/schema/metaproxy.xsd
1136 and can be used to verify or debug the XML structure of
1137 configuration files. For example, using the utility
1138 <filename>xmllint</filename>, syntax checking is done like this:
1140 xmllint --noout --schema xml/schema/metaproxy.xsd etc/config-local.xml
1141 xmllint --noout --relaxng xml/schema/metaproxy.rng etc/config-local.xml
1143 (A recent version of <literal>libxml2</literal> is required, as
1144 support for XML Schemas is a relatively recent addition.)
1147 You can of course use any other RelaxNG or XML Schema compliant tool
1155 <chapter id="multidb">
1156 <title>Virtual databases and multi-database searching</title>
1160 <title>Introductory notes</title>
1162 Two of Metaproxy's filters are concerned with multiple-database
1163 operations. Of these, <literal>virt_db</literal> can work alone
1164 to control the routing of searches to one of a number of servers,
1165 while <literal>multi</literal> can work together with
1166 <literal>virt_db</literal> to perform multi-database searching, merging
1167 the results into a unified result-set - ``metasearch in a box''.
1170 The interaction between
1171 these two filters is necessarily complex: it reflects the real,
1172 irreducible complexity of multi-database searching in a protocol such
1173 as Z39.50 that separates initialization from searching, and in
1174 which the database to be searched is not known at initialization
1178 It's possible to use these filters without understanding the
1179 details of their functioning and the interaction between them; the
1180 next two sections of this chapter are ``HOW-TO'' guides for doing
1181 just that. However, debugging complex configurations will require
1182 a deeper understanding, which the last two sections of this
1183 chapters attempt to provide.
1188 <section id="multidb.virt_db">
1189 <title>Virtual databases with the <literal>virt_db</literal> filter</title>
1191 Working alone, the purpose of the
1192 <literal>virt_db</literal>
1193 filter is to route search requests to one of a selection of
1194 back-end databases. In this way, a single Z39.50 endpoint
1195 (running Metaproxy) can provide access to several different
1196 underlying services, including those that would otherwise be
1197 inaccessible due to firewalls. In many useful configurations, the
1198 back-end databases are local to the Metaproxy installation, but
1199 the software does not enforce this, and any valid Z39.50 servers
1200 may be used as back-ends.
1203 For example, a <literal>virt_db</literal>
1204 filter could be set up so that searches in the virtual database
1205 ``lc'' are forwarded to the Library of Congress bibliographic
1206 catalogue server, and searches in the virtual database ``marc''
1207 are forwarded to the toy database of MARC records that Index Data
1208 hosts for testing purposes. A <literal>virt_db</literal>
1209 configuration to make this switch would look like this:
1211 <screen><![CDATA[<filter type="virt_db">
1213 <database>lc</database>
1214 <target>z3950.loc.gov:7090/voyager</target>
1217 <database>marc</database>
1218 <target>indexdata.com/marc</target>
1220 </filter>]]></screen>
1222 As well as being useful in it own right, this filter also provides
1223 the foundation for multi-database searching.
1228 <section id="multidb.multi">
1229 <title>Multi-database search with the <literal>multi</literal> filter</title>
1231 To arrange for Metaproxy to broadcast searches to multiple back-end
1232 servers, the configuration needs to include two components: a
1233 <literal>virt_db</literal>
1234 filter that specifies multiple
1235 <literal><target></literal>
1236 elements, and a subsequent
1237 <literal>multi</literal>
1238 filter. Here, for example, is a complete configuration that
1239 broadcasts searches to both the Library of Congress catalogue and
1240 Index Data's tiny testing database of MARC records:
1242 <screen><![CDATA[<?xml version="1.0"?>
1243 <metaproxy xmlns="http://indexdata.com/metaproxy" version="1.0">
1244 <start route="start"/>
1247 <filter type="frontend_net">
1248 <threads>10</threads>
1251 <filter type="virt_db">
1253 <database>lc</database>
1254 <target>z3950.loc.gov:7090/voyager</target>
1257 <database>marc</database>
1258 <target>indexdata.com/marc</target>
1261 <database>all</database>
1262 <target>z3950.loc.gov:7090/voyager</target>
1263 <target>indexdata.com/marc</target>
1266 <filter type="multi"/>
1267 <filter type="z3950_client">
1268 <timeout>30</timeout>
1270 <filter type="bounce"/>
1273 </metaproxy>]]></screen>
1276 <literal>virt_db</literal>
1277 filter that specifies multiple
1278 <literal><target></literal>
1279 elements but without a subsequent
1280 <literal>multi</literal>
1281 filter yields surprising and undesirable results, as will be
1282 described below. Don't do that.)
1285 Metaproxy can be invoked with this configuration as follows:
1287 <screen>../src/metaproxy --config config-simple-multi.xml</screen>
1289 And thereafter, Z39.50 clients can connect to the running server
1290 (on port 9000, as specified in the configuration) and search in
1291 any of the databases
1292 <literal>lc</literal> (the Library of Congress catalogue),
1293 <literal>marc</literal> (Index Data's test database of MARC records)
1295 <literal>all</literal> (both of these). As an example, a session
1296 using the YAZ command-line client <literal>yaz-client</literal> is
1297 here included (edited for brevity and clarity):
1299 <screen><![CDATA[$ yaz-client @:9000
1303 Search was a success.
1304 Number of hits: 10000, setno 1
1308 Search was a success.
1309 Number of hits: 10, setno 3
1313 Search was a success.
1314 Number of hits: 10010, setno 4
1317 [marc]Record type: USmarc
1320 005 00000000000000.0
1321 008 910710c19910701nju 00010 eng
1325 100 10 $a Jack Collins
1326 245 10 $a How to program a computer
1332 [VOYAGER]Record type: USmarc
1334 005 20041229102447.0
1335 008 030910s2004 caua 000 0 eng
1336 035 $a (DLC) 2003112666
1337 906 $a 7 $b cbc $c orignew $d 4 $e epcn $f 20 $g y-gencatlg
1338 925 0 $a acquire $b 1 shelf copy $x policy default
1339 955 $a pc10 2003-09-10 $a pv12 2004-06-23 to SSCD; $h sj05 2004-11-30 $e sj05 2004-11-30 to Shelf.
1342 040 $a DLC $c DLC $d DLC
1343 050 00 $a MLCM 2004/03312 (G)
1344 245 10 $a 007, everything or nothing : $b Prima's official strategy guide / $c created by Kaizen Media Group.
1345 246 3 $a Double-O-seven, everything or nothing
1346 246 30 $a Prima's official strategy guide
1347 260 $a Roseville, CA : $b Prima Games, $c c2004.
1348 300 $a 161 p. : $b col. ill. ; $c 28 cm.
1349 500 $a "Platforms: Nintendo GameCube, Macintosh, PC, PlayStation 2 computer entertainment system, Xbox"--P. [4] of cover.
1350 650 0 $a Video games.
1351 710 2 $a Kaizen Media Group.
1352 856 42 $3 Publisher description $u http://www.loc.gov/catdir/description/random052/2003112666.html
1357 As can be seen, the first record in the result set is from the
1358 Index Data test database, and the second from the Library of
1359 Congress database. The result-set continues alternating records
1360 round-robin style until the point where one of the databases'
1361 records are exhausted.
1364 This example uses only two back-end databases; more may be used.
1365 There is no limitation imposed on the number of databases that may
1366 be metasearched in this way: issues of resource usage and
1367 administrative complexity dictate the practical limits.
1370 What happens when one of the databases doesn't respond? By default,
1371 the entire multi-database search fails, and the appropriate
1372 diagnostic is returned to the client. This is usually appropriate
1373 during development, when technicians need maximum information, but
1374 can be inconvenient in deployment, when users typically don't want
1375 to be bothered with problems of this kind and prefer just to get
1376 the records from the databases that are available. To obtain this
1377 latter behavior add an empty
1378 <literal><hideunavailable></literal>
1380 <literal>multi</literal> filter:
1382 <screen><![CDATA[ <filter type="multi">
1384 </filter>]]></screen>
1386 Under this regime, an error is reported to the client only if
1387 <emphasis>all</emphasis> the databases in a multi-database search
1393 <section id="multidb.what">
1394 <title>What's going on?</title>
1396 <title>Lark's vomit</title>
1398 This section goes into a level of technical detail that is
1399 probably not necessary in order to configure and use Metaproxy.
1400 It is provided only for those who like to know how things work.
1401 You should feel free to skip on to the next section if this one
1402 doesn't seem like fun.
1406 Hold on tight - this may get a little hairy.
1409 In the general course of things, a Z39.50 Init request may carry
1410 with it an otherInfo packet of type <literal>VAL_PROXY</literal>,
1411 whose value indicates the address of a Z39.50 server to which the
1412 ultimate connection is to be made. (This otherInfo packet is
1413 supported by YAZ-based Z39.50 clients and servers, but has not yet
1414 been ratified by the Maintenance Agency and so is not widely used
1415 in non-Index Data software. We're working on it.)
1416 The <literal>VAL_PROXY</literal> packet functions
1417 analogously to the absoluteURI-style Request-URI used with the GET
1418 method when a web browser asks a proxy to forward its request: see
1420 <ulink url="http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html#sec5.1.2"
1421 >Request-URI</ulink>
1423 <ulink url="http://www.w3.org/Protocols/rfc2616/rfc2616.html"
1424 >the HTTP 1.1 specification</ulink>.
1427 Within Metaproxy, Search requests that are part of the same
1428 session as an Init request that carries a
1429 <literal>VAL_PROXY</literal> otherInfo are also annotated with the
1430 same information. The role of the <literal>virt_db</literal>
1431 filter is to rewrite this otherInfo packet dependent on the
1432 virtual database that the client wants to search.
1435 When Metaproxy receives a Z39.50 Init request from a client, it
1436 doesn't immediately forward that request to the back-end server.
1437 Why not? Because it doesn't know <emphasis>which</emphasis>
1438 back-end server to forward it to until the client sends a Search
1439 request that specifies the database that it wants to search in.
1440 Instead, it just treasures the Init request up in its heart; and,
1441 later, the first time the client does a search on one of the
1442 specified virtual databases, a connection is forged to the
1443 appropriate server and the Init request is forwarded to it. If,
1444 later in the session, the same client searches in a different
1445 virtual database, then a connection is forged to the server that
1446 hosts it, and the same cached Init request is forwarded there,
1450 All of this clever Init-delaying is done by the
1451 <literal>frontend_net</literal> filter. The
1452 <literal>virt_db</literal> filter knows nothing about it; in
1453 fact, because the Init request that is received from the client
1454 doesn't get forwarded until a Search request is received, the
1455 <literal>virt_db</literal> filter (and the
1456 <literal>z3950_client</literal> filter behind it) doesn't even get
1457 invoked at Init time. The <emphasis>only</emphasis> thing that a
1458 <literal>virt_db</literal> filter ever does is rewrite the
1459 <literal>VAL_PROXY</literal> otherInfo in the requests that pass
1463 It is possible for a <literal>virt_db</literal> filter to contain
1465 <literal><target></literal>
1466 elements. What does this mean? Only that the filter will add
1467 multiple <literal>VAL_PROXY</literal> otherInfo packets to the
1468 Search requests that pass through it. That's because the virtual
1469 DB filter is dumb, and does exactly what it's told - no more, no
1471 If a Search request with multiple <literal>VAL_PROXY</literal>
1472 otherInfo packets reaches a <literal>z3950_client</literal>
1473 filter, this is an error. That filter doesn't know how to deal
1474 with multiple targets, so it will either just pick one and search
1475 in it, or (better) fail with an error message.
1478 The <literal>multi</literal> filter comes to the rescue! This is
1479 the only filter that knows how to deal with multiple
1480 <literal>VAL_PROXY</literal> otherInfo packets, and it does so by
1481 making multiple copies of the entire Search request: one for each
1482 <literal>VAL_PROXY</literal>. Each of these new copies is then
1483 passed down through the remaining filters in the route. (The
1484 copies are handled in parallel though the
1485 spawning of new threads.) Since the copies each have only one
1486 <literal>VAL_PROXY</literal> otherInfo, they can be handled by the
1487 <literal>z3950_client</literal> filter, which happily deals with
1488 each one individually. When the results of the individual
1489 searches come back up to the <literal>multi</literal> filter, it
1490 merges them into a single Search response, which is what
1491 eventually makes it back to the client.
1496 <section id="multidb.picture">
1497 <title>A picture is worth a thousand words (but only five hundred on 64-bit architectures)</title>
1501 <imagedata fileref="multi.pdf" format="PDF" scale="50"/>
1504 <imagedata fileref="multi.png" format="PNG"/>
1507 <!-- Fall back if none of the images can be used -->
1509 [Here there should be a diagram showing the progress of
1510 packages through the filters during a simple virtual-database
1511 search and a multi-database search, but is seems that your
1512 tool chain has not been able to include the diagram in this
1516 <!-- ### This used to work with an older version of DocBook
1518 <para>Caption: progress of packages through filters.</para>
1521 </inlinemediaobject>
1528 <chapter id="extensions">
1529 <title>Writing extensions for Metaproxy</title>
1530 <para>### To be written</para>
1536 <chapter id="classes">
1537 <title>Classes in the Metaproxy source code</title>
1541 <title>Introductory notes</title>
1543 <emphasis>Stop! Do not read this!</emphasis>
1544 You won't enjoy it at all. You should just skip ahead to
1545 <xref linkend="reference"/>,
1547 <!-- The remainder of this paragraph is lifted verbatim from
1548 Douglas Adams' _Hitch Hiker's Guide to the Galaxy_, chapter 8 -->
1549 you things you really need to know, like the fact that the
1550 fabulously beautiful planet Bethselamin is now so worried about
1551 the cumulative erosion by ten billion visiting tourists a year
1552 that any net imbalance between the amount you eat and the amount
1553 you excrete whilst on the planet is surgically removed from your
1554 bodyweight when you leave: so every time you go to the lavatory it
1555 is vitally important to get a receipt.
1558 This chapter contains documentation of the Metaproxy source code, and is
1559 of interest only to maintainers and developers. If you need to
1560 change Metaproxy's behavior or write a new filter, then you will most
1561 likely find this chapter helpful. Otherwise it's a waste of your
1562 good time. Seriously: go and watch a film or something.
1563 <citetitle>This is Spinal Tap</citetitle> is particularly good.
1566 Still here? OK, let's continue.
1569 In general, classes seem to be named big-endianly, so that
1570 <literal>FactoryFilter</literal> is not a filter that filters
1571 factories, but a factory that produces filters; and
1572 <literal>FactoryStatic</literal> is a factory for the statically
1573 registered filters (as opposed to those that are dynamically
1578 <section id="individual.classes">
1579 <title>Individual classes</title>
1581 The classes making up the Metaproxy application are here listed by
1582 class-name, with the names of the source files that define them in
1587 <title><literal>mp::FactoryFilter</literal>
1588 (<filename>factory_filter.cpp</filename>)</title>
1590 A factory class that exists primarily to provide the
1591 <literal>create()</literal> method, which takes the name of a
1592 filter class as its argument and returns a new filter of that
1593 type. To enable this, the factory must first be populated by
1594 calling <literal>add_creator()</literal> for static filters (this
1595 is done by the <literal>FactoryStatic</literal> class, see below)
1596 and <literal>add_creator_dyn()</literal> for filters loaded
1602 <title><literal>mp::FactoryStatic</literal>
1603 (<filename>factory_static.cpp</filename>)</title>
1605 A subclass of <literal>FactoryFilter</literal> which is
1606 responsible for registering all the statically defined filter
1607 types. It does this by knowing about all those filters'
1608 structures, which are listed in its constructor. Merely
1609 instantiating this class registers all the static classes. It is
1610 for the benefit of this class that <literal>struct
1611 metaproxy_1_filter_struct</literal> exists, and that all the filter
1612 classes provide a static object of that type.
1617 <title><literal>mp::filter::Base</literal>
1618 (<filename>filter.cpp</filename>)</title>
1620 The virtual base class of all filters. The filter API is, on the
1621 surface at least, extremely simple: two methods.
1622 <literal>configure()</literal> is passed an XML DOM tree representing
1623 that part of the configuration file that pertains to this filter
1624 instance, and is expected to walk that tree extracting relevant
1625 information. And <literal>process()</literal> processes a
1626 package (see below). That surface simplicity is a bit
1627 misleading, as <literal>process()</literal> needs to know a lot
1628 about the <literal>Package</literal> class in order to do
1634 <title><literal>mp::filter::AuthSimple</literal>,
1635 <literal>Backend_test</literal>, etc.
1636 (<filename>filter_auth_simple.cpp</filename>,
1637 <filename>filter_backend_test.cpp</filename>, etc.)</title>
1639 Individual filters. Each of these is implemented by a header and
1640 a source file, named <filename>filter_*.hpp</filename> and
1641 <filename>filter_*.cpp</filename> respectively. All the header
1642 files should be pretty much identical, in that they declare the
1643 class, including a private <literal>Rep</literal> class and a
1644 member pointer to it, and the two public methods.
1647 The source file for each filter needs to supply:
1652 A definition of the private <literal>Rep</literal> class.
1657 Some boilerplate constructors and destructors.
1662 A <literal>configure()</literal> method that uses the
1663 appropriate XML fragment.
1668 Most important, the <literal>process()</literal> method that
1669 does all the actual work.
1676 <title><literal>mp::Package</literal>
1677 (<filename>package.cpp</filename>)</title>
1679 Represents a package on its way through the series of filters
1680 that make up a route. This is essentially a Z39.50 or SRU APDU
1681 together with information about where it came from, which is
1682 modified as it passes through the various filters.
1687 <title><literal>mp::Pipe</literal>
1688 (<filename>pipe.cpp</filename>)</title>
1690 This class provides a compatibility layer so that we have an IPC
1691 mechanism that works the same under Unix and Windows. It's not
1692 particularly exciting.
1697 <title><literal>mp::RouterChain</literal>
1698 (<filename>router_chain.cpp</filename>)</title>
1705 <title><literal>mp::RouterFleXML</literal>
1706 (<filename>router_flexml.cpp</filename>)</title>
1713 <title><literal>mp::Session</literal>
1714 (<filename>session.cpp</filename>)</title>
1721 <title><literal>mp::ThreadPoolSocketObserver</literal>
1722 (<filename>thread_pool_observer.cpp</filename>)</title>
1729 <title><literal>mp::util</literal>
1730 (<filename>util.cpp</filename>)</title>
1732 A namespace of various small utility functions and classes,
1733 collected together for convenience. Most importantly, includes
1734 the <literal>mp::util::odr</literal> class, a wrapper for YAZ's
1740 <title><literal>mp::xml</literal>
1741 (<filename>xmlutil.cpp</filename>)</title>
1743 A namespace of various XML utility functions and classes,
1744 collected together for convenience.
1750 <section id="other.source.files">
1751 <title>Other Source Files</title>
1753 In addition to the Metaproxy source files that define the classes
1754 described above, there are a few additional files which are
1755 briefly described here:
1759 <term><literal>metaproxy_prog.cpp</literal></term>
1762 The main function of the <command>metaproxy</command> program.
1767 <term><literal>ex_router_flexml.cpp</literal></term>
1770 Identical to <literal>metaproxy_prog.cpp</literal>: it's not clear why.
1775 <term><literal>test_*.cpp</literal></term>
1778 Unit-tests for various modules.
1784 ### Still to be described:
1785 <literal>ex_filter_frontend_net.cpp</literal>,
1786 <literal>filter_dl.cpp</literal>,
1787 <literal>plainfile.cpp</literal>,
1788 <literal>tstdl.cpp</literal>.
1794 <reference id="reference">
1795 <title>Reference</title>
1798 The material in this chapter is drawn directly from the individual
1799 manual entries. In particular, the Metaproxy invocation section is
1800 available using <command>man metaproxy</command>, and the section
1801 on each individual filter is available using the name of the filter
1802 as the argument to the <command>man</command> command.
1809 <!-- Keep this comment at the end of the file
1814 sgml-minimize-attributes:nil
1815 sgml-always-quote-attributes:t
1818 sgml-parent-document: nil
1819 sgml-local-catalogs: nil
1820 sgml-namecase-general:t