1 <?xml version="1.0" standalone="no"?>
2 <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1//EN"
3 "http://www.oasis-open.org/docbook/xml/4.1/docbookx.dtd"
5 <!ENTITY local SYSTEM "local.ent">
6 <!ENTITY manref SYSTEM "manref.xml">
7 <!ENTITY progref SYSTEM "progref.xml">
8 <!ENTITY % common SYSTEM "common/common.ent">
10 <!-- Next line allows imagedata/@format="PDF" and is taken from
11 http://lists.oasis-open.org/archives/docbook/200303/msg00163.html
13 <!ENTITY % local.notation.class "| PDF">
14 <!-- Next line is necessary for some XML parsers, for reasons I
15 don't understand. I got this from
16 http://lists.oasis-open.org/archives/docbook/200303/msg00180.html
18 <!NOTATION PDF SYSTEM "PDF">
20 <!-- $Id: book.xml,v 1.47 2007-01-08 12:27:27 marc Exp $ -->
23 <title>Metaproxy - User's Guide and Reference</title>
25 <firstname>Adam</firstname><surname>Dickmeiss</surname>
28 <firstname>Marc</firstname><surname>Cromme</surname>
31 <firstname>Mike</firstname><surname>Taylor</surname>
34 <year>2005-2007</year>
35 <holder>Index Data ApS</holder>
39 Metaproxy is a universal router, proxy and encapsulated
40 metasearcher for information retrieval protocols. It accepts,
41 processes, interprets and redirects requests from IR clients using
42 standard protocols such as
43 <ulink url="&url.z39.50;">ANSI/NISO Z39.50</ulink>
44 (and in the future <ulink url="&url.sru;">SRU</ulink>
45 and <ulink url="&url.srw;">SRW</ulink>), as
46 well as functioning as a limited
47 <ulink url="&url.http;">HTTP</ulink> server.
48 Metaproxy is configured by an XML file which
49 specifies how the software should function in terms of routes that
50 the request packets can take through the proxy, each step on a
51 route being an instantiation of a filter. Filters come in many
52 types, one for each operation: accepting Z39.50 packets, logging,
53 query transformation, multiplexing, etc. Further filter-types can
54 be added as loadable modules to extend Metaproxy functionality,
58 Metaproxy is <emphasis>not</emphasis> open-source software, but
59 may be freely downloaded, unpacked, inspected, built and run for
60 evaluation purposes. Deployment requires a separate, commercial,
66 <imagedata fileref="common/id.png" format="PNG"/>
69 <imagedata fileref="common/id.eps" format="EPS"/>
76 <chapter id="introduction">
77 <title>Introduction</title>
81 <ulink url="&url.metaproxy;">Metaproxy</ulink>
82 is a stand alone program that acts as a universal router, proxy and
83 encapsulated metasearcher for information retrieval protocols such
84 as <ulink url="&url.z39.50;">Z39.50</ulink>, and in the future
85 <ulink url="&url.sru;">SRU</ulink> and <ulink url="&url.srw;">SRW</ulink>.
86 To clients, it acts as a server of these protocols: it can be searched,
87 records can be retrieved from it, etc.
88 To servers, it acts as a client: it searches in them,
89 retrieves records from them, etc. it satisfies its clients'
90 requests by transforming them, multiplexing them, forwarding them
91 on to zero or more servers, merging the results, transforming
92 them, and delivering them back to the client. In addition, it
93 acts as a simple <ulink url="&url.http;">HTTP</ulink> server; support
94 for further protocols can be added in a modular fashion, through the
95 creation of new filters.
100 Fish, bananas, cold pyjamas,
101 Mutton, beef and trout!
102 - attributed to Cole Porter.
105 Metaproxy is a more capable alternative to
106 <ulink url="&url.yazproxy;">YAZ Proxy</ulink>,
107 being more powerful, flexible, configurable and extensible. Among
108 its many advantages over the older, more pedestrian work are
109 support for multiplexing (encapsulated metasearching), routing by
110 database name, authentication and authorization and serving local
111 files via HTTP. Equally significant, its modular architecture
112 facilitites the creation of pluggable modules implementing further
116 This manual will describe how to install Metaproxy
117 before giving an overview of its architecture, then discussing the
118 key concept of a filter in some depth and giving an overview of
119 the various filter types, then discussing the configuration file
120 format. After this come several optional chapters which may be
121 freely skipped: a detailed discussion of virtual databases and
122 multi-database searching, some notes on writing extensions
123 (additional filter types) and a high-level description of the
124 source code. Finally comes the reference guide, which contains
125 instructions for invoking the <command>metaproxy</command>
126 program, and detailed information on each type of filter,
131 <chapter id="license">
132 <title>The Metaproxy License</title>
133 <orderedlist numeration="arabic">
136 You are allowed to download this software for evaluation purposes.
137 You can unpack it, build it, run it, see how it works and how it fits
138 your needs, all at zero cost.
143 You may NOT deploy the software. For the purposes of this license,
144 deployment means running it for any purpose other than evaluation,
145 whether or not you or anyone else makes a profit from doing so. If
146 you wish to deploy the software, you must first contact Index Data and
147 arrange to purchase a DEPLOYMENT LICENCE. If you are unsure
148 whether or not your proposed use of the software constitutes
149 deployment, email us at <literal>info@indexdata.com</literal>
155 You may modify your copy of the software (fix bugs, add features)
156 if you need to. We encourage you to send your changes back to us for
157 integration into the master copy, but you are not obliged to do so. You
158 may NOT pass your changes on to any other party.
163 There is NO WARRANTY for this software, to the extent permitted by
164 applicable law. We provide the software ``as is'' without warranty of
165 any kind, either expressed or implied, including, but not limited to, the
166 implied warranties of MERCHANTABILITY and FITNESS FOR A
167 PARTICULAR PURPOSE. The entire risk as to the quality and
168 performance of the software is with you. Should the software prove
169 defective, you assume the cost of all necessary servicing, repair or
170 correction. In no event unless required by applicable law will we be
171 liable to you for damages, arising out of the use of the software,
172 including but not limited to loss of data or data being rendered
178 All rights to the software are reserved by Index Data except where
179 this license explicitly says otherwise.
185 <chapter id="installation">
186 <title>Installation</title>
188 Metaproxy depends on the following tools/libraries:
190 <varlistentry><term><ulink url="&url.yazplusplus;">YAZ++</ulink></term>
193 This is a C++ library based on <ulink url="&url.yaz;">YAZ</ulink>.
197 <varlistentry><term><ulink url="&url.libxslt;">Libxslt</ulink></term>
199 <para>This is an XSLT processor - based on
200 <ulink url="&url.libxml2;">Libxml2</ulink>. Both Libxml2 and
201 Libxslt must be installed with the development components
202 (header files, etc.) as well as the run-time libraries.
206 <varlistentry><term><ulink url="&url.boost;">Boost</ulink></term>
209 The popular C++ library. Initial versions of Metaproxy
210 was built with 1.33.0. Version 1.33.1 works too.
217 In order to compile Metaproxy a modern C++ compiler is
218 required. Boost, in particular, requires the C++ compiler
219 to facilitate the newest features. Refer to Boost
220 <ulink url="&url.boost.compilers.status;">Compiler Status</ulink>
221 for more information.
224 We have successfully built Metaproxy using the compilers
225 <ulink url="&url.gcc;">GCC</ulink> version 4.0 and
226 <ulink url="&url.vstudio;">Microsoft Visual Studio</ulink> 2003/2005.
229 <section id="installation.unix">
230 <title>Installation on Unix (from Source)</title>
232 Here is a quick step-by-step guide on how to compile all the
233 tools that Metaproxy uses. Only few systems have none of the required
234 tools binary packages. If, for example, Libxml2/libxslt are already
235 installed as development packages use those (and omit compilation).
242 gunzip -c libxml2-version.tar.gz|tar xf -
250 gunzip -c libxslt-version.tar.gz|tar xf -
261 gunzip -c yaz-version.tar.gz|tar xf -
269 gunzip -c yazpp-version.tar.gz|tar xf -
280 gunzip -c boost-version.tar.gz|tar xf -
291 gunzip -c metaproxy-version.tar.gz|tar xf -
300 <section id="installation.debian">
301 <title>Installation on Debian GNU/Linux</title>
303 All dependencies for Metaproxy are available as
304 <ulink url="&url.debian;">Debian</ulink>
305 packages for the sarge (stable in 2005) and etch (testing in 2005)
309 The procedures for Debian based systems, such as
310 <ulink url="&url.ubuntu;">Ubuntu</ulink> is probably similar
313 There is currently no official Debian package for YAZ++.
314 And the Debian package for YAZ is probably too old.
315 Update the <filename>/etc/apt/sources.list</filename>
316 to include the Index Data repository.
317 See YAZ' <ulink url="&url.yaz.download.debian;">Download Debian</ulink>
318 for more information.
321 apt-get install libxslt1-dev
322 apt-get install libyazpp-dev
323 apt-get install libboost-dev
324 apt-get install libboost-thread-dev
325 apt-get install libboost-date-time-dev
326 apt-get install libboost-program-options-dev
327 apt-get install libboost-test-dev
330 With these packages installed, the usual configure + make
331 procedure can be used for Metaproxy as outlined in
332 <xref linkend="installation.unix"/>.
336 <section id="installation.windows">
337 <title>Installation on Windows</title>
339 Metaproxy can be compiled with Microsoft
340 <ulink url="&url.vstudio;">Visual Studio</ulink>.
341 Version 2003 (C 7.1) and 2005 (C 8.0) is known to work.
343 <section id="installation.windows.boost">
346 Get Boost from its <ulink url="&url.boost;">home page</ulink>.
347 You also need Boost Jam (an alternative to make).
348 That's also available from the Boost home page.
349 The files to be downloaded are called something like:
350 <filename>boost_1_33-1.exe</filename>
352 <filename>boost-jam-3.1.12-1-ntx86.zip</filename>.
353 Unpack Boost Jam first. Put <filename>bjam.exe</filename>
354 in your system path. Make a command prompt and ensure
355 it can be found automatically. If not check the PATH.
356 The Boost .exe is a self-extracting exe with
357 complete source for Boost. Compile that source with
358 Boost Jam (An alternative to Make).
359 The compilation takes a while.
360 For Visual Studio 2003, use
362 bjam "-sTOOLS=vc-7_1"
364 Here <literal>vc-7_1</literal> refers to a "Toolset" (compiler system).
365 For Visual Studio 2005, use
367 bjam "-sTOOLS=vc-8_0"
369 To install the libraries in a common place, use
371 bjam "-sTOOLS=vc-7_1" install
373 (or vc-8_0 for VS 2005).
376 By default, the Boost build process installs the resulting
377 libraries + header files in
378 <literal>\boost\lib</literal>, <literal>\boost\include</literal>.
381 For more information about installing Boost refer to the
382 <ulink url="&url.boost.getting.started;">getting started</ulink>
387 <section id="installation.windows.libxslt">
388 <title>Libxslt</title>
390 <ulink url="&url.libxslt;">Libxslt</ulink> can be downloaded
392 <ulink url="&url.libxml2.download.win32;">here</ulink>.
395 Libxslt has other dependencies, but these can all be downloaded
396 from the same site. Get the following:
397 iconv, zlib, libxml2, libxslt.
401 <section id="installation.windows.yaz">
404 <ulink url="&url.yaz;">YAZ</ulink> can be downloaded
406 <ulink url="&url.yaz.download.win32;">here</ulink>.
410 <section id="installation.windows.yazplusplus">
413 Get <ulink url="&url.yazplusplus;">YAZ++</ulink> as well.
414 Version 1.0 or later is required. For now get it from
416 <ulink url="&url.snapshot.download;">Snapshot area</ulink>.
419 YAZ++ includes NMAKE makefiles, similar to those found in the
424 <section id="installation.windows.metaproxy">
425 <title>Metaproxy</title>
427 Metaproxy is shipped with NMAKE makefiles as well - similar
428 to those found in the YAZ++/YAZ packages. Adjust this Makefile
429 to point to the proper locations of Boost, Libxslt, Libxml2,
430 zlib, iconv, yaz and yazpp.
434 <varlistentry><term><literal>DEBUG</literal></term>
436 If set to 1, the software is
437 compiled with debugging libraries (code generation is
438 multi-threaded debug DLL).
439 If set to 0, the software is compiled with release libraries
440 (code generation is multi-threaded DLL).
445 <term><literal>BOOST</literal></term>
448 Boost install location
454 <term><literal>BOOST_VERSION</literal></term>
457 Boost version (replace . with _).
463 <term><literal>BOOST_TOOLSET</literal></term>
472 <term><literal>LIBXSLT_DIR</literal>,
473 <literal>LIBXML2_DIR</literal> ..</term>
476 Specify the locations of Libxslt, libiconv, libxml2 and
485 After successful compilation you'll find
486 <literal>metaproxy.exe</literal> in the
487 <literal>bin</literal> directory.
495 <chapter id="architecture">
496 <title>The Metaproxy Architecture</title>
498 The Metaproxy architecture is based on three concepts:
499 the <emphasis>package</emphasis>,
500 the <emphasis>route</emphasis>
501 and the <emphasis>filter</emphasis>.
505 <term>Packages</term>
508 A package is request or response, encoded in some protocol,
509 issued by a client, making its way through Metaproxy, send to or
510 received from a server, or sent back to the client.
513 The core of a package is the protocol unit - for example, a
514 Z39.50 Init Request or Search Response, or an SRU searchRetrieve
515 URL or Explain Response. In addition to this core, a package
516 also carries some extra information added and used by Metaproxy
520 In general, packages are doctored as they pass through
521 Metaproxy. For example, when the proxy performs authentication
522 and authorization on a Z39.50 Init request, it removes the
523 authentication credentials from the package so that they are not
524 passed onto the back-end server; and when search-response
525 packages are obtained from multiple servers, they are merged
526 into a single unified package that makes its way back to the
535 Packages make their way through routes, which can be thought of
536 as programs that operate on the package data-type. Each
537 incoming package initially makes its way through a default
538 route, but may be switched to a different route based on various
539 considerations. Routes are made up of sequences of filters (see
548 Filters provide the individual instructions within a route, and
549 effect the necessary transformations on packages. A particular
550 configuration of Metaproxy is essentially a set of filters,
551 described by configuration details and arranged in order in one
552 or more routes. There are many kinds of filter - about a dozen
553 at the time of writing with more appearing all the time - each
554 performing a specific function and configured by different
558 The word ``filter'' is sometimes used rather loosely, in two
559 different ways: it may be used to mean a particular
560 <emphasis>type</emphasis> of filter, as when we speak of ``the
561 auth_simple filter'' or ``the multi filter''; or it may be used
562 to be a specific <emphasis>instance</emphasis> of a filter
563 within a Metaproxy configuration. For example, a single
564 configuration will often contain multiple instances of the
565 <literal>z3950_client</literal> filter. In
566 operational terms, of these is a separate filter. In practice,
567 context always make it clear which sense of the word ``filter''
571 Extensibility of Metaproxy is primarily through the creation of
572 plugins that provide new filters. The filter API is small and
573 conceptually simple, but there are many details to master. See
575 <link linkend="extensions">extensions</link>.
581 Since packages are created and handled by the system itself, and
582 routes are conceptually simple, most of the remainder of this
583 document concentrates on filters. After a brief overview of the
584 filter types follows, along with some thoughts on possible future
591 <chapter id="filters">
592 <title>Filters</title>
596 <title>Introductory notes</title>
598 It's useful to think of Metaproxy as an interpreter providing a small
599 number of primitives and operations, but operating on a very
600 complex data type, namely the ``package''.
603 A package represents a Z39.50 or SRU/W request (whether for Init,
604 Search, Scan, etc.) together with information about where it came
605 from. Packages are created by front-end filters such as
606 <literal>frontend_net</literal> (see below), which reads them from
607 the network; other front-end filters are possible. They then pass
608 along a route consisting of a sequence of filters, each of which
609 transforms the package and may also have side-effects such as
610 generating logging. Eventually, the route will yield a response,
611 which is sent back to the origin.
614 There are many kinds of filter: some that are defined statically
615 as part of Metaproxy, and others may be provided by third parties
616 and dynamically loaded. They all conform to the same simple API
617 of essentially two methods: <function>configure()</function> is
618 called at startup time, and is passed an XML DOM tree representing that
619 part of the configuration file that pertains to this filter
620 instance: it is expected to walk that tree extracting relevant
621 information; and <function>process()</function> is called every
622 time the filter has to processes a package.
625 While all filters provide the same API, there are different modes
626 of functionality. Some filters are sources: they create
628 (<literal>frontend_net</literal>);
629 others are sinks: they consume packages and return a result
630 (<literal>backend_test</literal>,
631 <literal>bounce</literal>,
632 <literal>http_file</literal>,
633 <literal>z3950_client</literal>);
634 the others are true filters, that read, process and pass on the
635 packages they are fed
636 (<literal>auth_simple</literal>,
637 <literal>log</literal>,
638 <literal>multi</literal>,
639 <literal>query_rewrite</literal>,
640 <literal>record_transform</literal>,
641 <literal>session_shared</literal>,
642 <literal>sru_z3950</literal>,
643 <literal>template</literal>,
644 <literal>virt_db</literal>).
649 <section id="overview.filter.types">
650 <title>Overview of filter types</title>
652 We now briefly consider each of the types of filter supported by
653 the core Metaproxy binary. This overview is intended to give a
654 flavor of the available functionality; more detailed information
655 about each type of filter is included below in
656 <link linkend="filterref"
657 >the reference guide to Metaproxy filters</link>.
660 The filters are here named by the string that is used as the
661 <literal>type</literal> attribute of a
662 <literal><filter></literal> element in the configuration
663 file to request them, with the name of the class that implements
664 them in parentheses. (The classname is not needed for normal
665 configuration and use of Metaproxy; it is useful only to
669 The filters are here listed in alphabetical order:
676 New virt_db-alike that does inteligent peer choice, explain merging,
677 adds FD&N to explain. Keeps init responses (like "virt_db Classic"),
678 makes routing choices based on local explain knowledge. Ref IDDI
681 Filter to convert Explain Classic to ZeeRex.
683 CQL2PQF (which needs augmented ZeeRex) - MARC for Talis.
687 Figure out what additional information we need in:
688 ZeeRex (check against D3.1)
689 Init request (e.g. loop detection)
690 Query package (e.g. number of hops)
691 Query response (e.g. record source)
696 <title><literal>auth_simple</literal>
697 (mp::filter::AuthSimple)</title>
699 Simple authentication and authorization. The configuration
700 specifies the name of a file that is the user register, which
701 lists <varname>username</varname>:<varname>password</varname>
702 pairs, one per line, colon separated. When a session begins, it
703 is rejected unless username and passsword are supplied, and match
704 a pair in the register. The configuration file may also specific
705 the name of another file that is the target register: this lists
706 lists <varname>username</varname>:<varname>dbname</varname>,<varname>dbname</varname>...
707 sets, one per line, with multiple database names separated by
708 commas. When a search is processed, it is rejected unless the
709 database to be searched is one of those listed as available to
715 <title><literal>backend_test</literal>
716 (mp::filter::Backend_test)</title>
718 A partial sink that provides dummy responses in the manner of the
719 <literal>yaz-ztest</literal> Z39.50 server. This is useful only
720 for testing. Seriously, you don't need this. Pretend you didn't
721 even read this section.
726 <title><literal>bounce</literal>
727 (mp::filter::Bounce)</title>
729 A sink that swallows <emphasis>all packages</emphasis>,
730 and returns them almost unprocessed.
731 It never sends any package of any type further down the row, but
732 sets Z39.50 packages to Z_Close, and HTTP_Request packages to
733 HTTP_Response err code 400 packages, and adds a suitable bounce
735 The bounce filter is usually added at end of each filter chain
736 config.xml to prevent infinite hanging of for example HTTP
737 requests packages when only the Z39.50 client partial sink
738 filter is found in the
744 <title><literal>frontend_net</literal>
745 (mp::filter::FrontendNet)</title>
747 A source that accepts Z39.50 connections from a port
748 specified in the configuration, reads protocol units, and
749 feeds them into the next filter in the route. When the result is
750 received, it is returned to the original origin.
755 <title><literal>http_file</literal>
756 (mp::filter::HttpFile)</title>
758 A partial sink which swallows only HTTP_Request packages, and
759 returns the contents of files from the local
760 filesystem in response to HTTP requests.
761 It lets Z39.50 packages and all other forthcoming package types
764 does mean that Metaproxy is also a Web-server in its spare time. So
765 far it does not contain either an email-reader or a Lisp
766 interpreter, but that day is surely coming.)
771 <title><literal>load_balance</literal>
772 (mp::filter::LoadBalance)</title>
774 Performs load balancing for incoming Z39.50 init requests.
775 It is used together with the <literal>virt_db</literal> filter,
776 but unlike the <literal>multi</literal> filter it does send an
777 entire session to only one of the virtual backends. The
778 <literal>load_balance</literal> filter is assuming that
779 all backend targets have equal content, and chooses the backend
780 with least load cost for a new session.
783 This filter is experimental and yet not mature for heavy load
791 <title><literal>log</literal>
792 (mp::filter::Log)</title>
794 Writes logging information to standard output, and passes on
795 the package unchanged. A log file name can be specified, as well
796 as multiple different logging formats.
801 <title><literal>multi</literal>
802 (mp::filter::Multi)</title>
804 Performs multi-database searching.
806 <link linkend="multidb">the extended discussion</link>
807 of virtual databases and multi-database searching below.
812 <title><literal>query_rewrite</literal>
813 (mp::filter::QueryRewrite)</title>
815 Rewrites Z39.50 Type-1 and Type-101 (``RPN'') queries by a
816 three-step process: the query is transliterated from Z39.50
817 packet structures into an XML representation; that XML
818 representation is transformed by an XSLT stylesheet; and the
819 resulting XML is transliterated back into the Z39.50 packet
826 <title><literal>record_transform</literal>
827 (mp::filter::RecordTransform)</title>
829 This filter acts only on Z3950 present requests, and let all
830 other types of packages and requests pass untouched. It's use is
831 twofold: blocking Z3950 present requests, which the backend
832 server does not understand and can not honor, and transforming
833 the present syntax and elementset name according to the rules
834 specified, to fetch only existing record formats, and transform
835 them on the fly to requested record syntaxes.
840 <title><literal>session_shared</literal>
841 (mp::filter::SessionShared)</title>
843 When this is finished, it will implement global sharing of
844 result sets (i.e. between threads and therefore between
845 clients), yielding performance improvements especially when
846 incoming requests are from a stateless environment such as a
847 web-server, in which the client process representing a session
848 might be any one of many. However:
852 This filter is not yet completed.
858 <title><literal>sru_z3950</literal>
859 (mp::filter::SRUtoZ3950)</title>
861 This filter transforms valid
862 SRU GET/POST/SOAP searchRetrieve requests to Z3950 init, search,
863 and present requests, and wraps the
864 received hit counts and XML records into suitable SRU response
866 The <literal>sru_z3950</literal> filter processes also SRU
867 GET/POST/SOAP explain requests, returning
868 either the absolute minimum required by the standard, or a full
869 pre-defined ZeeReX explain record.
871 <ulink url="&url.zeerex.explain;">ZeeReX Explain</ulink>
872 standard pages and the
873 <ulink url="&url.sru.explain;">SRU Explain</ulink> pages
874 for more information on the correct explain syntax.
875 SRU scan requests are not supported yet.
880 <title><literal>template</literal>
881 (mp::filter::Template)</title>
883 Does nothing at all, merely passing the packet on. (Maybe it
884 should be called <literal>nop</literal> or
885 <literal>passthrough</literal>?) This exists not to be used, but
886 to be copied - to become the skeleton of new filters as they are
887 written. As with <literal>backend_test</literal>, this is not
888 intended for civilians.
893 <title><literal>virt_db</literal>
894 (mp::filter::VirtualDB)</title>
896 Performs virtual database selection: based on the name of the
897 database in the search request, a server is selected, and its
898 address added to the request in a <literal>VAL_PROXY</literal>
899 otherInfo packet. It will subsequently be used by a
900 <literal>z3950_client</literal> filter.
902 <link linkend="multidb">the extended discussion</link>
903 of virtual databases and multi-database searching below.
908 <title><literal>z3950_client</literal>
909 (mp::filter::Z3950Client)</title>
911 A partial sink which swallows only Z39.50 packages.
912 It performs Z39.50 searching and retrieval by proxying the
913 packages that are passed to it. Init requests are sent to the
914 address specified in the <literal>VAL_PROXY</literal> otherInfo
915 attached to the request: this may have been specified by client,
916 or generated by a <literal>virt_db</literal> filter earlier in
917 the route. Subsequent requests are sent to the same address,
918 which is remembered at Init time in a Session object.
919 HTTP_Request packages and all other forthcoming package types
920 are passed untouched.
926 <title><literal>zeerex_explain</literal>
927 (mp::filter::ZeerexExplain)</title>
929 This filter acts as a sink for
930 Z39.50 explain requests, returning a static ZeeReX
931 Explain XML record from the config section. All other packages
934 <ulink url="&url.zeerex.explain;">ZeeReX Explain</ulink>
936 for more information on the correct explain syntax.
940 This filter is not yet completed.
949 <section id="future.directions">
950 <title>Future directions</title>
952 Some other filters that do not yet exist, but which would be
953 useful, are briefly described. These may be added in future
954 releases (or may be created by third parties, as loadable
960 <term><literal>frontend_cli</literal> (source)</term>
963 Command-line interface for generating requests.
968 <term><literal>sru_client</literal> (sink)</term>
971 SRU/GET and SRU/SOAP searching and retrieval.
976 <term><literal>opensearch_client</literal> (sink)</term>
979 A9 OpenSearch searching and retrieval.
989 <chapter id="configuration">
990 <title>Configuration: the Metaproxy configuration file format</title>
994 <title>Introductory notes</title>
996 If Metaproxy is an interpreter providing operations on packages, then
997 its configuration file can be thought of as a program for that
998 interpreter. Configuration is by means of a single XML file, the name
999 of which is supplied as the sole command-line argument to the
1000 <command>metaproxy</command> program. (See
1001 <link linkend="progref">the reference guide</link>
1002 below for more information on invoking Metaproxy.)
1006 <section id="overview.xml.structure">
1007 <title>Overview of the config file XML structure</title>
1009 All elements and attributes are in the namespace
1010 <ulink url="http://indexdata.com/metaproxy"/>.
1011 This is most easily achieved by setting the default namespace on
1012 the top-level element, as here:
1015 <metaproxy xmlns="http://indexdata.com/metaproxy" version="1.0">
1018 The top-level element is <metaproxy>. This contains a
1019 <start> element, a <filters> element and a
1020 <routes> element, in that order. <filters> is
1021 optional; the other two are mandatory. All three are
1025 The <start> element is empty, but carries a
1026 <literal>route</literal> attribute, whose value is the name of
1027 route at which to start running - analogous to the name of the
1028 start production in a formal grammar.
1031 If present, <filters> contains zero or more <filter>
1032 elements. Each filter carries a <literal>type</literal> attribute
1033 which specifies what kind of filter is being defined
1034 (<literal>frontend_net</literal>, <literal>log</literal>, etc.)
1035 and contain various elements that provide suitable configuration
1036 for a filter of its type. The filter-specific elements are
1038 <link linkend="filterref">the reference guide below</link>.
1039 Filters defined in this part of the file must carry an
1040 <literal>id</literal> attribute so that they can be referenced
1044 <routes> contains one or more <route> elements, each
1045 of which must carry an <literal>id</literal> element. One of the
1046 routes must have the ID value that was specified as the start
1047 route in the <start> element's <literal>route</literal>
1048 attribute. Each route contains zero or more <filter>
1049 elements. These are of two types. They may be empty, but carry a
1050 <literal>refid</literal> attribute whose value is the same as the
1051 <literal>id</literal> of a filter previously defined in the
1052 <filters> section. Alternatively, a route within a filter
1053 may omit the <literal>refid</literal> attribute, but contain
1054 configuration elements similar to those used for filters defined
1055 in the <filters> section. (In other words, each filter in a
1056 route may be included either by reference or by physical
1062 <section id="example.configuration">
1063 <title>An example configuration</title>
1065 The following is a small, but complete, Metaproxy configuration
1066 file (included in the distribution as
1067 <literal>metaproxy/etc/config1.xml</literal>).
1068 This file defines a very simple configuration that simply proxies
1069 to whatever back-end server the client requests, but logs each
1070 request and response. This can be useful for debugging complex
1071 client-server dialogues.
1073 <screen><![CDATA[<?xml version="1.0"?>
1074 <metaproxy xmlns="http://indexdata.com/metaproxy" version="1.0">
1075 <start route="start"/>
1077 <filter id="frontend" type="frontend_net">
1080 <filter id="backend" type="z3950_client">
1085 <filter refid="frontend"/>
1086 <filter type="log"/>
1087 <filter refid="backend"/>
1088 <filter type="bounce"/>
1094 It works by defining a single route, called
1095 <literal>start</literal>, which consists of a sequence of four
1096 filters. The first and last of these are included by reference:
1097 their <literal><filter></literal> elements have
1098 <literal>refid</literal> attributes that refer to filters defined
1099 within the prior <literal><filters></literal> section. The
1100 middle filter is included inline in the route.
1103 The four filters in the route are as follows: first, a
1104 <literal>frontend_net</literal> filter accepts Z39.50 requests
1105 from any host on port 9000; then these requests are passed through
1106 a <literal>log</literal> filter that emits a message for each
1107 request; they are then fed into a <literal>z3950_client</literal>
1108 filter, which forwards all Z39.50 requests to the client-specified
1109 back-end Z39.509 server. Those Z39.50 packages are returned by the
1110 <literal>z3950_client</literal> filter, with the response data
1111 filled by the external Z39.50 server targeted.
1112 All non-Z39.50 packages are passed through to the
1113 <literal>bounce</literal> filter, which definitely bounces
1114 everything, including fish, bananas, cold pyjamas,
1115 mutton, beef and trout packages.
1116 When the response arrives, it is handed
1117 back to the <literal>log</literal> filter, which emits another
1118 message; and then to the <literal>frontend_net</literal> filter,
1119 which returns the response to the client.
1122 <section id="checking.xml.syntax">
1123 <title>Config file syntax checking</title>
1125 The distribution contains RelaxNG Compact and XML syntax checking
1126 files, as well as XML Schema files. These are found in the
1129 xml/schema/metaproxy.rnc
1130 xml/schema/metaproxy.rng
1131 xml/schema/metaproxy.xsd
1133 and can be used to verify or debug the XML structure of
1134 configuration files. For example, using the utility
1135 <filename>xmllint</filename>, syntax checking is done like this:
1137 xmllint --noout --schema xml/schema/metaproxy.xsd etc/config-local.xml
1138 xmllint --noout --relaxng xml/schema/metaproxy.rng etc/config-local.xml
1140 (A recent version of <literal>libxml2</literal> is required, as
1141 support for XML Schemas is a relatively recent addition.)
1144 You can of course use any other RelaxNG or XML Schema compliant tool
1152 <chapter id="multidb">
1153 <title>Virtual databases and multi-database searching</title>
1157 <title>Introductory notes</title>
1159 Two of Metaproxy's filters are concerned with multiple-database
1160 operations. Of these, <literal>virt_db</literal> can work alone
1161 to control the routing of searches to one of a number of servers,
1162 while <literal>multi</literal> can work together with
1163 <literal>virt_db</literal> to perform multi-database searching, merging
1164 the results into a unified result-set - ``metasearch in a box''.
1167 The interaction between
1168 these two filters is necessarily complex: it reflects the real,
1169 irreducible complexity of multi-database searching in a protocol such
1170 as Z39.50 that separates initialization from searching, and in
1171 which the database to be searched is not known at initialization
1175 It's possible to use these filters without understanding the
1176 details of their functioning and the interaction between them; the
1177 next two sections of this chapter are ``HOW-TO'' guides for doing
1178 just that. However, debugging complex configurations will require
1179 a deeper understanding, which the last two sections of this
1180 chapters attempt to provide.
1185 <section id="multidb.virt_db">
1186 <title>Virtual databases with the <literal>virt_db</literal> filter</title>
1188 Working alone, the purpose of the
1189 <literal>virt_db</literal>
1190 filter is to route search requests to one of a selection of
1191 back-end databases. In this way, a single Z39.50 endpoint
1192 (running Metaproxy) can provide access to several different
1193 underlying services, including those that would otherwise be
1194 inaccessible due to firewalls. In many useful configurations, the
1195 back-end databases are local to the Metaproxy installation, but
1196 the software does not enforce this, and any valid Z39.50 servers
1197 may be used as back-ends.
1200 For example, a <literal>virt_db</literal>
1201 filter could be set up so that searches in the virtual database
1202 ``lc'' are forwarded to the Library of Congress bibliographic
1203 catalogue server, and searches in the virtual database ``marc''
1204 are forwarded to the toy database of MARC records that Index Data
1205 hosts for testing purposes. A <literal>virt_db</literal>
1206 configuration to make this switch would look like this:
1208 <screen><![CDATA[<filter type="virt_db">
1210 <database>lc</database>
1211 <target>z3950.loc.gov:7090/voyager</target>
1214 <database>marc</database>
1215 <target>indexdata.com/marc</target>
1217 </filter>]]></screen>
1219 As well as being useful in it own right, this filter also provides
1220 the foundation for multi-database searching.
1225 <section id="multidb.multi">
1226 <title>Multi-database search with the <literal>multi</literal> filter</title>
1228 To arrange for Metaproxy to broadcast searches to multiple back-end
1229 servers, the configuration needs to include two components: a
1230 <literal>virt_db</literal>
1231 filter that specifies multiple
1232 <literal><target></literal>
1233 elements, and a subsequent
1234 <literal>multi</literal>
1235 filter. Here, for example, is a complete configuration that
1236 broadcasts searches to both the Library of Congress catalogue and
1237 Index Data's tiny testing database of MARC records:
1239 <screen><![CDATA[<?xml version="1.0"?>
1240 <metaproxy xmlns="http://indexdata.com/metaproxy" version="1.0">
1241 <start route="start"/>
1244 <filter type="frontend_net">
1245 <threads>10</threads>
1248 <filter type="virt_db">
1250 <database>lc</database>
1251 <target>z3950.loc.gov:7090/voyager</target>
1254 <database>marc</database>
1255 <target>indexdata.com/marc</target>
1258 <database>all</database>
1259 <target>z3950.loc.gov:7090/voyager</target>
1260 <target>indexdata.com/marc</target>
1263 <filter type="multi"/>
1264 <filter type="z3950_client">
1265 <timeout>30</timeout>
1267 <filter type="bounce"/>
1270 </metaproxy>]]></screen>
1273 <literal>virt_db</literal>
1274 filter that specifies multiple
1275 <literal><target></literal>
1276 elements but without a subsequent
1277 <literal>multi</literal>
1278 filter yields surprising and undesirable results, as will be
1279 described below. Don't do that.)
1282 Metaproxy can be invoked with this configuration as follows:
1284 <screen>../src/metaproxy --config config-simple-multi.xml</screen>
1286 And thereafter, Z39.50 clients can connect to the running server
1287 (on port 9000, as specified in the configuration) and search in
1288 any of the databases
1289 <literal>lc</literal> (the Library of Congress catalogue),
1290 <literal>marc</literal> (Index Data's test database of MARC records)
1292 <literal>all</literal> (both of these). As an example, a session
1293 using the YAZ command-line client <literal>yaz-client</literal> is
1294 here included (edited for brevity and clarity):
1296 <screen><![CDATA[$ yaz-client @:9000
1300 Search was a success.
1301 Number of hits: 10000, setno 1
1305 Search was a success.
1306 Number of hits: 10, setno 3
1310 Search was a success.
1311 Number of hits: 10010, setno 4
1314 [marc]Record type: USmarc
1317 005 00000000000000.0
1318 008 910710c19910701nju 00010 eng
1322 100 10 $a Jack Collins
1323 245 10 $a How to program a computer
1329 [VOYAGER]Record type: USmarc
1331 005 20041229102447.0
1332 008 030910s2004 caua 000 0 eng
1333 035 $a (DLC) 2003112666
1334 906 $a 7 $b cbc $c orignew $d 4 $e epcn $f 20 $g y-gencatlg
1335 925 0 $a acquire $b 1 shelf copy $x policy default
1336 955 $a pc10 2003-09-10 $a pv12 2004-06-23 to SSCD; $h sj05 2004-11-30 $e sj05 2004-11-30 to Shelf.
1339 040 $a DLC $c DLC $d DLC
1340 050 00 $a MLCM 2004/03312 (G)
1341 245 10 $a 007, everything or nothing : $b Prima's official strategy guide / $c created by Kaizen Media Group.
1342 246 3 $a Double-O-seven, everything or nothing
1343 246 30 $a Prima's official strategy guide
1344 260 $a Roseville, CA : $b Prima Games, $c c2004.
1345 300 $a 161 p. : $b col. ill. ; $c 28 cm.
1346 500 $a "Platforms: Nintendo GameCube, Macintosh, PC, PlayStation 2 computer entertainment system, Xbox"--P. [4] of cover.
1347 650 0 $a Video games.
1348 710 2 $a Kaizen Media Group.
1349 856 42 $3 Publisher description $u http://www.loc.gov/catdir/description/random052/2003112666.html
1354 As can be seen, the first record in the result set is from the
1355 Index Data test database, and the second from the Library of
1356 Congress database. The result-set continues alternating records
1357 round-robin style until the point where one of the databases'
1358 records are exhausted.
1361 This example uses only two back-end databases; more may be used.
1362 There is no limitation imposed on the number of databases that may
1363 be metasearched in this way: issues of resource usage and
1364 administrative complexity dictate the practical limits.
1367 What happens when one of the databases doesn't respond? By default,
1368 the entire multi-database search fails, and the appropriate
1369 diagnostic is returned to the client. This is usually appropriate
1370 during development, when technicians need maximum information, but
1371 can be inconvenient in deployment, when users typically don't want
1372 to be bothered with problems of this kind and prefer just to get
1373 the records from the databases that are available. To obtain this
1374 latter behavior add an empty
1375 <literal><hideunavailable></literal>
1377 <literal>multi</literal> filter:
1379 <screen><![CDATA[ <filter type="multi">
1381 </filter>]]></screen>
1383 Under this regime, an error is reported to the client only if
1384 <emphasis>all</emphasis> the databases in a multi-database search
1390 <section id="multidb.what">
1391 <title>What's going on?</title>
1393 <title>Lark's vomit</title>
1395 This section goes into a level of technical detail that is
1396 probably not necessary in order to configure and use Metaproxy.
1397 It is provided only for those who like to know how things work.
1398 You should feel free to skip on to the next section if this one
1399 doesn't seem like fun.
1403 Hold on tight - this may get a little hairy.
1406 In the general course of things, a Z39.50 Init request may carry
1407 with it an otherInfo packet of type <literal>VAL_PROXY</literal>,
1408 whose value indicates the address of a Z39.50 server to which the
1409 ultimate connection is to be made. (This otherInfo packet is
1410 supported by YAZ-based Z39.50 clients and servers, but has not yet
1411 been ratified by the Maintenance Agency and so is not widely used
1412 in non-Index Data software. We're working on it.)
1413 The <literal>VAL_PROXY</literal> packet functions
1414 analogously to the absoluteURI-style Request-URI used with the GET
1415 method when a web browser asks a proxy to forward its request: see
1417 <ulink url="http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html#sec5.1.2"
1418 >Request-URI</ulink>
1420 <ulink url="http://www.w3.org/Protocols/rfc2616/rfc2616.html"
1421 >the HTTP 1.1 specification</ulink>.
1424 Within Metaproxy, Search requests that are part of the same
1425 session as an Init request that carries a
1426 <literal>VAL_PROXY</literal> otherInfo are also annotated with the
1427 same information. The role of the <literal>virt_db</literal>
1428 filter is to rewrite this otherInfo packet dependent on the
1429 virtual database that the client wants to search.
1432 When Metaproxy receives a Z39.50 Init request from a client, it
1433 doesn't immediately forward that request to the back-end server.
1434 Why not? Because it doesn't know <emphasis>which</emphasis>
1435 back-end server to forward it to until the client sends a Search
1436 request that specifies the database that it wants to search in.
1437 Instead, it just treasures the Init request up in its heart; and,
1438 later, the first time the client does a search on one of the
1439 specified virtual databases, a connection is forged to the
1440 appropriate server and the Init request is forwarded to it. If,
1441 later in the session, the same client searches in a different
1442 virtual database, then a connection is forged to the server that
1443 hosts it, and the same cached Init request is forwarded there,
1447 All of this clever Init-delaying is done by the
1448 <literal>frontend_net</literal> filter. The
1449 <literal>virt_db</literal> filter knows nothing about it; in
1450 fact, because the Init request that is received from the client
1451 doesn't get forwarded until a Search request is received, the
1452 <literal>virt_db</literal> filter (and the
1453 <literal>z3950_client</literal> filter behind it) doesn't even get
1454 invoked at Init time. The <emphasis>only</emphasis> thing that a
1455 <literal>virt_db</literal> filter ever does is rewrite the
1456 <literal>VAL_PROXY</literal> otherInfo in the requests that pass
1460 It is possible for a <literal>virt_db</literal> filter to contain
1462 <literal><target></literal>
1463 elements. What does this mean? Only that the filter will add
1464 multiple <literal>VAL_PROXY</literal> otherInfo packets to the
1465 Search requests that pass through it. That's because the virtual
1466 DB filter is dumb, and does exactly what it's told - no more, no
1468 If a Search request with multiple <literal>VAL_PROXY</literal>
1469 otherInfo packets reaches a <literal>z3950_client</literal>
1470 filter, this is an error. That filter doesn't know how to deal
1471 with multiple targets, so it will either just pick one and search
1472 in it, or (better) fail with an error message.
1475 The <literal>multi</literal> filter comes to the rescue! This is
1476 the only filter that knows how to deal with multiple
1477 <literal>VAL_PROXY</literal> otherInfo packets, and it does so by
1478 making multiple copies of the entire Search request: one for each
1479 <literal>VAL_PROXY</literal>. Each of these new copies is then
1480 passed down through the remaining filters in the route. (The
1481 copies are handled in parallel though the
1482 spawning of new threads.) Since the copies each have only one
1483 <literal>VAL_PROXY</literal> otherInfo, they can be handled by the
1484 <literal>z3950_client</literal> filter, which happily deals with
1485 each one individually. When the results of the individual
1486 searches come back up to the <literal>multi</literal> filter, it
1487 merges them into a single Search response, which is what
1488 eventually makes it back to the client.
1493 <section id="multidb.picture">
1494 <title>A picture is worth a thousand words (but only five hundred on 64-bit architectures)</title>
1498 <imagedata fileref="multi.pdf" format="PDF" scale="50"/>
1501 <imagedata fileref="multi.png" format="PNG"/>
1504 <!-- Fall back if none of the images can be used -->
1506 [Here there should be a diagram showing the progress of
1507 packages through the filters during a simple virtual-database
1508 search and a multi-database search, but is seems that your
1509 tool chain has not been able to include the diagram in this
1513 <!-- ### This used to work with an older version of DocBook
1515 <para>Caption: progress of packages through filters.</para>
1518 </inlinemediaobject>
1525 <chapter id="extensions">
1526 <title>Writing extensions for Metaproxy</title>
1527 <para>### To be written</para>
1533 <chapter id="classes">
1534 <title>Classes in the Metaproxy source code</title>
1538 <title>Introductory notes</title>
1540 <emphasis>Stop! Do not read this!</emphasis>
1541 You won't enjoy it at all. You should just skip ahead to
1542 <link linkend="refguide">the reference guide</link>,
1544 <!-- The remainder of this paragraph is lifted verbatim from
1545 Douglas Adams' _Hitch Hiker's Guide to the Galaxy_, chapter 8 -->
1546 you things you really need to know, like the fact that the
1547 fabulously beautiful planet Bethselamin is now so worried about
1548 the cumulative erosion by ten billion visiting tourists a year
1549 that any net imbalance between the amount you eat and the amount
1550 you excrete whilst on the planet is surgically removed from your
1551 bodyweight when you leave: so every time you go to the lavatory it
1552 is vitally important to get a receipt.
1555 This chapter contains documentation of the Metaproxy source code, and is
1556 of interest only to maintainers and developers. If you need to
1557 change Metaproxy's behavior or write a new filter, then you will most
1558 likely find this chapter helpful. Otherwise it's a waste of your
1559 good time. Seriously: go and watch a film or something.
1560 <citetitle>This is Spinal Tap</citetitle> is particularly good.
1563 Still here? OK, let's continue.
1566 In general, classes seem to be named big-endianly, so that
1567 <literal>FactoryFilter</literal> is not a filter that filters
1568 factories, but a factory that produces filters; and
1569 <literal>FactoryStatic</literal> is a factory for the statically
1570 registered filters (as opposed to those that are dynamically
1575 <section id="individual.classes">
1576 <title>Individual classes</title>
1578 The classes making up the Metaproxy application are here listed by
1579 class-name, with the names of the source files that define them in
1584 <title><literal>mp::FactoryFilter</literal>
1585 (<filename>factory_filter.cpp</filename>)</title>
1587 A factory class that exists primarily to provide the
1588 <literal>create()</literal> method, which takes the name of a
1589 filter class as its argument and returns a new filter of that
1590 type. To enable this, the factory must first be populated by
1591 calling <literal>add_creator()</literal> for static filters (this
1592 is done by the <literal>FactoryStatic</literal> class, see below)
1593 and <literal>add_creator_dyn()</literal> for filters loaded
1599 <title><literal>mp::FactoryStatic</literal>
1600 (<filename>factory_static.cpp</filename>)</title>
1602 A subclass of <literal>FactoryFilter</literal> which is
1603 responsible for registering all the statically defined filter
1604 types. It does this by knowing about all those filters'
1605 structures, which are listed in its constructor. Merely
1606 instantiating this class registers all the static classes. It is
1607 for the benefit of this class that <literal>struct
1608 metaproxy_1_filter_struct</literal> exists, and that all the filter
1609 classes provide a static object of that type.
1614 <title><literal>mp::filter::Base</literal>
1615 (<filename>filter.cpp</filename>)</title>
1617 The virtual base class of all filters. The filter API is, on the
1618 surface at least, extremely simple: two methods.
1619 <literal>configure()</literal> is passed an XML DOM tree representing
1620 that part of the configuration file that pertains to this filter
1621 instance, and is expected to walk that tree extracting relevant
1622 information. And <literal>process()</literal> processes a
1623 package (see below). That surface simplicity is a bit
1624 misleading, as <literal>process()</literal> needs to know a lot
1625 about the <literal>Package</literal> class in order to do
1631 <title><literal>mp::filter::AuthSimple</literal>,
1632 <literal>Backend_test</literal>, etc.
1633 (<filename>filter_auth_simple.cpp</filename>,
1634 <filename>filter_backend_test.cpp</filename>, etc.)</title>
1636 Individual filters. Each of these is implemented by a header and
1637 a source file, named <filename>filter_*.hpp</filename> and
1638 <filename>filter_*.cpp</filename> respectively. All the header
1639 files should be pretty much identical, in that they declare the
1640 class, including a private <literal>Rep</literal> class and a
1641 member pointer to it, and the two public methods.
1644 The source file for each filter needs to supply:
1649 A definition of the private <literal>Rep</literal> class.
1654 Some boilerplate constructors and destructors.
1659 A <literal>configure()</literal> method that uses the
1660 appropriate XML fragment.
1665 Most important, the <literal>process()</literal> method that
1666 does all the actual work.
1673 <title><literal>mp::Package</literal>
1674 (<filename>package.cpp</filename>)</title>
1676 Represents a package on its way through the series of filters
1677 that make up a route. This is essentially a Z39.50 or SRU APDU
1678 together with information about where it came from, which is
1679 modified as it passes through the various filters.
1684 <title><literal>mp::Pipe</literal>
1685 (<filename>pipe.cpp</filename>)</title>
1687 This class provides a compatibility layer so that we have an IPC
1688 mechanism that works the same under Unix and Windows. It's not
1689 particularly exciting.
1694 <title><literal>mp::RouterChain</literal>
1695 (<filename>router_chain.cpp</filename>)</title>
1702 <title><literal>mp::RouterFleXML</literal>
1703 (<filename>router_flexml.cpp</filename>)</title>
1710 <title><literal>mp::Session</literal>
1711 (<filename>session.cpp</filename>)</title>
1718 <title><literal>mp::ThreadPoolSocketObserver</literal>
1719 (<filename>thread_pool_observer.cpp</filename>)</title>
1726 <title><literal>mp::util</literal>
1727 (<filename>util.cpp</filename>)</title>
1729 A namespace of various small utility functions and classes,
1730 collected together for convenience. Most importantly, includes
1731 the <literal>mp::util::odr</literal> class, a wrapper for YAZ's
1737 <title><literal>mp::xml</literal>
1738 (<filename>xmlutil.cpp</filename>)</title>
1740 A namespace of various XML utility functions and classes,
1741 collected together for convenience.
1747 <section id="other.source.files">
1748 <title>Other Source Files</title>
1750 In addition to the Metaproxy source files that define the classes
1751 described above, there are a few additional files which are
1752 briefly described here:
1756 <term><literal>metaproxy_prog.cpp</literal></term>
1759 The main function of the <command>metaproxy</command> program.
1764 <term><literal>ex_router_flexml.cpp</literal></term>
1767 Identical to <literal>metaproxy_prog.cpp</literal>: it's not clear why.
1772 <term><literal>test_*.cpp</literal></term>
1775 Unit-tests for various modules.
1781 ### Still to be described:
1782 <literal>ex_filter_frontend_net.cpp</literal>,
1783 <literal>filter_dl.cpp</literal>,
1784 <literal>plainfile.cpp</literal>,
1785 <literal>tstdl.cpp</literal>.
1792 <reference id="refguide">
1793 <title>Reference guide</title>
1795 The material in this chapter is drawn directly from the individual
1796 manual entries. In particular, the Metaproxy invocation section is
1797 available using <command>man metaproxy</command>, and the section
1798 on each individual filter is available using the name of the filter
1799 as the argument to the <command>man</command> command.
1805 <!-- Keep this comment at the end of the file
1810 sgml-minimize-attributes:nil
1811 sgml-always-quote-attributes:t
1814 sgml-parent-document: nil
1815 sgml-local-catalogs: nil
1816 sgml-namecase-general:t