From: Mike Taylor Date: Thu, 2 Feb 2006 18:21:39 +0000 (+0000) Subject: None of these text-files are needed any longer, as they have been X-Git-Tag: METAPROXY.0.9~10 X-Git-Url: http://sru.miketaylor.org.uk/cgi-bin?a=commitdiff_plain;h=0da777d2f0c621766d4a6ad3c299e9a5b9fcfd45;p=metaproxy-moved-to-github.git None of these text-files are needed any longer, as they have been merged, consolidated and upgraded into DocBook source. --- diff --git a/doc/README b/doc/README deleted file mode 100644 index d4f4af2..0000000 --- a/doc/README +++ /dev/null @@ -1,17 +0,0 @@ -$Id: README,v 1.2 2006-01-17 10:35:12 mike Exp $ - -This directory contains documentation generated by human hands. In -contrast, ../dox contains documentation automatically generated by -doxygen from the API declarations. - -Eventually, everything in this directory will become an integrated YP2 -manual written in DocBook, like the YAZ and Zebra manuals. For now, -though, we have separate files, which might perhaps best be read in -the specified order: - -README -- this file. -filters -- lists and briefly describes the various filters. -config -- describes the YP2 configuration file. -multidb -- virtual databases and multi-db searching (complicated!) -classes -- describes some of the classes in the YP2 source code. - diff --git a/doc/classes b/doc/classes deleted file mode 100644 index 5c99ce2..0000000 --- a/doc/classes +++ /dev/null @@ -1,145 +0,0 @@ -$Id: classes,v 1.4 2006-01-17 09:26:44 mike Exp $ - - -A Hitch-Hiker's Guide to the YP2 Classes -======================================== - -Introductory Notes ------------------- - -In general, classes seem to be named big-endianly, so that -"FactoryFilter" is not a filter for factories, but a factory that -produces filters, and "FactoryStatic" is a factory for the statically -registered filters (as opposed to those that are dynamically loaded). - - -Classes -======= - -yp::FactoryFilter ------------------ - -A factory class that exists primarily to provide the create() method, -which takes the name of a filter class as its argument and returns a -new filter of that type. To enable this, the factory must first be -populated by calling add_creator() for static filters (this is done by -the FactoryStatic class, see below) and add_creator_dyn() for filters -loaded dynamically. - -yp2::FactoryStatic ------------------- - -A subclass of FactoryFilter which is responsible for registering all -the statically defined filter types. It does this by knowing about -all those filters' structures, which are listed in its constructor. -Merely instantiating this class registers all the static classes. It -is for the benefit of this class that struct yp2_filter_struct exists, -and that all the filter classes provide a static object of that type. - -yp2::filter::Base ------------------ - -The virtual base class of all filters. The filter API is, on the -surface at least, extremely simple: two methods. configure() is -passed a DOM tree representing that part of the configuration file -that pertains to this filter instance, and is expected to walk that -tree extracting relevant information. And process() processes a -Package (see below). That surface simplicitly is a bit misleading, as -process() needs to know a lot about the Package class in order to do -anything useful. - -yp2::filter::AuthSimple, Backend_test, FrontendNet, etc. --------------------------------------------------------- - -Individual filters. Each of these is implemented by a header and a -source file, named filter_*.hpp and filter_*.cpp respectively. All -the header files should be pretty much identical, in that they declare -the class, including a private Rep class and a member pointer to it, -and the two public methods. The only extra information in any filter -header is additional private types and members (the latter should -really all be in the Rep anyway). - -The source file for each filter needs to supply: -- a definition of the private Rep class -- some boilerplate constructors and destructors -- a configure() method that uses the appropriate XML fragment -- most important, the prcoess() method that does all the actual work. - -In a better world, each of the filters (nine so far) would be -documented separately. Very, very briefly, though: -- auth_simple: authentication against a plain-text file -- backend_test: trivial ztest-like dummy hardwired back-end -- frontend_net: receives incoming Z39.50 and SRW PSUs -- log: writes log messages -- multi: fans out searches to multiple back-ends -- session_shared: global sharing of cached result-sets -- template: NOT TO BE USED: just a template source file for new filters -- virt_db: chooses which server/db to use based on database name -- z3950_client: forwards PDUs to a nominated Z39.50 server - -### What is "filter_dl.cpp"? It's to do with dynamic loading, but in -what way and to what extent does it function as a filter? - -See the file "filters" for more information. - - -yp2::Package ------------- - -Represents a package on its way through the series of filters that -make up a route. This is essentially a Z39.50 or SRU APDU together -with information about where it came from, which is modified as it -passes through the various filters. - - -yp2::Pipe ---------- - -This class provides a compatibility layer so that we have an IPC -mechanism that works the same under Unix and Windows. It's not -particularly exciting. - - -yp2::Router and yp2::RouterChain --------------------------------- - -### - - -yp2::RouterFleXML ------------------ - -### - - -yp2::Session ------------- - -### - -yp2::ThreadPoolSocketObserver ------------------------------ - -### - -yp2::util ---------- - -A namespace of various small utility functions and classes, collected -together for convenience. Most importantly, includes the -yp2::util::odr class, a wrapper for YAZ's ODR facilities. - - -yp2::xml ---------- - -A namespace of various XML utility functions and classes, collected -together for convenience. - - -Other Source Files -================== - -ex_router_flexml.cpp -- the main function of the proxy. -yp2_prog.cpp -- identical to the above: it's not clear why. -test_*.cpp -- unit-tests for various modules. diff --git a/doc/config b/doc/config deleted file mode 100644 index 8c924b4..0000000 --- a/doc/config +++ /dev/null @@ -1,113 +0,0 @@ -$Id: config,v 1.3 2006-01-18 10:29:08 mike Exp $ - -The Guerilla Guide to Configuring YP2 -===================================== - -Introduction: invocation, schema and namespace ----------------------------------------------- - -YP2 configuration is by means of a single XML file, the name of which -is supplied as the sole command-line argument to the "yp2" binary. -This document briefly describes the format of that configuration file. - -An XML schema, config.xsd, is provided for validating configuration -files. It can be used by (among other tools) the "xmllint" program -supplied as part of the libxml2 distribution: - - xmllint --noout --schema config.xsd config1.xml - -(A recent version of libxml2 is required, as support for XML Schemas -is a relatively recent addition.) - -All elements and attributes are in the namespace - http://indexdata.dk/yp2/config/1 -This is most easily achieved by setting the default namespace on the -top-level element, as here: - - - - -Overview of XML structure -------------------------- - -The top-level element is . This contains a element, a - element and a element, in that order. is -optional; the other two are mandatory. All three are non-repeatable. - - is empty, but carries a "route" attribute, whose value is the -name of route at which to start running -- a bit like the name of the -start production in a grammar. - -If present, contains zero or more elements; filters -carry a "type" attribute and contain various elements that provide -suitable configuration for filters of that type. The filter-specific -elements are described below. Filters defined in this part of the -file must carry an "id" attribute so that they can be referenced from -elsewhere. - - contains zero or more elements, each of which must -carry an "id" element. One of the routes must have the id that was -specified for the start route in the element's "route" -attribute. Each route contains zero or more elements. These -are of two types. They may be empty, but carry a "refid" attribute -whose value is the same as the "id" of a filter previously defined in -the section. Alternatively, a route within a filter may -omit the "refid" attribute, but contain configuration elements similar -to those used for filters defined in the section. - - -Filter configuration --------------------- - -All elements have in common that they must carry a "type" -attribute whose value is one of the supported ones, listed in the -schema file and discussed below. In additional, s occurring -the section must have an "id" attribute, and those occurring -within a route must have either a "refid" attribute referencing a -previously defined filter or contain its own configuration -information. - -In general, each filter recognises different configuration elements -within its element, as each filter has different functionality. These -are as follows: - - - 10 - @:9000 - - - - 30 - - - - B - - - - - loc - z3950.loc.gov:7090/voyager - - - idgils - indexdata.dk/gils - - - - - ../etc/example.simple-auth - - - - - - ### Not yet resolved - - - - ### Not yet defined - - - - diff --git a/doc/filters b/doc/filters deleted file mode 100644 index c4ce443..0000000 --- a/doc/filters +++ /dev/null @@ -1,115 +0,0 @@ -$Id: filters,v 1.1 2006-01-17 10:43:14 mike Exp $ - - -YP2 Filter Classes for Superhuman Geniuses -========================================== - -Introductory Notes ------------------- - -It's useful to think of a YP2 configuration file as a program (written -in XML, but that's an implementation detail) providing a small number -of primitives and operations, but operating on a very complex data -type, namely the Package. A package represents a Z39.50 or SRW/U -request (whether for Init, Search, Scan, etc.) together with -information about where it came from. Packages come from front-end -filters such as frontend_end, which reads them from the network; other -front-end filters are possible. They then pass along a route -consisting of a sequence of filters, each of which transforms the -package and may also have side-effects such as generating logging. -Eventually, the route will yield a response, which is sent back to the -origin. - -There are many kinds of filter: some that are defined statically as -part of YP2, and other that may be provided by third parties and -dynamically loaded. They all conform to the same simple API of -essentially two methods: configure() is passed a DOM tree representing -that part of the configuration file that pertains to this filter -instance, and is expected to walk that tree extracting relevant -information; and process() processes a Package. - -While all filters provide the same API, there are different modes of -functionality. Some filters are sources -- they create packages -(frontend_net); others are sinks -- they consume packages and return a -result (z3950_client, backend_test); the others are true filters, that -read process and pass on the packages they are fed (auth_simple, log, -multi, session_shared, template, virt_db). - - -Filters -------- - -The filters are here named by the string that is used as the "type" -attribute of a element in the configuration file to request -them, with the class that implements them in parentheses. - -auth_simple (yp2::filter::AuthSimple) - - Simple authentication and authorisation. The configuration - specifies the name of a file that is the user register, which - lists username:password pairs, one per line, colon separated. - When a session begins, it is rejected unless username and - passsword are supplied, and match a pair in the register. - -backend_test (yp2::filter::Backend_test) - - A sink that provides dummy responses in the manner of the - "yaz-ztest" Z39.50 server. This is useful only for testing. - -frontend_net (yp2::filter::FrontendNet) - - A source that accepts Z39.50 and SRW connections from a port - specified in the configuration, reads protocol units, and - feeds them into the next filter, eventually returning the - result to the origin. - -log (yp2::filter::Log) - - Writes logging information to standard output, and passes on - the package unchanged. - -multi (yp2::filter::Multi) - - Performs multicast searching. See the extended discussion - in the file "multidb". - -session_shared (yp2::filter::SessionShared) - - When this is finished, it will implement global sharing of - result sets (i.e. between threads and therefore between - clients), but it's not yet done. - -template (yp2::filter::Template) - - Does nothing at all, merely passing the packet on. (Maybe it - should be called "nop" or "passthrough"?) This exists not to - be used, but to be copied -- to become the skeleton of new - filters as they are written. - -virt_db (yp2::filter::Virt_db) - - Performs virtual database selection. See the extended - discussion in the file "multidb". - -z3950_client (yp2::filter::Z3950Client) - - Performs Z39.50 searching and retrieval by proxying the - packages that are passed to it. Init requests are sent to the - address specified in the VAL_PROXY otherInfo attached to the - request: this may have been specified by client, or generated - by a virt_db filter earlier in the route. Subsequent requests - are sent to the same address, which is remembered at Init time - in a Session object. - - -Directions ----------- - -Some other filters that do not yet exist but would be useful: -frontend_cli (source) - command-line interface for generating requests. -srw2z3950 (filter) - translate SRW requests into Z39.50 requests. -srw_client (sink) - performs SRW searching and retrieval. -sru_client (sink) - performs SRU searching and retrieval. -opensearch_client (sink) - A9 OpenSearch searching and retrieval. - - diff --git a/doc/multidb b/doc/multidb deleted file mode 100644 index a70fc78..0000000 --- a/doc/multidb +++ /dev/null @@ -1,18 +0,0 @@ -$Id: multidb,v 1.1 2006-01-17 11:10:30 mike Exp $ - - -YP2 Virtual Databases and Multicast Searching -============================================= - -Two of YP2's filters are concerned with multiple-database operations. -Of these, "virt_db" can work alone to control the routing of searches -to one of a number of servers, while "multi" can work with the output -of "virt_db" to perform multicast searching, merging the results into -a unified result-set. The interaction between these two filters is -necessarily complex, reflecting the real complexity of multicast -searching in a protocol such as Z39.50 that separates initialisation -from searching, with the database to search known only during the -latter operation. - -### Much, much more to say! -