1 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook V4.4//EN"
2 "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd" [
3 <!ENTITY copyright SYSTEM "copyright.xml">
4 <!ENTITY % idcommon SYSTEM "common/common.ent">
7 <refentry id="ref-zoom">
9 <productname>Metaproxy</productname>
10 <info><orgname>Index Data</orgname></info>
14 <refentrytitle>zoom</refentrytitle>
15 <manvolnum>3mp</manvolnum>
16 <refmiscinfo class="manual">Metaproxy Module</refmiscinfo>
20 <refname>zoom</refname>
21 <refpurpose>Metaproxy ZOOM Module</refpurpose>
25 <title>DESCRIPTION</title>
27 This filter implements a generic client based on
28 <ulink url="&url.yaz.zoom;">ZOOM</ulink> of YAZ.
29 The client implements the protocols that ZOOM C does: Z39.50, SRU
30 (GET, POST, SOAP) and SOLR .
34 This filter only deals with Z39.50 on input. The following services
35 are supported: init, search, present and close. The backend target
36 is selected based on the database as part search and
37 <emphasis>not</emphasis> as part of init.
41 This filter is an alternative to the z3950_client filter but also
42 shares properties of the virt_db - in that the target is selected
43 for a specific database
47 The ZOOM filter relies on a target profile description, which is
48 XML based. It picks the profile for a given database from a web service
49 or it may be locally given for each unique database (AKA virtual database
50 in virt_db). Target profiles are directly and indrectly given as part
51 of the <literal>torus</literal> element in the configuration.
57 <title>CONFIGURATION</title>
59 The configuration consists of five parts: <literal>torus</literal>,
60 <literal>fieldmap</literal>, <literal>cclmap</literal>,
61 <literal>contentProxy</literal> and <literal>log</literal>.
66 The <literal>torus</literal> element specifies target profiles
67 and takes the following content:
71 <term>attribute <literal>url</literal></term>
74 URL of Web service to be used when fetch target profiles from
75 a remote service (Torus normally).
78 The sequence <literal>%query</literal> is replaced with a CQL
79 query for the Torus search.
82 The special sequence <literal>%realm</literal> is replaced by value
83 of attribute <literal>realm</literal> or by realm DATABASE argument.
86 The special sequence <literal>%db</literal> is replaced with
87 a single database while searching. Note that this sequence
88 is no longer needed, because the <literal>%query</literal> can already
89 query for a single database by using CQL query
90 <literal>udb==...</literal>.
95 <term>attribute <literal>content_url</literal></term>
98 URL of Web service to be used to fetch target profile
99 for a given database (udb) of type content. Semantics otherwise like
100 <literal>url</literal> attribute above.
105 <term>attribute <literal>realm</literal></term>
108 The default realm value. Used for %realm in URL, unless
109 specified in DATABASE argument.
114 <term>attribute <literal>proxy</literal></term>
117 HTTP proxy to bse used for fetching target profiles.
122 <term>attribute <literal>xsldir</literal></term>
125 Directory that is searched for XSL stylesheets. Stylesheets
126 are specified in the target profile by the
127 <literal>transform</literal> element.
132 <term>attribute <literal>element_transform</literal></term>
135 Specifies the element that triggers retrieval and transform using
136 the parameters elementSet, recordEncoding, requestSyntax, transform
137 from the target profile. Default value
138 is "pz2", due to the fact that for historical reasons the
139 common format is that used in Pazpar2.
144 <term>attribute <literal>element_raw</literal></term>
147 Specifies an element that triggers retrieval using the
148 parameters elementSet, recordEncoding, requestSyntax from the
149 target profile. Same actions as for element_transform, but without
150 the XSL transform. Useful for debugging.
151 The default value is "raw".
156 <term>attribute <literal>explain_xsl</literal></term>
159 Specifies a stylesheet that converts one or more Torus records
160 to ZeeExplain records. The content of recordData is assumed to be
161 holding each Explain record.
166 <term>element <literal>records</literal></term>
169 Local target profiles. This element may includes zero or
170 more <literal>record</literal> elements (one per target
171 profile). See section TARGET PROFILE.
177 <refsect2 id="fieldmap">
178 <title>fieldmap</title>
180 The <literal>fieldmap</literal> may be specified zero or more times and
181 specifies the map from CQL fields to CCL fields and takes the
186 <term>attribute <literal>cql</literal></term>
189 CQL field that we are mapping "from".
194 <term>attribute <literal>ccl</literal></term>
197 CCL field that we are mapping "to".
203 <refsect2 id="cclmap_base">
204 <title>cclmap</title>
206 The third part of the configuration consists of zero or more
207 <literal>cclmap</literal> elements that specifies
208 <emphasis>base</emphasis> CCL profile to be used for all targets.
209 This configuration, thus, will be combined with cclmap-definitions
210 from the target profile.
214 <title>contentProxy</title>
216 The <literal>contentProxy</literal> element controls content proxy'in.
218 is optional and must only be defined if content proxy'ing is enabled.
222 <term>attribute <literal>server</literal></term>
225 Specifies the content proxy host. The host is of the form
226 host[:port]. That is without a method (such as HTTP) and optional
232 <term>attribute <literal>tmp_file</literal></term>
235 Specifies a filename of a session file for content proxy'ing. The
236 file should be an absolute filename that includes
237 <literal>XXXXXX</literal> which is replaced by a unique filename
238 using the mkstemp(3) system call. The default value of this
239 setting is <literal>/tmp/cf.XXXXXX.p</literal>.
248 The <literal>log</literal> element controls logging for the
253 <term>attribute <literal>apdu</literal></term>
256 If the value of apdu is "true", then protocol packages
257 (APDUs and HTTP packages) from the ZOOM filter will be
258 logged to the yaz_log system. A value of "false" will
259 not perform logging of protocol packages (the default
268 <title>QUERY HANDLING</title>
270 The ZOOM filter accepts three query types: RPN(Type-1), CCL and
274 Queries are converted in two separate steps. In the first step
275 the input query is converted to RPN/Type-1. This is always
276 the common internal format between step 1 and step 2.
277 In step 2 the query is converted to the native query type of the target.
280 Step 1: for RPN, the query is passed un-modified to the target.
283 Step 1: for CCL, the query is converted to RPN via
284 <link linkend="cclmap"><literal>cclmap</literal></link> elements part of
285 the target profile as well as
286 <link linkend="cclmap_base">base CCL maps</link>.
289 Step 1: For CQL, the query is converted to CCL. The mappings of
290 CQL fields to CCL fields are handled by
291 <link linkend="fieldmap"><literal>fieldmap</literal></link>
292 elements as part of the target profile. The resulting query, CCL,
293 is the converted to RPN using the schema mentioned earlier (via
294 <literal>cclmap</literal>).
297 Step 2: If the target is Z39.50-based, it is passed verbatim (RPN).
298 If the target is SRU-based, the RPN will be converted to CQL.
299 If the target is SOLR-based, the RPN will be converted to SOLR's query
305 <title>SORTING</title>
307 The ZOOM module actively handle CQL sorting - using the SORTBY parameter
308 which was introduced in SRU version 1.2. The conversion from SORTBY clause
309 to native sort for some target is driven by the two parameters:
310 <link linkend="sortStrategy"><literal>sortStrategy</literal></link>
311 and <link linkend="sortmap"><literal>sortmap_</literal><replaceable>field</replaceable></link>.
314 If a sort field that does not have an equivalent
315 <literal>sortmap_</literal>-mapping is passed un-modified through the
316 conversion. It doesn't throw a diagnostic.
321 <title>TARGET PROFILE</title>
323 The ZOOM module is driven by a number of settings that specifies how
324 to handle each target.
325 Note that unknown elements are silently <emphasis>ignored</emphasis>.
328 The elements, in alphabetical order, are:
332 <term id="zoom-torus-authentication">authentication</term><listitem>
334 Authentication parameters to be sent to the target. For
335 Z39.50 targets, this will be sent as part of the
336 Init Request. Authentication consists of two components: username
337 and password, separated by a slash.
340 If this value is omitted or empty no authentication information is sent.
345 <varlistentry id="cclmap">
346 <term>cclmap_<replaceable>field</replaceable></term><listitem>
348 This value specifies CCL field (qualifier) definition for some
349 field. For Z39.50 targets this most likely will specify the
350 mapping to a numeric use attribute + a structure attribute.
351 For SRU targets, the use attribute should be string based, in
352 order to make the RPN to CQL conversion work properly (step 2).
358 <term>cfAuth</term><listitem>
360 When cfAuth is defined, its value will be used as authentication
361 to backend target and authentication setting will be specified
362 as part of a database. This is like a "proxy" for authentication and
363 is used for Connector Framework based targets.
369 <term id="zoom-torus-cfproxy">cfProxy</term><listitem>
371 Specifies HTTP proxy for the target in the form
372 <replaceable>host</replaceable>:<replaceable>port</replaceable>.
378 <term>cfSubDB</term><listitem>
380 Specifies sub database for a Connector Framework based target.
385 <varlistentry id="zoom-torus-contentConnector">
386 <term>contentConnector</term><listitem>
388 Specifies a database for content-based proxy'ing.
394 <term>elementSet</term><listitem>
396 Specifies the elementSet to be sent to the target if record
397 transform is enabled (not to be confused' with the record_transform
398 module). The record transform is enabled only if the client uses
399 record syntax = XML and a element set determined by
400 the <literal>element_transform</literal> /
401 <literal>element_raw</literal> from the configuration.
402 By default that is the element sets <literal>pz2</literal>
403 and <literal>raw</literal>.
404 If record transform is not enabled, this setting is
405 not used and the element set specified by the client
412 <term>literalTransform</term><listitem>
414 Specifies a XSL stylesheet to be used if record
415 transform is anabled; see description of elementSet.
416 The XSL transform is only used if the element set is set to the
417 value of <literal>element_transform</literal> in the configuration.
420 The value of literalTransform is the XSL - string encoded.
426 <term>piggyback</term><listitem>
428 A value of 1/true is a hint to the ZOOM module that this Z39.50
429 target supports piggyback searches, ie Search Response with
430 records. Any other value (false) will prevent the ZOOM module
431 to make use of piggyback (all records part of Present Response).
437 <term>queryEncoding</term><listitem>
439 If this value is defined, all queries will be converted
440 to this encoding. This should be used for all Z39.50 targets that
441 do not use UTF-8 for query terms.
447 <term>recordEncoding</term><listitem>
449 Specifies the character encoding of records that are returned
450 by the target. This is primarily used for targets were records
451 are not UTF-8 encoded already. This setting is only used
452 if the record transform is enabled (see description of elementSet).
458 <term>requestSyntax</term><listitem>
460 Specifies the record syntax to be specified for the target
461 if record transform is enabled; see description of elementSet.
462 If record transform is not enabled, the record syntax of the
463 client is passed verbatim to the target.
468 <varlistentry id="sortmap">
469 <term>sortmap_<replaceable>field</replaceable></term><listitem>
471 This value the native field for a target. The form of the value is
472 given by <link linkend="sortStrategy">sortStrategy</link>.
477 <varlistentry id="sortStrategy">
478 <term>sortStrategy</term><listitem>
480 Specifies sort strategy for a target. One of:
481 <literal>z3950</literal>, <literal>type7</literal>,
482 <literal>cql</literal>, <literal>sru11</literal> or
483 <literal>embed</literal>. The <literal>embed</literal> chooses type-7
484 or CQL sortby depending on whether Type-1 or CQL is
485 actually sent to the target.
491 <term>sru</term><listitem>
493 If this setting is set, it specifies that the target is web service
494 based and must be one of : <literal>get</literal>,
495 <literal>post</literal>, <literal>soap</literal>
496 or <literal>solr</literal>.
502 <term>sruVersion</term><listitem>
504 Specifies the SRU version to use. It unset, version 1.2 will be
505 used. Some servers do not support this version, in which case
506 version 1.1 or even 1.0 could be set it.
512 <term>transform</term><listitem>
514 Specifies a XSL stylesheet filename to be used if record
515 transform is anabled; see description of elementSet.
516 The XSL transform is only used if the element set is set to the
517 value of <literal>element_transform</literal> in the configuration.
523 <term>udb</term><listitem>
525 This value is required and specifies the unique database for
526 this profile . All target profiles should hold a unique database.
531 <varlistentry id="urlRecipe">
532 <term>urlRecipe</term><listitem>
534 The value of this field is a string that generates a dynamic link
535 based on record content. If the resulting string is non-zero in length
536 a new field, <literal>metadata</literal> with attribute
537 <literal>type="generated-url"</literal> is generated.
538 The contents of this field is the result of the URL recipe conversion.
539 The urlRecipe value may refer to an existing metadata element by
540 ${field[pattern/result/flags]}, which will take content
541 of field and perform a regular expression conversion using the pattern
542 given. For example: <literal>${md-title[\s+/+/g]}</literal> takes
543 metadata element <literal>title</literal> and converts one or more
544 spaces to a plus character.
547 If the <link linkend="zoom-torus-contentConnector">contentConnector</link>
548 setting also defined, the resulting value is
549 augmented with a session string as well as host name of the
550 content proxy server.
556 <term>zurl</term><listitem>
558 This is setting is mandatory and specifies the ZURL of the
559 target in the form of host/database. The HTTP method should
560 not be provided as this is guessed from the "sru" attribute value.
567 <title>DATABASE parameters</title>
569 Extra information may be carried in the Z39.50 Database or SRU path,
570 such as authentication to be passed to backend etc. Some of
571 the parameters override TARGET profile values. The format is
574 udb,parm1=value1&parm2=value2&...
577 Where udb is the unique database recognised by the backend and parm1,
578 value1, .. are parameters to be passed. The following describes the
579 supported parameters. Like form values in HTTP the parameters and
580 values are URL encoded. The separator, though, between udb and parameters
581 is a comma rather than a question mark. What follows question mark are
582 HTTP arguments (in this case SRU arguments).
589 Specifies user to be passed to backend. If this parameter is
590 omitted, the user will be taken from TARGET profile setting
591 <link linkend="zoom-torus-authentication">
592 <literal>authentication</literal>
599 <term>password</term>
602 Specifies password to be passed to backend. If this parameters is
603 omitted, the password will be taken from TARGET profile setting
604 <link linkend="zoom-torus-authentication">
605 <literal>authentication</literal>
615 Specifies one or more proxies for backend. If this parameter is
616 omitted, the proxy will be taken from TARGET profile setting
617 <link linkend="zoom-torus-cfproxy">
618 <literal>cfProxy</literal></link>.
619 The parameter is a list of comma-separated host:port entries.
620 Bost host and port must be given for each proxy.
625 <term>cproxysession</term>
628 Session ID for content proxy. This parameter is, generally,
629 not used by anything but the content proxy itself.
634 <term>nocproxy</term>
637 If this parameter is specified, content-proyxing is disabled
646 Session realm to be used for this target, changed the resulting
647 URL to be used for getting a target profile, by changing the
648 value that gets substituted for the %realm string.
656 All parameters that has prefix x, dash are passed verbatim
664 <title>SCHEMA</title>
665 <literallayout><xi:include
666 xi:href="../xml/schema/filter_zoom.rnc"
668 xmlns:xi="http://www.w3.org/2001/XInclude" />
673 <title>EXAMPLES</title>
675 The following configuration illustrates most of the
680 url="http://torus.indexdata.com/src/records/?query=udb%3D%db"
681 proxy="localhost:3128"
683 <fieldmap cql="cql.anywhere"/>
684 <fieldmap cql="cql.serverChoice"/>
685 <fieldmap cql="dc.creator" ccl="au"/>
686 <fieldmap cql="dc.title" ccl="ti"/>
687 <fieldmap cql="dc.subject" ccl="su"/>
691 <attr type="u" value="12"/>
692 <attr type="s" value="107"/>
704 <title>SEE ALSO</title>
707 <refentrytitle>metaproxy</refentrytitle>
708 <manvolnum>1</manvolnum>
713 <refentrytitle>virt_db</refentrytitle>
714 <manvolnum>3mp</manvolnum>
722 <!-- Keep this comment at the end of the file