1 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook V4.4//EN"
2 "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd" [
3 <!ENTITY copyright SYSTEM "copyright.xml">
4 <!ENTITY % idcommon SYSTEM "common/common.ent">
7 <refentry id="ref-zoom">
9 <productname>Metaproxy</productname>
10 <info><orgname>Index Data</orgname></info>
14 <refentrytitle>zoom</refentrytitle>
15 <manvolnum>3mp</manvolnum>
16 <refmiscinfo class="manual">Metaproxy Module</refmiscinfo>
20 <refname>zoom</refname>
21 <refpurpose>Metaproxy ZOOM Module</refpurpose>
25 <title>DESCRIPTION</title>
27 This filter implements a generic client based on
28 <ulink url="&url.yaz.zoom;">ZOOM</ulink> of YAZ.
29 The client implements the protocols that ZOOM C does: Z39.50, SRU
30 (GET, POST, SOAP) and SOLR .
34 This filter only deals with Z39.50 on input. The following services
35 are supported: init, search, present and close. The backend target
36 is selected based on the database as part search and
37 <emphasis>not</emphasis> as part of init.
41 This filter is an alternative to the z3950_client filter but also
42 shares properties of the virt_db - in that the target is selected
43 for a specific database
47 The ZOOM filter relies on a target profile description, which is
48 XML based. It picks the profile for a given database from a web service
49 or it may be locally given for each unique database (AKA virtual database
50 in virt_db). Target profiles are directly and indrectly given as part
51 of the <literal>torus</literal> element in the configuration.
57 <title>CONFIGURATION</title>
59 The configuration consists of six parts: <literal>torus</literal>,
60 <literal>fieldmap</literal>, <literal>cclmap</literal>,
61 <literal>contentProxy</literal>, <literal>log</literal>
62 and <literal>zoom</literal>.
67 The <literal>torus</literal> element specifies target profiles
68 and takes the following content:
72 <term>attribute <literal>url</literal></term>
75 URL of Web service to be used when fetch target profiles from
76 a remote service (Torus normally).
79 The sequence <literal>%query</literal> is replaced with a CQL
80 query for the Torus search.
83 The special sequence <literal>%realm</literal> is replaced by value
84 of attribute <literal>realm</literal> or by realm DATABASE argument.
87 The special sequence <literal>%db</literal> is replaced with
88 a single database while searching. Note that this sequence
89 is no longer needed, because the <literal>%query</literal> can already
90 query for a single database by using CQL query
91 <literal>udb==...</literal>.
96 <term>attribute <literal>content_url</literal></term>
99 URL of Web service to be used to fetch target profile
100 for a given database (udb) of type content. Semantics otherwise like
101 <literal>url</literal> attribute above.
106 <term>attribute <literal>realm</literal></term>
109 The default realm value. Used for %realm in URL, unless
110 specified in DATABASE argument.
115 <term>attribute <literal>proxy</literal></term>
118 HTTP proxy to bse used for fetching target profiles.
123 <term>attribute <literal>xsldir</literal></term>
126 Directory that is searched for XSL stylesheets. Stylesheets
127 are specified in the target profile by the
128 <literal>transform</literal> element.
133 <term>attribute <literal>element_transform</literal></term>
136 Specifies the element that triggers retrieval and transform using
137 the parameters elementSet, recordEncoding, requestSyntax, transform
138 from the target profile. Default value
139 is "pz2", due to the fact that for historical reasons the
140 common format is that used in Pazpar2.
145 <term>attribute <literal>element_raw</literal></term>
148 Specifies an element that triggers retrieval using the
149 parameters elementSet, recordEncoding, requestSyntax from the
150 target profile. Same actions as for element_transform, but without
151 the XSL transform. Useful for debugging.
152 The default value is "raw".
157 <term>attribute <literal>explain_xsl</literal></term>
160 Specifies a stylesheet that converts one or more Torus records
161 to ZeeExplain records. The content of recordData is assumed to be
162 holding each Explain record.
167 <term>element <literal>records</literal></term>
170 Local target profiles. This element may includes zero or
171 more <literal>record</literal> elements (one per target
172 profile). See section TARGET PROFILE.
178 <refsect2 id="fieldmap">
179 <title>fieldmap</title>
181 The <literal>fieldmap</literal> may be specified zero or more times and
182 specifies the map from CQL fields to CCL fields and takes the
187 <term>attribute <literal>cql</literal></term>
190 CQL field that we are mapping "from".
195 <term>attribute <literal>ccl</literal></term>
198 CCL field that we are mapping "to".
204 <refsect2 id="cclmap_base">
205 <title>cclmap</title>
207 The third part of the configuration consists of zero or more
208 <literal>cclmap</literal> elements that specifies
209 <emphasis>base</emphasis> CCL profile to be used for all targets.
210 This configuration, thus, will be combined with cclmap-definitions
211 from the target profile.
215 <title>contentProxy</title>
217 The <literal>contentProxy</literal> element controls content proxy'in.
219 is optional and must only be defined if content proxy'ing is enabled.
223 <term>attribute <literal>server</literal></term>
226 Specifies the content proxy host. The host is of the form
227 host[:port]. That is without a method (such as HTTP) and optional
233 <term>attribute <literal>tmp_file</literal></term>
236 Specifies a filename of a session file for content proxy'ing. The
237 file should be an absolute filename that includes
238 <literal>XXXXXX</literal> which is replaced by a unique filename
239 using the mkstemp(3) system call. The default value of this
240 setting is <literal>/tmp/cf.XXXXXX.p</literal>.
249 The <literal>log</literal> element controls logging for the
254 <term>attribute <literal>apdu</literal></term>
257 If the value of apdu is "true", then protocol packages
258 (APDUs and HTTP packages) from the ZOOM filter will be
259 logged to the yaz_log system. A value of "false" will
260 not perform logging of protocol packages (the default
271 The <literal>zoom</literal> element controls settings for the
276 <term>attribute <literal>timeout</literal></term>
279 Is an integer that specifies, in seconds, how long an operation
280 may take before ZOOM gives up. Default value is 40.
289 <title>QUERY HANDLING</title>
291 The ZOOM filter accepts three query types: RPN(Type-1), CCL and
295 Queries are converted in two separate steps. In the first step
296 the input query is converted to RPN/Type-1. This is always
297 the common internal format between step 1 and step 2.
298 In step 2 the query is converted to the native query type of the target.
301 Step 1: for RPN, the query is passed un-modified to the target.
304 Step 1: for CCL, the query is converted to RPN via
305 <link linkend="cclmap"><literal>cclmap</literal></link> elements part of
306 the target profile as well as
307 <link linkend="cclmap_base">base CCL maps</link>.
310 Step 1: For CQL, the query is converted to CCL. The mappings of
311 CQL fields to CCL fields are handled by
312 <link linkend="fieldmap"><literal>fieldmap</literal></link>
313 elements as part of the target profile. The resulting query, CCL,
314 is the converted to RPN using the schema mentioned earlier (via
315 <literal>cclmap</literal>).
318 Step 2: If the target is Z39.50-based, it is passed verbatim (RPN).
319 If the target is SRU-based, the RPN will be converted to CQL.
320 If the target is SOLR-based, the RPN will be converted to SOLR's query
326 <title>SORTING</title>
328 The ZOOM module actively handle CQL sorting - using the SORTBY parameter
329 which was introduced in SRU version 1.2. The conversion from SORTBY clause
330 to native sort for some target is driven by the two parameters:
331 <link linkend="sortStrategy"><literal>sortStrategy</literal></link>
332 and <link linkend="sortmap"><literal>sortmap_</literal><replaceable>field</replaceable></link>.
335 If a sort field that does not have an equivalent
336 <literal>sortmap_</literal>-mapping is passed un-modified through the
337 conversion. It doesn't throw a diagnostic.
342 <title>TARGET PROFILE</title>
344 The ZOOM module is driven by a number of settings that specifies how
345 to handle each target.
346 Note that unknown elements are silently <emphasis>ignored</emphasis>.
349 The elements, in alphabetical order, are:
353 <term id="zoom-torus-authentication">authentication</term><listitem>
355 Authentication parameters to be sent to the target. For
356 Z39.50 targets, this will be sent as part of the
357 Init Request. Authentication consists of two components: username
358 and password, separated by a slash.
361 If this value is omitted or empty no authentication information is sent.
366 <varlistentry id="cclmap">
367 <term>cclmap_<replaceable>field</replaceable></term><listitem>
369 This value specifies CCL field (qualifier) definition for some
370 field. For Z39.50 targets this most likely will specify the
371 mapping to a numeric use attribute + a structure attribute.
372 For SRU targets, the use attribute should be string based, in
373 order to make the RPN to CQL conversion work properly (step 2).
379 <term>cfAuth</term><listitem>
381 When cfAuth is defined, its value will be used as authentication
382 to backend target and authentication setting will be specified
383 as part of a database. This is like a "proxy" for authentication and
384 is used for Connector Framework based targets.
390 <term id="zoom-torus-cfproxy">cfProxy</term><listitem>
392 Specifies HTTP proxy for the target in the form
393 <replaceable>host</replaceable>:<replaceable>port</replaceable>.
399 <term>cfSubDB</term><listitem>
401 Specifies sub database for a Connector Framework based target.
406 <varlistentry id="zoom-torus-contentConnector">
407 <term>contentConnector</term><listitem>
409 Specifies a database for content-based proxy'ing.
415 <term>elementSet</term><listitem>
417 Specifies the elementSet to be sent to the target if record
418 transform is enabled (not to be confused' with the record_transform
419 module). The record transform is enabled only if the client uses
420 record syntax = XML and a element set determined by
421 the <literal>element_transform</literal> /
422 <literal>element_raw</literal> from the configuration.
423 By default that is the element sets <literal>pz2</literal>
424 and <literal>raw</literal>.
425 If record transform is not enabled, this setting is
426 not used and the element set specified by the client
433 <term>literalTransform</term><listitem>
435 Specifies a XSL stylesheet to be used if record
436 transform is anabled; see description of elementSet.
437 The XSL transform is only used if the element set is set to the
438 value of <literal>element_transform</literal> in the configuration.
441 The value of literalTransform is the XSL - string encoded.
447 <term>piggyback</term><listitem>
449 A value of 1/true is a hint to the ZOOM module that this Z39.50
450 target supports piggyback searches, ie Search Response with
451 records. Any other value (false) will prevent the ZOOM module
452 to make use of piggyback (all records part of Present Response).
458 <term>queryEncoding</term><listitem>
460 If this value is defined, all queries will be converted
461 to this encoding. This should be used for all Z39.50 targets that
462 do not use UTF-8 for query terms.
468 <term>recordEncoding</term><listitem>
470 Specifies the character encoding of records that are returned
471 by the target. This is primarily used for targets were records
472 are not UTF-8 encoded already. This setting is only used
473 if the record transform is enabled (see description of elementSet).
479 <term>requestSyntax</term><listitem>
481 Specifies the record syntax to be specified for the target
482 if record transform is enabled; see description of elementSet.
483 If record transform is not enabled, the record syntax of the
484 client is passed verbatim to the target.
489 <varlistentry id="sortmap">
490 <term>sortmap_<replaceable>field</replaceable></term><listitem>
492 This value the native field for a target. The form of the value is
493 given by <link linkend="sortStrategy">sortStrategy</link>.
498 <varlistentry id="sortStrategy">
499 <term>sortStrategy</term><listitem>
501 Specifies sort strategy for a target. One of:
502 <literal>z3950</literal>, <literal>type7</literal>,
503 <literal>cql</literal>, <literal>sru11</literal> or
504 <literal>embed</literal>. The <literal>embed</literal> chooses type-7
505 or CQL sortby depending on whether Type-1 or CQL is
506 actually sent to the target.
512 <term>sru</term><listitem>
514 If this setting is set, it specifies that the target is web service
515 based and must be one of : <literal>get</literal>,
516 <literal>post</literal>, <literal>soap</literal>
517 or <literal>solr</literal>.
523 <term>sruVersion</term><listitem>
525 Specifies the SRU version to use. It unset, version 1.2 will be
526 used. Some servers do not support this version, in which case
527 version 1.1 or even 1.0 could be set it.
533 <term>transform</term><listitem>
535 Specifies a XSL stylesheet filename to be used if record
536 transform is anabled; see description of elementSet.
537 The XSL transform is only used if the element set is set to the
538 value of <literal>element_transform</literal> in the configuration.
544 <term>udb</term><listitem>
546 This value is required and specifies the unique database for
547 this profile . All target profiles should hold a unique database.
552 <varlistentry id="urlRecipe">
553 <term>urlRecipe</term><listitem>
555 The value of this field is a string that generates a dynamic link
556 based on record content. If the resulting string is non-zero in length
557 a new field, <literal>metadata</literal> with attribute
558 <literal>type="generated-url"</literal> is generated.
559 The contents of this field is the result of the URL recipe conversion.
560 The urlRecipe value may refer to an existing metadata element by
561 ${field[pattern/result/flags]}, which will take content
562 of field and perform a regular expression conversion using the pattern
563 given. For example: <literal>${md-title[\s+/+/g]}</literal> takes
564 metadata element <literal>title</literal> and converts one or more
565 spaces to a plus character.
568 If the <link linkend="zoom-torus-contentConnector">contentConnector</link>
569 setting also defined, the resulting value is
570 augmented with a session string as well as host name of the
571 content proxy server.
577 <term>zurl</term><listitem>
579 This is setting is mandatory and specifies the ZURL of the
580 target in the form of host/database. The HTTP method should
581 not be provided as this is guessed from the "sru" attribute value.
588 <title>DATABASE parameters</title>
590 Extra information may be carried in the Z39.50 Database or SRU path,
591 such as authentication to be passed to backend etc. Some of
592 the parameters override TARGET profile values. The format is
595 udb,parm1=value1&parm2=value2&...
598 Where udb is the unique database recognised by the backend and parm1,
599 value1, .. are parameters to be passed. The following describes the
600 supported parameters. Like form values in HTTP the parameters and
601 values are URL encoded. The separator, though, between udb and parameters
602 is a comma rather than a question mark. What follows question mark are
603 HTTP arguments (in this case SRU arguments).
610 Specifies user to be passed to backend. If this parameter is
611 omitted, the user will be taken from TARGET profile setting
612 <link linkend="zoom-torus-authentication">
613 <literal>authentication</literal>
620 <term>password</term>
623 Specifies password to be passed to backend. If this parameters is
624 omitted, the password will be taken from TARGET profile setting
625 <link linkend="zoom-torus-authentication">
626 <literal>authentication</literal>
636 Specifies one or more proxies for backend. If this parameter is
637 omitted, the proxy will be taken from TARGET profile setting
638 <link linkend="zoom-torus-cfproxy">
639 <literal>cfProxy</literal></link>.
640 The parameter is a list of comma-separated host:port entries.
641 Bost host and port must be given for each proxy.
646 <term>cproxysession</term>
649 Session ID for content proxy. This parameter is, generally,
650 not used by anything but the content proxy itself.
655 <term>nocproxy</term>
658 If this parameter is specified, content-proyxing is disabled
667 Session realm to be used for this target, changed the resulting
668 URL to be used for getting a target profile, by changing the
669 value that gets substituted for the %realm string.
677 All parameters that has prefix x, dash are passed verbatim
685 <title>SCHEMA</title>
686 <literallayout><xi:include
687 xi:href="../xml/schema/filter_zoom.rnc"
689 xmlns:xi="http://www.w3.org/2001/XInclude" />
694 <title>EXAMPLES</title>
696 The following configuration illustrates most of the
701 url="http://torus.indexdata.com/src/records/?query=udb%3D%db"
702 proxy="localhost:3128"
704 <fieldmap cql="cql.anywhere"/>
705 <fieldmap cql="cql.serverChoice"/>
706 <fieldmap cql="dc.creator" ccl="au"/>
707 <fieldmap cql="dc.title" ccl="ti"/>
708 <fieldmap cql="dc.subject" ccl="su"/>
712 <attr type="u" value="12"/>
713 <attr type="s" value="107"/>
726 <title>SEE ALSO</title>
729 <refentrytitle>metaproxy</refentrytitle>
730 <manvolnum>1</manvolnum>
735 <refentrytitle>virt_db</refentrytitle>
736 <manvolnum>3mp</manvolnum>
744 <!-- Keep this comment at the end of the file