1 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook V4.4//EN"
2 "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd" [
3 <!ENTITY copyright SYSTEM "copyright.xml">
4 <!ENTITY % idcommon SYSTEM "common/common.ent">
7 <refentry id="ref-zoom">
9 <productname>Metaproxy</productname>
10 <info><orgname>Index Data</orgname></info>
14 <refentrytitle>zoom</refentrytitle>
15 <manvolnum>3mp</manvolnum>
16 <refmiscinfo class="manual">Metaproxy Module</refmiscinfo>
20 <refname>zoom</refname>
21 <refpurpose>Metaproxy ZOOM Module</refpurpose>
25 <title>DESCRIPTION</title>
27 This filter implements a generic client based on
28 <ulink url="&url.yaz.zoom;">ZOOM</ulink> of YAZ.
29 The client implements the protocols that ZOOM C does: Z39.50, SRU
30 (GET, POST, SOAP) and SOLR .
34 This filter only deals with Z39.50 on input. The following services
35 are supported: init, search, present and close. The backend target
36 is selected based on the database as part search and
37 <emphasis>not</emphasis> as part of init.
41 This filter is an alternative to the z3950_client filter but also
42 shares properties of the virt_db - in that the target is selected
43 for a specific database
47 The ZOOM filter relies on a target profile description, which is
48 XML based. It picks the profile for a given database from a web service
49 or it may be locally given for each unique database (AKA virtual database
50 in virt_db). Target profiles are directly and indrectly given as part
51 of the <literal>torus</literal> element in the configuration.
57 <title>CONFIGURATION</title>
59 The configuration consists of six parts: <literal>torus</literal>,
60 <literal>fieldmap</literal>, <literal>cclmap</literal>,
61 <literal>contentProxy</literal>, <literal>log</literal>
62 and <literal>zoom</literal>.
67 The <literal>torus</literal> element specifies target profiles
68 and takes the following content:
72 <term>attribute <literal>url</literal></term>
75 URL of Web service to be used when fetch target profiles from
76 a remote service (Torus normally).
79 The sequence <literal>%query</literal> is replaced with a CQL
80 query for the Torus search.
83 The special sequence <literal>%realm</literal> is replaced by value
84 of attribute <literal>realm</literal> or by realm DATABASE argument.
87 The special sequence <literal>%db</literal> is replaced with
88 a single database while searching. Note that this sequence
89 is no longer needed, because the <literal>%query</literal> can already
90 query for a single database by using CQL query
91 <literal>udb==...</literal>.
96 <term>attribute <literal>content_url</literal></term>
99 URL of Web service to be used to fetch target profile
100 for a given database (udb) of type content. Semantics otherwise like
101 <literal>url</literal> attribute above.
106 <term>attribute <literal>realm</literal></term>
109 The default realm value. Used for %realm in URL, unless
110 specified in DATABASE argument.
115 <term>attribute <literal>proxy</literal></term>
118 HTTP proxy to bse used for fetching target profiles.
123 <term>attribute <literal>xsldir</literal></term>
126 Directory that is searched for XSL stylesheets. Stylesheets
127 are specified in the target profile by the
128 <literal>transform</literal> element.
133 <term>attribute <literal>element_transform</literal></term>
136 Specifies the element that triggers retrieval and transform using
137 the parameters elementSet, recordEncoding, requestSyntax, transform
138 from the target profile. Default value
139 is "pz2", due to the fact that for historical reasons the
140 common format is that used in Pazpar2.
145 <term>attribute <literal>element_raw</literal></term>
148 Specifies an element that triggers retrieval using the
149 parameters elementSet, recordEncoding, requestSyntax from the
150 target profile. Same actions as for element_transform, but without
151 the XSL transform. Useful for debugging.
152 The default value is "raw".
157 <term>attribute <literal>explain_xsl</literal></term>
160 Specifies a stylesheet that converts one or more Torus records
161 to ZeeExplain records. The content of recordData is assumed to be
162 holding each Explain record.
167 <term>attribute <literal>record_xsl</literal></term>
170 Specifies a stylesheet that converts retrieval records after
171 transform/literal operations.
176 <term>element <literal>records</literal></term>
179 Local target profiles. This element may includes zero or
180 more <literal>record</literal> elements (one per target
181 profile). See section TARGET PROFILE.
187 <refsect2 id="fieldmap">
188 <title>fieldmap</title>
190 The <literal>fieldmap</literal> may be specified zero or more times and
191 specifies the map from CQL fields to CCL fields and takes the
196 <term>attribute <literal>cql</literal></term>
199 CQL field that we are mapping "from".
204 <term>attribute <literal>ccl</literal></term>
207 CCL field that we are mapping "to".
213 <refsect2 id="cclmap_base">
214 <title>cclmap</title>
216 The third part of the configuration consists of zero or more
217 <literal>cclmap</literal> elements that specifies
218 <emphasis>base</emphasis> CCL profile to be used for all targets.
219 This configuration, thus, will be combined with cclmap-definitions
220 from the target profile.
224 <title>contentProxy</title>
226 The <literal>contentProxy</literal> element controls content proxy'in.
228 is optional and must only be defined if content proxy'ing is enabled.
232 <term>attribute <literal>server</literal></term>
235 Specifies the content proxy host. The host is of the form
236 host[:port]. That is without a method (such as HTTP) and optional
242 <term>attribute <literal>tmp_file</literal></term>
245 Specifies a filename of a session file for content proxy'ing. The
246 file should be an absolute filename that includes
247 <literal>XXXXXX</literal> which is replaced by a unique filename
248 using the mkstemp(3) system call. The default value of this
249 setting is <literal>/tmp/cf.XXXXXX.p</literal>.
258 The <literal>log</literal> element controls logging for the
263 <term>attribute <literal>apdu</literal></term>
266 If the value of apdu is "true", then protocol packages
267 (APDUs and HTTP packages) from the ZOOM filter will be
268 logged to the yaz_log system. A value of "false" will
269 not perform logging of protocol packages (the default
280 The <literal>zoom</literal> element controls settings for the
285 <term>attribute <literal>timeout</literal></term>
288 Is an integer that specifies, in seconds, how long an operation
289 may take before ZOOM gives up. Default value is 40.
298 <title>QUERY HANDLING</title>
300 The ZOOM filter accepts three query types: RPN(Type-1), CCL and
304 Queries are converted in two separate steps. In the first step
305 the input query is converted to RPN/Type-1. This is always
306 the common internal format between step 1 and step 2.
307 In step 2 the query is converted to the native query type of the target.
310 Step 1: for RPN, the query is passed un-modified to the target.
313 Step 1: for CCL, the query is converted to RPN via
314 <link linkend="cclmap"><literal>cclmap</literal></link> elements part of
315 the target profile as well as
316 <link linkend="cclmap_base">base CCL maps</link>.
319 Step 1: For CQL, the query is converted to CCL. The mappings of
320 CQL fields to CCL fields are handled by
321 <link linkend="fieldmap"><literal>fieldmap</literal></link>
322 elements as part of the target profile. The resulting query, CCL,
323 is the converted to RPN using the schema mentioned earlier (via
324 <literal>cclmap</literal>).
327 Step 2: If the target is Z39.50-based, it is passed verbatim (RPN).
328 If the target is SRU-based, the RPN will be converted to CQL.
329 If the target is SOLR-based, the RPN will be converted to SOLR's query
335 <title>SORTING</title>
337 The ZOOM module actively handle CQL sorting - using the SORTBY parameter
338 which was introduced in SRU version 1.2. The conversion from SORTBY clause
339 to native sort for some target is driven by the two parameters:
340 <link linkend="sortStrategy"><literal>sortStrategy</literal></link>
341 and <link linkend="sortmap"><literal>sortmap_</literal><replaceable>field</replaceable></link>.
344 If a sort field that does not have an equivalent
345 <literal>sortmap_</literal>-mapping is passed un-modified through the
346 conversion. It doesn't throw a diagnostic.
351 <title>TARGET PROFILE</title>
353 The ZOOM module is driven by a number of settings that specifies how
354 to handle each target.
355 Note that unknown elements are silently <emphasis>ignored</emphasis>.
358 The elements, in alphabetical order, are:
362 <term id="zoom-torus-authentication">authentication</term><listitem>
364 Authentication parameters to be sent to the target. For
365 Z39.50 targets, this will be sent as part of the
366 Init Request. Authentication consists of two components: username
367 and password, separated by a slash.
370 If this value is omitted or empty no authentication information is sent.
375 <varlistentry id="cclmap">
376 <term>cclmap_<replaceable>field</replaceable></term><listitem>
378 This value specifies CCL field (qualifier) definition for some
379 field. For Z39.50 targets this most likely will specify the
380 mapping to a numeric use attribute + a structure attribute.
381 For SRU targets, the use attribute should be string based, in
382 order to make the RPN to CQL conversion work properly (step 2).
388 <term>cfAuth</term><listitem>
390 When cfAuth is defined, its value will be used as authentication
391 to backend target and authentication setting will be specified
392 as part of a database. This is like a "proxy" for authentication and
393 is used for Connector Framework based targets.
399 <term id="zoom-torus-cfproxy">cfProxy</term><listitem>
401 Specifies HTTP proxy for the target in the form
402 <replaceable>host</replaceable>:<replaceable>port</replaceable>.
408 <term>cfSubDB</term><listitem>
410 Specifies sub database for a Connector Framework based target.
415 <varlistentry id="zoom-torus-contentConnector">
416 <term>contentConnector</term><listitem>
418 Specifies a database for content-based proxy'ing.
424 <term>elementSet</term><listitem>
426 Specifies the elementSet to be sent to the target if record
427 transform is enabled (not to be confused' with the record_transform
428 module). The record transform is enabled only if the client uses
429 record syntax = XML and a element set determined by
430 the <literal>element_transform</literal> /
431 <literal>element_raw</literal> from the configuration.
432 By default that is the element sets <literal>pz2</literal>
433 and <literal>raw</literal>.
434 If record transform is not enabled, this setting is
435 not used and the element set specified by the client
442 <term>literalTransform</term><listitem>
444 Specifies a XSL stylesheet to be used if record
445 transform is anabled; see description of elementSet.
446 The XSL transform is only used if the element set is set to the
447 value of <literal>element_transform</literal> in the configuration.
450 The value of literalTransform is the XSL - string encoded.
456 <term>piggyback</term><listitem>
458 A value of 1/true is a hint to the ZOOM module that this Z39.50
459 target supports piggyback searches, ie Search Response with
460 records. Any other value (false) will prevent the ZOOM module
461 to make use of piggyback (all records part of Present Response).
467 <term>queryEncoding</term><listitem>
469 If this value is defined, all queries will be converted
470 to this encoding. This should be used for all Z39.50 targets that
471 do not use UTF-8 for query terms.
477 <term>recordEncoding</term><listitem>
479 Specifies the character encoding of records that are returned
480 by the target. This is primarily used for targets were records
481 are not UTF-8 encoded already. This setting is only used
482 if the record transform is enabled (see description of elementSet).
488 <term>requestSyntax</term><listitem>
490 Specifies the record syntax to be specified for the target
491 if record transform is enabled; see description of elementSet.
492 If record transform is not enabled, the record syntax of the
493 client is passed verbatim to the target.
498 <varlistentry id="sortmap">
499 <term>sortmap_<replaceable>field</replaceable></term><listitem>
501 This value the native field for a target. The form of the value is
502 given by <link linkend="sortStrategy">sortStrategy</link>.
507 <varlistentry id="sortStrategy">
508 <term>sortStrategy</term><listitem>
510 Specifies sort strategy for a target. One of:
511 <literal>z3950</literal>, <literal>type7</literal>,
512 <literal>cql</literal>, <literal>sru11</literal> or
513 <literal>embed</literal>. The <literal>embed</literal> chooses type-7
514 or CQL sortby depending on whether Type-1 or CQL is
515 actually sent to the target.
521 <term>sru</term><listitem>
523 If this setting is set, it specifies that the target is web service
524 based and must be one of : <literal>get</literal>,
525 <literal>post</literal>, <literal>soap</literal>
526 or <literal>solr</literal>.
532 <term>sruVersion</term><listitem>
534 Specifies the SRU version to use. It unset, version 1.2 will be
535 used. Some servers do not support this version, in which case
536 version 1.1 or even 1.0 could be set it.
542 <term>transform</term><listitem>
544 Specifies a XSL stylesheet filename to be used if record
545 transform is anabled; see description of elementSet.
546 The XSL transform is only used if the element set is set to the
547 value of <literal>element_transform</literal> in the configuration.
553 <term>udb</term><listitem>
555 This value is required and specifies the unique database for
556 this profile . All target profiles should hold a unique database.
561 <varlistentry id="urlRecipe">
562 <term>urlRecipe</term><listitem>
564 The value of this field is a string that generates a dynamic link
565 based on record content. If the resulting string is non-zero in length
566 a new field, <literal>metadata</literal> with attribute
567 <literal>type="generated-url"</literal> is generated.
568 The contents of this field is the result of the URL recipe conversion.
569 The urlRecipe value may refer to an existing metadata element by
570 ${field[pattern/result/flags]}, which will take content
571 of field and perform a regular expression conversion using the pattern
572 given. For example: <literal>${md-title[\s+/+/g]}</literal> takes
573 metadata element <literal>title</literal> and converts one or more
574 spaces to a plus character.
577 If the <link linkend="zoom-torus-contentConnector">contentConnector</link>
578 setting also defined, the resulting value is
579 augmented with a session string as well as host name of the
580 content proxy server.
586 <term>zurl</term><listitem>
588 This is setting is mandatory and specifies the ZURL of the
589 target in the form of host/database. The HTTP method should
590 not be provided as this is guessed from the "sru" attribute value.
597 <title>DATABASE parameters</title>
599 Extra information may be carried in the Z39.50 Database or SRU path,
600 such as authentication to be passed to backend etc. Some of
601 the parameters override TARGET profile values. The format is
604 udb,parm1=value1&parm2=value2&...
607 Where udb is the unique database recognised by the backend and parm1,
608 value1, .. are parameters to be passed. The following describes the
609 supported parameters. Like form values in HTTP the parameters and
610 values are URL encoded. The separator, though, between udb and parameters
611 is a comma rather than a question mark. What follows question mark are
612 HTTP arguments (in this case SRU arguments).
619 Specifies user to be passed to backend. If this parameter is
620 omitted, the user will be taken from TARGET profile setting
621 <link linkend="zoom-torus-authentication">
622 <literal>authentication</literal>
629 <term>password</term>
632 Specifies password to be passed to backend. If this parameters is
633 omitted, the password will be taken from TARGET profile setting
634 <link linkend="zoom-torus-authentication">
635 <literal>authentication</literal>
645 Specifies one or more proxies for backend. If this parameter is
646 omitted, the proxy will be taken from TARGET profile setting
647 <link linkend="zoom-torus-cfproxy">
648 <literal>cfProxy</literal></link>.
649 The parameter is a list of comma-separated host:port entries.
650 Bost host and port must be given for each proxy.
655 <term>cproxysession</term>
658 Session ID for content proxy. This parameter is, generally,
659 not used by anything but the content proxy itself.
664 <term>nocproxy</term>
667 If this parameter is specified, content-proyxing is disabled
676 Session realm to be used for this target, changed the resulting
677 URL to be used for getting a target profile, by changing the
678 value that gets substituted for the %realm string.
686 All parameters that has prefix x, dash are passed verbatim
694 <title>SCHEMA</title>
695 <literallayout><xi:include
696 xi:href="../xml/schema/filter_zoom.rnc"
698 xmlns:xi="http://www.w3.org/2001/XInclude" />
703 <title>EXAMPLES</title>
705 The following configuration illustrates most of the
710 url="http://torus.indexdata.com/src/records/?query=%query"
711 proxy="localhost:3128"
713 <fieldmap cql="cql.anywhere"/>
714 <fieldmap cql="cql.serverChoice"/>
715 <fieldmap cql="dc.creator" ccl="au"/>
716 <fieldmap cql="dc.title" ccl="ti"/>
717 <fieldmap cql="dc.subject" ccl="su"/>
721 <attr type="u" value="12"/>
722 <attr type="s" value="107"/>
735 <title>SEE ALSO</title>
738 <refentrytitle>metaproxy</refentrytitle>
739 <manvolnum>1</manvolnum>
744 <refentrytitle>virt_db</refentrytitle>
745 <manvolnum>3mp</manvolnum>
753 <!-- Keep this comment at the end of the file