1 <chapter id="querymodel">
2 <!-- $Id: querymodel.xml,v 1.13 2006-06-23 12:40:33 marc Exp $ -->
3 <title>Query Model</title>
5 <sect1 id="querymodel-overview">
6 <title>Query Model Overview</title>
9 <sect2 id="querymodel-query-languages">
10 <title>Query Languages</title>
13 Zebra is born as a networking Information Retrieval engine adhering
14 to the international standards
15 <ulink url="&url.z39.50;">Z39.50</ulink> and
16 <ulink url="&url.sru;">SRU</ulink>,
18 <literal>type-1 Reverse Polish Notation (RPN)</literal> query
20 Unfortunately, this model has only defined a binary
21 encoded representation, which is used as transport packaging in
22 the Z39.50 protocol layer. This representation is not human
23 readable, nor defines any convenient way to specify queries.
26 Since the <literal>type-1 (RPN)</literal>
27 query structure has no direct, useful string
28 representation, every origin application needs to provide some
29 form of mapping from a local query notation or representation to it.
33 <sect3 id="querymodel-query-languages-pqf">
34 <title>Prefix Query Format (PQF)</title>
37 Index Data has defined a textual representaion in the
38 <literal>Prefix Query Format</literal>, short
39 <literal>PQF</literal>, which mappes
40 <literal>one-to-one</literal> to binary encoded
41 <literal>type-1 RPN</literal> query packages.
42 It has been adopted by other
43 parties developing Z39.50 software, and is often referred to as
44 <literal>Prefix Query Notation</literal>, or in short
45 <literal>PQN</literal>. See
46 <xref linkend="querymodel-pqf"/> for further explanaitions and
47 descriptions of Zebra's capabilities.
51 <sect3 id="querymodel-query-languages-cql">
52 <title>Common Query Language (CQL)</title>
54 The query model of the <literal>type-1 RPN</literal>,
55 expressed in <literal>PQF/PQN</literal> is natively supported.
56 On the other hand, the default <literal>SRU</literal>
57 webservices <literal>Common Query Language</literal>
58 <ulink url="&url.cql;">CQL</ulink> is not natively supported.
61 Zebra can be configured to understand and map CQL to PQF. See
62 <xref linkend="querymodel-cql-to-pqf"/>.
68 <sect2 id="querymodel-operation-types">
69 <title>Operation types</title>
71 Zebra supports all of the three different
72 <literal>Z39.50/SRU</literal> operations defined in the
73 standards: <literal>explain</literal>, <literal>search</literal>,
74 and <literal>scan</literal>. A short description of the
75 functionality and purpose of each is quite in order here.
78 <sect3 id="querymodel-operation-type-explain">
79 <title>Explain Operation</title>
81 The <emphasis>syntax</emphasis> of Z39.50/SRU queries is
82 well known to any client, but the specific
83 <emphasis>semantics</emphasis> - taking into account a
84 particular servers functionalities and abilities - must be
85 discovered from case to case. Enters the
86 <literal>explain</literal> operation, which provides the means
88 <emphasis>fields</emphasis> (also called
89 <emphasis>indexes</emphasis> or <emphasis>access points</emphasis>
90 are provided, which default parameter the server uses, which
91 retrieve document formats are defined, and which specific parts
92 of the general query model are supported.
95 The Z39.50 embeddes the <literal>explain</literal> operation
97 <literal>search</literal> in the magic
98 <literal>IR-Explain-1</literal> database;
99 see <xref linkend="querymodel-exp1"/>.
102 In SRU, <literal>explain</literal> is an entirely seperate
103 operation, which returns an <literal>Zeerex
104 XML</literal> record according to the
105 structure defined by the protocol.
108 In both cases, the information gathered through
109 <literal>explain</literal> operations can be used to
110 auto-configure a client user interface to the servers
115 <sect3 id="querymodel-operation-type-search">
116 <title>Search Operation</title>
118 Search and retrieve interactions are the raison d'ĂȘtre.
119 They are used to query the remote database and
120 return search result documents. Search queries span from
121 simple free text searches to nested complex boolean queries,
122 targeting specific indexes, and possibly enhanced with many
123 query semantic specifications. Search interactions are the heart
124 and soul of Z39.50/SRU servers.
128 <sect3 id="querymodel-operation-type-scan">
129 <title>Scan Operation</title>
131 The <literal>scan</literal> operation is a helper functionality,
132 which operates on one index or access point a time.
136 the means to investigate the content of specific indexes.
137 Scanning an index returns a handfull of terms actually fond in
138 the indexes, and in addition the <literal>scan</literal>
139 operation returns th enumber of documents indexed by each term.
140 A search client can use this information to propose proper
141 spelling of search terms, to auto-fill search boxes, or to
142 display controlled vocabularies.
151 <sect1 id="querymodel-pqf">
152 <title>Prefix Query Format syntax and semantics</title>
154 The <ulink url="&url.yaz.pqf;">PQF grammer</ulink>
155 is documented in the YAZ manual, and shall not be
156 repeated here. This textual PQF representation
157 is always during search mapped to the equivalent Zebra internal
161 <sect2 id="querymodel-pqf-tree">
162 <title>PQF tree structure</title>
164 The PQF parse tree - or the equivalent textual representation -
165 may start with one specification of the
166 <emphasis>attribute set</emphasis> used. Following is a query
168 consists of <emphasis>atomic query parts (APT)</emphasis> or
169 <emphasis>named result sets</emphasis>, eventually
170 paired by <emphasis>boolean binary operators</emphasis>, and
171 finally <emphasis>recursively combined </emphasis> into
175 <sect3 id="querymodel-attribute-sets">
176 <title>Attribute sets</title>
178 Attribute sets define the exact meaning and semantics of queries
179 issued. Zebra comes with some predefined attribute set
180 definitions, others can easily be defined and added to the
185 <table id="querymodel-attribute-sets-table"
186 frame="all" rowsep="1" colsep="1" align="center">
188 <caption>Attribute sets predefined in Zebra</caption>
192 <td>Attribute set</td>
201 <td><literal>Explain</literal></td>
202 <td><literal>exp-1</literal></td>
203 <td>Special attribute set used on the special automagic
204 <literal>IR-Explain-1</literal> database to gain information on
205 server capabilities, database names, and database
210 <td><literal>Bib1</literal></td>
211 <td><literal>bib-1</literal></td>
212 <td>Standard PQF query language attribute set which defines the
213 semantics of Z39.50 searching. In addition, all of the
214 non-use attributes (type 2-9) define the hard-wired
220 <td><literal>GILS</literal></td>
221 <td><literal>gils</literal></td>
222 <td>Extention to the <literal>Bib1</literal> attribute set.</td>
227 <td><literal>IDXPATH</literal></td>
228 <td><literal>idxpath</literal></td>
229 <td>Hardwired XPATH like attribute set, only available for
230 indexing with the GRS record model</td>
239 The <literal>use attributes (type 1)</literal> mappings the
240 predefined attribute sets are found in the
241 attribute set configuration files <filename>tab/*.att</filename>.
245 The Zebra internal query processing is modeled after
246 the <literal>Bib1</literal> attribute set, and the non-use
247 attributes type 2-6 are hard-wired in. It is therefore essential
248 to be familiar with <xref linkend="querymodel-bib1-nonuse"/>.
252 <sect3 id="querymodel-boolean-operators">
253 <title>Boolean operators</title>
255 A pair of subquery trees, or of atomic queries, is combined
256 using the standard boolean operators into new query trees.
259 <table id="querymodel-boolean-operators-table"
260 frame="all" rowsep="1" colsep="1" align="center">
262 <caption>Boolean operators</caption>
271 <tr><td><literal>@and</literal></td>
272 <td>binary <literal>AND</literal> operator</td>
273 <td>Set intersection of two atomic queries hit sets</td>
275 <tr><td><literal>@or</literal></td>
276 <td>binary <literal>OR</literal> operator</td>
277 <td>Set union of two atomic queries hit sets</td>
279 <tr><td><literal>@not</literal></td>
280 <td>binary <literal>AND NOT</literal> operator</td>
281 <td>Set complement of two atomic queries hit sets</td>
283 <tr><td><literal>@prox</literal></td>
284 <td>binary <literal>PROXIMY</literal> operator</td>
285 <td>Set intersection of two atomic queries hit sets. In
286 addition, the intersection set is purged for all
287 documents which do not satisfy the requested query
288 term proximity. Usually a proper subset of the AND
295 For example, we can combine the terms
296 <emphasis>information</emphasis> and <emphasis>retrieval</emphasis>
297 into different searches in the default index of the default
298 attribute set as follows.
299 Querying for the union of all documents containing the
300 terms <emphasis>information</emphasis> OR
301 <emphasis>retrieval</emphasis>:
303 Z> find @or information retrieval
307 Querying for the intersection of all documents containing the
308 terms <emphasis>information</emphasis> AND
309 <emphasis>retrieval</emphasis>:
310 The hit set is a subset of the coresponding
313 Z> find @and information retrieval
317 Querying for the intersection of all documents containing the
318 terms <emphasis>information</emphasis> AND
319 <emphasis>retrieval</emphasis>, taking proximity into account:
320 The hit set is a subset of the coresponding
323 Z> find @prox 0 3 0 2 k 2 information retrieval
325 See <ulink url="&url.yaz.pqf;">PQF grammer</ulink> for details.
328 Querying for the intersection of all documents containing the
329 terms <emphasis>information</emphasis> AND
330 <emphasis>retrieval</emphasis>, in the same order and near each
331 other as described in the term list
332 The hit set is a subset of the coresponding
335 Z> find "information retrieval"
341 <sect3 id="querymodel-atomic-queries">
342 <title>Atomic queries (APT)</title>
344 Atomic queries are the query parts which work on one acess point
345 only. These consist of <literal>an attribute list</literal>
346 followed by a <literal>single term</literal> or a
347 <literal>quoted term list</literal>, and are often called
348 <emphasis>Attributes-Plus-Terms (APT)</emphasis> queries.
351 Unsupplied non-use attributes type 2-9 are either inherited from
352 higher nodes in the query tree, or are set to Zebra's default values.
353 See <xref linkend="querymodel-bib1"/> for details.
356 <table id="querymodel-atomic-queries-table"
357 frame="all" rowsep="1" colsep="1" align="center">
359 <caption>Atomic queries</caption>
362 <tr><td>one</td><td>two</td></tr>
367 <td><emphasis>attribute list</emphasis></td>
368 <td>List of <literal>orthogonal</literal> attributes</td>
369 <td>Any of the orthogonal attribute types may be omitted,
370 these are inherited from higher query tree nodes, or if not
371 inherited, are set to the default Zebra configuration values.
375 <td><emphasis>term</emphasis></td>
376 <td>single <literal>term</literal>
377 or <literal>quoted term list</literal> </td>
378 <td>Here the search terms or list of search terms is added
384 Querying for the term <emphasis>information</emphasis> in the
385 default index using the default attribite set, the server choice
386 of access point/index, and the default non-use attributes.
392 Equivalent query fully specified including all default values:
394 Z> find @attrset bib-1 @attr 1=1017 @attr 2=3 @attr 3=3 @attr 4=1 @attr 5=100 @attr 6=1 information
399 Finding all documents which have the term
400 <emphasis>debussy</emphasis> in the title field.
402 Z> find @attr 1=4 debussy
409 <sect3 id="querymodel-resultset">
410 <title>Named Result Sets</title>
412 Named result sets are supported in Zebra, and result sets can be
413 used as operands without limitations.
416 After the execution of a search, the result set is available at
417 the server, such that the client can use it for subsequent
418 searches or retrieval requests. The Z30.50 standard actually
419 stresses the fact that result sets are voliatile. It may cease
420 to exist at any time point after search, and the server will
421 send a diagnostic to the effect that the requested
422 result set does not exist any more.
426 Defining a named result set and re-using it in the next query,
427 using <literal>yaz-client</literal>.
429 Z> f @attr 1=4 mozart
431 Number of hits: 43, setno 1
433 Z> f @and @set 1 @attr 1=4 amadeus
435 Number of hits: 14, setno 2
437 Z> f @attr 1=1016 beethoven
439 Number of hits: 26, setno 3
445 Named result sets are only supported by the Z39.50 protocol.
446 The SRU web service is stateless, and therefore the notion of
447 named result sets does not exist when acessing a Zebra server by
453 <sect3 id="querymodel-use-string">
454 <title>Zebra's special access point of type 'string'</title>
456 The numeric <literal>use (type 1)</literal> attribute is usually
457 refered to from a given
458 attribute set. In addition, Zebra let you use
459 <emphasis>any internal index
460 name defined in your configuration</emphasis>
461 as use atribute value. This is a great feature for
462 debugging, and when you do
463 not need the complecity of defined use attribute values. It is
464 the preferred way of accessing Zebra indexes directly.
467 Finding all documents which have the term list "information
468 retrieval" in an Zebra index, using it's internal full string
469 name. Scanning the same index.
471 Z> find @attr 1=sometext "information retrieval"
472 Z> scan @attr 1=sometext aterm
476 Searching or scanning
477 the bib-1 use attribute 54 using it's string name:
479 Z> find @attr 1=Code-language eng
480 Z> scan @attr 1=Code-language ""
484 It is possible to search
485 in any silly string index - if it's defined in your
486 indexation rules and can be parsed by the PQF parser.
487 This is definitely not the recommended use of
488 this facility, as it might confuse your users with some very
491 Z> find @attr 1=silly/xpath/alike[@index]/name "information retrieval"
495 See also <xref linkend="querymodel-pqf-apt-mapping"/> for details, and
496 <xref linkend="server-sru"/>
497 for the SRU PQF query extention using string names as a fast
502 <sect3 id="querymodel-use-xpath">
503 <title>Zebra's special access point of type 'XPath'
504 for GRS filters</title>
506 As we have seen above, it is possible (albeit seldom a great
508 <ulink url="http://www.w3.org/TR/xpath">XPath 1.0</ulink> based
509 search by defining <literal>use (type 1)</literal>
510 <emphasis>string</emphasis> attributes which in appearence
511 <emphasis>resemble XPath queries</emphasis>. There are two
512 problems with this approach: first, the XPath-look-alike has to
513 be defined at indexation time, no new undefined
514 XPath queries can entered at search time, and second, it might
515 confuse users very much that an XPath-alike index name in fact
516 gets populated from a possible entirely different XML element
517 than it pretends to access.
520 When using the <literal>GRS Record Model</literal>
521 (see <xref linkend="record-model-grs"/>), we have the
522 possibility to embed <emphasis>life</emphasis>
524 in the PQF queries, which are here called
525 <literal>use (type 1)</literal> <emphasis>xpath</emphasis>
526 attributes. You must enable the
527 <literal>xpath enable</literal> directive in your
528 <literal>.abs</literal> config files.
531 Only a <emphasis>very</emphasis> restricted subset of the
532 <ulink url="http://www.w3.org/TR/xpath">XPath 1.0</ulink>
533 standard is supported as the GRS record model is simpler than
534 a full XML DOM structure. See the following examples for
538 Finding all documents which have the term "content"
539 inside a text node found in a specific XML DOM
540 <emphasis>subtree</emphasis>, whose starting element is
543 Z> find @attr 1=/root content
544 Z> find @attr 1=/root/first content
546 <emphasis>Notice that the
547 XPath must be absolute, i.e., must start with '/', and that the
548 XPath <literal>decendant-or-self</literal> axis followed by a
549 text node selection <literal>text()</literal> is implicitly
550 appended to the stated XPath.
552 It follows that the above searches are interpreted as:
554 Z> find @attr 1=/root//text() content
555 Z> find @attr 1=/root/first//text() content
560 Searching inside attribute strings is possible:
562 Z> find @attr 1=/link/@creator morten
567 Filter the adressing XPath by a predicate working on exact
569 attributes (in the XML sense) can be done: return all those docs which
570 have the term "english" contained in one of all text subnodes of
571 the subtree defined by the XPath
572 <literal>/record/title[@lang='en']</literal>. And similar
575 Z> find @attr 1=/record/title[@lang='en'] english
576 Z> find @attr 1=/link[@creator='sisse'] sibelius
577 Z> find @attr 1=/link[@creator='sisse']/description[@xml:lang='da'] sibelius
582 Combining numeric indexes, boolean expressions,
583 and xpath based searches is possible:
585 Z> find @attr 1=/record/title @and foo bar
586 Z> find @and @attr 1=/record/title foo @attr 1=4 bar
590 Escaping PQF keywords and other non-parseable XPath constructs
591 with <literal>'{ }'</literal> to prevent syntax errors:
593 Z> find @attr {1=/root/first[@attr='danish']} content
594 Z> find @attr {1=/record/@set} oai
598 It is worth mentioning that these dynamic performed XPath
599 queries are a performance bottelneck, as no optimized
600 specialized indexes can be used. Therefore, avoid the use of
601 this facility when speed is essential, and the database content
602 size is medium to large.
609 <sect2 id="querymodel-exp1">
610 <title>Explain Attribute Set</title>
612 The Z39.50 standard defines the
613 <ulink url="&url.z39.50.explain;">Explain</ulink> attribute set
614 <literal>Exp-1</literal>, which is used to discover information
615 about a server's search semantics and functional capabilities
616 Zebra exposes a "classic"
617 Explain database by base name <literal>IR-Explain-1</literal>, which
618 is populated with system internal information.
621 The attribute-set <literal>exp-1</literal> consists of a single
622 <literal>use attribute (type 1)</literal>.
625 In addition, the non-Use
626 <literal>bib-1</literal> attributes, that is, the types
627 <literal>Relation</literal>, <literal>Position</literal>,
628 <literal>Structure</literal>, <literal>Truncation</literal>,
629 and <literal>Completeness</literal> are imported from
630 the <literal>bib-1</literal> attribute set, and may be used
631 within any explain query.
634 <sect3 id="querymodel-exp1-use">
635 <title>Use Attributes (type = 1)</title>
637 The following Explain search atributes are supported:
638 <literal>ExplainCategory</literal> (@attr 1=1),
639 <literal>DatabaseName</literal> (@attr 1=3),
640 <literal>DateAdded</literal> (@attr 1=9),
641 <literal>DateChanged</literal>(@attr 1=10).
644 A search in the use attribute <literal>ExplainCategory</literal>
645 supports only these predefined values:
646 <literal>CategoryList</literal>, <literal>TargetInfo</literal>,
647 <literal>DatabaseInfo</literal>, <literal>AttributeDetails</literal>.
650 See <filename>tab/explain.att</filename> and the
651 <ulink url="&url.z39.50;">Z39.50</ulink> standard
652 for more information.
657 <title>Explain searches with yaz-client</title>
659 Classic Explain only defines retrieval of Explain information
660 via ASN.1. Pratically no Z39.50 clients supports this. Fortunately
661 they don't have to - Zebra allows retrieval of this information
663 <literal>SUTRS</literal>, <literal>XML</literal>,
664 <literal>GRS-1</literal> and <literal>ASN.1</literal> Explain.
668 List supported categories to find out which explain commands are
672 Z> find @attr exp1 1=1 categorylist
679 Get target info, that is, investigate which databases exist at
680 this server endpoint:
683 Z> find @attr exp1 1=1 targetinfo
694 List all supported databases, the number of hits
695 is the number of databases found, which most commonly are the
697 the <literal>Default</literal> and the
698 <literal>IR-Explain-1</literal> databases.
701 Z> find @attr exp1 1=1 databaseinfo
708 Get database info record for database <literal>Default</literal>.
711 Z> find @and @attr exp1 1=1 databaseinfo @attr exp1 1=3 Default
713 Identical query with explicitly specified attribute set:
716 Z> find @attrset exp1 @and @attr 1=1 databaseinfo @attr 1=3 Default
721 Get attribute details record for database
722 <literal>Default</literal>.
723 This query is very useful to study the internal Zebra indexes.
724 If records have been indexed using the <literal>alvis</literal>
725 XSLT filter, the string representation names of the known indexes can be
729 Z> find @and @attr exp1 1=1 attributedetails @attr exp1 1=3 Default
731 Identical query with explicitly specified attribute set:
734 Z> find @attrset exp1 @and @attr 1=1 attributedetails @attr 1=3 Default
741 <sect2 id="querymodel-bib1">
742 <title>Bib1 Attribute Set</title>
744 Most of the information contained in this section is an excerpt of
745 the <literal>ATTRIBUTE SET BIB-1 (Z39.50-1995)
747 found at <ulink url="&url.z39.50.attset.bib1.1995;">. The BIB-1
748 Attribute Set Semantics</ulink> from 1995, also in an updated
749 <ulink url="&url.z39.50.attset.bib1;">Bib-1
750 Attribute Set</ulink>
751 version from 2003. Index Data is not the copyright holder of this
752 information, except for the configuration details, the listing of
753 Zebra's capabilities, and the example queries.
757 <sect3 id="querymodel-bib1-use">
758 <title>Use Attributes (type 1)</title>
761 A use attribute specifies an access point for any atomic query.
762 These acess points are highly dependent on the attribute set used
763 in the query, and are user configurable using the following
764 default configuration files:
765 <filename>tab/bib1.att</filename>,
766 <filename>tab/dan1.att</filename>,
767 <filename>tab/explain.att</filename>, and
768 <filename>tab/gils.att</filename>.
769 New attribute sets can be added by adding new
770 <filename>tab/*.att</filename> configuration files, which need to
771 be sourced in the main configuration <filename>zebra.cfg</filename>.
775 In addition, Zebra allows the acess of
776 <emphasis>internal index names</emphasis> and <emphasis>dynamic
777 XPath</emphasis> as use attributes; see
778 <xref linkend="querymodel-use-string"/> and
779 <xref linkend="querymodel-use-xpath"/>.
783 Phrase search for <emphasis>information retrieval</emphasis> in
784 the title-register, scanning the same register afterwards:
786 Z> find @attr 1=4 "information retrieval"
787 Z> scan @attr 1=4 information
795 <sect2 id="querymodel-bib1-nonuse">
796 <title>Zebra general Bib1 Non-Use Attributes (type 2-6)</title>
798 <sect3 id="querymodel-bib1-relation">
799 <title>Relation Attributes (type 2)</title>
802 Relation attributes describe the relationship of the access
804 of the relation) to the search term as qualified by the attributes (right
805 side of the relation), e.g., Date-publication <= 1975.
808 <table id="querymodel-bib1-relation-table"
809 frame="all" rowsep="1" colsep="1" align="center">
811 <caption>Relation Attributes (type 2)</caption>
826 <td>Less than or equal</td>
836 <td>Greater or equal</td>
841 <td>Greater than</td>
866 <td>AlwaysMatches</td>
874 The relation attributes
875 <literal>1-5</literal> are supported and work exactly as
877 All ordering operations are based on a lexicographical ordering,
878 <emphasis>expect</emphasis> when the
879 <literal>structure attribute numeric (109)</literal> is used. In
880 this case, ordering is numerical. See
881 <xref linkend="querymodel-bib1-structure"/>.
883 Z> find @attr 1=Title @attr 2=1 music
885 Number of hits: 11745, setno 1
887 Z> find @attr 1=Title @attr 2=2 music
889 Number of hits: 11771, setno 2
891 Z> find @attr 1=Title @attr 2=3 music
893 Number of hits: 532, setno 3
895 Z> find @attr 1=Title @attr 2=4 music
897 Number of hits: 11463, setno 4
899 Z> find @attr 1=Title @attr 2=5 music
901 Number of hits: 11419, setno 5
906 The relation attribute
907 <literal>Relevance (102)</literal> is supported, see
908 <xref linkend="administration-ranking"/> for full information.
912 Ranked search for <emphasis>information retrieval</emphasis> in
915 Z> find @attr 1=4 @attr 2=102 "information retrieval"
920 The relation attribute
921 <literal>AlwaysMatches (103)</literal> is in the default
923 supported in conjecture with structure attribute
924 <literal>Phrase (1)</literal> (which may be omitted by
926 It can be configured to work with other structure attributes,
927 see the configuration file
928 <filename>tab/default.idx</filename> and
929 <xref linkend="querymodel-pqf-apt-mapping"/>.
932 <literal>AlwaysMatches (103)</literal> is a
933 great way to discover how many documents have been indexed in a
934 given field. The search term is ignored, but needed for correct
935 PQF syntax. An empty search term may be supplied.
937 Z> find @attr 1=Title @attr 2=103 ""
938 Z> find @attr 1=Title @attr 2=103 @attr 4=1 ""
945 <sect3 id="querymodel-bib1-position">
946 <title>Position Attributes (type 3)</title>
949 The position attribute specifies the location of the search term
950 within the field or subfield in which it appears.
953 <table id="querymodel-bib1-position-table"
954 frame="all" rowsep="1" colsep="1" align="center">
956 <caption>Position Attributes (type 3)</caption>
966 <td>First in field </td>
971 <td>First in subfield</td>
976 <td>Any position in field</td>
984 The position attribute values <literal>first in field (1)</literal>,
985 and <literal>first in subfield(2)</literal> are unsupported.
986 Using them does not trigger an error, but silent defaults to
987 <literal>any position in field (3)</literal>.
992 <sect3 id="querymodel-bib1-structure">
993 <title>Structure Attributes (type 4)</title>
996 The structure attribute specifies the type of search
997 term. This causes the search to be mapped on
998 different Zebra internal indexes, which must have been defined
1003 The possible values of the
1004 <literal>structure attribute (type 4)</literal> can be defined
1005 using the configuration file <filename>
1006 tab/default.idx</filename>.
1007 The default configuration is summerized in this table.
1010 <table id="querymodel-bib1-structure-table"
1011 frame="all" rowsep="1" colsep="1" align="center">
1013 <caption>Structure Attributes (type 4)</caption>
1043 <td>Date (normalized)</td>
1053 <td>Date (un-normalized)</td>
1055 <td>unsupported</td>
1058 <td>Name (normalized) </td>
1060 <td>unsupported</td>
1063 <td>Name (un-normalized) </td>
1065 <td>unsupported</td>
1070 <td>unsupported</td>
1078 <td>Free-form-text</td>
1083 <td>Document-text</td>
1088 <td>Local-number</td>
1095 <td>unsupported</td>
1098 <td>Numeric string</td>
1107 The structure attribute values
1108 <literal>Word list (6)</literal>
1109 is supported, and maps to the boolean <literal>AND</literal>
1110 combination of words supplied. The word list is useful when
1111 google-like bag-of-word queries need to be translated from a GUI
1112 query language to PQF. For example, the following queries
1115 Z> find @attr 1=Title @attr 4=6 "mozart amadeus"
1116 Z> find @attr 1=Title @and mozart amadeus
1121 The structure attribute value
1122 <literal>Free-form-text (105)</literal> and
1123 <literal>Document-text (106)</literal>
1124 are supported, and map both to the boolean <literal>OR</literal>
1125 combination of words supplied. The following queries
1128 Z> find @attr 1=Body-of-text @attr 4=105 "bach salieri teleman"
1129 Z> find @attr 1=Body-of-text @attr 4=106 "bach salieri teleman"
1130 Z> find @attr 1=Body-of-text @or bach @or salieri teleman
1132 This <literal>OR</literal> list of terms is very usefull in
1133 combination with relevance ranking:
1135 Z> find @attr 1=Body-of-text @attr 2=102 @attr 4=105 "bach salieri teleman"
1140 The structure attribute value
1141 <literal>Local number (107)</literal>
1142 is supported, and maps always to the Zebra internal document ID,
1143 irrespectively which use attribute is specified. The following queries
1144 have exactly the same unique record in the hit set:
1146 Z> find @attr 4=107 10
1147 Z> find @attr 1=4 @attr 4=107 10
1148 Z> find @attr 1=1010 @attr 4=107 10
1154 the GILS schema (<literal>gils.abs</literal>), the
1155 west-bounding-coordinate is indexed as type <literal>n</literal>,
1156 and is therefore searched by specifying
1157 <emphasis>structure</emphasis>=<emphasis>Numeric String</emphasis>.
1158 To match all those records with west-bounding-coordinate greater
1159 than -114 we use the following query:
1161 Z> find @attr 4=109 @attr 2=5 @attr gils 1=2038 -114
1165 The exact mapping between PQF queries and Zebra internal indexes
1166 and index types is explained in
1167 <xref linkend="querymodel-pqf-apt-mapping"/>.
1172 <sect3 id="querymodel-bib1-truncation">
1173 <title>Truncation Attributes (type = 5)</title>
1176 The truncation attribute specifies whether variations of one or
1177 more characters are allowed between serch term and hit terms, or
1178 not. Using non-default truncation attributes will broaden the
1179 document hit set of a search query.
1182 <table id="querymodel-bib1-truncation-table"
1183 frame="all" rowsep="1" colsep="1" align="center">
1185 <caption>Truncation Attributes (type 5)</caption>
1195 <td>Right truncation </td>
1200 <td>Left truncation</td>
1205 <td>Left and right truncation</td>
1210 <td>Do not truncate</td>
1215 <td>Process # in search term</td>
1233 The truncation attribute values 1-3 perform the obvious way:
1235 Z> scan @attr 1=Body-of-text schnittke
1241 Z> find @attr 1=Body-of-text @attr 5=1 schnittke
1243 Number of hits: 95, setno 7
1245 Z> find @attr 1=Body-of-text @attr 5=2 schnittke
1247 Number of hits: 81, setno 6
1249 Z> find @attr 1=Body-of-text @attr 5=3 schnittke
1251 Number of hits: 95, setno 8
1256 The truncation attribute value
1257 <literal>Process # in search term (101)</literal> is a
1258 poor-man's regular expression search. It maps
1259 each <literal>#</literal> to <literal>.*</literal>, and
1260 performes then a <literal>Regexp-1 (102)</literal> regular
1261 expression search. The following two queries are equivalent:
1263 Z> find @attr 1=Body-of-text @attr 5=101 schnit#ke
1264 Z> find @attr 1=Body-of-text @attr 5=102 schnit.*ke
1266 Number of hits: 89, setno 10
1271 The truncation attribute value
1272 <literal>Regexp-1 (102)</literal> is a normal regular search,
1273 see <xref linkend="querymodel-regular"/> for details.
1275 Z> find @attr 1=Body-of-text @attr 5=102 schnit+ke
1276 Z> find @attr 1=Body-of-text @attr 5=102 schni[a-t]+ke
1281 The truncation attribute value
1282 <literal>Regexp-2 (103) </literal> is a Zebra specific extention
1283 which allows <emphasis>fuzzy</emphasis> matches. One single
1284 error in spelling of search terms is allowed, i.e., a document
1285 is hit if it includes a term which can be mapped to the used
1286 search term by one character substitution, addition, deletion or
1289 Z> find @attr 1=Body-of-text @attr 5=100 schnittke
1291 Number of hits: 81, setno 14
1293 Z> find @attr 1=Body-of-text @attr 5=103 schnittke
1295 Number of hits: 103, setno 15
1301 <sect3 id="querymodel-bib1-completeness">
1302 <title>Completeness Attributes (type = 6)</title>
1306 The <literal>Completeness Attributes (type = 6)</literal>
1307 is used to specify that a given search term or term list is either
1308 part of the terms of a given index/field
1309 (<literal>Incomplete subfield (1)</literal>), or is
1310 what literally is found in the entire field's index
1311 (<literal>Complete field (3)</literal>).
1314 <table id="querymodel-bib1-completeness-table"
1315 frame="all" rowsep="1" colsep="1" align="center">
1316 <caption>Completeness Attributes (type = 6)</caption>
1319 <td>Completeness</td>
1326 <td>Incomplete subfield</td>
1331 <td>Complete subfield</td>
1333 <td>depreciated</td>
1336 <td>Complete field</td>
1344 The <literal>Completeness Attributes (type = 6)</literal>
1345 is only partially and conditionally
1346 supported in the sense that it is ignored if the hit index is
1347 not of structure <literal>type="w"</literal> or
1348 <literal>type="p"</literal>.
1351 <literal>Incomplete subfield (1)</literal> is the default, and
1353 register <literal>type="w"</literal>, whereas
1354 <literal>Complete field (3)</literal> triggers
1355 search and scan in index <literal>type="p"</literal>.
1358 The <literal>Complete subfield (2)</literal> is a reminiscens
1359 from the happy <literal>MARC</literal>
1360 binary format days. Zebra does not support it, but maps silently
1361 to <literal>Complete field (3)</literal>.
1365 The exact mapping between PQF queries and Zebra internal indexes
1366 and index types is explained in
1367 <xref linkend="querymodel-pqf-apt-mapping"/>.
1375 <sect1 id="querymodel-zebra">
1376 <title>Advanced Zebra PQF Features</title>
1378 The Zebra internal query engine has been extended to specific needs
1379 not covered by the <literal>bib-1</literal> attribute set query
1380 model. These extentions are <emphasis>non-standard</emphasis>
1381 and <emphasis>non-portable</emphasis>: most functional extentions
1382 are modeled over the <literal>bib-1</literal> attribute set,
1383 defining type 7-9 attributes.
1384 There are also the speciel
1385 <literal>string</literal> type index names for the
1386 <literal>idxpath</literal> attribute set.
1389 <sect2 id="querymodel-zebra-attr-allrecords">
1390 <title>Zebra specific retrieval of all records</title>
1392 Zebra defines a hardwired <literal>string</literal> index name
1393 called <literal>_ALLRECORDS</literal>. It matches any record
1394 contained in the database, if used in conjunction with
1395 the relation attribute
1396 <literal>AlwaysMatches (103)</literal>.
1399 The <literal>_ALLRECORDS</literal> index name is used for total database
1400 export. The search term is ignored, it may be empty.
1402 Z> find @attr 1=_ALLRECORDS @attr 2=103 ""
1406 Combination with other index types can be made. For example, to
1407 find all records which are <emphasis>not</emphasis> indexed in
1408 the <literal>Title</literal> register, issue one of the two
1411 Z> find @not @attr 1=_ALLRECORDS @attr 2=103 "" @attr 1=Title @attr 2=103 ""
1412 Z> find @not @attr 1=_ALLRECORDS @attr 2=103 "" @attr 1=4 @attr 2=103 ""
1416 The special string index <literal>_ALLRECORDS</literal> is
1417 experimental, and the provided functionality and syntax may very
1418 well change in future releases of Zebra.
1423 <sect2 id="querymodel-zebra-attr-search">
1424 <title>Zebra specific Search Extentions to all Attribute Sets</title>
1426 Zebra extends the Bib1 attribute types, and these extentions are
1427 recognized regardless of attribute
1428 set used in a <literal>search</literal> operation query.
1431 <table id="querymodel-zebra-attr-search-table"
1432 frame="all" rowsep="1" colsep="1" align="center">
1434 <caption>Zebra Search Attribute Extentions</caption>
1440 <td>Zebra version</td>
1445 <td>Embedded Sort</td>
1457 <td>Rank Weight</td>
1463 <td>Approx Limit</td>
1469 <td>Term Reference</td>
1477 <sect3 id="querymodel-zebra-attr-sorting">
1478 <title>Zebra Extention Embedded Sort Attribute (type 7)</title>
1481 The embedded sort is a way to specify sort within a query - thus
1482 removing the need to send a Sort Request separately. It is both
1483 faster and does not require clients to deal with the Sort
1488 All ordering operations are based on a lexicographical ordering,
1489 <emphasis>expect</emphasis> when the
1490 <literal>structure attribute numeric (109)</literal> is used. In
1491 this case, ordering is numerical. See
1492 <xref linkend="querymodel-bib1-structure"/>.
1496 The possible values after attribute <literal>type 7</literal> are
1497 <literal>1</literal> ascending and
1498 <literal>2</literal> descending.
1499 The attributes+term (APT) node is separate from the
1500 rest and must be <literal>@or</literal>'ed.
1501 The term associated with APT is the sorting level in integers,
1502 where <literal>0</literal> means primary sort,
1503 <literal>1</literal> means secondary sort, and so forth.
1504 See also <xref linkend="administration-ranking"/>.
1507 For example, searching for water, sort by title (ascending)
1509 Z> find @or @attr 1=1016 water @attr 7=1 @attr 1=4 0
1513 Or, searching for water, sort by title ascending, then date descending
1515 Z> find @or @or @attr 1=1016 water @attr 7=1 @attr 1=4 0 @attr 7=2 @attr 1=30 1
1519 <sect3 id="querymodel-zebra-attr-estimation">
1520 <title>Zebra Extention Term Set Attribute (type 8)</title>
1523 The Term Set feature is a facility that allows a search to store
1524 hitting terms in a "pseudo" resultset; thus a search (as usual) +
1525 a scan-like facility. Requires a client that can do named result
1526 sets since the search generates two result sets. The value for
1527 attribute 8 is the name of a result set (string). The terms in
1528 the named term set are returned as SUTRS records.
1531 For example, searching for u in title, right truncated, and
1532 storing the result in term set named 'aset'
1534 Z> find @attr 5=1 @attr 1=4 @attr 8=aset u
1538 The model has one serious flaw: we don't know the size of term
1539 set. Experimental. Do not use in production code.
1542 <sect3 id="querymodel-zebra-attr-weight">
1543 <title>Zebra Extention Rank Weight Attribute (type 9)</title>
1546 Rank weight is a way to pass a value to a ranking algorithm - so
1547 that one APT has one value - while another as a different one.
1548 See also <xref linkend="administration-ranking"/>.
1551 For example, searching for utah in title with weight 30 as well
1552 as any with weight 20:
1554 Z> find @attr 2=102 @or @attr 9=30 @attr 1=4 utah @attr 9=20 utah
1558 <sect3 id="querymodel-zebra-attr-limit">
1559 <title>Zebra Extention Approximative Limit Attribute (type 9)</title>
1562 Newer Zebra versions normally estemiates hit count for every APT
1563 (leaf) in the query tree. These hit counts are returned as part of
1564 the searchResult-1 facility in the binary encoded Z39.50 search
1568 By setting a limit for the APT we can make Zebra turn into
1569 approximate hit count when a certain hit count limit is
1570 reached. A value of zero means exact hit count.
1573 For example, we might be intersted in exact hit count for a, but
1574 for b we allow hit count estimates for 1000 and higher.
1576 Z> find @and a @attr 9=1000 b
1580 The estimated hit count fascility makes searches faster, as one
1581 only needs to process large hit lists partially.
1584 This facility clashes with rank weight, because there all
1585 documents in the hit lists need to be examined for scoring and
1587 It is an experimental
1588 extention. Do not use in production code.
1591 <sect3 id="querymodel-zebra-attr-termref">
1592 <title>Zebra Extention Term Reference Attribute (type 10)</title>
1595 Zebra supports the <literal>searchResult-1</literal> facility.
1596 If the <literal>Term Reference Attribute (type 10)</literal> is
1597 given, that specifies a subqueryId value returned as part of the
1598 search result. It is a way for a client to name an APT part of a
1608 Experimental. Do not use in production code.
1615 <sect2 id="querymodel-zebra-attr-scan">
1616 <title>Zebra specific Scan Extentions to all Attribute Sets</title>
1618 Zebra extends the Bib1 attribute types, and these extentions are
1619 recognized regardless of attribute
1620 set used in a <literal>scan</literal> operation query.
1622 <table id="querymodel-zebra-attr-scan-table"
1623 frame="all" rowsep="1" colsep="1" align="center">
1625 <caption>Zebra Scan Attribute Extentions</caption>
1631 <td>Zebra version</td>
1636 <td>Result Set Narrow</td>
1642 <td>Approximative Limit</td>
1650 <sect3 id="querymodel-zebra-attr-narrow">
1651 <title>Zebra Extention Result Set Narrow (type 8)</title>
1654 If attribute <literal>Result Set Narrow (type 8)</literal>
1655 is given for <literal>scan</literal>, the value is the name of a
1656 result set. Each hit count in <literal>scan</literal> is
1657 <literal>@and</literal>'ed with the result set given.
1660 Consider for example
1661 the case of scanning all title fields around the
1662 scanterm <emphasis>mozart</emphasis>, then refining the scan by
1663 issuing a filtering query for <emphasis>amadeus</emphasis> to
1664 restric the scan to the result set of the query:
1666 Z> scan @attr 1=4 mozart
1669 mozartforskningen (1)
1673 Z> f @attr 1=4 amadeus
1675 Number of hits: 15, setno 2
1677 Z> scan @attr 1=4 @attr 8=2 mozart
1680 mozartforskningen (0)
1688 Experimental. Do not use in production code.
1691 <sect3 id="querymodel-zebra-attr-approx">
1692 <title>Zebra Extention Approximative Limit (type 9)</title>
1695 The <literal>Zebra Extention Approximative Limit (type
1696 9)</literal> is a way to enable approx
1697 hit counts for <literal>scan</literal> hit counts, in the same
1698 way as for <literal>search</literal> hit counts.
1707 Experimental and buggy. Definitely not to be used in production code.
1714 <sect2 id="querymodel-idxpath">
1715 <title>Zebra special IDXPATH Attribute Set for GRS indexing</title>
1717 The attribute-set <literal>idxpath</literal> consists of a single
1718 <literal>Use (type 1)</literal> attribute. All non-use attributes
1722 This feature is enabled when defining the
1723 <literal>xpath enable</literal> option in the GRS filter
1724 <literal>*.abs</literal> configuration files. If one wants to use
1725 the special <literal>idxpath</literal> numeric attribute set, the
1726 main Zebra configuraiton file <filename>zebra.cfg</filename>
1727 directive <literal>attset: idxpath.att</literal> must be enabled.
1729 <warning>The <literal>idxpath</literal> is depreciated, may not be
1730 supported in future Zebra versions, and should definitely
1731 not be used in production code.
1734 <sect3 id="querymodel-idxpath-use">
1735 <title>IDXPATH Use Attributes (type = 1)</title>
1737 This attribute set allows one to search GRS filter indexed
1738 records by XPATH like structured index names.
1741 <warning>The <literal>idxpath</literal> option defines hard-coded
1742 index names, which might clash with your own index names.
1745 <table id="querymodel-idxpath-use-table"
1746 frame="all" rowsep="1" colsep="1" align="center">
1748 <caption>Zebra specific IDXPATH Use Attributes (type 1)</caption>
1753 <td>String Index</td>
1759 <td>XPATH Begin</td>
1761 <td>_XPATH_BEGIN</td>
1762 <td>depreciated</td>
1768 <td>depreciated</td>
1771 <td>XPATH CData</td>
1773 <td>_XPATH_CDATA</td>
1774 <td>depreciated</td>
1777 <td>XPATH Attribute Name</td>
1779 <td>_XPATH_ATTR_NAME</td>
1780 <td>depreciated</td>
1783 <td>XPATH Attribute CData</td>
1785 <td>_XPATH_ATTR_CDATA</td>
1786 <td>depreciated</td>
1793 See <filename>tab/idxpath.att</filename> for more information.
1796 Search for all documents starting with root element
1797 <literal>/root</literal> (either using the numeric or the string
1800 Z> find @attrset idxpath @attr 1=1 @attr 4=3 root/
1801 Z> find @attr idxpath 1=1 @attr 4=3 root/
1802 Z> find @attr 1=_XPATH_BEGIN @attr 4=3 root/
1806 Search for all documents where specific nested XPATH
1807 <literal>/c1/c2/../cn</literal> exists. Notice the very
1808 counter-intuitive <emphasis>reverse</emphasis> notation!
1810 Z> find @attrset idxpath @attr 1=1 @attr 4=3 cn/cn-1/../c1/
1811 Z> find @attr 1=_XPATH_BEGIN @attr 4=3 cn/cn-1/../c1/
1815 Search for CDATA string <emphasis>text</emphasis> in any element
1817 Z> find @attrset idxpath @attr 1=1016 text
1818 Z> find @attr 1=_XPATH_CDATA text
1822 Search for CDATA string <emphasis>anothertext</emphasis> in any
1825 Z> find @attrset idxpath @attr 1=1015 anothertext
1826 Z> find @attr 1=_XPATH_ATTR_CDATA anothertext
1830 Search for all documents with have an XML element node
1831 including an XML attribute named <emphasis>creator</emphasis>
1833 Z> find @attrset idxpath @attr 1=3 @attr 4=3 creator
1834 Z> find @attr 1=_XPATH_ATTR_NAME @attr 4=3 creator
1838 Combining usual <literal>bib-1</literal> attribut set searches
1839 with <literal>idxpath</literal> attribute set searches:
1841 Z> find @and @attr idxpath 1=1 @attr 4=3 link/ @attr 1=4 mozart
1842 Z> find @and @attr 1=_XPATH_BEGIN @attr 4=3 link/ @attr 1=_XPATH_CDATA mozart
1846 Scanning is supportet on all <literal>idxpath</literal>
1847 indexes, both specified as numeric use attributes, or as string
1850 Z> scan @attrset idxpath @attr 1=1016 text
1851 Z> scan @attr 1=_XPATH_ATTR_CDATA anothertext
1852 Z> scan @attrset idxpath @attr 1=3 @attr 4=3 ''
1860 <sect2 id="querymodel-pqf-apt-mapping">
1861 <title>Mapping from PQF atomic APT queries to Zebra internal
1862 register indexes</title>
1864 The rules for PQF APT mapping are rather tricky to grasp in the
1865 first place. We deal first with the rules for deciding which
1866 internal register or string index to use, according to the use
1867 attribute or access point specified in the query. Thereafter we
1868 deal with the rules for tetermining the correct structure type of
1872 <sect3 id="querymodel-pqf-apt-mapping-accesspoint">
1873 <title>Mapping of PQF APT access points</title>
1875 Zebra understands four fundamental different types of access
1876 points, of which only the
1877 <emphasis>numeric use attribute</emphasis> type access points
1878 are defined by the <ulink url="&url.z39.50;">Z39.50</ulink>
1880 All other access point types are Zebra specific, and non-portable.
1883 <table id="querymodel-zebra-mapping-accesspoint-types"
1884 frame="all" rowsep="1" colsep="1" align="center">
1886 <caption>Acces point name</caption>
1889 <td>Acess Point</td>
1897 <td>Use attibute</td>
1899 <td>[1-9][1-9]*</td>
1900 <td>directly mapped to string index name</td>
1903 <td>String index name</td>
1905 <td>[a-zA-Z](\-?[a-zA-Z0-9])*</td>
1906 <td>normalized name is used as internal string index name</td>
1909 <td>Zebra internal index name</td>
1911 <td>_[a-zA-Z](_?[a-zA-Z0-9])*</td>
1912 <td>hardwired internal string index name</td>
1915 <td>XPATH special index</td>
1918 <td>special xpath search for GRS indexed records</td>
1924 <literal>Attribute set names</literal> and
1925 <literal>string index names</literal> are normalizes
1926 according to the following rules: all <emphasis>single</emphasis>
1927 hyphens <literal>'-'</literal> are stripped, and all upper case
1928 letters are folded to lower case.</para>
1931 <emphasis>Numeric use attributes</emphasis> are mapped
1932 to the Zebra internal
1933 string index according to the attribute set defintion in use.
1934 The default attribute set is <literal>Bib-1</literal>, and may be
1935 omitted in the PQF query. According to normalization and numeric
1936 use attribute mapping, it follows that the following
1937 PQF queries are considered equivalent (assuming the default
1938 configuration has not been altered):
1940 Z> find @attr 1=Body-of-text serenade
1941 Z> find @attr 1=bodyoftext serenade
1942 Z> find @attr 1=BodyOfText serenade
1943 Z> find @attr 1=bO-d-Y-of-tE-x-t serenade
1944 Z> find @attr 1=1010 serenade
1945 Z> find @attrset Bib-1 @attr 1=1010 serenade
1946 Z> find @attrset bib1 @attr 1=1010 serenade
1947 Z> find @attrset Bib1 @attr 1=1010 serenade
1948 Z> find @attrset b-I-b-1 @attr 1=1010 serenade
1953 The <emphasis>numerical</emphasis>
1954 <literal>use attributes (type 1)</literal>
1955 are interpreted according to the
1956 attribute sets which have been loaded in the
1957 <literal>zebra.cfg</literal> file, and are matched against specific
1958 fields as specified in the <literal>.abs</literal> file which
1959 describes the profile of the records which have been loaded.
1960 If no use attribute is provided, a default of Bib-1 Any is
1962 The predefined <literal>use attribute sets</literal>
1963 can be reconfigured by tweaking the configuration files
1964 <filename>tab/*.att</filename>, and
1965 new attribute sets can be defined by adding similar files in the
1966 configuration path <literal>profilePath</literal> of the server.
1970 <literal>String indexes</literal> can be acessed directly,
1971 independently which attribute set is in use. These are just
1972 ignored. The above mentioned name normalization applies.
1973 <literal>String index names</literal> are defined in the
1974 used indexing filter configuration files, for example in the
1975 <literal>GRS</literal>
1976 <filename>*.abs</filename> configuration files, or in the
1977 <literal>alvis</literal> filter XSLT indexing stylesheets.
1981 <literal>Zebra internal indexes</literal> can be acessed directly,
1982 according to the same rules as the user defined
1983 <literal>string indexes</literal>. The only difference is that
1984 <literal>Zebra internal indexe names</literal> are hardwired,
1986 must start with the character <literal>'_'</literal>.
1990 Finally, <literal>XPATH</literal> access points are only
1991 available using the <literal>GRS</literal> filter for indexing.
1992 These acees point names must start with the character
1993 <literal>'/'</literal>, they are <emphasis>not
1994 normalized</emphasis>, but passed unaltered to the Zebra internal
1995 XPATH engine. See <xref linkend="querymodel-use-xpath"/>.
2003 <sect3 id="querymodel-pqf-apt-mapping-structuretype">
2004 <title>Mapping of PQF APT structure and type</title>
2008 <!-- see in util/zebramap.c
2011 if (completeness_value == 2 || completeness_value == 3)
2017 *sort_flag =(sort_relation_value > 0) ? 1 : 0;
2018 *search_type = "phrase";
2019 strcpy(rank_type, "void");
2020 if (relation_value == 102)
2022 if (weight_value == -1)
2024 sprintf(rank_type, "rank,w=%d,u=%d", weight_value, use_value);
2026 if (relation_value == 103)
2028 *search_type = "always";
2036 switch (structure_value)
2038 case 6: /* word list */
2039 *search_type = "and-list";
2041 case 105: /* free-form-text */
2042 *search_type = "or-list";
2044 case 106: /* document-text */
2045 *search_type = "or-list";
2048 case 1: /* phrase */
2050 case 108: /* string */
2051 *search_type = "phrase";
2053 case 107: /* local-number */
2054 *search_type = "local";
2057 case 109: /* numeric string */
2059 *search_type = "numeric";
2063 *search_type = "phrase";
2067 *search_type = "phrase";
2071 *search_type = "phrase";
2075 *search_type = "phrase";
2087 If a <emphasis>Structure</emphasis> attribute of
2088 <emphasis>Phrase</emphasis> is used in conjunction with a
2089 <emphasis>Completeness</emphasis> attribute of
2090 <emphasis>Complete (Sub)field</emphasis>, the term is matched
2091 against the contents of the phrase (long word) register, if one
2092 exists for the given <emphasis>Use</emphasis> attribute.
2093 A phrase register is created for those fields in the
2094 <literal>.abs</literal> file that contains a
2095 <literal>p</literal>-specifier.
2096 <!-- ### whatever the hell _that_ is -->
2100 If <emphasis>Structure</emphasis>=<emphasis>Phrase</emphasis> is
2101 used in conjunction with <emphasis>Incomplete Field</emphasis> - the
2102 default value for <emphasis>Completeness</emphasis>, the
2103 search is directed against the normal word registers, but if the term
2104 contains multiple words, the term will only match if all of the words
2105 are found immediately adjacent, and in the given order.
2106 The word search is performed on those fields that are indexed as
2107 type <literal>w</literal> in the <literal>.abs</literal> file.
2111 If the <emphasis>Structure</emphasis> attribute is
2112 <emphasis>Word List</emphasis>,
2113 <emphasis>Free-form Text</emphasis>, or
2114 <emphasis>Document Text</emphasis>, the term is treated as a
2115 natural-language, relevance-ranked query.
2116 This search type uses the word register, i.e. those fields
2117 that are indexed as type <literal>w</literal> in the
2118 <literal>.abs</literal> file.
2122 If the <emphasis>Structure</emphasis> attribute is
2123 <emphasis>Numeric String</emphasis> the term is treated as an integer.
2124 The search is performed on those fields that are indexed
2125 as type <literal>n</literal> in the <literal>.abs</literal> file.
2129 If the <emphasis>Structure</emphasis> attribute is
2130 <emphasis>URx</emphasis> the term is treated as a URX (URL) entity.
2131 The search is performed on those fields that are indexed as type
2132 <literal>u</literal> in the <literal>.abs</literal> file.
2136 If the <emphasis>Structure</emphasis> attribute is
2137 <emphasis>Local Number</emphasis> the term is treated as
2138 native Zebra Record Identifier.
2142 If the <emphasis>Relation</emphasis> attribute is
2143 <emphasis>Equals</emphasis> (default), the term is matched
2144 in a normal fashion (modulo truncation and processing of
2145 individual words, if required).
2146 If <emphasis>Relation</emphasis> is <emphasis>Less Than</emphasis>,
2147 <emphasis>Less Than or Equal</emphasis>,
2148 <emphasis>Greater than</emphasis>, or <emphasis>Greater than or
2149 Equal</emphasis>, the term is assumed to be numerical, and a
2150 standard regular expression is constructed to match the given
2152 If <emphasis>Relation</emphasis> is <emphasis>Relevance</emphasis>,
2153 the standard natural-language query processor is invoked.
2157 For the <emphasis>Truncation</emphasis> attribute,
2158 <emphasis>No Truncation</emphasis> is the default.
2159 <emphasis>Left Truncation</emphasis> is not supported.
2160 <emphasis>Process # in search term</emphasis> is supported, as is
2161 <emphasis>Regxp-1</emphasis>.
2162 <emphasis>Regxp-2</emphasis> enables the fault-tolerant (fuzzy)
2163 search. As a default, a single error (deletion, insertion,
2164 replacement) is accepted when terms are matched against the register
2171 <sect2 id="querymodel-regular">
2172 <title>Zebra Regular Expressions in Truncation Attribute (type = 5)</title>
2175 Each term in a query is interpreted as a regular expression if
2176 the truncation value is either <emphasis>Regxp-1 (@attr 5=102)</emphasis>
2177 or <emphasis>Regxp-2 (@attr 5=103)</emphasis>.
2178 Both query types follow the same syntax with the operands:
2181 <table id="querymodel-regular-operands-table"
2182 frame="all" rowsep="1" colsep="1" align="center">
2184 <caption>Regular Expression Operands</caption>
2187 <tr><td>one</td><td>two</td></tr>
2192 <td><literal>x</literal></td>
2193 <td>Matches the character <literal>x</literal>.</td>
2196 <td><literal>.</literal></td>
2197 <td>Matches any character.</td>
2200 <td><literal>[ .. ]</literal></td>
2201 <td>Matches the set of characters specified;
2202 such as <literal>[abc]</literal> or <literal>[a-c]</literal>.</td>
2208 The above operands can be combined with the following operators:
2211 <table id="querymodel-regular-operators-table"
2212 frame="all" rowsep="1" colsep="1" align="center">
2213 <caption>Regular Expression Operators</caption>
2216 <tr><td>one</td><td>two</td></tr>
2221 <td><literal>x*</literal></td>
2222 <td>Matches <literal>x</literal> zero or more times.
2223 Priority: high.</td>
2226 <td><literal>x+</literal></td>
2227 <td>Matches <literal>x</literal> one or more times.
2228 Priority: high.</td>
2231 <td><literal>x?</literal></td>
2232 <td> Matches <literal>x</literal> zero or once.
2233 Priority: high.</td>
2236 <td><literal>xy</literal></td>
2237 <td> Matches <literal>x</literal>, then <literal>y</literal>.
2238 Priority: medium.</td>
2241 <td><literal>x|y</literal></td>
2242 <td> Matches either <literal>x</literal> or <literal>y</literal>.
2246 <td><literal>( )</literal></td>
2247 <td>The order of evaluation may be changed by using parentheses.</td>
2253 If the first character of the <literal>Regxp-2</literal> query
2254 is a plus character (<literal>+</literal>) it marks the
2255 beginning of a section with non-standard specifiers.
2256 The next plus character marks the end of the section.
2257 Currently Zebra only supports one specifier, the error tolerance,
2258 which consists one digit.
2262 Since the plus operator is normally a suffix operator the addition to
2263 the query syntax doesn't violate the syntax for standard regular
2268 For example, a phrase search with regular expressions in
2269 the title-register is performed like this:
2271 Z> find @attr 1=4 @attr 5=102 "informat.* retrieval"
2276 Combinations with other attributes are possible. For example, a
2277 ranked search with a regular expression:
2279 Z> find @attr 1=4 @attr 5=102 @attr 2=102 "informat.* retrieval"
2287 The RecordType parameter in the <literal>zebra.cfg</literal> file, or
2288 the <literal>-t</literal> option to the indexer tells Zebra how to
2289 process input records.
2290 Two basic types of processing are available - raw text and structured
2291 data. Raw text is just that, and it is selected by providing the
2292 argument <literal>text</literal> to Zebra. Structured records are
2293 all handled internally using the basic mechanisms described in the
2294 subsequent sections.
2295 Zebra can read structured records in many different formats.
2301 <sect1 id="querymodel-cql-to-pqf">
2302 <title>Server Side CQL to PQF Query Translation</title>
2305 <literal><cql2rpn>l2rpn.txt</cql2rpn></literal>
2306 YAZ Frontend Virtual
2307 Hosts option, one can configure
2308 the YAZ Frontend CQL-to-PQF
2309 converter, specifying the interpretation of various
2310 <ulink url="&url.cql;">CQL</ulink>
2311 indexes, relations, etc. in terms of Type-1 query attributes.
2312 <!-- The yaz-client config file -->
2315 For example, using server-side CQL-to-PQF conversion, one might
2316 query a zebra server like this:
2319 yaz-client localhost:9999
2321 Z> find text=(plant and soil)
2324 and - if properly configured - even static relevance ranking can
2325 be performed using CQL query syntax:
2328 Z> find text = /relevant (plant and soil)
2334 By the way, the same configuration can be used to
2335 search using client-side CQL-to-PQF conversion:
2336 (the only difference is <literal>querytype cql2rpn</literal>
2338 <literal>querytype cql</literal>, and the call specifying a local
2342 yaz-client -q local/cql2pqf.txt localhost:9999
2343 Z> querytype cql2rpn
2344 Z> find text=(plant and soil)
2350 Exhaustive information can be found in the
2351 Section "Specification of CQL to RPN mappings" in the YAZ manual.
2352 <ulink url="http://www.indexdata.dk/yaz/doc/tools.tkl#tools.cql.map">
2353 http://www.indexdata.dk/yaz/doc/tools.tkl#tools.cql.map</ulink>,
2354 and shall therefore not be repeated here.
2359 <ulink url="http://www.loc.gov/z3950/agency/zing/cql/dc-indexes.html">
2360 http://www.loc.gov/z3950/agency/zing/cql/dc-indexes.html</ulink>
2361 for the Maintenance Agency's work-in-progress mapping of Dublin Core
2362 indexes to Attribute Architecture (util, XD and BIB-2)
2372 <!-- Keep this comment at the end of the file
2377 sgml-minimize-attributes:nil
2378 sgml-always-quote-attributes:t
2381 sgml-parent-document: "zebra.xml"
2382 sgml-local-catalogs: nil
2383 sgml-namecase-general:t