Adam Dickmeiss [Fri, 22 Dec 2006 13:48:20 +0000 (13:48 +0000)]
Added function atoizint with a atoi/atoll wrapper.
Adam Dickmeiss [Fri, 22 Dec 2006 12:14:25 +0000 (12:14 +0000)]
Add timing for zebra_start/zebra_stop.
Adam Dickmeiss [Fri, 22 Dec 2006 12:13:39 +0000 (12:13 +0000)]
Use atoll/atoi for ascii to zint conversion in alvis filter.
Adam Dickmeiss [Wed, 20 Dec 2006 14:29:04 +0000 (14:29 +0000)]
Do not warn for truncmax limit
Adam Dickmeiss [Wed, 20 Dec 2006 14:19:21 +0000 (14:19 +0000)]
Do not truncate more than at most 'truncmax' terms in one single
term with right-truncation/regular attribute. By default, 'truncmax',
is 10000.
Adam Dickmeiss [Wed, 20 Dec 2006 08:37:49 +0000 (08:37 +0000)]
Fix reference to uninitialized var (bug introduced in previous commit).
Adam Dickmeiss [Tue, 19 Dec 2006 16:57:38 +0000 (16:57 +0000)]
Using zebra::index::field:s rather than zebra::sort::field for retrieval
of sort keys.
Adam Dickmeiss [Tue, 19 Dec 2006 13:55:05 +0000 (13:55 +0000)]
Fix bad ptr in yaz_log debug msg
Adam Dickmeiss [Tue, 19 Dec 2006 13:17:34 +0000 (13:17 +0000)]
Avoid file sync for read-only files.
Adam Dickmeiss [Tue, 19 Dec 2006 00:25:41 +0000 (00:25 +0000)]
Changed assert in isamb, since we have larger keys
Adam Dickmeiss [Mon, 18 Dec 2006 23:40:06 +0000 (23:40 +0000)]
Implemented sorting via the ISAMB system. To enable, use sortindex:b
in zebra.cfg. The sort keys can also be fetched using zebra::sort:field .
Adam Dickmeiss [Sat, 16 Dec 2006 09:04:28 +0000 (09:04 +0000)]
Disable check: bug #730
Adam Dickmeiss [Fri, 15 Dec 2006 21:11:37 +0000 (21:11 +0000)]
Define YAZ_HAVE_XML2
Adam Dickmeiss [Wed, 13 Dec 2006 13:05:45 +0000 (13:05 +0000)]
Fixed bug #767: Zebra indexing stylesheet fails to work. The problem
was that the root node was not of type element and split_level = 0 (deal with
whole record / no split). We make split level 0 a special case (again) and
use extract_full for this.
Adam Dickmeiss [Tue, 12 Dec 2006 17:33:35 +0000 (17:33 +0000)]
More digits in output for total docs
Adam Dickmeiss [Tue, 12 Dec 2006 15:23:45 +0000 (15:23 +0000)]
Rename isb to isb_postings.
Adam Dickmeiss [Tue, 12 Dec 2006 15:22:26 +0000 (15:22 +0000)]
Possible fix of two bugs (1) bug #759: Crash during retrieval of some
records (2) bug #765.
Adam Dickmeiss [Tue, 12 Dec 2006 13:54:33 +0000 (13:54 +0000)]
Added gnuplot scripts for dictisam timings, filesystems comparisons.
Adam Dickmeiss [Tue, 12 Dec 2006 13:51:23 +0000 (13:51 +0000)]
All output to stdout for easy feet into gnuplot. The following is now
logged: user,real,sys for dict+isam, postings, documents, distinct
words, new words, dict splits, isamb leaf splits, isamb interior splits.
Adam Dickmeiss [Tue, 12 Dec 2006 13:46:41 +0000 (13:46 +0000)]
Added functions isamb_set_int_count, isamb-set_cache_size,
isamb_get_int_splits, isamb_get_leaf_splits, iasmb_set_int_count.
Adam Dickmeiss [Tue, 12 Dec 2006 13:42:23 +0000 (13:42 +0000)]
Added statistics functions dict_get_no_{lookup,insert,split}.
Adam Dickmeiss [Mon, 11 Dec 2006 17:08:03 +0000 (17:08 +0000)]
Remove unused variable (dst_buf)
Adam Dickmeiss [Mon, 11 Dec 2006 15:08:55 +0000 (15:08 +0000)]
Allow MARC line format to be read.
Adam Dickmeiss [Mon, 11 Dec 2006 10:02:14 +0000 (10:02 +0000)]
Using timing utility
Adam Dickmeiss [Mon, 11 Dec 2006 09:50:36 +0000 (09:50 +0000)]
Added a utility for doing timings.
Adam Dickmeiss [Sun, 10 Dec 2006 21:02:28 +0000 (21:02 +0000)]
More appropriate usage
Adam Dickmeiss [Sun, 10 Dec 2006 21:00:56 +0000 (21:00 +0000)]
Make MARC indexer with different ISAM strategy.
Adam Dickmeiss [Sun, 10 Dec 2006 20:59:52 +0000 (20:59 +0000)]
Use rand() twice on platforms where RAND_MAX < 2^16.
Adam Dickmeiss [Sun, 10 Dec 2006 11:54:29 +0000 (11:54 +0000)]
Added new test which makes variations to ISAM entry size.
Adam Dickmeiss [Sun, 10 Dec 2006 11:54:11 +0000 (11:54 +0000)]
Produce .dat files with bench1. prefix
Adam Dickmeiss [Sun, 10 Dec 2006 11:49:16 +0000 (11:49 +0000)]
Allow size of ISAM entry to be specified in bench test
Adam Dickmeiss [Sat, 9 Dec 2006 08:03:57 +0000 (08:03 +0000)]
Added some performance test utilities.
Adam Dickmeiss [Thu, 7 Dec 2006 21:13:56 +0000 (21:13 +0000)]
Added some ISAMB benchmark code
Adam Dickmeiss [Thu, 7 Dec 2006 19:23:56 +0000 (19:23 +0000)]
Allow different values for cache parameter for isamb_open: 0=normal,
1=caching of pages, -1 testing (NULL ISAM).
Adam Dickmeiss [Thu, 7 Dec 2006 10:01:32 +0000 (10:01 +0000)]
Remove @XSLT_LIBS@ from idzebra-config --libs / --lalibs output. XSLT
libs is part of @YAZLIB@ already.
Adam Dickmeiss [Wed, 6 Dec 2006 22:52:38 +0000 (22:52 +0000)]
Use zint for numbers in update statistics.
Adam Dickmeiss [Wed, 6 Dec 2006 13:00:12 +0000 (13:00 +0000)]
Added new object check_res.
Adam Dickmeiss [Wed, 6 Dec 2006 10:26:54 +0000 (10:26 +0000)]
Added more to list of options.
Adam Dickmeiss [Wed, 6 Dec 2006 10:26:40 +0000 (10:26 +0000)]
Deal with a picky ambiguity with configs like a.b where both suffix and
prefix are allowed. Both a and b must be checked.
Adam Dickmeiss [Tue, 5 Dec 2006 14:06:29 +0000 (14:06 +0000)]
Fixed bug #445: zebra.cfg does not check for valid directives.
Adam Dickmeiss [Tue, 5 Dec 2006 09:26:37 +0000 (09:26 +0000)]
Provide defaults for profilePath and modulePath for the zebra{srv,idx}.
Adam Dickmeiss [Tue, 5 Dec 2006 09:26:04 +0000 (09:26 +0000)]
Reformat a bit.
Adam Dickmeiss [Tue, 5 Dec 2006 09:24:47 +0000 (09:24 +0000)]
Deal with NULL profile_path.
Adam Dickmeiss [Tue, 5 Dec 2006 09:24:31 +0000 (09:24 +0000)]
For Res(ources), maintain reference count (poor man's smart ptr).
Adam Dickmeiss [Tue, 5 Dec 2006 08:14:59 +0000 (08:14 +0000)]
Experiment with alternative reckeys store.
Adam Dickmeiss [Tue, 5 Dec 2006 08:14:47 +0000 (08:14 +0000)]
Only add first in field marker if there field has term(s) in it
Adam Dickmeiss [Sun, 3 Dec 2006 16:05:13 +0000 (16:05 +0000)]
Fixed bug #739: Extended service update fails with Alvis filter.
This change makes it possible to have register and shadow as same
directory. The size limits is only counted for each file type (so if
shadow is 10M and register is 20M the total size of directory is 30M).
Adam Dickmeiss [Sun, 3 Dec 2006 15:55:02 +0000 (15:55 +0000)]
Removed unused var
Adam Dickmeiss [Thu, 30 Nov 2006 11:03:57 +0000 (11:03 +0000)]
Removed a few heavily used yaz_log calls
Adam Dickmeiss [Thu, 30 Nov 2006 10:33:18 +0000 (10:33 +0000)]
Added support for specification of approximative limits for whole query.
This is specified as attribute type 12. Semantics is the same as
estimatehits in zebra.cfg.
Adam Dickmeiss [Thu, 30 Nov 2006 10:29:23 +0000 (10:29 +0000)]
Change IDs for the special retrieval stuff.
Adam Dickmeiss [Wed, 29 Nov 2006 18:06:57 +0000 (18:06 +0000)]
Disable xelm path ! case code in case of index_name (att in 1.3) being NULL.
It's not documented and not in use for Zebra 2. The .abs reader can
never generate it so it's dead code which just slows indexing down.
Adam Dickmeiss [Wed, 29 Nov 2006 10:31:29 +0000 (10:31 +0000)]
Cut memory for sorting in half when threaded sort is in effect.
Adam Dickmeiss [Wed, 29 Nov 2006 10:24:01 +0000 (10:24 +0000)]
Version 2.0.7.
Marc Cromme [Wed, 29 Nov 2006 09:01:53 +0000 (09:01 +0000)]
shortened zebra::meta:.sysno lement XML format
Marc Cromme [Tue, 28 Nov 2006 14:37:45 +0000 (14:37 +0000)]
added example of config file
Marc Cromme [Tue, 28 Nov 2006 14:18:26 +0000 (14:18 +0000)]
added entry on encoding directive
Marc Cromme [Tue, 28 Nov 2006 13:05:57 +0000 (13:05 +0000)]
added more information on character mapping
added entry on equivalent directive
added short examples
Adam Dickmeiss [Tue, 28 Nov 2006 11:42:13 +0000 (11:42 +0000)]
Towards 2.0.6.
Adam Dickmeiss [Tue, 28 Nov 2006 08:44:10 +0000 (08:44 +0000)]
Issue warning if staticrank is < 0. And set to 0
Adam Dickmeiss [Tue, 28 Nov 2006 08:43:53 +0000 (08:43 +0000)]
Added assert
Adam Dickmeiss [Mon, 27 Nov 2006 21:50:15 +0000 (21:50 +0000)]
Zebra 2 RPM with the following packages: idzebra-2.0, libidzebra-2.0,
libidzebra-2.0-modules, libidzebra-2.0-devel.
Adam Dickmeiss [Mon, 27 Nov 2006 14:27:12 +0000 (14:27 +0000)]
Require YAZ 2.1.40 or later
Adam Dickmeiss [Mon, 27 Nov 2006 10:40:33 +0000 (10:40 +0000)]
Avoid indexing of zebraInfo stuff. Problem is that explain.abs has lines
elm (4,2) targetInfo ExplainCategory
and it will zebraInfo child as ExplainCategory. This is now prevented
by making a zebraInfo node with no children.
Adam Dickmeiss [Mon, 27 Nov 2006 10:10:14 +0000 (10:10 +0000)]
Better debugging for indexing.. Introduction of zebra_log_dict_entry.
Adam Dickmeiss [Mon, 27 Nov 2006 10:09:48 +0000 (10:09 +0000)]
Fixed bug #736: Updates gets slower. The problem was that duplicate
name tags (with database name in it) was generated for attributeDetails.
This caused indexes to grow and grow for each update iteration.
Adam Dickmeiss [Mon, 27 Nov 2006 09:17:01 +0000 (09:17 +0000)]
Disable hit snippets for now. It does not work well and it slows retrieval
down a lot.
Adam Dickmeiss [Sat, 25 Nov 2006 09:15:19 +0000 (09:15 +0000)]
Fixed bug #735: Delete fails to process rest of file after a record that
does not exist,
Adam Dickmeiss [Fri, 24 Nov 2006 19:48:23 +0000 (19:48 +0000)]
Added new object files to list
Adam Dickmeiss [Fri, 24 Nov 2006 13:25:42 +0000 (13:25 +0000)]
Rephase a bit.
Adam Dickmeiss [Fri, 24 Nov 2006 13:25:02 +0000 (13:25 +0000)]
Mention --disable-shared option as a way to prevent modules to be
built.
Adam Dickmeiss [Fri, 24 Nov 2006 13:05:11 +0000 (13:05 +0000)]
Moved section about special retrieval zebra:: to Architecture chapter.
Added table with supported zebra:: elements.
Marc Cromme [Fri, 24 Nov 2006 12:21:31 +0000 (12:21 +0000)]
updated test since non-used attributes are not longer printed out
Adam Dickmeiss [Fri, 24 Nov 2006 11:35:23 +0000 (11:35 +0000)]
Using wrbuf rather than sprintf. Allow all syntaxes for ::data fetch.
Adam Dickmeiss [Thu, 23 Nov 2006 21:38:26 +0000 (21:38 +0000)]
Added check for sys/utsname.h.
Adam Dickmeiss [Thu, 23 Nov 2006 21:35:02 +0000 (21:35 +0000)]
Added Include of string.h
Adam Dickmeiss [Thu, 23 Nov 2006 21:34:22 +0000 (21:34 +0000)]
Omit test with multiple processes and update on FreeBSD. Bug #733.
Adam Dickmeiss [Thu, 23 Nov 2006 18:42:48 +0000 (18:42 +0000)]
Make buildconf.sh work for FreeBSD
Adam Dickmeiss [Thu, 23 Nov 2006 14:01:21 +0000 (14:01 +0000)]
Use size of rec->info stuff in meta record fetch. Failing that will
result in a SEGV or similar.
Marc Cromme [Thu, 23 Nov 2006 09:03:50 +0000 (09:03 +0000)]
implemented full metadata display of fast element set names
implemented more test of fast element set names, including zebra::meta and zebra::data element set names
updated documentation on fast element set names
Adam Dickmeiss [Wed, 22 Nov 2006 14:12:49 +0000 (14:12 +0000)]
Added include of yaz/xmalloc.h.
Adam Dickmeiss [Wed, 22 Nov 2006 14:06:53 +0000 (14:06 +0000)]
Remove function tl_start_log.
Adam Dickmeiss [Wed, 22 Nov 2006 11:13:17 +0000 (11:13 +0000)]
Added a few memset for structures that are later written to disk.
This makes valgind happy on platforms which aligns members in structs.
Adam Dickmeiss [Wed, 22 Nov 2006 10:26:12 +0000 (10:26 +0000)]
Added initializer value for two locals
Adam Dickmeiss [Wed, 22 Nov 2006 09:42:27 +0000 (09:42 +0000)]
For build script: -p enables profiling.
Adam Dickmeiss [Tue, 21 Nov 2006 22:17:49 +0000 (22:17 +0000)]
Get rid of SYSNO which is zint anyway. Removed various prototypes
of unused/undefined functions.
Adam Dickmeiss [Tue, 21 Nov 2006 17:55:47 +0000 (17:55 +0000)]
Added
Adam Dickmeiss [Tue, 21 Nov 2006 17:48:08 +0000 (17:48 +0000)]
Make threading for index sort configurable.
Adam Dickmeiss [Tue, 21 Nov 2006 14:54:12 +0000 (14:54 +0000)]
Remove log msg
Adam Dickmeiss [Tue, 21 Nov 2006 14:32:38 +0000 (14:32 +0000)]
Split it_key stuff into it_key.c. For indexing, sort key chunks in a separate
thread (POSIX threads only).
Adam Dickmeiss [Mon, 20 Nov 2006 13:59:35 +0000 (13:59 +0000)]
Increase reckeys hash size from 1023 to 32767.
Adam Dickmeiss [Mon, 20 Nov 2006 13:59:13 +0000 (13:59 +0000)]
Remove obsolete SORT_EXTRA code (no longer in use).
Adam Dickmeiss [Fri, 17 Nov 2006 17:32:45 +0000 (17:32 +0000)]
Contact address for Zebra is zebra-help@indexdata.dk
Marc Cromme [Fri, 17 Nov 2006 14:54:00 +0000 (14:54 +0000)]
updated docs to reflect change to 'zebra::index::' special elem set name
Marc Cromme [Fri, 17 Nov 2006 14:24:56 +0000 (14:24 +0000)]
added tests fror wrong element set names which are not recognized
Marc Cromme [Fri, 17 Nov 2006 13:47:22 +0000 (13:47 +0000)]
preventing later memory acess violation when one leaves retrieve.c:zebra_special_fetch by error condition before setting *rec_lenp by setting int len = 0; upon initialization.
Marc Cromme [Fri, 17 Nov 2006 08:20:17 +0000 (08:20 +0000)]
removing unnecessary comments
not setting the *rec_lenp = 0; at retrieve.c:127 to provoke test failure with segfault in the last unit test of test/api/t16.c , which is commented out. It seems like *rec_lenp = wrbuf_len(wrbuf); at retrieve.c:247 leaves rec_lenp in an uninitialized state, which causes a segfault later. More investigation needed.
Marc Cromme [Thu, 16 Nov 2006 21:58:54 +0000 (21:58 +0000)]
protected wrong buffer-length write by initializing to 0, but there is more debugging needed: there is still something fishy with the last out-commented test of test/api/t16.c.
Marc Cromme [Thu, 16 Nov 2006 20:44:28 +0000 (20:44 +0000)]
added tests for zebra::data, zebra::meta::sysno and zebra::index::title:p element sets. Last one segfaults and need further debugging, it is commented out.
Adam Dickmeiss [Thu, 16 Nov 2006 13:27:54 +0000 (13:27 +0000)]
For alvis filter make XML pass-through if stylesheet is attribute
is omitted (not specified).