fts: update docs and defaults
This commit is contained in:
Martin Weinelt
+26
-10
@@ -591,18 +591,27 @@ in
|
||||
memoryLimit = mkOption {
|
||||
type = types.nullOr types.int;
|
||||
default = null;
|
||||
example = 2000;
|
||||
example = 1024;
|
||||
description = ''
|
||||
Memory limit for the indexer process, in MiB.
|
||||
If null, leaves the default (which is rather low),
|
||||
and if 0, no limit.
|
||||
|
||||
When `null` the [`default_vsz_limit`](https://doc.dovecot.org/main/core/plugins/fts.html#fts_search_read_fallback)
|
||||
applies while with `0` no limit is applied.
|
||||
'';
|
||||
};
|
||||
|
||||
autoIndex = mkOption {
|
||||
type = types.bool;
|
||||
default = true;
|
||||
description = "Enable automatic indexing of messages as they are received or modified.";
|
||||
description = ''
|
||||
Enable automatic indexing of messages as they are received or modified.
|
||||
|
||||
:::{tip}
|
||||
Can be overridden per mailbox by setting `fts_autoindex` for
|
||||
{option}`mailserver.mailboxes`. By default the Junk and Trash folders
|
||||
are already excluded.
|
||||
:::
|
||||
'';
|
||||
};
|
||||
|
||||
fallback = mkOption {
|
||||
@@ -611,6 +620,8 @@ in
|
||||
description = ''
|
||||
Whether to fallback to slow non-indexed search, if FTS lookup and
|
||||
indexing have failed.
|
||||
|
||||
See <https://doc.dovecot.org/main/core/plugins/fts.html#fts_search_read_fallback>.
|
||||
'';
|
||||
};
|
||||
|
||||
@@ -635,10 +646,13 @@ in
|
||||
type = types.bool;
|
||||
default = false;
|
||||
description = ''
|
||||
If enabled, allows substring searches.
|
||||
See <https://doc.dovecot.org/main/core/plugins/fts_flatcurve.html#fts_flatcurve_substring_search>.
|
||||
Whether to allows substring searches. By default only prefix searches are supported.
|
||||
|
||||
Enabling this requires significant additional storage space.
|
||||
:::{warning}
|
||||
Enabling this significantly increases storage requirements.
|
||||
:::
|
||||
|
||||
See <https://doc.dovecot.org/main/core/plugins/fts_flatcurve.html#fts_flatcurve_substring_search>.
|
||||
'';
|
||||
};
|
||||
|
||||
@@ -651,7 +665,8 @@ in
|
||||
"Comments"
|
||||
];
|
||||
description = ''
|
||||
The list of headers to exclude.
|
||||
The list of headers to exclude while indexing.
|
||||
|
||||
See <https://doc.dovecot.org/main/core/plugins/fts.html#fts_header_excludes>.
|
||||
'';
|
||||
};
|
||||
@@ -664,8 +679,9 @@ in
|
||||
"stopwords"
|
||||
];
|
||||
description = ''
|
||||
The list of language filters to apply.
|
||||
See <https://doc.dovecot.org/main/core/plugins/fts.html#filter-configuration>.
|
||||
The list of [language filters] to apply.
|
||||
|
||||
[language filters]: https://doc.dovecot.org/main/core/plugins/fts.html#filter-configuration
|
||||
'';
|
||||
};
|
||||
};
|
||||
|
||||
+24
-22
@@ -4,7 +4,7 @@ Full text search
|
||||
By default, when your IMAP client searches for an email containing some
|
||||
text in its *body*, dovecot will read all your email sequentially. This
|
||||
is very slow and IO intensive. To speed body searches up, it is possible to
|
||||
*index* emails with a plugin to dovecot, ``fts_flatcurve``.
|
||||
*index* emails with the ``fts_flatcurve`` dovecot plugin.
|
||||
|
||||
Enabling full text search
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
@@ -20,48 +20,50 @@ To enable indexing for full text search here is an example configuration.
|
||||
enable = true;
|
||||
# index new email as they arrive
|
||||
autoIndex = true;
|
||||
# only query index
|
||||
fallback = false;
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
The ``fallback`` parameter tells dovecot to fail any body search query that cannot
|
||||
use an index. This prevents dovecot to fall back to the IO-intensive brute
|
||||
force search.
|
||||
Disabling the :option:`mailserver.fullTextSearch.fallback` option tells dovecot
|
||||
to fail any body search query that cannot use an index. This prevents Dovecot to
|
||||
fall back to the IO-intensive brute force search.
|
||||
|
||||
If you set ``autoIndex`` to ``false``, indices will be created when the IMAP client
|
||||
issues a search query, so latency will be high.
|
||||
If you set :option:`mailserver.fullTextSearch.autoIndex` to ``false``, indices
|
||||
will be created when the IMAP client issues a search query, so latency will
|
||||
be high.
|
||||
|
||||
Resource requirements
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Indices created by the full text search feature can take more disk
|
||||
space than the emails themselves. By default, they are kept in the
|
||||
emails location. When enabling the full text search feature, it is
|
||||
recommended to move indices in a different location, such as
|
||||
(``/var/lib/dovecot/indices``) by using the option
|
||||
``mailserver.indexDir``.
|
||||
Indices created by the full text search feature can take more disk space than
|
||||
the emails themselves. By default, they are kept within the maildir. When
|
||||
enabling the full text search feature, it is recommended to move indices in a
|
||||
different location, such as (``/var/lib/dovecot/indices``) by configuring
|
||||
:option:`mailserver.indexDir`.
|
||||
|
||||
.. warning::
|
||||
|
||||
When the value of the ``indexDir`` option is changed, all dovecot
|
||||
indices needs to be recreated: clients would need to resynchronize.
|
||||
When the value of the :option:`mailserver.indexDir` option is changed, all
|
||||
dovecot indices needs to be recreated: clients would need to resynchronize.
|
||||
|
||||
Indexation itself is rather resource intensive, in CPU, and for emails with
|
||||
large headers, in memory as well. Initial indexation of existing emails can take
|
||||
hours. If the indexer worker is killed or segfaults during indexation, it can
|
||||
be that it tried to allocate more memory than allowed. You can increase the memory
|
||||
limit by eg ``mailserver.fullTextSearch.memoryLimit = 2000`` (in MiB).
|
||||
hours. If the indexer worker is killed or segfaults during indexation, it can be
|
||||
that it tried to allocate more memory than allowed. You can increase the default
|
||||
memory limit through :option:`mailserver.fullTextSearch.memoryLimit`.
|
||||
|
||||
Mitigating resources requirements
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
You can:
|
||||
|
||||
* exclude some headers from indexation with ``mailserver.fullTextSearch.headerExcludes``
|
||||
* disable expensive token normalisation in ``mailserver.fullTextSearch.filters``
|
||||
* disable automatic indexation for some folders with
|
||||
``mailserver.fullTextSearch.autoIndexExclude``. Folders can be specified by
|
||||
name (``"Trash"``), by special use (``"\\Junk"``) or with a wildcard.
|
||||
* exclude some headers from indexation with :option:`mailserver.fullTextSearch.headerExcludes`
|
||||
* disable expensive token normalisation in :option:`mailserver.fullTextSearch.filters`
|
||||
* disable automatic indexation for individual mailboxes by overriding
|
||||
`fts_autoindex`_ on the mailbox level. This is exposed via
|
||||
:option:`mailserver.mailboxes`, where all default mailboxes are defined.
|
||||
|
||||
.. _fts_autoindex: https://doc.dovecot.org/main/core/plugins/fts.html#fts_autoindex
|
||||
|
||||
Reference in New Issue
Block a user