indexer skips downloading and parsing the documents with
Section matching the given
Every time a matching document expires and appears in the crawler queue,
indexer just marks the document
as fresh again by modifying
its next_index_time value.
Word and section information about the document remains untouched.
SkipIf can be useful for excluding
sites from revisiting when, for example, the sites are
temporarily not available.
The meaning of the first three optional parameters
is exactly the same as in the Allow
Section parameter specifies which
section is checked against the
It can also be a concatenation of multiple sections, composed
with help of
It's possible to use multiple patterns in
the same SkipIf command.