Description
URLSelectCacheSize
defines the number of URLs to load from the database
as crawler targets in a single portion. The bigger number makes
crawling performance better, however in case when
indexer crashes or is killed
or is terminated using Ctrl-Break,
some documents will get stuck in crawling queue and
won't be available for crawling for 4 hours.
Note:
To make stuck documents available for crawling
use indexer -a flag, in combination
with suitable
subsection control option, like
-s, -u, etc.
The default value is 128.