Revision as of 05:51, 27 December 2013

Configuration File

Crawler's personalities are driven to a great extent by means of configuration files.

There is no hard rule that dictates how these configuration files should be created - they could be text files, INI-files, XML-based files,...

Because .INI files are easy to understand for end-users, most of the pre-made personalities use .INI-based configuration files.

For more popular personalities, GUI-driven configuration tools might be provided, but in many cases, the GUI-development might lag, in which case the next easiest method to (re)configure a Crawler personality is to edit one or more configuration files.

INI file

Basic INI files are a loosely defined de-facto standard; more info can be found here.

The Crawler INI files have the following properties:

• Section and entry names are case-insensitive by default (but Crawler has built-in support for case-sensitive INI files should the need arise).
• Comment lines are supported. Prefixing a line with a '#' or a ';' makes it a comment line. In-line comments are not supported: a single line is either a comment line or it is not - comments on lines with data are not supported. For example:

entry = test # test

means to set entry to "test # test"

• Blank lines are allowed (and ignored)
• If an INI file is missing a section, then any entries without section are assumed to be in a default section [main]
• Duplicate names are allowed, and provide an 'override' mechanism. If an entry appears twice, the second appearance will 'win'.