Configuration File

Configuration File

Crawler's personalities are driven to a great extent by means of configuration files.

There is no hard rule that dictates how these configuration files should be created - they could be text files, INI-files, XML-based files,...

Because .INI files are easy to understand for end-users, most of the pre-made personalities use .INI-based configuration files.

For more popular personalities, GUI-driven configuration tools might be provided, but in many cases, the GUI-development might lag, in which case the next easiest method to (re)configure a Crawler personality is to edit one or more configuration files.

INI file

Basic INI files are a loosely defined de-facto standard; more info can be found here.

The Crawler INI files have the following properties:

• Section and entry names are case-insensitive
• Comment lines are supported. Prefixing a line with a '#' or a ';' makes it a comment line. In-line comments are not supported: a single line is either a comment line or it is not - comments on lines with data are not supported. For example:

entry = test # test

means to set entry to "test # test"